JP6003574B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program.

  Patent Document 1 discloses a step of dividing an input image into pixel groups having a predetermined size, a step of calculating a pixel group density in the divided pixel group, and an attention in the image among the pixel groups. Based on the relative density of the pixel of interest calculated based on the pixel group density of the pixel group to which the pixel belongs and the pixel group density of the pixel group adjacent to the pixel group to which the pixel of interest belongs, and the absolute density of the pixel of interest A step of calculating an output value of the pixel of interest, and character recognition, etc. at high speed by cutting out characters written with a pen or the like relatively darker than the background in the multi-valued image It is disclosed that an object such as a character graphic can be emphasized and the image size can be compressed without impairing the ease of understanding.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses an image luminance correction apparatus for correcting the luminance of a scanned image obtained by reading a book document placed on a contact glass by an image reading unit, and a block dividing unit that divides the scanned image into a plurality of blocks. The pixel having the highest luminance value among the pixels included in each of the blocks divided by the block dividing means is selected, and the luminance value of the pixel having the highest luminance value is regarded as the background value of each block. A luminance value to be subjected to luminance correction processing on the scanned image based on a background value obtained by smoothing the background value of each block detected by the background value detecting unit and the background value detected by the background value detecting unit. And an image brightness correction device characterized by comprising: a correction means; Even when the placement on the glass is uneven (when the top and bottom of the page are different from the contact glass), the brightness of the scanned image is corrected to a band based on the background value of a portion of the scanned image. Since the background value is detected in the local range and the brightness is corrected compared to the case where the image is corrected, it is possible to prevent black streaks and light shadows from appearing in the image after the brightness correction, and to reduce the influence of noise. It is disclosed that it becomes possible to suppress. Basically, a correction formula for each pixel is realized by “correction result = actual measurement value × designated luminance value ÷ representative luminance value” by using the maximum luminance value of each block obtained by dividing the fog condition into blocks.

  In Patent Document 3, when the image gradation correction technique is applied to backlight correction, there is a limit to improving local contrast in a bright region or a dark region. Divides one screen into a plurality of blocks, the average luminance calculation unit calculates the average luminance level in block units designated by the block timing generation unit, and the block unit correction amount calculation unit calculates in block units. From the average luminance level, the correction direction and the correction amount are calculated, the pixel unit correction amount calculation unit interpolates the correction amount in block units into the correction amount in each pixel unit in the block, and the gradation conversion unit It is disclosed that gradation conversion is performed by reading the final correction amount from the gradation conversion table using the correction amount in pixel units and the luminance data of the pixel read from the memory as addresses.

  Patent Document 4 has a problem that, when the backlight correction is performed, the color of the bright portion in the image is lost, and it is difficult to properly adjust the color balance of the entire image. A specific image detecting unit for detecting a region including at least a part of the specific image, a representative color calculating unit for calculating a representative color representing the specific image based on the pixels belonging to the detected region, a representative color A first correction curve acquisition unit that acquires a first correction curve for gradation correction based on brightness, and the representative color is corrected by the first correction curve, and the level for each element color that constitutes the corrected representative color A second correction curve acquisition unit that acquires a second correction curve for gradation correction for each element color based on a tone value; and gradations of pixels that belong to a color gamut that defines a dark part among pixels constituting the input image First correction to correct the value using the first correction curve When, it was configured to include a second correction unit that corrects using the second correction curve the tone values of element colors of the pixels constituting the input image is disclosed.

  Patent Document 5 discloses a specific image detection unit that detects a region including at least a part of a specific image in an input image, with the problem that a bright portion in the image has lost color when backlight correction is performed. A difference acquisition unit that acquires the difference between the brightness in the area detected by the specific image detection unit and the brightness of the background area in the input image, and acquires a correction curve for tone correction based on the difference And a correction unit that corrects a gradation value of a pixel that belongs to a color gamut that defines a dark portion among the pixels that constitute the input image using the correction curve. ing.

  Japanese Patent Laid-Open No. 2004-133867 has a problem of performing appropriate image correction according to the type of photographed document and the background color, and a digital camera photographs a document (characters, drawings, photographs, etc.) on a white plate as a photographing object. Luminance histogram and hue histogram from the acquired image, luminance histogram parameters indicating the maximum, minimum, and peak values in the acquired luminance histogram, and maximum, minimum, peak, average, and color differences in the hue histogram A hue histogram parameter indicating a variance value is extracted, and using the extracted luminance histogram parameter and hue histogram parameter, the type and background color of the photographed document are determined, and according to the determined document type and background color, It is disclosed that image processing for improving the visibility of a document is optimized.

  Patent Document 7 discloses a binarization processing apparatus capable of performing binarization accurately even when the brightness of an input image is uneven or the background density varies depending on the location. The input image is blocked for each predetermined area in the block forming circuit, the edge area of the graphic area is extracted in the edge area extracting circuit, and the density histogram of the edge area pixels is obtained from the edge extraction result. By calculating, the binarization threshold value calculation circuit calculates a binarization threshold value for each block, the background region extraction circuit extracts the background region for each block, and the binarization threshold value interpolation data calculation circuit calculates the background region. Binarization threshold interpolation data is calculated based on the average value of the pixel density values and the binarization threshold, and the block including the graphic region binarizes the input image based on the binarization threshold. There, the block of the background region that was performing the binarization based on the binarization threshold interpolation data is disclosed.

  In Patent Document 8, it is an object to remove a shadow portion without erasing characters located in the vicinity of the saddle stitch portion, and the hue of a background portion is specified in image data obtained by reading a spread original, and the specified hue is determined. When a pixel that can be regarded as a hue close to is extracted, and the lightness integral value obtained by integrating the lightness values for one line of pixel data strings along the Y-axis direction is obtained for all X coordinate values, a histogram H1 is obtained. Then, when a peak having a width wider than the peak width (for example, 5 mm) obtained by the formation of characters is extracted, a histogram H2 is obtained. Further, when smoothing processing is performed by inverting the brightness integral value, A histogram H3 is obtained, and then, for each of the pixel data strings of one line along the Y-axis direction, the peak of the histogram H3 is used as a saddle stitched shade portion. Flip be corrected to remove the shadow portion is disclosed.

  In Patent Document 9, it is an object to perform a stable binarization process on a shadowed image, an image including a wide figure region, or an image including a wide ground region. This image processing apparatus determines an original image in advance. A background lightness candidate detection unit that detects a lightness value candidate to be used as the lightness of the background of the original image for each partial region by dividing into a plurality of partial regions in accordance with the determined procedure, and the background of the original image using the detected candidates A background lightness distribution estimation unit that estimates lightness distribution, a threshold value calculation unit that calculates a threshold value for binarization processing of the original image based on the estimated background lightness distribution, and a calculated threshold value And a binarization unit that binarizes the original image.

  Patent Document 10 aims to provide an object detection device capable of detecting an object straddling the boundary between light and shadow with high accuracy, and the background side luminance distribution obtained by analyzing the background image is a bimodal distribution. When it is determined that, the image correction unit performs a luminance conversion for expanding the luminance distribution range of the pixel group distributed in the high luminance region to each pixel of the input image to generate a high luminance side corrected image, and A luminance conversion that expands the luminance distribution range of the pixel group distributed to each pixel of the input image to generate a low-luminance correction image, an average image of the high-luminance correction image and the low-luminance correction image, The edge extraction means extracts edge information from the average image, and the object detection means can detect the object based on the edge information and suppress the boundary edge between light and shadow while enhancing the contour edge of the object. , The detection accuracy can be improved It is shown.

Japanese Patent No. 3426189 Japanese Patent No. 4111697 JP 2005-341527 A JP 2009-290660 A JP 2009-290661 A JP 2005-260657 A JP 2005-165925 A JP 2005-115768 A JP 2007-249836 A JP 2011-028686 A

  It is an object of the present invention to provide an image processing apparatus and an image processing program that prevent the removal of a design in a document similar to the fog when removing the fog in the image. It is said.

The gist of the present invention for achieving the object lies in the inventions of the following items.
The invention according to claim 1 estimates the fogging amount, which is a difference between the luminance value of the original pixel and the luminance value of the pixel of the image, based on the receiving unit that receives the image and the image received by the receiving unit. By comparing the estimation means, the extraction means for extracting the area of the document image taken from the image received by the reception means, and the images inside and outside the area extracted by the extraction means, Judgment means for judging whether or not it is only in the area, and if the judgment means judges that fog is occurring only in the area, the amount of fogging Control means for controlling so as not to remove, measuring means for measuring the saturation value of the pixels in the image received by the receiving means, and received by the receiving means Determining means for determining a correction target value of a luminance value in the background in the image; the fog amount estimated by the estimating means; the correction target value determined by the determining means; and the saturation value measured by the measuring means An image processing apparatus comprising correction means for correcting the saturation value based on a distance from the chromatic color.

  According to a second aspect of the present invention, the region extracted by the extraction unit is a quadrangle, the determination unit compares the outside and inside images of the sides of the quadrangle, and the inside image is the rectangle. The image processing apparatus according to claim 1, wherein the image processing apparatus includes a region within a predetermined distance from the side and having a fog amount estimated by the estimation unit.

  The invention according to claim 3 is characterized in that the correction means corrects the saturation value to be reduced as the saturation value measured by the measurement means becomes closer to an achromatic color. Alternatively, the image processing apparatus according to 2.

  According to a fourth aspect of the present invention, there is provided judging means for judging whether or not to perform correction by the correcting means based on a difference between the fogging amount estimated by the estimating means and the correction target value determined by the determining means. 4. The image processing apparatus according to claim 1, wherein the correction unit performs correction when it is determined that the correction is performed by the determination unit. 5.

  According to a fifth aspect of the present invention, determination means for determining whether to perform correction by the correction means based on a difference between the fog amounts estimated by the estimation means in each pixel in the image received by the reception means. 4. The image processing apparatus according to claim 1, further comprising a correction unit configured to correct the correction unit when the determination unit determines to perform correction. 5. .

  The invention according to claim 6 estimates the fogging amount which is a difference between the luminance value of the original pixel and the luminance value of the pixel of the image based on the receiving unit which receives the image and the image received by the receiving unit. By comparing the estimation means, the extraction means for extracting the area of the document image taken from the image received by the reception means, and the images inside and outside the area extracted by the extraction means, Judgment means for judging whether or not it is only in the area, and if the judgment means judges that fog is occurring only in the area, the amount of fogging Control means for controlling so as not to remove, measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means, and receiving by the receiving means Determining means for determining a correction target value of the luminance value in the background in the obtained image; the fog amount estimated by the estimating means; the correction target value determined by the determining means; and the saturation value measured by the measuring means First correction means for correcting the saturation value based on the distance from the achromatic color, and the measurement means based on the fog amount estimated by the estimation means and the correction target value determined by the determination means An image processing apparatus comprising: a second correction unit that corrects the luminance value measured by the method.

  According to a seventh aspect of the present invention, the area extracted by the extracting means is a quadrangular shape, and the determining means compares the images inside and outside the sides of the quadrangular shape, and the inside image is the rectangular shape. The image processing apparatus according to claim 6, wherein the image processing apparatus includes a region within a predetermined distance from the side and having a fog amount estimated by the estimation unit.

  The invention according to claim 8 performs correction by the first correction unit and the second correction unit based on a difference between the fog amount estimated by the estimation unit and the correction target value determined by the determination unit. And determining means for determining whether or not the first correction means and the second correction means perform correction when the determination means determines that correction is to be performed. An image processing apparatus according to claim 6 or 7.

  The invention according to claim 9 is a correction by the first correction unit and the second correction unit based on a difference between the fog amounts estimated by the estimation unit in each pixel in the image received by the reception unit. The first correction unit and the second correction unit perform correction when it is determined by the determination unit that correction is to be performed. An image processing apparatus according to claim 6 or 7.

Rules invention of claim 10, in which the receiving means for receiving image, first extracting means for extracting an area from the image received by said receiving means, and preset the region extracted by the first extracting means Selection means for selecting based on the measurement means, measurement means for measuring the luminance value and saturation value of the pixels in the region selected by the selection means, and the reception means receiving from the luminance values measured by the measurement means Estimating means for estimating a function indicating the degree of fogging in the received image, second extracting means for extracting a region of a document image taken from the image received by the receiving means, and the second extracting means A judgment method for determining whether or not the fog is generated only in the area by comparing the image inside the area extracted by the above and the image outside the area. When the determination means determines that the fog is occurring only in the area, the control means for controlling so as not to remove the fogging amount, and the reception means A determining unit that determines a correction target value of a luminance value in a background in the image; a fog amount is calculated based on the function estimated by the estimating unit; the fog amount and a correction target value determined by the determining unit; Based on the distance from the achromatic color of the saturation value measured by the measurement means, a first correction means for correcting the saturation value and a fog amount based on the function estimated by the estimation means are calculated. And a second correcting means for correcting the brightness value measured by the measuring means based on the fogging amount and the correction target value determined by the determining means. That is an image processing apparatus.

According to an eleventh aspect of the present invention, the region extracted by the second extracting means is a quadrangular shape, and the determining means compares the outside and inside images of the sides of the square, and the inside image is the The image processing apparatus according to claim 10, wherein the image processing apparatus includes an area that is within a predetermined distance inward from a side of the quadrangle and whose fogging amount is estimated by the estimation unit.

  According to a twelfth aspect of the present invention, a fog amount is calculated based on the function estimated by the estimation unit, and the first correction is performed based on a difference between the fog amount and a correction target value determined by the determination unit. And a determination means for determining whether or not to perform correction by the second correction means. The first correction means and the second correction means are determined to be corrected by the determination means. The image processing apparatus according to claim 10, wherein correction is performed when the image processing is performed.

  According to a thirteenth aspect of the present invention, a fog amount is calculated based on the function estimated by the estimating means, and the first amount is calculated based on a difference between the fog amounts in each pixel in the image received by the receiving means. And determining means for determining whether to perform correction by the second correcting means, and the first correcting means and the second correcting means perform correction by the determining means. The image processing apparatus according to claim 10, wherein correction is performed when the determination is made.

  According to the invention of claim 14, the computer is configured to receive the image, and based on the image received by the receiving unit, the fogging amount that is the difference between the luminance value of the original pixel and the luminance value of the pixel of the image By comparing the estimation means for estimating the image, the extraction means for extracting the area of the document image taken from the image received by the reception means, and the images inside and outside the area extracted by the extraction means A determination means for determining whether or not the fog is generated only in the area; and when the determination means determines that the fog is generated only in the area A control means for controlling so as not to remove the fogging amount, a measuring means for measuring a saturation value of a pixel in the image received by the receiving means, and the receiving means. Determining means for determining the correction target value of the luminance value in the background in the received image, the fog amount estimated by the estimating means, the correction target value determined by the determining means, and the saturation measured by the measuring means An image processing program that functions as a correction unit that corrects a saturation value based on a distance from the achromatic color.

  According to the fifteenth aspect of the present invention, the amount of fogging that is the difference between the luminance value of the original pixel and the luminance value of the pixel of the image is determined based on the image received by the receiving unit and the receiving unit. By comparing the estimation means for estimating the image, the extraction means for extracting the area of the document image taken from the image received by the reception means, and the images inside and outside the area extracted by the extraction means A determination means for determining whether or not the fog is generated only in the area; and when the determination means determines that the fog is generated only in the area Control means for controlling not to remove the fogging amount, measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means, and the receiving A determination means for determining a correction target value of the luminance value in the background in the image received by the stage; a fog amount estimated by the estimation means; a correction target value determined by the determination means; and the measurement means Based on the first correction means for correcting the saturation value based on the distance of the saturation value from the achromatic color, the fog amount estimated by the estimation means and the correction target value determined by the determination means, An image processing program that functions as a second correction unit that corrects a luminance value measured by the measurement unit.

The invention of claim 16 causes a computer, and receiving means for receiving image, first extracting means for extracting an area from the image received by the receiving unit, the area extracted by said first extraction means previously From the selection means for selecting based on a predetermined rule, the measurement means for measuring the luminance value and the saturation value of the pixels in the area selected by the selection means, and the reception value from the luminance value measured by the measurement means Estimation means for estimating a function indicating the degree of fogging in the image received by the means, second extraction means for extracting a region of the document image taken from the image received by the reception means, and the second by comparing the images of the outside and within the extraction area region by the extracting means, the head has occurred, whether it is only within that region Determining means for determining, control means for controlling so as not to remove the fogging amount when the determining means determines that the fog is occurring only in the area; and the receiving means And determining means for determining a correction target value of the luminance value of the background in the image received by the calculation means, calculating the fog amount based on the function estimated by the estimation means, and determining the fog amount and the determination means First correction means for correcting the saturation value based on the correction target value and the distance from the achromatic color of the saturation value measured by the measurement means, and fogging based on the function estimated by the estimation means As a second correction unit that calculates the amount and corrects the luminance value measured by the measurement unit based on the fogging amount and the correction target value determined by the determination unit An image processing program for causing the performance.

  According to the image processing apparatus of the first aspect, in the case of removing the fog in the image, it is possible to prevent the removal of the design in the document similar to the fog.

  According to the image processing apparatus of the second aspect, it is possible to accurately determine whether or not the fog is generated only in the region as compared with the case where the present configuration is not provided.

  According to the image processing apparatus of the third aspect, it is possible to prevent the saturation from being excessively emphasized with respect to the achromatic color portion.

  According to the image processing apparatus of the fourth aspect, it is possible to determine whether or not to perform correction based on the difference between the fogging amount and the correction target value.

  According to the image processing apparatus of the fifth aspect, it is possible to determine whether or not to perform correction based on the difference between the fogging amounts in each pixel.

  According to the image processing apparatus of the sixth aspect, in the case of removing the fog in the image, it is possible to prevent the removal of the design in the document similar to the fog, and in the image In the case of correcting the luminance value in order to remove the fog, it is possible to prevent the luminance value from becoming too bright with respect to a portion other than the background.

  According to the image processing apparatus of the seventh aspect, it is possible to accurately determine whether or not the fog is generated only in the region as compared with the case where the present configuration is not provided.

  According to the image processing apparatus of the eighth aspect, it is possible to determine whether or not to perform correction based on the difference between the fogging amount and the correction target value.

  According to the image processing apparatus of the ninth aspect, it is possible to determine whether or not to perform correction based on the difference between the fogging amounts in each pixel.

  According to the image processing apparatus of the tenth aspect, in the case of removing the fog in the image, it is possible to prevent the design in the original similar to the fog from being removed and to be a target. When a chromatic color region exists in the image, it is possible to prevent the luminance value generated when removing the fog corresponding to the document color from becoming too bright in the entire image.

  According to the image processing apparatus of the eleventh aspect, it is possible to accurately determine whether or not the occurrence of the fog is only in the region as compared with the case where the present configuration is not provided.

  According to the image processing apparatus of the twelfth aspect, it is possible to determine whether or not to perform correction based on the difference between the fogging amount and the correction target value.

  According to the image processing apparatus of the thirteenth aspect, it is possible to determine whether or not to perform correction based on the difference between the fogging amounts in each pixel.

  According to the image processing program of the fourteenth aspect, when the fog in the image is removed, it is possible to prevent the design in the document similar to the fog from being removed.

  According to the image processing program of the fifteenth aspect, in the case of removing the fog in the image, it is possible to prevent the design in the document similar to the fog from being removed, and in the image In the case of correcting the luminance value in order to remove the fog, it is possible to prevent the luminance value from becoming too bright with respect to a portion other than the background.

  According to the image processing program of the sixteenth aspect, in the case of removing the fog in the image, it is possible to prevent the removal of the design in the document similar to the fog, and to be the target. When a chromatic color region exists in the image, it is possible to prevent the luminance value generated when removing the fog corresponding to the document color from becoming too bright in the entire image.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is a flowchart which shows the process example by 1st Embodiment. It is explanatory drawing which shows the process example by an image reception module. It is explanatory drawing which shows the process example by a fog amount estimation module, a pixel value measurement module, and a correction target value determination module. It is explanatory drawing which shows the example of a process by a saturation correction module. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is a flowchart which shows the process example by 2nd Embodiment. It is explanatory drawing which shows the example of a process by the saturation correction | amendment propriety module. It is a conceptual module block diagram about the structural example of 3rd Embodiment. It is a flowchart which shows the process example by 3rd Embodiment. It is explanatory drawing which shows the process example by an image reception module. It is explanatory drawing which shows the process example by a fog amount estimation module, a pixel value measurement module, and a correction target value determination module. It is explanatory drawing which shows the process example by a brightness correction module. It is a notional module block diagram about the structural example of 4th Embodiment. It is a flowchart which shows the process example by 4th Embodiment. It is a notional module block diagram about the structural example of 5th Embodiment. It is a flowchart which shows the process example by 5th Embodiment. It is explanatory drawing which shows the process example by an image reception module. It is explanatory drawing which shows the process example by a non-edge area | region extraction module. It is explanatory drawing which shows the process example by a non-edge area | region extraction module. It is explanatory drawing which shows the process example by a test | inspection area | region selection module. It is explanatory drawing which shows the example of a process by a pixel value measurement module. It is explanatory drawing which shows the example of a process by a pixel value measurement module. It is explanatory drawing which shows the process example by a pixel value distribution estimation module. It is explanatory drawing which shows the process example by a pixel value distribution estimation module. It is explanatory drawing which shows the process example by a brightness correction module. It is explanatory drawing which shows the process example by a brightness correction module. It is explanatory drawing which shows the process example by a brightness correction module. It is a notional module block diagram about the structural example of 6th Embodiment. It is a flowchart which shows the process example by 6th Embodiment. It is a notional module block diagram about the structural example of 7th Embodiment. It is a flowchart which shows the process example by 7th Embodiment. It is explanatory drawing which shows the example of a relationship between the scanning line of an image, and a line and function correspondence table. It is explanatory drawing which shows the example of the difference norm in a line and function correspondence table. It is a notional module block diagram about the structural example of 8th Embodiment. It is explanatory drawing which shows the example of the image made into object by 8th Embodiment. It is explanatory drawing which shows the example in the scanning line of the image made into object by 8th Embodiment. It is explanatory drawing which shows the example of a process by 8th Embodiment. It is explanatory drawing which shows the example of the difference norm in a line and function correspondence table. It is explanatory drawing which shows the example of a process by 8th Embodiment. It is explanatory drawing which shows the example of a process by 8th Embodiment. It is explanatory drawing which shows the example of a process by 8th Embodiment. It is a notional module block diagram about the structural example of 9th Embodiment. It is explanatory drawing which shows the example of the image made into object by 9th Embodiment. It is explanatory drawing which shows the process example by 9th Embodiment. It is explanatory drawing which shows the process example by 9th Embodiment. It is explanatory drawing which shows the process example by 9th Embodiment. It is explanatory drawing which shows the process example by 9th Embodiment. It is explanatory drawing which shows the process example by 9th Embodiment. It is explanatory drawing which shows the process example by 9th Embodiment. It is explanatory drawing which shows the technical example used as the premise of this Embodiment. It is explanatory drawing which shows the technical example used as the premise of this Embodiment. It is explanatory drawing which shows the technical example used as the premise of this Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

First, before explaining the present embodiment, a technique that is a premise thereof will be described. This description is intended to facilitate understanding of the present embodiment.
The photographing target document 5100 shown in the example of FIG. 51A is a business card and includes not only black and white but also a chromatic region. Specifically, the figure on the upper right in the photographing target document 5100 is a red mark (logo). The photographing target document 5100 is photographed by, for example, a digital camera, and a photographed image 5130 shown in the example of FIG. In this case, the fogged image 5120 shown in the example of FIG. This fogging image 5120 depends on the lighting, backlight, setting of the shooting target document 5100 (that is, the positional relationship between the shooting target document 5100 and the lighting and the shooting device), or the configuration of the lens in the digital camera (for example, the central portion of the image). Is bright and the surrounding part is dark). As shown in the example of FIG. 51 (b), the fogging image 5120 is a gradation image (an image whose gradation changes in steps) in which the right side is dark black and the left side is white. Therefore, as shown in the example of FIG. 51C, even in the captured image 5130, the right side is black, but the left side is close to the original image 5100. In addition, this fog is also called a shadow, a black fog, etc.

  Further, this fogging image may occur not only when shooting with a digital camera but also with a scanner or the like. For example, in a scanner (a so-called flood bed scanner), a pressure plate that can be freely opened and closed is provided on a contact glass, and after placing a document on the contact glass, the pressure plate is closed to scan the document. However, the original is not limited to a sheet, and a book original (book, booklet, etc.) may be handled as an original. In such a case, the book original is placed on the contact glass and the original is scanned. It will be. However, when a book document is used as a document, the page binding portion of the book document is lifted from the contact glass. In this way, when the page binding portion of the book document is lifted from the contact glass, the page binding portion is separated from the focal plane, so that image degradation called fog occurs in the scanned image of the lifted portion.

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. It is the control to be stored in the device. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. In addition, if it is before the target processing, it is used in accordance with the situation / state at that time or with the intention to be decided according to the situation / state up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description having the meaning of “do B when it is A” is used in the meaning of “determine whether or not it is A and do B when it is judged as A”. However, the case where it is not necessary to determine whether or not A is excluded.
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

  The image processing apparatus according to the first embodiment removes fog from an image. As shown in the example of FIG. 1, the image receiving module 110, the fog amount estimation module 120, the pixel value measurement module 130, A correction target value determination module 140, a saturation correction module 150, and an output module 160 are provided.

  The image reception module 110 is connected to the fog amount estimation module 120, receives an image, and passes the image to the fog amount estimation module 120. Accepting an image means, for example, reading an image with a scanner, a camera, etc., receiving an image from an external device via a communication line by fax, etc., a hard disk (in addition to what is built in a computer, via a network) And the like, and the like read out the images stored in the device etc.). The image is a multivalued image (including a color image). One image may be received or a plurality of images may be received. Further, the content of the image may be a document used for business (for example, the above-mentioned business card or the like), a pamphlet for advertisement, or written on a whiteboard. Further, the image receiving module 110 may perform correction by digital filter processing or the like. For example, there is camera shake correction. A processing example of the image reception module 110 will be described later with reference to FIG.

  The fog amount estimation module 120 is connected to the image reception module 110 and the pixel value measurement module 130, and based on the image received by the image reception module 110, the luminance value of the original pixel and the luminance value of the pixel of the image. Estimate the amount of fog that is the difference. Then, the estimated fog amount is passed to the correction target value determination module 140 and the saturation correction module 150. The fog amount estimation module 120 estimates, for example, the fog amount and inclination strength at each coordinate from the background color distribution of the photographing target document. As a specific example, the technique described in Patent Document 2 described above may be used. A luminance representative value at a predetermined quantization point of an image is generated, and an estimated luminance value of each coordinate is calculated from the surrounding luminance representative values. You may make it do.

  The pixel value measurement module 130 is connected to the fog amount estimation module 120 and the correction target value determination module 140, and measures the saturation value of the pixel in the image received by the image reception module 110. Then, the measured saturation value is passed to the correction target value determination module 140. Further, the pixel value measurement module 130 may scan the image and measure the saturation value at each pixel. For the scanning direction, scanning equivalent to that of the pixel value measurement module 1640 of the fifth embodiment to be described later may be performed. In addition to measuring the saturation value of the pixel, the luminance value of each pixel may be measured. Note that the pixel value includes a saturation value and a luminance value.

  The correction target value determination module 140 is connected to the pixel value measurement module 130 and the saturation correction module 150, and determines the correction target value of the luminance value in the background in the image received by the image reception module 110. Then, the determined correction target value is passed to the saturation correction module 150. The correction target value determination module 140 determines the correction target value of the luminance value when it is assumed that the entire surface is the document background, for example, from the fog amount estimated by the fog amount estimation module 120. The correction target value determination module 140 may calculate and determine a predetermined correction target value or a condition that satisfies a predetermined condition from the fog luminance distribution. More specifically, for example, (1) the luminance values of the pixels in the image are totaled to generate a luminance value histogram, and the pixels satisfying a predetermined condition (for example, pixels in the upper 10% from the brighter one) The brightness value may be used, or (2) the operator may set the brightness value.

The saturation correction module 150 is connected to the correction target value determination module 140 and the output module 160, and the fog amount estimated by the fog amount estimation module 120 (hereinafter also referred to as an estimated luminance value) and the correction target value determination module 140. Based on the determined correction target value and the distance of the saturation value measured by the pixel value measurement module 130 from the achromatic color, the saturation value is corrected. Then, the image with the saturation value corrected is passed to the output module 160. The image with the saturation value corrected is an image from which the fog is removed.
Further, the saturation correction module 150 may perform correction so that the saturation value correction amount decreases as the saturation value measured by the pixel value measurement module 130 becomes closer to an achromatic color. For example, with respect to the distance from the axis that is an achromatic color in the color space to the target saturation, the suppression rate (a parameter that reduces the saturation correction) on the axis takes a minimum value. The suppression rate may be calculated from the relationship.
Further, the saturation correction module 150 may correct the saturation value based on the difference between the correction target value and the fogging amount. Specifically, the saturation correction amount may be increased in conjunction with the magnitude of the difference between the correction target value and the fogging amount. More specifically, the difference may be increased by a first-order or higher monotonically increasing polynomial function.
The processing of the saturation correction module 150 will be described later using Equations (1), (2), and (3).

  The output module 160 is connected to the saturation correction module 150, receives an image whose saturation has been corrected by the saturation correction module 150, and outputs the image. To output an image is, for example, printing on a printing device such as a printer, displaying on a display device such as a display, transmitting an image on an image transmission device such as a fax, or image to an image storage device such as an image database. , Storing in a storage medium such as a memory card, passing to another information processing apparatus, and the like. Further, the image may be output in association with the image received by the image receiving module 110.

FIG. 2 is a flowchart illustrating a processing example according to the first exemplary embodiment.
In step S202, the image reception module 110 receives a target image.
FIG. 3 is an explanatory diagram illustrating an example of processing performed by the image reception module 110. The photographing target document 320 is equivalent to the photographing target document 5100 shown in the example of FIG. When this is photographed by the photographing apparatus 310, an image with fogging like a photographed image 330 is obtained. For example, after exchanging business cards, the photographing target document 320 is placed on a desk and photographed by the photographing device 310. The imaging device 310 may be a digital camera, a digital camera built in a portable information terminal device (for example, a mobile phone including a smartphone), a scanner, or the like.

In step S204, the fog amount estimation module 120 estimates the fog amount.
In step S206, the pixel value measurement module 130 measures the pixel value in the image.
In step S208, the correction target value determination module 140 determines a correction target value.
FIG. 4 is a three-dimensional representation of how the fogging amount is removed, and is an explanatory diagram showing a processing example by the fogging amount estimation module 120, the pixel value measurement module 130, and the correction target value determination module 140. The x-axis and y-axis indicate positions in the image, and the z-axis is a space indicating luminance values. The gray surface below the z-axis is a fogged image and is positioned so as to cover the entire image. Then, as it goes in the x-axis direction, the downward movement in the z-axis indicates that the fogging image becomes darker toward the right side. And the brightness | luminance of the image after correction | amendment of the surface of the thin lattice pattern on the upper side in az axis has shown. As shown in the example of FIG. 4, p ^* is the correction target value determined in step S <b ^> 208, for example, the luminance value of the pixels in the upper 10% from the brightest in the image as described above. In the case where the brightness is corrected in the next step S210, the fog is removed by increasing the brightness at each point (brightening, raising it on the z axis) so as to match the correction target value. In FIG. 4, the luminance is raised to the corrected intermediate point 422 at the intersection point 418 surrounded by the intersection points 412, 414, 416, and 418. The intersection points 412, 414 are on the line 432, the intersection points 416, 418 are on the line 434, the intersection points 412, 416 are on the line 436, and the intersection points 414, 418 are on the line 438. A luminance value darker than the background portion is measured by the pixel value measurement module 130 for the character portion (the character “A” in FIG. 4) that is a portion other than the background portion.

また、（３）式において、文書の読み易さを保つために、無彩色（例えば、白背景、黒文字等）の彩度は強調しすぎないようにしている。つまり、彩度が無彩色に近くなるほどｃｏｌｏｒＲが０に近くなるようにしている。
彩度補正モジュール１５０は、補正目標値とかぶり量の差分が大きくなると、彩度値の補正量を多くするように補正している。これは後述の（８）式のＣ（（８）式の２段目の式参照）が大きくなることである。より具体的には、差分に対して、（８）式の３段目の式のような１次以上の単調増加な多項式関数によって増加させるようにしてもよい。
なお、ステップＳ２１０において、彩度補正モジュール１５０は輝度値を補正するようにしてもよい。なお、輝度値の補正については、第３の実施の形態で用いられている輝度値の補正方法を用いるようにしてもよい。 In step S210, the saturation correction module 150 corrects the saturation of the image. The saturation correction module 150 increases the saturation according to the amount of fog estimated by the fog amount estimation module 120. In the L ^* ab space, the saturation components (a, b) are strengthened by the following equations (1), (2), (3), and the like. Here, C is the fogging amount. Further, C is the luminance correction amount C in the equation (8) described in the third embodiment. In these equations, saturation improvement corresponding to the luminance correction is applied to the chromatic color portion, and correction processing is not performed on the achromatic color portion. When the saturation component (a, b) is 128, it is assumed that the color is achromatic.

In the equation (3), the saturation of achromatic colors (for example, white background, black characters, etc.) is not excessively emphasized in order to maintain readability of the document. That is, colorR is made closer to 0 as the saturation is closer to an achromatic color.
The saturation correction module 150 corrects the saturation value correction amount to increase as the difference between the correction target value and the fogging amount increases. This is because C of the following formula (8) (see the second stage formula of formula (8)) becomes large. More specifically, the difference may be increased by a first-order or higher monotonically increasing polynomial function such as the expression in the third stage of Expression (8).
In step S210, the saturation correction module 150 may correct the luminance value. As for the correction of the luminance value, the luminance value correction method used in the third embodiment may be used.

  FIG. 5 is an explanatory diagram showing a processing example (experimental example) by the saturation correction module. When the image receiving module 110 receives the captured image 330 in step S202, the image in which the luminance and saturation are corrected in step S210 is the processed image 540 of the present embodiment. An image with only luminance corrected is a simple fog removal processed image 550. Thus, it can be seen that the chromatic portion of the processed image 540 of the present embodiment is reproduced more clearly than the simple fog removal processed image 550.

  In step S212, the output module 160 outputs an image with corrected saturation. For example, an image from which fog is removed by a printer is printed.

<Second Embodiment>
FIG. 6 is a conceptual module configuration diagram of a configuration example according to the second embodiment.
The image processing apparatus according to the second embodiment removes the fog from the image. As shown in the example of FIG. 6, the image reception module 110, the fog amount estimation module 120, the pixel value measurement module 130, A correction target value determination module 140, a saturation correction availability determination module 145, a saturation correction module 150, and an output module 160 are provided. In addition, the same code | symbol is attached | subjected to the site | part of the same kind as above-mentioned embodiment, and the overlapping description is abbreviate | omitted. However, even if it is the site | part which attached | subjected the same code | symbol, a different part from the above-mentioned embodiment adds description. Even when the description is added, the description of the overlapping part is omitted (the same applies to the description of the embodiment below).

  The correction target value determination module 140 is connected to the pixel value measurement module 130 and the saturation correction availability determination module 145. Although it is equivalent to the correction target value determination module 140 of the first embodiment, the determined correction target value is passed to the saturation correction availability determination module 145.

The saturation correction availability determination module 145 is connected to the correction target value determination module 140 and the saturation correction module 150. The saturation correction possibility determination module 145 always performs the correction process because the character portion other than the background may be too crushed when the gradient of the estimated luminance value and the difference between the estimated luminance value and the target luminance value are large. I don't want to.
Whether the saturation correction possibility determination module 145 performs correction by the saturation correction module 150 based on the difference between the fog amount estimated by the fog amount estimation module 120 and the correction target value determined by the correction target value determination module 140. Judge whether or not. This determination process will be described later using equation (4).
Further, the saturation correction availability determination module 145 performs correction by the saturation correction module 150 based on the difference between the fog amounts estimated by the fog amount estimation module 120 at each pixel in the image received by the image reception module 110. You may make it judge whether it performs. This determination process will be described later using equations (5) and (6).

  The saturation correction module 150 is connected to the saturation correction availability determination module 145 and the output module 160. Although it is equivalent to the saturation correction module 150 of the first embodiment, the correction is performed when it is determined by the saturation correction availability determination module 145 that the correction is made.

FIG. 7 is a flowchart illustrating a processing example according to the second exemplary embodiment. From step S702 to step S708, the processing after step S712 is equivalent to the flowchart illustrated in FIG.
In step S702, the image reception module 110 receives a target image.
In step S704, the fog amount estimation module 120 estimates the fog amount.
In step S706, the pixel value measurement module 130 measures the pixel value in the image.
In step S708, the correction target value determination module 140 determines a correction target value.

なお、ｅ_ｉｊは、座標（ｘ_ｉ，ｙ_ｊ）における推定輝度値であり、ｐ^＊ _ｉｊは、座標（ｘ_ｉ，ｙ_ｊ）における補正目標値を表している。
この（４）式は、推定輝度値と補正目標値の差分を利用するものである。図８は、彩度補正可否モジュール１４５による処理例を示す説明図である。図８の例で示すと、目標輝度（点線）と推定輝度（一点鎖線）との間隔の最大値である。

この（５）式は、各画素間における推定輝度値の差分を利用するものである。図８の例で示すと、推定輝度（一点鎖線）における最大値と最小値との間隔である。

この（６）式は、各画素間における推定輝度値の差分を距離で正規化して傾き量として利用するものである。図８の例で示すと、推定輝度（一点鎖線）の傾きである。 In step S710, the saturation correction availability determination module 145 determines whether to perform saturation correction processing. If so, the process proceeds to step S712. Otherwise, the process proceeds to step S714.
For example, when the calculated value E in any of the following formulas (4), (5), and (6) exceeds a predetermined value, correction is not performed.

Note that e _ij is an estimated luminance value at coordinates (x _i , y _j ), and p ^* _ij represents a correction target value at coordinates (x _i , y _j ).
This equation (4) uses the difference between the estimated luminance value and the correction target value. FIG. 8 is an explanatory diagram illustrating an example of processing performed by the saturation correction availability module 145. In the example of FIG. 8, it is the maximum value of the interval between the target luminance (dotted line) and the estimated luminance (dashed line).

This equation (5) uses the difference of the estimated luminance value between each pixel. In the example of FIG. 8, it is the interval between the maximum value and the minimum value in the estimated luminance (one-dot chain line).

This equation (6) is used as the amount of inclination by normalizing the difference of the estimated luminance value between the pixels with the distance. In the example of FIG. 8, it is the inclination of the estimated luminance (dashed line).

In step S712, the saturation correction module 150 corrects the saturation of the image.
In step S714, the output module 160 outputs an image with corrected saturation.

In the description of FIGS. 51A to 51C described above, a comparison with the first embodiment and the second embodiment will be described using the examples of FIGS. 51C to 51G. To do.
Even if the fog amount can be calculated correctly, the red figure on the upper right side loses saturation.
This will be described with reference to FIGS. 51C to 51G. As shown in the example of FIG. 51C, the luminance value when the captured image 5130 is scanned with the line 5132 is shown in a graph (the vertical axis indicates the luminance value, and the horizontal axis indicates the x coordinate in the scanning). As shown in the example of (d). That is, since the fogging image 5120 becomes darker toward the right, the luminance value decreases to the right. However, since there is the original luminance value in the original image, the luminance is as shown in the example of the graph of FIG. Distributions 5142 to 5158 indicate a stepwise luminance distribution. In addition, when the saturation value in the case of scanning with the line 5132 is shown in the graph, it is as shown in the example of FIG. That is, the saturation decay rate decreases to the right.
Here, when only the luminance is corrected based on the fogging amount, the resulting image becomes a corrected image 5160 as shown in the example of FIG. That is, the right portion of the corrected image 5160 is light gray even if it is a black character portion. In addition, the red figure in the upper right will lose saturation.
In order to restore the original document 5100 that is the original document, it is necessary to increase the saturation in proportion to the correction value of the luminance value, as shown in the example of FIG. In the first embodiment and the second embodiment, by correcting the saturation value, it is possible to prevent the reproducibility of the chromatic color portion from being lowered.

<Third Embodiment>
Next, a third embodiment will be described.
First, before explaining the third embodiment and the fourth embodiment, the technology that is the premise thereof will be explained. This description is intended to facilitate understanding of the third embodiment and the fourth embodiment.
The photographing target document 5200 shown in the example of FIG. 52A is a business card and is printed only in black and white.
The photographing target document 5200 is photographed by, for example, a digital camera, and a photographed image 5230 shown in the example of FIG. In this case, the fogged image 5220 shown in the example of FIG. As shown in the example of FIG. 52B, the fogging image 5220 is a gradation image in which the right side is dark black and the left side is light black. Therefore, as shown in the example of FIG. 52C, even in the captured image 5230, the right side is black, but the left side is close to the image of the original shooting target document 5210.
Further, as described above, this fogging image may occur not only when shooting with a digital camera but also with a scanner or the like.

  The image processing apparatus according to the third embodiment removes the fog from the image. As shown in the example of FIG. 9, the image reception module 110, the fog estimation module 120, the pixel value measurement module 130, A correction target value determination module 140, a correction module 910, and an output module 160 are provided. The correction module 910 includes a saturation correction module 150 and a luminance correction module 950.

  The pixel value measurement module 130 is connected to the fogging amount estimation module 120 and the correction target value determination module 140, and measures the pixel value of the pixel in the image received by the image reception module 110. Then, the measured pixel value is passed to the correction target value determination module 140. The pixel value measurement module 130 scans the image and measures the pixel value in each pixel.

  The correction target value determination module 140 is connected to the pixel value measurement module 130 and the correction module 910, and determines the correction target value of the luminance value of the background in the image received by the image reception module 110. Then, the determined correction target value is passed to the correction module 910.

The brightness correction module 950 is in the correction module 910 connected to the correction target value determination module 140 and the output module 160, and the fog amount estimated by the fog amount estimation module 120 and the correction determined by the correction target value determination module 140. Based on the target value, the luminance value measured by the pixel value measurement module 130 is corrected. Then, the image whose luminance value is corrected is passed to the saturation correction module 150 or the output module 160. The image whose luminance value is corrected is an image from which the fog is removed.
Further, the luminance correction module 950 uses a difference between the fog amount estimated by the fog amount estimation module 120 and the correction target value determined by the correction target value determination module 140 as a basic correction amount, and uses the basic correction amount as a pixel value measurement module. The brightness value may be changed based on the coefficient based on the difference between the brightness value measured by 130 and the fogging amount, and the brightness value may be corrected based on the changed basic correction amount. This process will be described later using equations (7) and (8).

Further, the luminance correction module 950 may be a coefficient that decreases the basic correction amount when the difference between the luminance value measured by the pixel value measurement module 130 and the fogging amount increases. “Reducing the basic correction amount” means “increasing the correction amount of the basic correction amount” and means that the correction amount of the measured luminance value is smaller than the correction amount in the background portion. . For example, the removal of the fogging amount is reduced for black characters and the like. This process will be described later using equation (8).
Further, the luminance correction module 950 may not correct the luminance value when the difference between the luminance value measured by the pixel value measuring module 130 and the fogging amount is greater than or equal to a predetermined threshold value. This process will be described later using equation (8).
Further, the luminance correction module 950 may change the basic correction amount by a coefficient based on the difference between the luminance value measured by the pixel value measurement module 130 and the fogging amount and the basic correction amount. This process will be described later using equation (9).

The saturation correction module 150 is in the correction module 910 connected to the correction target value determination module 140 and the output module 160.
Either the saturation correction module 150 or the luminance correction module 950 may be processed first (if the saturation correction module 150 processes first, the luminance correction module 950 processes the image of the processing result). When the luminance correction module 950 processes first, the saturation correction module 150 processes the image of the processing result), and the processing may be performed in parallel.

  The output module 160 is connected to the correction module 910, receives an image corrected by the saturation correction module 150 and the luminance correction module 950, and outputs the image.

FIG. 10 is a flowchart illustrating a processing example according to the third exemplary embodiment.
In step S1002, the image receiving module 110 receives a target image.
FIG. 11 is an explanatory diagram illustrating an example of processing performed by the image reception module 110. The shooting target document 1120 is equivalent to the shooting target document 5200 shown in the example of FIG. When this is photographed by the photographing apparatus 310, an image with fogging like a photographed image 1130 is obtained. For example, after the business card is exchanged, the photographing target document 1120 is placed on the desk, and the photographing apparatus 310 shoots. The imaging device 310 may be a digital camera, a digital camera built in a portable information terminal device (for example, a mobile phone including a smartphone), a scanner, or the like.

In step S1004, the fog amount estimation module 120 estimates the fog amount.
In step S1006, the pixel value measurement module 130 measures the pixel value in the image.
In step S1008, the correction target value determination module 140 determines a correction target value.
FIG. 12 is a three-dimensional representation of how the fogging amount is removed, and is an explanatory diagram showing a processing example by the fogging amount estimation module 120, the pixel value measurement module 130, and the correction target value determination module 140. The x-axis and y-axis indicate positions in the image, and the z-axis is a space indicating luminance values. The gray surface below the z-axis is a fogged image and is positioned so as to cover the entire image. Then, as it goes in the x-axis direction, the downward movement in the z-axis indicates that the fogging image becomes darker toward the right side. And the brightness | luminance of the image after correction | amendment of the surface of the thin lattice pattern on the upper side in az axis has shown. As shown in the example of FIG. 12, p ^* is the correction target value determined in step S1008, for example, the luminance value of the pixels in the upper 10% from the brighter side in the image as described above. In the next step S1012, the fog is removed by increasing the brightness at each point (increasing the brightness on the z axis) so as to match this. FIG. 12 shows that the brightness of the intersection 1218 surrounded by the intersections 1212, 1214, 1216, and 1218 is raised to the corrected intermediate point 1222. The intersection points 1212 and 1214 are on the line 1232, the intersection points 1216 and 1218 are on the line 1234, the intersection points 1212 and 1216 are on the line 1236, and the intersection points 1214 and 1218 are on the line 1238. The pixel value measurement module 130 measures a luminance value darker than the background portion of the character portion (the character “A” in FIG. 12) other than the background portion.

In step S1010, the saturation correction module 150 corrects the saturation of the image.
In step S1012, the luminance correction module 950 corrects the luminance of the image. The brightness correction module 950 checks the difference between the estimated brightness value and the measured brightness value. If the difference is on the dark side (if the result of subtracting the measured luminance value from the estimated luminance is a positive value), the correction amount is further suppressed as compared with the background portion. In other words, the luminance value is not increased more than necessary for portions such as characters.
FIG. 13 is an explanatory diagram illustrating an example of processing performed by the luminance correction module 950. FIG. 13A is an example of a graph showing the state of an image before correction by the luminance correction module 950, and FIG. 13B is an example of a graph showing the state of an image after correction by the luminance correction module 950. is there. 13A and 13B, the correction target value determined by the correction target value determination module 140 is represented by a horizontal dotted line. In the graph shown in the example of FIG. 13A, the luminance measured by the pixel value measurement module 130 is indicated by a solid line, and the fog amount (estimated luminance value) estimated by the fog amount estimation module 120 is indicated by a one-dot chain line. . In the graph shown in the example of FIG. 13B, the luminance measured by the pixel value measurement module 130 is indicated by a gray solid line, and the luminance value corrected by the luminance correction module 950 is indicated by a black solid line. In other words, the luminance of the background portion is increased to the correction target value (increasing or decreasing the luminance), and the portion other than the background is increased in luminance, but the amount is smaller than the amount increased in the background portion. Therefore, the portions other than the base are not thinner than necessary.

したがって、補正後の輝度値ｎｅｗ＿ｖは、次の（８）式のように計算される。

なお、（８）式内のＣは、前述の補正量のことである。 Whether it is a ground part or a part other than the ground part,
If (estimated luminance value−measured luminance value) <threshold th, it is determined to be a background portion,
If (estimated luminance value−measured luminance value) ≧ threshold th, it is determined that the portion is other than the background. Here, the threshold th is a value that can be determined to be a portion (for example, a character) other than the background when viewed by human eyes, and is a predetermined value.
And about the part judged to be the ground part,
Correction amount = (correction target value−estimated luminance value)
(This correction amount is the basic correction amount), and for the parts that are determined to be other than the background,
Correction amount = (correction target value−estimated luminance value) × correction rate R
Calculate Then, the correction amount is added to the measured luminance value. The correction rate R is a value of 1 or less, and is calculated as in the following equation (7). The greater the difference between the measured luminance value (v) and the estimated luminance value (e), the closer the correction rate (R), which is a coefficient, to a value of zero. Further, 0 becomes the lower limit by the max () expression outside the expression (7), and no correction is made when the difference exceeds a predetermined value. That is, the correction rate (R) is 0, and no correction is performed (the measured luminance value is left as it is). The correction strength parameter ρ is a predetermined value.

Therefore, the corrected luminance value new_v is calculated as in the following equation (8).

Note that C in the equation (8) is the above-described correction amount.

（９）式は、補正目標値である輝度値と推定輝度値の差分である（ｐ^＊−ｅ）が大きい場合には、推定輝度値と測定輝度値の差が小さくても補正抑制を大きくするような式となっている。なお、τは倍率補正である。例えば、中間輝度値の１２８などを採用してもよい。 Further, the brightness correction module 950 may adopt the following equation (9) instead of the equation (7).

When the difference between the luminance value that is the correction target value and the estimated luminance value (p ^* −e) is large, the expression (9) increases correction suppression even if the difference between the estimated luminance value and the measured luminance value is small. It is a formula to do. Note that τ is a magnification correction. For example, an intermediate luminance value of 128 may be employed.

  In step S1014, the output module 160 outputs the corrected image. For example, an image from which fog is removed by a printer is printed.

<Fourth embodiment>
FIG. 14 is a conceptual module configuration diagram of a configuration example according to the fourth embodiment.
The image processing apparatus according to the fourth embodiment removes the fog from the image. As shown in the example of FIG. 14, the image receiving module 110, the fog amount estimating module 120, the pixel value measuring module 130, A correction target value determination module 140, a correction availability determination module 1445, a correction module 910, and an output module 160 are included. The correction module 910 includes a saturation correction module 150 and a luminance correction module 950.
The correction target value determination module 140 is connected to the pixel value measurement module 130 and the correction possibility determination module 1445. Although it is equivalent to the correction target value determination module 140 of the first embodiment, the determined correction target value is passed to the correction possibility determination module 1445.

The correction possibility determination module 1445 is connected to the correction target value determination module 140 and the correction module 910. The correction possibility determination module 1445 does not always perform correction processing when the slope of the estimated luminance value or the difference between the estimated luminance value and the target luminance value is large, because the character portion other than the background may be too crushed. I am doing so.
Based on the difference between the fog amount estimated by the fog amount estimation module 120 and the correction target value determined by the correction target value determination module 140, the correction possibility determination module 1445 performs correction by the saturation correction module 150 and the luminance correction module 950. It is determined whether or not to perform. This determination process is equivalent to the determination process using the equation (4) by the saturation correction availability determination module 145 described above.
The correction possibility determination module 1445 also determines the saturation correction module 150 and the luminance correction module 950 based on the difference between the fog amounts estimated by the fog amount estimation module 120 at each pixel in the image received by the image reception module 110. It may be determined whether or not correction is performed. This determination process is the same as the determination process using the equations (5) and (6) by the saturation correction availability determination module 145 described above.

The saturation correction module 150 in the correction module 910 is equivalent to the saturation correction module 150 of the third embodiment, but performs correction when it is determined by the correction availability determination module 1445 that correction is performed.
The luminance correction module 950 in the correction module 910 is equivalent to the luminance correction module 950 of the third embodiment, but performs correction when it is determined by the correction availability determination module 1445 that correction is performed.

FIG. 15 is a flowchart illustrating a processing example according to the fourth exemplary embodiment. From step S1502 to step S1508, the processing after step S1512 performs the same processing as the flowchart illustrated in FIG.
In step S1502, the image reception module 110 receives a target image.
In step S1504, the fog amount estimation module 120 estimates the fog amount.
In step S1506, the pixel value measurement module 130 measures a pixel value in the image.
In step S1508, the correction target value determination module 140 determines a correction target value.

In step S1510, the correction possibility determination module 1445 determines whether or not correction processing is to be performed. If so, the process proceeds to step S1512. Otherwise, the process proceeds to step S1516.
For example, when the calculated value E in any one of the above-mentioned formulas (4), (5), and (6) exceeds a predetermined value, the correction is not performed in the example of FIG. This is equivalent to step S710 in the flowchart shown in FIG.

In step S1512, the saturation correction module 150 corrects the saturation of the image.
In step S1514, the luminance correction module 950 corrects the luminance of the image.
In step S1516, the output module 160 outputs the corrected image.

In the description of FIGS. 52A to 52C described above, the comparison with the third embodiment and the fourth embodiment will be described using the examples of FIGS. 52C to 52F.
Even if the fog amount can be calculated correctly, the difference between the background portion and the character portion becomes small because the character portion has a luminance of around 0 (black). Here, if the luminance value is increased by the same amount as that of the background portion, the character portion is also corrected in a brighter direction than necessary.
This will be described with reference to FIGS. 52C to 52F. As shown in the example of FIG. 52C, the luminance value when the captured image 5230 is scanned with the line 5232 is shown in the graph as shown in the example of FIG. That is, since the fog image 5220 becomes darker toward the right, the luminance value decreases to the right. However, since there is the original luminance value in the original image, as shown in the example of the graph in FIG. The distributions 5242 to 5258 indicate a stepwise luminance distribution. Here, as shown in the example of FIG. 52F, when each luminance is corrected so that the luminance value of the background portion (luminance distributions 5244, 5248, 5252, 5256) becomes the target luminance, the portion other than the background (luminance distribution) 5246, 5250, and 5254) are also corrected as indicated by the solid lines of the corrected luminance distributions 5274 to 5280. The result is a corrected image 5260 as shown in the example of FIG. That is, the right portion of the corrected image 5260 is light gray even if it is a black character portion. This is because the fog removal correction equivalent to that of the background portion is performed on the portion other than the background portion.
On the other hand, in the third embodiment and the fourth embodiment, a correction amount is applied to a portion other than the background, as indicated by the dotted line in the original corrected luminance distribution 5282 to 5288 in the example of FIG. The number is reduced and the reproducibility is prevented from being lowered.

<Fifth embodiment>
Next, a fifth embodiment will be described.
First, before explaining the fifth embodiment and the sixth embodiment, the technology that is the premise thereof will be explained. This description is intended to facilitate understanding of the fifth embodiment and the sixth embodiment.
The photographing target document 5310 shown in the example of FIG. 53A is a business card, and includes not only black and white but also chromatic areas. Specifically, a region 5312 in the photographing target document 5310 has a red background, a region 5314 has a white background, and a graphic 5316 is a red mark (graphic). This photographing target document 5310 is photographed by, for example, a digital camera, and a photographed image 5330 shown in the example of FIG. 53 (c) is obtained. In this case, the fogged image 5320 shown in the example of FIG. The cause of generation of the fog image 5320 is as described above. As shown in the example of FIG. 53B, the fogging image 5320 is a gradation image in which the upper right is dark black and the lower left is white. Therefore, as shown in the example of FIG. 53C, even in the captured image 5330, the upper right is black, but the lower left is close to the image of the original document 5310.
Further, as described above, this fogging image may occur not only when shooting with a digital camera but also with a scanner or the like.

FIG. 16 is a conceptual module configuration diagram of a configuration example according to the fifth embodiment.
The image processing apparatus according to the fifth embodiment removes fog from an image. As shown in the example of FIG. 16, the image reception module 110, the non-edge area extraction module 1620, and the inspection area selection module 1630 are used. A pixel value measurement module 1640, a pixel value distribution estimation module 1650, a correction target value determination module 140, a correction module 910, and an output module 160. The correction module 910 includes a saturation correction module 150 and a luminance correction module 950.

  The image reception module 110 is connected to the non-edge region extraction module 1620, receives an image, and passes the image to the non-edge region extraction module 1620. An example of processing performed by the image receiving module 110 will be described later with reference to FIG.

The non-edge area extraction module 1620 is connected to the image reception module 110 and the inspection area selection module 1630. The non-edge area extraction module 1620 extracts an area from the image received by the image reception module 110 and passes the extracted area to the inspection area selection module 1630.
Further, the non-edge region extraction module 1620 extracts a region having a size greater than or equal to a predetermined size, and if the region cannot be extracted, the inspection region selection module 1630, the pixel value measurement module 1640, the pixel value distribution estimation Processing of the module 1650, the correction target value determination module 140, the correction module 910, and the output module 160 may not be performed. In this case, a message indicating that the region could not be extracted (that is, the fog could not be removed) may be output.
Here, the extraction target area is an area where color values are gentle and continuous. In other words, it is a region without edges or a region surrounded by edges.
A processing example of the non-edge region extraction module 1620 will be described later with reference to FIGS.

The inspection area selection module 1630 is connected to the non-edge area extraction module 1620 and the pixel value measurement module 1640. The inspection area selection module 1630 selects an area extracted by the non-edge area extraction module 1620 based on a predetermined rule, and passes the selected area to the pixel value measurement module 1640.
Further, the predetermined rule in the inspection region selection module 1630 may be determined by the size of the region extracted by the non-edge region extraction module 1620. The inspection area selection module 1630 may be further determined by the brightness or saturation of the area extracted by the non-edge area extraction module 1620 as a predetermined rule. The inspection area selection module 1630 is further determined as a predetermined rule by one or more of the variance value of the saturation of the area extracted by the non-edge area extraction module 1620, the position in the image, and the size of the outer periphery. You may make it.
A processing example of the inspection area selection module 1630 will be described later with reference to FIG.

The pixel value measurement module 1640 is connected to the inspection region selection module 1630 and the pixel value distribution estimation module 1650. The pixel value measurement module 1640 measures the pixel values (luminance, saturation) of the pixels in the area selected by the inspection area selection module 1630 and passes the measurement results to the pixel value distribution estimation module 1650. In the measurement of the pixel value, the pixel value (luminance, saturation) is measured while scanning the image in a predetermined direction.
In the processing performed by the pixel value measurement module 1640, the scanning direction in which the pixel value is measured may be any one or more of a horizontal direction, a vertical direction, an oblique direction, and an elliptical shape. There may be two types of scanning directions such as a combination of horizontal and vertical directions, a combination of upper right diagonal direction and lower right diagonal direction, or one type of scanning direction so as to scan in an elliptical shape. Further, it may be a combination of three or more scanning directions.
A processing example of the pixel value measurement module 1640 will be described later with reference to FIGS.

The pixel value distribution estimation module 1650 is connected to the pixel value measurement module 1640 and the correction target value determination module 140. The pixel value distribution estimation module 1650 estimates a function indicating the degree of fogging in the image received by the image receiving module 110 from the pixel value measured by the pixel value measuring module 1640, and sends the estimated function to the correction module 910. hand over. The function indicating the degree of fogging may be a linear function or a function of second or higher order.
Further, the pixel value distribution estimation module 1650 may estimate a function that passes through pixel values (luminance values) of pixels in the region selected by the inspection region selection module 1630.
A processing example of the pixel value distribution estimation module 1650 will be described later with reference to FIGS.

  The correction target value determination module 140 is connected to the pixel value distribution estimation module 1650 and the correction module 910, determines the correction target value of the luminance value in the background in the image received by the image reception module 110, and determines the determination. The correction target value is passed to the correction module 910.

The luminance correction module 950 in the correction module 910 removes the fog from the image received by the image reception module 110 based on the function estimated by the pixel value distribution estimation module 1650, and outputs an image from which the fog has been removed. To 160. That is, the fog amount is calculated based on the function estimated by the pixel value distribution estimation module 1650, and measured by the pixel value measurement module 1640 based on the fog amount and the correction target value determined by the correction target value determination module 140. Correct the brightness value.
Also, the luminance correction module 950 has a plurality of scanning directions by the pixel value measurement module 1640, and the fog value at a position where the scanning directions intersect is estimated by the pixel value distribution estimation module 1650 for the scanning. It may be calculated on the basis of a value obtained by the function. For pixels that are not on the scan line, the fog value may be calculated according to the distance from the pixel using the fog value at the pixel on the scan line. Note that the fogging value is the difference between the pixel value of the received image and the original background value (pixel value of the image taken without fogging).
A processing example of the luminance correction module 950 will be described later with reference to FIGS. 26, 27, and 28.

  The saturation correction module 150 in the correction module 910 calculates the fog amount based on the function estimated by the pixel value distribution estimation module 1650, the correction amount and the correction target value determined by the correction target value determination module 140. The saturation value is corrected based on the distance from the achromatic color of the saturation value measured by the pixel value measurement module 1640. A process equivalent to the saturation correction module 150 in the above-described embodiment is performed. Further, the calculation equivalent to the brightness correction module 950 is performed for calculating the fogging amount based on the function estimated by the pixel value distribution estimation module 1650. Further, the processing result of the luminance correction module 950 may be used. Conversely, the luminance correction module 950 may divert the processing result obtained by the saturation correction module 150.

  The output module 160 is connected to the correction module 910, receives the image corrected (fog removal) by the saturation correction module 150 and the luminance correction module 950, and outputs the image.

FIG. 17 is a flowchart illustrating a processing example according to the fifth exemplary embodiment.
In step S1702, the image reception module 110 receives the target image.
FIG. 18 is an explanatory diagram illustrating an example of processing performed by the image reception module 110. The shooting target document 1820 is equivalent to the shooting target document 5310 shown in the example of FIG. The shooting target document 1820 includes an area with a red background on the left side, an area with a white background on the right side, and a red mark area on the upper right. When this is photographed by the photographing apparatus 310, an image with fogging is obtained as a photographed image 1830. For example, after the business card is exchanged, the photographing target document 1820 is placed on the desk and the photographing apparatus 310 shoots. The imaging device 310 may be a digital camera, a digital camera built in a portable information terminal device (for example, a mobile phone including a smartphone), a scanner, or the like.

In step S1704, the non-edge region extraction module 1620 extracts a continuous region where the color value changes gently.
19 and 20 are explanatory diagrams illustrating an example of processing performed by the non-edge region extraction module 1620. For example, the non-edge region extraction module 1620 includes a Sobel filter module 1910, a binarization module 1920, a black / white reversal module 1930, a labeling module 1940, and a small size removal module 1950.
The Sobel filter module 1910 is connected to the binarization module 1920. The Sobel filter module 1910 performs a Sobel filter process on the captured image 1830 received in step S1702, and binarizes the resulting image. To the conversion module 1920. The Sobel filter process is, for example, a process for detecting edges of vertical lines or horizontal lines, and is performed using a filter. Of course, lines other than vertical lines and horizontal lines can be detected depending on the design of the filter.

The binarization module 1920 is connected to the Sobel filter module 1910 and the black and white inversion module 1930. The binarization module 1920 receives an image in which an edge is detected by the Sobel filter module 1910, performs binarization processing on the image, The resulting binarization result image is passed to the black and white reversal module 1930. The binarization process here may adopt a known binarization process. By this processing, the edge portion becomes black and the other portions become white.
The black / white reversal module 1930 is connected to the binarization module 1920 and the labeling module 1940, receives the image binarized by the binarization module 1920, performs black / white reversal processing on the image, and performs the black / white reversal processing. The result image is passed to the labeling module 1940. By this processing, the edge portion becomes white and the other portions become black.

The labeling module 1940 is connected to the black-and-white reversal module 1930 and the small size removal module 1950. The labeling module 1940 receives an image that has been black-and-white reversed by the black-and-white reversal module 1930, performs a labeling process on the image, and outputs a result of the labeling process. Pass to small size removal module 1950. By this process, the same label is attached to the continuous black areas. Therefore, a region that is not an edge can be extracted by extracting a region having the same label.
The small size removal module 1950 is connected to the labeling module 1940, receives the image labeled by the labeling module 1940, and removes noise that is an area below or smaller than a predetermined size. As a result, this process extracts an area that is larger than or larger than a predetermined size. The area of the region to which the same label is attached may be obtained by counting the number of pixels to which the label is attached or by calculating the area of the circumscribed rectangle of the region.
FIG. 20 shows an example of an image obtained as a result of performing the above processing on the captured image 1830 shown in the example of FIG. The area image 2010 is an area with a red background on the left side of the captured image 1830, the area image 2020 is an area with a white background on the right side of the captured image 1830, and the area image 2030 is a red color on the upper right of the captured image 1830. This is the mark area. However, these images are binary images. In addition, these images have a role as a mask image in which an area can be extracted from the photographed image 1830 by performing an AND process (logical product process) with the photographed image 1830.
Note that the non-edge region extraction module 1620 may extract regions by other processing instead of the processing by the module configuration shown in the example of FIG. For example, processing for integrating regions of the same color may be performed. Note that the same color may include not only a completely identical color but also a color having a predetermined relationship. Specifically, a pixel is selected from the image and is in contact with the selected pixel, and a predetermined relationship with the color of the selected pixel (for example, between the two colors in the color space) The pixels of a color having a distance of less than or less than a predetermined value) are integrated, and then the same integration process is repeatedly performed on the integrated pixels to extract a region. Good.

ここで、（１０）式の右辺のｗ_ｊは重みであり、正の数の他に、０、負の数であってもよい。０の場合は、その特徴量を採用しないことを意味する。負の数の場合は、特徴量が逆の方向に作用することになる。この重みｗ_ｊは、予め定めた値である。予め定めた値として、操作者が設定した値であってもよいし、正解の選択肢が決まっている教師データを用いて予め学習を行った結果の値であってもよい。
（１０）式の右辺のｘ_iｊは特徴量である。検査領域選択モジュール１６３０が、これらの特徴量を各領域（ｉ）から抽出する。
ｘ_i０は、領域の幅若しくは高さのいずれか、又は幅×高さ（いわゆる外接矩形の面積）であってもよい。
ｘ_i１は、領域の面積である。
ｘ_i２は、領域の画素密度である。
ｘ_i３は、領域内の画素の輝度（例えば、Ｌ^＊ａｂ色空間の場合はＬ^＊の値）の平均値である。
ｘ_i４は、領域内の画素の彩度（例えば、Ｌ^＊ａｂ色空間の場合はａ，ｂの値）の平均値である。
ｘ_i５は、領域内の画素の彩度の分散値である。
ｘ_i６は、領域（マスク画像）の重心とステップＳ１７０２で受け付けた画像の中心との距離である。
ｘ_i７は、領域の外周輪郭の長さ／外接矩形の周囲長である。 In step S1706, the examination region selection module 1630 extracts a region suitable for estimating the fog inclination.
The inspection area selection module 1630 extracts the feature amount of the area extracted in step S1704. Then, according to the discriminant function of equation (10), the value of equation (10) is calculated for each region, and the region (here, the aforementioned mask image) is selected based on the value. For example, the area having the maximum value is selected from the areas extracted in step S1704.

Here, w _{j on} the right side of the equation (10) is a weight, and may be 0 or a negative number in addition to a positive number. In the case of 0, it means that the feature value is not adopted. In the case of a negative number, the feature amount acts in the opposite direction. This weight w _j is a predetermined value. The predetermined value may be a value set by the operator, or may be a value obtained as a result of learning in advance using teacher data in which correct answer options are determined.
X _{ij on} the right side of the equation (10) is a feature quantity. The inspection area selection module 1630 extracts these feature amounts from each area (i).
x _i0 may be either the width or height of the region, or width × height (the area of a so-called circumscribed rectangle).
x _i1 is the area of the region.
x _i2 is the pixel density of the region.
x _i3, the luminance of the pixels in the area ^(e.g., in the case of ^L * ab color space value of ^{L *)} is an average value of.
x _i4 is an average value of the saturations of the pixels in the region (for example, values of a and b in the case of the L ^* ab color space).
x _i5 is the variance value of the saturation of the pixels in the region.
x _i6 is the distance between the center of gravity of the region (mask image) and the center of the image received in step S1702.
x _i7 is the length of the outer peripheral contour of the region / the perimeter of the circumscribed rectangle.

Note that the feature amounts listed here are examples, and other feature amounts may be used. The average value may be a value that represents the region, and may be a mode value, a median value, or the like.
Further, it is not necessary to use all the feature amounts exemplified here, and any one of them may be selected and used. For example, an identification function using only one of x _i0 , x _i1 , and x _i2 may be used. This is because a large region is suitable for estimating a function indicating the degree of fogging.
Further, x _i3 and x _i4 may be added to select the background area of the document. This is because the background region is suitable for estimating a function indicating the degree of fogging. This is because the background area is generally white and thus becomes an area of high luminance and low saturation. The weight (w ₄ ) of saturation (x _i4 ) is a negative number.

The above feature quantities can be used for business cards, documents, whiteboards, etc. that have a white background. For example, an image of a document with a landscape image pasted on half the area of a business card. In some cases (the area of the white background is slightly narrow), or in the case of an image taken of a whiteboard in backlight (when the surrounding area of the image is brighter), it may be insufficient .
Therefore, a feature amount of x _i5 or less may be added.
Landscape paintings generally have more variation in saturation than the background. Therefore, x _i5 may be adopted so that an area having a small saturation variance value can be selected so as not to select a landscape image. The weight (w ₅ ) of the saturation dispersion value (x _i5 ) is a negative number.
In general, the target area at the time of photographing is intentionally set at the center of the image. Accordingly, x _i6 may be adopted so that a region whose center of gravity (which may be the center of the region) is closer to the center of the image can be selected. The weight (w ₆ ) of this distance (x _i6 ) is a negative number.
The outer periphery of the landscape image area is more uneven than the outer periphery of the circumscribed rectangle. Moreover, the background part of a business card is rectangular, and its outer periphery is often linear. Therefore, x _i7 may be adopted so that a region having a short outer peripheral contour can be selected so as not to select a landscape image. Note that the weight (w ₇ ) of the outer peripheral contour (x _i7 ) is a negative number.

領域画像２０２０に対して識別関数の計算結果は、（１２）式のように２．０である。

領域画像２０３０に対して識別関数の計算結果は、（１３）式のように０．２である。

そして、（１４）式に示すように、識別関数の算出値が最大値となるものを選択する。

この場合、（１５）式に示すように、領域画像２０２０の領域が選択されることになる。

FIG. 21 is an explanatory diagram illustrating an example of processing performed by the inspection region selection module 1630. The calculation result of the discriminant function for the region image 2010 is 1.5 as shown in the equation (11).

The calculation result of the discriminant function for the region image 2020 is 2.0 as shown in the equation (12).

The calculation result of the discriminant function for the region image 2030 is 0.2 as shown in equation (13).

Then, as shown in the equation (14), the one having the maximum discriminant function calculated value is selected.

In this case, as shown in the equation (15), the region of the region image 2020 is selected.

In step S1708, the pixel value measurement module 1640 measures the pixel value by scanning the line according to a predetermined rule.
22 and 23 are explanatory diagrams illustrating an example of processing performed by the pixel value measurement module 1640.
A region extraction image 2200 shown in the example of FIG. 22A is an image obtained by AND processing of the captured image 1830 received in step S1702 and the region image 2020 selected in step S1706. That is, a region with a white background on the right side is extracted from the captured image 1830. The region extraction image 2200 is scanned at predetermined intervals (for example, equal intervals) horizontally and vertically. For example, scanning is performed with the lines 2202 to 2218 in the horizontal direction and the lines 2222 to 2242 in the vertical direction. As a result of the scanning, if the luminance value of each pixel is represented in a graph, the scanning of the line 2202 results in a graph as in the example of FIG. That is, the graph is composed of only invalid data without scanning the selected area. In the scanning of the line 2212, a graph like the example of FIG. That is, since a part of the selected area is scanned, the graph includes valid data, and the luminance value is a graph with a lower right. This is because the fogging here is generally darker in the upper right direction.
Whether the data is valid or invalid is determined by scanning an image of the area selected by the inspection area selection module 1630 (mask image, area image 2020 shown in the example of FIG. 21). It can be carried out. The position of the black part in the area image 2020 is a position where there is valid data in the area extraction image 2200, and the position of the white part in the area image 2020 is handled as a position where there is invalid data in the area extraction image 2200. Also, here, invalid data is treated with a luminance value of zero.

  Further, as a predetermined rule in scanning, not only scanning as illustrated in FIG. 22A but also scanning in an oblique direction from upper left to lower right and from upper right as illustrated in the example of FIG. You may make it perform the scanning of the diagonal direction to a lower left at a predetermined space | interval. Further, elliptical scanning as shown in the example of FIG. 23B may be performed at predetermined intervals. Here, the ellipse may include a circle. This scanning direction may be selected by the operator according to the inclination of the brightness of the fog, or may be a scanning that is predetermined according to the photographing apparatus.

In step S1710, the pixel value distribution estimation module 1650 estimates the degree of fog inclination (a function indicating the degree of fog) from the measured set of pixel values. The function here will be described by taking a linear function as an example.
24 and 25 are explanatory diagrams illustrating processing examples performed by the pixel value distribution estimation module 1650.
As shown in the example of FIG. 24A, a function indicating the degree of fogging in the vertical scanning is represented by Expression (16), and a function indicating the degree of fogging in the horizontal scanning is represented by Expression (17).

そして、有効なデータがないラインにおいても、その関数を適用すればよい。したがって、例えば、図２４（ｃ）の例に示すようにライン２２０２上の推定点２４３２〜２４４０における輝度値を、図２５（ｃ）の例に示すように関数を適用して、推定点２４３２〜２４４０における輝度値とすればよい。このことは、推定した関数を画像全体に適用していることになる。
縦方向のラインを用いた関数の推定も、前述した横方向のラインを用いた関数の推定と同等に行う。 In the example of FIG. 24B, lines with valid data are indicated by solid lines, and lines without valid data (lines with only invalid data) are indicated by dotted lines. Lines having valid data include lines 2204 to 2216 and lines 2232 to 2240, and lines having no valid data include lines 2202 and 2218 and lines 2222 to 2230 and 2242.
The example of FIG. 25A shows a graph in a line with valid data. A similar graph can be shown for each of the lines 2204 to 2216. For these, a function that passes the luminance value is estimated using a least square method or the like. For example, the function in the horizontal line can be expressed as in equation (18). When the number of effective data is less than a predetermined number in each line, the function is estimated without using the data. In the example of FIG. 25B, the formula (18) is represented by a one-dot chain line.

The function may be applied even to a line that does not have valid data. Therefore, for example, the luminance values at the estimated points 2432 to 2440 on the line 2202 as shown in the example of FIG. 24C are applied, and the function is applied as shown in the example of FIG. The luminance value at 2440 may be used. This means that the estimated function is applied to the entire image.
The function estimation using the vertical line is performed in the same manner as the function estimation using the horizontal line described above.

  In step S1712, the correction target value determination module 140 determines a correction target value. Processing equivalent to that of the correction target value determination module 140 in the above-described embodiment is performed. In addition to the above-described determination method, the correction target value is determined by, for example, totaling luminance histograms of a plurality of intersection points (for example, intersection points in the region selected by the inspection region selection module 1630). It is good also as a luminance value of the defined intersection (for example, intersection in the upper 10% from the brighter one).

  In step S1714, the saturation correction module 150 corrects the saturation of the image. A fog amount is calculated based on the function estimated in step S1710, and based on the fog amount, the correction target value determined in step S1712, and the distance from the achromatic color of the saturation value measured in step S1708, Correct the saturation value. A process equivalent to the saturation correction module 150 in the above-described embodiment is performed. In addition, regarding the calculation of the fogging amount based on the function estimated by the pixel value distribution estimation module 1650, processing equivalent to step S1716 is performed.

（１９）式は、ライン２２０６における関数ｒ_ｉとライン２２２６における関数ｃ_ｊの交点２６１２（ｘ_ｉ，ｙ_ｉ）における値の平均値である。交点２６１４における輝度値も、ライン２２０６における関数ｒ_ｉとライン２２２８における関数ｃ_ｊ＋１の交点２６１４（ｘ_ｉ＋１，ｙ_ｊ）における値の平均値である。交点２６１６における輝度値も、ライン２２０８における関数ｒ_ｉ＋１とライン２２２６における関数ｃ_ｊの交点２６１６（ｘ_ｉ，ｙ_ｊ＋１）における値の平均値である。交点２６１８における輝度値も、ライン２２０８における関数ｒ_ｉ＋１とライン２２２８における関数ｃ_ｊ＋１の交点２６１８（ｘ_ｉ＋１，ｙ_ｊ＋１）における値の平均値である。このように、ラインの交点にある画素の輝度値は、横方向の関数値と縦方向の関数値の平均値とする。 In step S1716, the brightness correction module 950 corrects the brightness of the image. In other words, the fog removal amount is calculated using the degree of inclination of the fog (the function estimated in step S1710), and the pixel value at each coordinate is corrected. The correction here is to remove the fog from the image and generate the background (original image).
26, 27, and 28 are explanatory diagrams illustrating processing examples performed by the luminance correction module 950.
In the example of FIG. 26, four intersections in the region 2610 (that is, the intersection 2612 is an intersection of the line 2206 and the line 2226, the intersection 2614 is an intersection of the line 2206 and the line 2228, and the intersection 2616 is an intersection of the line 2208 and the line 2226. The calculation of the fogging amount at an intersection point (intersection point 2618 is an intersection point of line 2208 and line 2228) will be described with reference to FIG. Note that these intersections are intersections in the area treated as invalid data.
The luminance value at the intersection point 2612 is calculated using equation (19).

Expression (19) is an average value of values at the intersection 2612 (x _i , y _i ) of the function r _i in the line 2206 and the function c _j in the line 2226. The luminance value at the intersection point 2614 is also an average value of values at the intersection point 2614 (x _{i + 1} , y _j ) of the function r _i in the line 2206 and the function c _{j + 1} in the line 2228. The luminance value at the intersection 2616 is also an average value of the values at the intersection 2616 (x _i , y _{j + 1} ) of the function r _{i + 1} in the line 2208 and the function c _j in the line 2226. The luminance value at the intersection 2618 is also an average value of values at the intersection 2618 (x _{i + 1} , y _{j + 1} ) of the function r _{i + 1} in the line 2208 and the function c _{j + 1} in the line 2228. As described above, the luminance value of the pixel at the intersection of the lines is an average value of the horizontal function value and the vertical function value.

中間点２７２２の位置を（ｘ，ｙ）としている。なお、（２０）式の右辺のＷ（（ｘ，ｙ），（ｘ_ｉ，ｙ_ｉ））は距離重みを表している。つまり、中間点２７２２からの交点（交点２６１２、２６１４、２６１６、２６１８）までの距離による重みであり、４つの交点の重みの合計が１となる値であり、例えば、距離の逆数を用いた値である。したがって、各交点から等距離にある場合は、各交点における輝度の１／４を加えた値となる。 And the brightness | luminance in the pixel which is not such an intersection, for example, the intermediate point 2722 shown in FIG. Expression (20) is for calculating the luminance at the intermediate point 2722 by the luminance of the four surrounding intersections. Therefore, the calculation by the expression (20) in the pixels other than the intersection is performed after the luminance value at the intersection is calculated.

The position of the intermediate point 2722 is (x, y). Note that W ((x, y), (x _i , y _i )) on the right side of the equation (20) represents a distance weight. That is, it is a weight based on the distance from the intermediate point 2722 to the intersection (intersection points 2612, 2614, 2616, and 2618), and the sum of the weights of the four intersections is 1. For example, a value using the reciprocal of the distance It is. Therefore, when equidistant from each intersection, the value is obtained by adding 1/4 of the luminance at each intersection.

なお、ｐ^＊は、補正目標値となる輝度値である。
図２８は、かぶり量を除去した様子を３次元的に表現したものである。ｘ軸、ｙ軸は画像の位置を示し、ｚ軸は輝度値を示す空間である。灰色の面がかぶりの画像であり、画像全体を覆っているように位置している。そして、薄い格子模様の面が補正した後の画像の輝度を示している。ｐ^＊は、図２８の例に示すように、前述（１）に示した画像における明るい輝度値である。これに合わせるように、各点における輝度を上げるようにしてかぶりを除去している。図２８では、中間点２７２２の輝度を引き上げていることを示している。 Then, the fogging amount is calculated by the equation (21).

Note that p ^* is a luminance value that is a correction target value.
FIG. 28 is a three-dimensional representation of how the fogging amount is removed. The x-axis and y-axis indicate the position of the image, and the z-axis is a space indicating the luminance value. The gray surface is a fogged image and is positioned so as to cover the entire image. And the brightness | luminance of the image after the surface of a thin grid | pattern pattern corrected is shown. As shown in the example of FIG. 28, p ^* is a bright luminance value in the image shown in (1) above. In accordance with this, the fog is removed by increasing the luminance at each point. FIG. 28 shows that the brightness of the intermediate point 2722 is raised.

  In step S1718, the output module 160 outputs the corrected image. For example, an image from which fog is removed by a printer is printed.

<Sixth Embodiment>
FIG. 29 is a conceptual module configuration diagram illustrating an exemplary configuration according to the sixth embodiment.
The image processing apparatus according to the sixth embodiment removes the fog from the image. As shown in the example of FIG. 29, the image reception module 110, the non-edge area extraction module 1620, and the inspection area selection module 1630 A pixel value measurement module 1640, a pixel value distribution estimation module 1650, a correction target value determination module 140, a correction availability determination module 1445, a correction module 910, and an output module 160. The correction module 910 includes a saturation correction module 150 and a luminance correction module 950.

The correction possibility determination module 1445 is connected to the correction target value determination module 140 and the correction module 910, calculates the fog amount based on the function estimated by the pixel value distribution estimation module 1650, and the fog amount and the correction target value. Based on the difference from the correction target value determined by the determination module 140, it is determined whether or not correction by the saturation correction module 150 and the luminance correction module 950 is to be performed.
Further, the correction possibility determination module 1445 calculates the fog amount based on the function estimated by the pixel value distribution estimation module 1650, and based on the difference between the fog amounts in each pixel in the image received by the image reception module 110. Thus, it may be determined whether or not correction by the saturation correction module 150 and the luminance correction module 950 is performed.
The saturation correction module 150 in the correction module 910 is equivalent to the saturation correction module 150 of the fifth embodiment, but performs correction when it is determined by the correction availability determination module 1445 that correction is performed.
The luminance correction module 950 in the correction module 910 is equivalent to the luminance correction module 950 of the fifth embodiment, but performs correction when it is determined by the correction availability determination module 1445 that correction is performed.

FIG. 30 is a flowchart illustrating a processing example according to the sixth exemplary embodiment.
In step S3002, the image reception module 110 receives a target image.
In step S3004, the non-edge region extraction module 1620 extracts a continuous region where the color value changes gently.
In step S3006, the inspection region selection module 1630 extracts a region suitable for estimating the fog inclination.
In step S3008, the pixel value measurement module 1640 measures the pixel value by scanning the line according to a predetermined rule.
In step S3010, the pixel value distribution estimation module 1650 estimates the degree of fog inclination from the measured set of pixel values.
In step S3012, the correction target value determination module 140 determines a correction target value.
In step S3014, the correction possibility determination module 1445 determines whether or not to perform correction processing. If so, the process proceeds to step S3016. Otherwise, the process proceeds to step S3020.
In step S3016, the saturation correction module 150 corrects the saturation of the image.
In step S3018, the brightness correction module 950 corrects the brightness of the image.
In step S3020, the output module 160 outputs the corrected image.

In the description of FIGS. 53A to 53C described above, comparison with the fifth embodiment and the sixth embodiment will be described using the examples of FIGS. 53C to 53F. As shown in the example of FIG. 53C, when the brightness value when the photographed image 5330 is scanned with the line 5332 is shown in the graph, it is as shown in the example of FIG. 53D. That is, since the fogged image 5320 becomes darker toward the right, the luminance value decreases to the right. However, since there is the original luminance value in the original image, the luminance is as shown in the example of the graph in FIG. A distribution 5342, a luminance distribution 5344, a luminance distribution 5346, and a luminance distribution 5348 indicate stepwise luminance distributions. Here, as in the example of FIG. 53 (f), when each luminance is corrected to the target luminance, the result is a corrected image 5340 as shown in the example of FIG. 53 (e). A portion such as a region 5312 where the background is not white becomes too bright and an image in which red is also lost is obtained. This is because the chromatic portion has a luminance value lower than that of the white background, but “a whole sheet of white paper document” is targeted.
On the other hand, in the fifth embodiment and the sixth embodiment, the area 5314 is selected, the fog removal amount is calculated based on the area 5314, and the image is corrected. A portion such as a region 5312 whose background is not white suppresses the brightness from becoming too bright compared to the corrected image 5340 shown in the example of FIG. 53 (e).

<Seventh embodiment>
FIG. 31 is a conceptual module configuration diagram of a configuration example according to the seventh embodiment.
The image processing apparatus according to the seventh embodiment removes the fog from the image. As shown in the example of FIG. 31, the quasi-flat area extraction module 3110, the sampling area selection module 3120, and the fog amount estimation module. 3130. A luminance fog correction module 3140 is provided.

  The quasi-flat region extraction module 3110 includes an edge enhancement module 3112, a binarization module 3114, an inversion module 3116, and a labeling module 3118, and is connected to the sampling region selection module 3120. The quasi-flat area extraction module 3110 extracts a quasi-flat area as a background from the target image. Here, the quasi-flat region refers to a non-edge region. The edge enhancement module 3112 performs edge enhancement on the target image. Existing edge enhancement techniques may be used. The binarization module 3114 binarizes the image on which edge enhancement has been performed. The inversion module 3116 inverts the binarized image into black and white. The labeling module 3118 labels the binary image that has been reversed in black and white. As a result, a plurality of non-edge regions are extracted. Note that the edge enhancement module 3112 corresponds to the Sobel filter module 1910 of the fifth embodiment described above, and the binarization module 3114 corresponds to the binarization module 1920 of the fifth embodiment described above. The module 3116 corresponds to the monochrome inversion module 1930 of the fifth embodiment, the labeling module 3118 corresponds to the labeling module 1940 of the fifth embodiment, and a small size removal module 1950 is added. May be.

  The sampling area selection module 3120 includes a color analysis module 3122, an identification function module 3124, and an area selection module 3126, and is connected to the quasi-flat area extraction module 3110 and the fogging amount estimation module 3130. The sampling area selection module 3120 selects a non-edge area suitable for fog amount estimation from a plurality of non-edge areas extracted by the quasi-flat area extraction module 3110. That is, the background area is selected. The color analysis module 3122 analyzes the color of the non-edge region. That is, from the coordinates of the non-edge region, the pixel value of that coordinate in the original image of interest is extracted, and the color of the non-edge region is analyzed. The discriminant function module 3124 applies a discriminant function to the color of the non-edge region to calculate a value indicating the background region. The area selection module 3126 selects a non-edge area suitable for fogging amount estimation using the value calculated by the discrimination function module 3124.

The fog amount estimation module 3130 includes a line sampling module 3132, a multi-model function estimation module 3134, and a planar luminance estimation module 3136, and is connected to the sampling area selection module 3120 and the luminance fog correction module 3140. The fog amount estimation module 3130 corresponds to the “fogging amount estimation process” in the above-described embodiment. The line sampling module 3132 sets a plurality of predetermined scanning lines (lines) and acquires color values (luminance values) inside and outside the area selected by the area selection module 3126.
The multi-model function estimation module 3134 calculates information amounts of a plurality of functions for estimating the fog amount for each scanning line in the target image, and selects a function to be applied to each scanning line based on the information amount. To do. Then, a combination of functions is selected based on the information amount of the function for each selected scanning line and the difference norm between the plurality of functions. In selecting a function, a function that should not be selected may be excluded based on the distribution of luminance values of pixels for each scanning line, and a function may be selected from the remaining functions. “A plurality of functions for estimating the fogging amount”, “information amount of function”, “selecting a function”, “selecting a combination of functions” and the like will be described later. The planar luminance estimation module 3136 estimates the fogging amount by applying the combination of functions selected by the multi-model function estimation module 3134 for each scanning line.

The luminance fog correction module 3140 includes a coordinate-by-coordinate correction amount calculation module 3142, a luminance correction module 3144, and a saturation correction module 3146, and is connected to the fog amount estimation module 3130. The luminance fog correction module 3140 removes the fog amount from the target image. The correction amount calculation module 3142 for each coordinate calculates the correction amount of each pixel in the target image. The luminance correction module 3144 corrects the luminance of the target image based on the correction amount calculated by the coordinate-specific correction amount calculation module 3142. The saturation correction module 3146 corrects the saturation of the target image based on the correction amount calculated by the correction amount calculation module 3142 for each coordinate.
Note that the “fogging amount estimation processing” in the above-described embodiment may be replaced with the processing by the fogging amount estimation module 3130 in the seventh embodiment.

FIG. 32 is a flowchart illustrating a processing example according to the seventh exemplary embodiment.
In step S3202, the quasi-flat region extraction module 3110 controls the edge enhancement module 3112, the binarization module 3114, the inversion module 3116, and the labeling module 3118 to extract a quasi-flat region from the target image.
In step S3204, the color analysis module 3122 analyzes the color of the quasi-flat region.
In step S3206, the discriminant function module 3124 applies the discriminant function to the quasi-flat region.
In step S3208, the region selection module 3126 selects a target region.
In step S <b> 3210, the line sampling module 3132 samples the image using scan lines.
In step S3212, the multi-model function estimation module 3134 performs estimation using a multi-model function.
In step S3214, the planar brightness estimation module 3136 performs brightness estimation using a model function.
In step S3216, the luminance fog correction module 3140 controls the correction amount calculation module 3142, the luminance correction module 3144, and the saturation correction module 3146 described above to correct the image using the estimated fog amount.

ｒ、ｃの関数（（１６）式、（１７）式）の形式から、明度分布ｈ^＊として正しく推定できる上限は（２３）式のようになる。

つまり、ｘ、ｙに関する２次形式以上の多項式分布や、当然ながら指数関数族などの非多項式分布も使われていない。平面（２次元）のかぶりは、非線形な分布と考えられる。そこで、第７の実施の形態では、かぶり量を推定するのに、より高次元な関数モデルで対応する。 Hereinafter, processing by the fog amount estimation module 3130 will be described. FIG. 33 is an explanatory diagram showing an example of the relationship between an image scanning line and a line / function correspondence table 3350.
The line sampling module 3132 scans the target image 3300. That is, a predetermined scanning line is set, and a pixel value on the scanning line is acquired. Then, the fogging amount is estimated. In the above-described embodiments (from the first embodiment to the sixth embodiment), as shown in the equations (16) and (17), the linear equation is used. Estimating. The lightness correction target value h is calculated as shown in equation (22).

From the format of the functions of r and c (equations (16) and (17)), the upper limit that can be correctly estimated as the brightness distribution h ^{* is} as shown in equation (23).

That is, a polynomial distribution of a quadratic or higher-order form regarding x and y and a non-polynomial distribution such as an exponential family are naturally not used. Planar (two-dimensional) fogging is considered a non-linear distribution. Therefore, in the seventh embodiment, a higher-dimensional function model is used to estimate the fogging amount.

このモデルで真の関数との誤差がガウス分布にしたがうとして、最尤推定解を計算すると（２６）式、（２７）式、（２８）式のようになる。

第７の実施の形態では、多項式、非線形関数の線形和とする。（２９）式は多項式であり、（３０）式はシグモイド関数であり、（３１）式はガウシアン関数であり、（３０）式、（３１）式は非線形の式である。

それぞれの関数集合を（３２）式、（３３）式、（３４）式で表現する。ただし、非線形の式である（３０）式、（３１）式には、φ_０（ｘ）＝２５５も加える（（３３）式、（３４）式）。

The fogging amount is estimated by selecting from a predetermined function set from the pixel values obtained by scanning. That is, the function models of the functions r and c in the horizontal and vertical directions (formulas (16) and (17) in the above embodiment) are expanded.
With respect to the parameters α and β, linear models such as the equations (24) and (25) are set. This is in order to obtain a solution analytically.

Assuming that the error from the true function in this model follows a Gaussian distribution, the maximum likelihood estimation solution is calculated as shown in equations (26), (27), and (28).

In the seventh embodiment, a linear sum of a polynomial and a nonlinear function is used. Equation (29) is a polynomial, Equation (30) is a sigmoid function, Equation (31) is a Gaussian function, and Equations (30) and (31) are nonlinear equations.

Each function set is expressed by Expression (32), Expression (33), and Expression (34). However, φ ₀ (x) = 255 is also added to the equations (30) and (31), which are nonlinear equations (the equations (33) and (34)).

なお、（３５）式のＡＩＣ値が小さいほどモデルとして適した関数であることを示している。 Thus, a maximum likelihood estimation solution was obtained for each model. Next, which function should be selected will be described.
Here, an example will be described in which selection is performed using AIC (Akaike Information Criterion). AIC is the model-dependent portion of the expected amount of information on the amount of information in the Cullback library, transformed into a state in which up to 1 / n order (n: number of samples) terms are left in statistical asymptotic theory (processing similar to Taylor expansion of functions). Is approximated by a sample average. As a feature, the approximation part of the 1 / n-order term means that “the more the number of parameters, the more penalty is given”.
Using this AIC, the expressions (35) and (36) are calculated from the expressions (24), (26), and (28).

Note that the smaller the AIC value in the equation (35), the more suitable the model is.

A line / function correspondence table 3350 shown in the example of FIG. 33B shows each scanning line in the image 3300 (in the example of FIG. 33A, a line segment with an arrow), and is arranged on the scanning line as a column. A function set (function model) to be applied to a pixel group is arranged. Each cell stores an AIC value. That is, the AIC value of each function is calculated for the pixel group of each scanning line, and the AIC value is stored. A circle (◯) in the line / function correspondence table 3350 indicates the AIC minimum value for each scanning line. That is, the best function model is determined for each scanning line using AIC.
However, here, each scanning line is estimated too much independently. That is, simply selecting a function model indicating the AIC minimum value for each scanning line is not suitable for estimating the fogging amount. As a matter of fact, the assumed shadow cast is not clear and is vague.

つまり、図３４（ｂ）のライン・関数対応テーブル３３５０では、走査線ｒ_０と隣接している走査線毎の関数の差分ノルム３４５２を算出する。
そして、２つの走査線分の評価値（Ｊ）を、「隣接走査線のＡＩＣ値２つと関数の差分ノルム値の和」で定義し、（３８）式のように表現する。（３８）式は、各走査線のＡＩＣ値と関数の差分ノルムによって定義されている。

これをＮ個の走査線に拡張した次の規準値を最小にする組み合わせを採用する（（３９）式、（４０）式参照）。

関数ノルムは例えば（４１）式で定める。

このように、近傍する走査線同士で各走査線の関数モデル同士の近似度（ここでは、「関数の差分ノルム」）を全て算出し、全ての走査線、又は近傍の複数区間内の走査線で最も有効な組み合わせを選択する。 Therefore, the function model selection method can be performed in series as follows.
FIG. 34 is an explanatory diagram showing an example of the difference norm in the line / function correspondence table 3350.
“The assumed shadow cast is not clear and is blurry”, so “the function of the near scanning line is relatively similar in shape”. Therefore, the similarity of shape is defined by “the difference norm of the function”. Here, “the difference norm of the function” in r _{i + 1} and r _i is the expression (37).

That is, in the line function correspondence table 3350 of FIG. 34 (b), calculates the difference norm 3452 functions for each scan line that is adjacent to the scanning line _{r 0.}
Then, the evaluation value (J) of two scanning lines is defined by “the sum of two AIC values of adjacent scanning lines and the difference norm value of the function”, and is expressed as in equation (38). Equation (38) is defined by the AIC value of each scanning line and the difference norm of the function.

A combination that minimizes the next reference value, which is expanded to N scanning lines, is employed (see equations (39) and (40)).

The function norm is determined by, for example, equation (41).

In this way, the degree of approximation between the function models of each scanning line (here, “function difference norm”) is calculated among the neighboring scanning lines, and all the scanning lines or scanning lines in a plurality of neighboring sections are calculated. Select the most effective combination.

Next, it can be expected that a high-dimensional function diverges when the effective area is narrow.
Based on the distribution of luminance values of pixels for each scanning line, function models that should not be selected may be excluded and function models may be selected from the remaining functions. Specifically, the function model of the scanning line that does not pass through the region selected by the region selection module 3126 may be excluded. In addition, the rate at which the region is included is calculated for each scanning line, and if it is smaller or smaller than a predetermined threshold value, the function model of the scanning line may be excluded. Further, for each function model, a value group that the function model diverges is calculated in advance, and when the divergence value group is included in the scan line, the function model that diverges in the scan line may be excluded. Good.

<Eighth Embodiment>
FIG. 35 is a conceptual module configuration diagram illustrating an exemplary configuration according to the eighth embodiment.
The image processing apparatus according to the eighth embodiment removes fog from an image, and as shown in the example of FIG. 35, a quasi-flat area extraction module 3110, a sampling area selection module 3120, and a fog amount estimation module. 3530 and a luminance fog correction module 3140. That is, the fog amount estimation module 3130 of the seventh embodiment is changed to a fog amount estimation module 3530. In particular, it is suitable when an image composed of a plurality of background colors is targeted. In the eighth embodiment, a plurality of background-like regions are extracted from a target image, and a plurality of sampling lines (scanning lines) in a predetermined direction passing through the plurality of regions are prepared. Even when the background colors of a plurality of regions are different, after performing differential function approximation of data using a difference value from adjacent pixels as an input, the background color of each region is calculated from the integral form.

The quasi-flat region extraction module 3110 includes an edge enhancement module 3112, a binarization module 3114, an inversion module 3116, and a labeling module 3118, and is connected to the sampling region selection module 3120.
The sampling area selection module 3120 includes a color analysis module 3122, an identification function module 3124, and an area selection module 3126, and is connected to the quasi-flat area extraction module 3110 and the fog amount estimation module 3530.

The fog amount estimation module 3530 includes a life sampling module 3532, a multi-model global differential function estimation module 3534, a local area constant estimation module 3536, and a planar luminance estimation module 3538. A sampling area selection module 3120 and a luminance fog correction module 3140. Connected with. This corresponds to the “fogging amount estimation process” in the above-described embodiment. The line sampling module 3532 sets a plurality of predetermined scanning lines (lines) and acquires color values (luminance values) inside and outside the area selected by the area selection module 3126. Then, the difference from the adjacent pixel is calculated for each scanning line. The difference is, of course, the difference between the color value of the target pixel and the color value of the adjacent pixel (adjacent on the scanning line).
The multi-model global differential function estimation module 3534 is a plurality of functions applied to the pixel difference calculated by the life sampling module 3532, and the plurality of functions for estimating the fogging amount for each scanning line in the target image. The information amount is calculated, and a function to be applied to each scanning line is selected based on the information amount. Then, a combination of functions is selected based on the information amount of the function for each selected scanning line and the difference norm between the plurality of functions. That is, a suitable function (regression function) is estimated from among a plurality of function models having differences as inputs. These processes are performed for each area selected by the area selection module 3126.
The local domain constant estimation module 3536 calculates an integral function of the function selected by the multi-model global differential function estimation module 3534 and estimates a constant term of the integral function. This constant term is the sum of the constant of the fog amount of the entire area selected by the area selection module 3126 and the specific amount of the background color of the area. That is, the line function of the function estimated for each selected region is calculated, and the constant term is estimated.
The planar luminance estimation module 3538 applies the combination of the function selected by the multi-model global differential function estimation module 3534 and the constant term estimated by the local area constant estimation module 3536 for each scanning line in the area, thereby applying the fog amount. Is estimated. As a result, the regression is executed with respect to the difference between the plurality of regions, so that the fog spreading over the entire target image can be estimated by the constant term.

  The luminance fog correction module 3140 includes a coordinate-by-coordinate correction amount calculation module 3142, a luminance correction module 3144, and a saturation correction module 3146, and is connected to the fog amount estimation module 3530.

図３６（ｂ）の例に示すかぶり画像３６２０は、画像３６１０を撮影した際に発生したかぶりを示す像である。かぶり量は、（４３）式のようになる。

図３６（ｃ）の例に示す撮影画像（観測画像）３６３０は、画像３６１０を実際に撮影した撮影画像（観測画像）３６３０である。つまり、画像３６１０とかぶり画像３６２０が合成された画像である。つまり、ここでの画素値は、（４４）式のようになる。

第８の実施の形態は、観測値（撮影画像（観測画像）３６３０）から非線形なかぶり（かぶり画像３６２０）を平坦にするための除去量を推定する。 FIG. 36 is an explanatory diagram illustrating an example of an image targeted by the eighth embodiment.
An image 3610 shown in the example of FIG. 36A is a target image, and has five regions with different backgrounds (an orange background region S ₀ c ₀ , a light green background region S). ₁ c ₁ , a light blue background region S ₂ c ₂ , a pink background region S ₃ c ₃ , and a white background region S _NS c _NS ). Here, the original pixel value at the coordinates x and y is as shown in equation (42).

A fogging image 3620 shown in the example of FIG. 36B is an image showing the fogging that occurs when the image 3610 is captured. The fogging amount is as shown in equation (43).

A captured image (observed image) 3630 shown in the example of FIG. 36C is a captured image (observed image) 3630 obtained by actually capturing the image 3610. That is, the image 3610 and the cover image 3620 are combined. That is, the pixel value here is as shown in Equation (44).

In the eighth embodiment, the removal amount for flattening the non-linear fog (the fog image 3620) is estimated from the observed value (the photographed image (observed image) 3630).

FIG. 37 is an explanatory diagram illustrating an example of scanning lines of an image targeted by the eighth embodiment. In the examples shown in FIGS. 37 (a), (b), and (c), scanning lines 3710, 3720, and 3730 are shown in the images shown in the examples of FIGS. 36 (a), (b), and (c), respectively. Is. The pixel values on each scanning line are as shown in Equations (45), (46), and (47).

図３８（ｂ）の例に示す撮影画像（観測画像）３６３０の輝度差分値データを取得する。撮影画像（観測画像）３６３０の走査線３７２０における輝度差分値データは、（４９）式のようになる。なお、（４９）式の最右辺にある２つの式のうち、上の式はエッジ部分を示しており、下の式はエッジ部分以外を示している。

非エッジ領域マスク３８１０（（４８）式）と非エッジ領域マスク３８１０（（４９）式）との論理積（ＡＮＤ）画像を生成して、図３８（ｃ）の例に示す差分値データ３８４０を生成する。これは、微分不可であるエッジを除外した後の差分値データである。そして、差分値データ３８４０の走査線３８５０上における輝度差分値データは、（５０）式のようになる。

FIG. 38 is an explanatory diagram illustrating a processing example according to the eighth exemplary embodiment.
In the eighth embodiment, the fact that the component related to the increase / decrease in equation (47) exists only on the shadow side (cover image) is used.
For this purpose, first, the non-edge region mask 3810 shown in the example of FIG. Non-edge region mask 3810 is an image generated by inversion module 3116. The pixel value here is given by equation (48).

The luminance difference value data of the captured image (observed image) 3630 shown in the example of FIG. The luminance difference value data on the scanning line 3720 of the photographed image (observed image) 3630 is expressed by equation (49). Of the two expressions on the rightmost side of the expression (49), the upper expression indicates the edge part, and the lower expression indicates other than the edge part.

A logical product (AND) image of the non-edge region mask 3810 (expression (48)) and the non-edge region mask 3810 (expression (49)) is generated, and the difference value data 3840 shown in the example of FIG. Generate. This is difference value data after excluding an edge that cannot be differentiated. Then, the luminance difference value data on the scanning line 3850 of the difference value data 3840 is expressed by the equation (50).

（２９）式、（３０）式、（３１）式、（５１）式、（５２）式、（５３）式のそれぞれの関数集合を、以下の（３２）式、（３３）式、（３４）式、（５４）式、（５５）式、（５６）式で表現する。

つまり、第７の実施の形態における関数集合を（５４）式、（５５）式、（５６）式で置き換える。 The partial differential terms of each function group of the equations (29), (30), and (31) used in the seventh embodiment are the following equations (51), (52), and (53). It becomes like the formula.

Equations (29), (30), (31), (51), (52), and (53) are expressed as the following function sets (32), (33), and (34), respectively. ), (54), (55), and (56).

That is, the function set in the seventh embodiment is replaced with the formulas (54), (55), and (56).

ここまでで、最尤推定結果であるパラメタαを推定できる。ただし、（５８）式における定数項部分（α_０）だけ未知のままである。 FIG. 39 is an explanatory diagram showing an example of the difference norm in the line / function correspondence table. That is, it is equivalent to FIG. 34 described in the seventh embodiment. However, the differential data on the scanning line is shown as a row of the line / function correspondence table 3950, and differential functions to which the differential data on the scanning line is applied are arranged as columns. Each cell stores an AIC value. That is, the function model is defined in a first-order differential space, and the model selection method is equivalent to that of the seventh embodiment. Then, the equation (57) is selected. The equation (58) is derived by integrating the equation (57).

Thus far, the parameter α which is the maximum likelihood estimation result can be estimated. However, only the constant term part (α ₀ ) in the equation (58) remains unknown.

全てのラベル領域（図４０の例に示す画像４０００では５つの領域）で定数項を推定する。この定数項は、影かぶりの定数量と背景色の固有量の和に相当する。この定数項を参照にして、対象としている画像の背景部の領域を探す。
そこで、影かぶりの均等化のための補正量を決める必要がある。図４１は、第８の実施の形態による処理例を示す説明図である。
まず、対象としている画像内で一番明度の高い領域（一般的には白色の背景部）の定数項（ｃ＾ｌ）を全面で採用するという仮定をする。その上で、影かぶりの分布の予め定められたレベル（中央値、平均値、最頻値、高明度側から予め定められた率に位置する点など）を基準にして差分量を決める。（６０）式の左辺は、領域固有色を白部分に合わせた状態に相当する。

図４１（ｂ）の例では、走査線毎のかぶり量を示している。 FIG. 40 is an explanatory diagram illustrating a processing example according to the eighth exemplary embodiment.
A constant term is estimated using the equation (59) from the estimation results of multiple lines (selection of a combination of function models) for each region.

Constant terms are estimated in all label regions (five regions in the image 4000 shown in the example of FIG. 40). This constant term corresponds to the sum of the constant amount of shadow cast and the inherent amount of background color. By referring to this constant term, the background area of the target image is searched.
Therefore, it is necessary to determine a correction amount for equalizing shadow fog. FIG. 41 is an explanatory diagram of a processing example according to the eighth embodiment.
First, it is assumed that the constant term (c ^ l) of the region with the highest brightness (generally a white background portion) in the target image is adopted over the entire surface. Then, a difference amount is determined based on a predetermined level (median value, average value, mode value, point located at a predetermined rate from the high brightness side) of the shadow fog distribution. The left side of the equation (60) corresponds to a state in which the region specific color is matched with the white portion.

In the example of FIG. 41B, the fogging amount for each scanning line is shown.

領域画像４２２０に対して識別関数の計算結果は、（６２）式のように２．０である。

領域画像４２３０に対して識別関数の計算結果は、（６３）式のように０．２である。

そして、（６４）式に示すように、識別関数の算出値が閾値ＴＨ以上となるマスク画像を選択する。この例の場合は、（６５）式のようになり、領域画像４２１０、領域画像４２２０が選択される。これによって、傾斜かぶりを推定するのに十分な領域が選択されることとなる。なお、重みｗは、予め定められた値に設定したり、正解の選択肢が決まっている教師データから学習させるようにしてもよい。

FIG. 42 is an explanatory diagram of an example of processing according to the eighth embodiment. This is equivalent to FIG. 21 described in the fifth embodiment.
As shown in the equation (10), by using a discriminant function of weighted linear sums of a plurality of feature amounts, all those satisfying the conditions in the region sufficient for estimating the most inclined fog are selected.
The calculation result of the discriminant function for the area image 4210 is 1.5 as shown in the equation (61).

The calculation result of the discriminant function for the region image 4220 is 2.0 as shown in Equation (62).

The calculation result of the discriminant function for the area image 4230 is 0.2 as shown in the equation (63).

Then, as shown in the equation (64), a mask image in which the calculated value of the discrimination function is equal to or greater than the threshold value TH is selected. In this example, the equation (65) is obtained, and the region image 4210 and the region image 4220 are selected. As a result, a region sufficient to estimate the tilt fog is selected. The weight w may be set to a predetermined value or may be learned from teacher data in which correct answer options are determined.

<Ninth embodiment>
FIG. 43 is a conceptual module configuration diagram illustrating an exemplary configuration according to the ninth embodiment.
The image processing apparatus according to the ninth embodiment removes the fog from the image. As shown in the example of FIG. 43, the quasi-flat area extraction module 3110, the sampling area selection module 3120, and the fog amount estimation module. 3530, a fogging presence / absence processing module 4310, and a luminance fogging correction module 3140. That is, the fogging presence / absence processing module 4310 is inserted between the fogging amount estimation module 3530 and the luminance fogging correction module 3140 of the eighth embodiment described above. 43, in addition to the module configuration shown in the example of FIG. 43, before the “fogging amount removal process” in the embodiments up to the eighth embodiment described above (not only immediately before, for example, the fogging amount is estimated. The processing by the fogging presence / absence processing module 4310 may be performed before the processing.
The ninth embodiment is particularly suitable when there is a possibility that gradation or the like similar to a fog as a design is included in a document. For example, the original (such as a business card) in the target image may have a gray gradation as a design. In the above-described embodiment, it is difficult to distinguish the fog from the gradation as the design.

In the first place, when fogging occurs, as a natural phenomenon, for example, when a photograph placed on a desk is photographed with the camera looking down from above, light blocking by the camera or hand often affects. In this case, the same fog as the original is generated around the original to be photographed. According to whether or not such a phenomenon has occurred, the ninth embodiment determines whether it is a design in a document or a cover.
This will be described with reference to the example of FIG. FIG. 44 is an explanatory diagram illustrating an example of an image targeted by the ninth embodiment.
The example in FIG. 44A shows a case where a document 4420 (white uniform document) that has not been subjected to gradation or the like as a design is photographed. There are desk surfaces. The example in FIG. 44 (b) shows a fogging image 4430 in the case of shooting in that state.
And when it actually image | photographs, it becomes the picked-up image 4440 as shown in the example of FIG.44 (c). In other words, the original document 4420 is white and uniform, but the document image 4450 obtained by photographing the document 4420 has fog, and the background also has fog.
On the other hand, the photographed image 4460 shown in the example of FIG. 44D has no fog, but the document has a gradation as a design, and the document image 4470 has a gradation similar to the fog. . In this case, no fog occurs on the background of the document image 4470.

The quasi-flat region extraction module 3110 includes an edge enhancement module 3112, a binarization module 3114, an inversion module 3116, and a labeling module 3118, and is connected to the sampling region selection module 3120.
The sampling area selection module 3120 includes a color analysis module 3122, an identification function module 3124, and an area selection module 3126, and is connected to the quasi-flat area extraction module 3110 and the fog amount estimation module 3530.
The fog amount estimation module 3530 includes a life sampling module 3532, a multi-model global differential function estimation module 3534, a local area constant estimation module 3536, and a planar luminance estimation module 3538. A sampling area selection module 3120 and a fogging presence / absence processing module 4310 Connected with.

The fogging presence / absence processing module 4310 includes a document area extraction module 4312, a fogging presence / absence determination module 4314, and a control module 4316, and is connected to a fogging amount estimation module 3530 and a luminance fogging correction module 3140. The fogging presence / absence processing module 4310 determines whether fogging has occurred in the target image (whether there is a gradation or the like in the document), and includes a fog removal process (including a fogging amount estimation process). Control may be performed.
The document area extraction module 4312 extracts a document image area captured from a target image. The area extracted by the document area extraction module 4312 may be a quadrangle. The processing by the document area extraction module 4312 will be described later with reference to FIG.

The fogging presence / absence determination module 4314 compares the image extracted by the document area extraction module 4312 with the image outside the area, and determines whether or not the fogging has occurred only in that area. . “Comparing images inside and outside the region” means calculating the similarity between the lightness distribution inside the region and the lightness distribution outside the region. Note that the distribution of saturation may be used instead of the distribution of brightness. The distribution similarity refers to the similarity in the shape of the graph composed of the values of lightness or saturation at the position of the X coordinate or the Y coordinate, and specifically, the X coordinate or the Y coordinate in a predetermined range. If the difference in brightness or saturation is within a predetermined range at the position, it may be determined that they are similar. The case where the fog is generated only in the area is a case where a gradation design is applied to the document to be photographed. That is, the gradation is determined to be fogging, and fog removal processing is performed to prevent the original design from being damaged.
Further, the fogging presence / absence determination module 4314 compares the outside and inside images of the quadrangular side as the region, and the inside image is within a predetermined distance from the quadrangular side to the inside. An area in which the fog amount is estimated by the fog amount estimation module 3530 may be included. Here, the “region where the fogging amount is estimated” is a region where the fogging correction is necessary and the fogging amount is larger than zero. When there is no area where the fogging amount is estimated within a predetermined distance from the sides of the quadrangular (when no fogging has occurred within that distance), it is not a comparison target (in the example of FIG. 48, “ Side that is said to be "unquestioned").

  When the fogging presence / absence judgment module 4314 determines that the fog is generated only in the area, the control module 4316 performs control so as not to remove the fogging amount. That is, the process by the luminance fog correction module 3140 is not performed. In the case of an exception (when it is determined by the fogging presence / absence determination module 4314 that fogging has occurred not only within the area but also outside the area), control is performed to remove the fogging amount. . That is, the processing by the luminance fog correction module 3140 is performed.

  The luminance fog correction module 3140 includes a coordinate-by-coordinate correction amount calculation module 3142, a luminance correction module 3144, and a saturation correction module 3146, and is connected to the fogging presence / absence processing module 4310.

FIG. 45 is an explanatory diagram showing a processing example according to the ninth exemplary embodiment (document area extraction module 4312).
As an example of document area extraction processing performed by the document area extraction module 4312, a technique of “detecting a straight line by Hough transform and detecting the one having the largest area among four vertices forming a quadrilateral” will be described.
The image shown in the example of FIG. 45A is a photographed image, and fogging has occurred throughout.
The image shown in the example of FIG. 45B is an image in which the image of FIG. 45A is edge-enhanced and binarized, and then black and white are reversed.
The image shown in the example of FIG. 45C is an image obtained as a result of extracting a straight line having a predetermined length or more from the image of FIG. 45B by the Hough transform process.
The image shown in the example of FIG. 45D is an image obtained as a result of extracting the vertices of the quadrangle having the maximum area among the four vertices forming a quadrangle with the straight lines in the image of FIG. The circles in the figure are the four vertices extracted.
The image shown in the example of FIG. 45 (e) shows a region composed of four vertices extracted as in the image of FIG. 45 (d).
The image shown in the example of FIG. 45 (f) is an image obtained by extracting the region of FIG. 45 (e) from the image of FIG. 45 (a). Only the image portion of the document (business card) is extracted.

46 and 47 are explanatory diagrams illustrating a processing example according to the ninth embodiment. The photographed image 4460 shown in the example of FIG. 46A shows an image when there is gradation in the document (original image 4470) and no fogging has occurred. In this case, the shape of the brightness value distribution is different at the boundary with the background portion. Specifically, the distribution of density values in the scanning line 4612 that scans the inside of the document image 4470 is as shown by a graph 4622 shown in the example of FIG. The distribution of density values in 4614 is as shown by a graph 4624 in the example of FIG.
A photographed image 4440 shown in the example of FIG. 47A shows an image when there is no gradation in the document (original image 4450) and fogging occurs. In this case, at the boundary with the background portion, although the brightness value is shifted, the shape of the distribution is the same. Specifically, the distribution of density values in the scanning line 4712 that scans within the document image 4450 is as shown by a graph 4722 in the example of FIG. The distribution of density values in 4714 is like a graph 4724 shown in the example of FIG.

なお、ｃ（ｆ，ｇ）は、関数ｆ、ｇの定数項のズレ量である。例えば、第８の実施の形態の領域毎の定数成分の推定量同士の差分を採用すればよい。 An example of processing performed by the fogging presence / absence determination module 4314 will be described.
As a premise, a manuscript is placed on a desk (office desk or the like) using a camera or the like and photographed. It is desirable that the background desk or the like has a uniform density.
(1) Measure the brightness distribution inside the document. Existing methods may be used.
(2) The area where the fog is to be removed (that is, the area where the fog amount is specified) is specified. For the identification, the method according to the above-described embodiment may be used.
(3) The density value on the scanning line A that passes through the area where the fogging amount is specified in (2) is acquired. The region through which the scanning line A passes is a region that is within a predetermined distance from the side of the quadrilateral extracted by the document region extraction module 4312 and includes the region in which the fog amount is estimated by the fog amount estimation module 3530. . There is a scanning line 4612 shown in the example of FIG. The density value on the scanning line B passing outside the quadrangular side adjacent to the scanning line A is acquired. There is a scanning line 4614 shown in the example of FIG. Then, the shape of the density distribution is compared. For example, when the difference between the density values at the respective positions on the scanning lines is within a predetermined range, it is determined that the shapes of the density distributions are similar (overall fog occurs). If the difference between the density values at each position on the scanning line is not within a predetermined range, the shape of the density distribution is not similar (a design similar to fogging is applied in the document). Judgment)
For example, the similarity of distributions may be evaluated by using the norm value of the difference function obtained by correcting the deviation of the constant terms of the distribution functions f and g on the scanning lines inside and outside the document. .

Note that c (f, g) is a deviation amount of the constant terms of the functions f and g. For example, what is necessary is just to employ | adopt the difference of the estimation amount of the constant component for every area | region of 8th Embodiment.

A processing example according to the ninth embodiment (a fogging presence / absence determination module 4314) will be described with reference to FIGS. The example of FIG. 48 is equivalent to the example of FIG. 46 (when there is a design similar to the fog in the document), and the example of FIG. 49 is equivalent to the example of FIG. 47 (when the fog occurs).
In the example of FIG. 48A, a region 4850 surrounded by a thick line is a correction candidate region in the document image 4470 (a region in which the fogging amount in the quadrilateral extracted by the document region extraction module 4312 is specified).
Out-of-region comparison target line segments 4812, 4816, and 4818 are density distribution shape match confirmation targets, and an out-of-region comparison target line segment 4814 is a don't care side. The out-of-region comparison target line segments 4812, 4816, and 4818 have a region 4850 within a predetermined distance on the inner side from the side of the quadrangular original image 4470 extracted by the original region extraction module 4312. This is because the target line segment 4814 has no area 4850 within a predetermined distance.
Then, the scanning line passing through the out-of-region comparison target line segments 4812, 4816 and 4818 and the density value in the scanning line parallel to the scanning line and passing through the region 4850 are extracted.
Here, in the out-of-region comparison target line segments 4812, 4816, and 4818 other than the don't care side, the shape of the density value is all inconsistent. In other words, the background estimation result is a flat surface with no inclination (no fogging). The out-of-region comparison target line segment 4814 that is a don't care side matches, but this side is not the target. Accordingly, the photographed image 4460 is not subjected to the fog removal process (substantially nothing).
On the other hand, in the example of FIG. 49A, a region 4950 surrounded by a thick line is a correction candidate region in the document image 4450 (a region in which the fogging amount in the rectangle extracted by the document region extraction module 4312 is specified). .
Out-of-region comparison target line segments 4912, 4916, and 4918 are density distribution shape matching confirmation targets, and an out-of-region comparison target line segment 4914 is a don't care side. This is equivalent to the example of FIG.
Then, the scanning line passing through the out-of-region comparison target line segments 4912, 4916, and 4918 and the density value in the scanning line that is parallel to the scanning line and passes through the region 4950 are extracted.
Here, the shape of the density value is the same in the non-region comparison line segments 4912, 4916, and 4918 other than the don't care side. That is, the estimation result of the background is a plane with inclination (where fogging occurs). The non-region comparison target line segment 4914 that is a don't care side also matches, but this side is not the target. Therefore, the photographed image 4440 is subjected to fog removal processing. In this case, the fogging amount of the captured image 4440 may be removed using the fogging amount in the region 4950 (that is, the region in which the fogging amount is specified in the document image 4450 without including the background).

In addition, when there is both a match and a disagreement other than in the don't care area, it may be determined that there is a match (the occurrence of fogging) if there is only one match. This will be described with reference to the example of FIG.
For example, it is assumed that when a business card or paper document placed on a desk is photographed with a portable camera, a cover such as an area 5060 is partially generated by the hand and the camera body. In this case, the sides 5012, 5014, and 5018 are don't care sides, but the sides 5016 have the same density shape. That is, the estimation result of the background is a plane with inclination (where fogging occurs). Therefore, the photographed image 5040 is subjected to fog removal processing.

  With reference to FIG. 54, a hardware configuration example of the image processing apparatus according to the above-described embodiment will be described. The configuration shown in FIG. 54 is configured by a personal computer (PC), for example, and shows a hardware configuration example including a data reading unit 5417 such as a scanner and a data output unit 5418 such as a printer.

  The CPU (Central Processing Unit) 5401 includes various modules described in the above-described embodiments, that is, the fog estimation module 120, the pixel value measurement module 130, the correction target value determination module 140, the saturation correction availability determination module 145, Saturation correction module 150, luminance correction module 950, correction availability determination module 1445, non-edge area extraction module 1620, inspection area selection module 1630, pixel value measurement module 1640, pixel value distribution estimation module 1650, Sobel filter module 1910, 2 Value conversion module 1920, monochrome inversion module 1930, labeling module 1940, small size removal module 1950, quasi-flat area extraction module 3110, edge enhancement module Module 3112, binarization module 3114, inversion module 3116, labeling module 3118, sampling area selection module 3120, color analysis module 3122, discrimination function module 3124, area selection module 3126, fogging amount estimation module 3130, line sampling module 3132, multi Model function estimation module 3134, plane luminance estimation module 3136, luminance fog correction module 3140, coordinate-by-coordinate correction amount calculation module 3142, luminance correction module 3144, saturation correction module 3146, fog amount estimation module 3530, life sampling module 3532, multi-model Global differential function estimation module 3534, local domain constant estimation module 3536, planar luminance estimation module 3538, or Ri determination processing module 4310, the original region extraction module 4312, head state determining module 4314, a control unit for executing processing in accordance with the computer program describing the execution sequence of the modules such as the control module 4316.

  A ROM (Read Only Memory) 5402 stores programs used by the CPU 5401, operation parameters, and the like. A RAM (Random Access Memory) 5403 stores programs used in the execution of the CPU 5401, parameters that change as appropriate in the execution, and the like. These are connected to each other by a host bus 5404 including a CPU bus.

  The host bus 5404 is connected to an external bus 5406 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 5405.

  A keyboard 5408 and a pointing device 5409 such as a mouse are input devices operated by an operator. The display 5410 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information.

  An HDD (Hard Disk Drive) 5411 has a built-in hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 5401 and information. The hard disk stores the received image, the estimated fog amount, the measured pixel value (luminance value and saturation value), the determined correction target value, the image of the extracted area, the estimated function, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 5412 reads out data or a program recorded on a removable recording medium 5413 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and reads the data or program into an interface 5407 and an external bus 5406. , The bridge 5405, and the RAM 5403 connected via the host bus 5404. The removable recording medium 5413 can also be used as a data recording area similar to the hard disk.

  The connection port 5414 is a port for connecting the external connection device 5415 and has a connection unit such as USB, IEEE1394. The connection port 5414 is connected to the CPU 5401 and the like via the interface 5407, the external bus 5406, the bridge 5405, the host bus 5404, and the like. A communication unit 5416 is connected to the network and executes data communication processing with the outside. The data reading unit 5417 is a scanner, for example, and executes document reading processing. The data output unit 5418 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration of the image processing apparatus shown in FIG. 54 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 54, and the modules described in the present embodiment are executed. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line In addition, a plurality of systems shown in FIG. 54 may be connected to each other through communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine (an image processing apparatus having any two or more functions of a scanner, a printer, a copying machine, a fax machine, etc.).

  Note that the above-described various embodiments may be combined (for example, adding or replacing a module in one embodiment in another embodiment), and processing contents of each module The technique described in the background art may be employed.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray Disc (registered trademark), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) Etc. are included.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

The above-described embodiment can also be understood as follows.
[A1]
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value of the pixels in the area selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the pixel value measured by the measuring means;
An image processing apparatus comprising: a removing unit that removes the fog from the image received by the receiving unit based on the function estimated by the estimating unit.
[A2]
The image according to [A1], wherein the extraction unit extracts an area larger than or larger than a predetermined size, and if the area cannot be extracted, the processing after the selection unit is not performed. Processing equipment.
[A3]
The image processing apparatus according to [A1] or [A2], wherein the predetermined rule in the selection unit is determined by a size of an area extracted by the extraction unit.
[A4]
The image processing apparatus according to [A3], wherein the selection unit is further determined as a predetermined rule by brightness or saturation of a region extracted by the extraction unit.
[A5]
The selection means is further defined as a predetermined rule by one or more of a variance value of saturation of the region extracted by the extraction means, a position in the image, and a size of the outer periphery. The image processing apparatus according to [A3] or [A4].
[A6]
The measurement means has at least one of a horizontal direction, a vertical direction, an oblique direction, and an elliptical shape as a scanning direction for measuring a pixel value. Any one of [A1] to [A5] The image processing apparatus described.
[A7]
The image processing apparatus according to any one of [A1] to [A6], wherein the estimation unit estimates a function that passes through a luminance value of a pixel in the region selected by the selection unit.
[A8]
The removing means has a plurality of scanning directions by the measuring means, and a fog value at a position where the scanning directions intersect is a value obtained by a plurality of functions estimated by the estimating means for the scanning. The image processing device according to [A7], wherein the image processing device is calculated on the basis of [A7].
[A9]
Computer
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value of the pixels in the area selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the pixel value measured by the measuring means;
An image processing program for functioning as a removing unit for removing fog from an image received by the receiving unit based on the function estimated by the estimating unit.

[B1]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a luminance value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
An image processing apparatus comprising: a correction unit that corrects the brightness value measured by the measurement unit based on the fog amount estimated by the estimation unit and the correction target value determined by the determination unit.
[B2]
The correction means uses a difference between the fog amount estimated by the estimation means and the correction target value determined by the determination means as a basic correction amount, and uses the basic correction amount as a luminance value measured by the measurement means and the fogging The image processing apparatus according to [B1], wherein the brightness value is corrected based on the changed basic correction amount by changing a coefficient based on a difference from the amount.
[B3]
The image processing according to [B2], wherein the correction unit uses a coefficient that decreases the basic correction amount when a difference between the luminance value measured by the measurement unit and the fogging amount increases. apparatus.
[B4]
The correction means does not correct the brightness value when the difference between the brightness value measured by the measurement means and the fogging amount is greater than or equal to a predetermined threshold value [B2] or [ B3].
[B5]
The correction means changes the basic correction amount according to a difference between a luminance value measured by the measurement means and the fogging amount and a coefficient based on the basic correction amount. [B2] to [B4] The image processing apparatus according to any one of the above.
[B6]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fogging amount estimated by the estimation means and the correction target value determined by the determination means;
The image processing apparatus according to any one of [B1] to [B5], wherein the correction unit performs correction when the determination unit determines that correction is performed.
[B7]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fog amounts estimated by the estimation means in each pixel in the image received by the reception means;
The image processing apparatus according to any one of [B1] to [B5], wherein the correction unit performs correction when the determination unit determines that correction is performed.
[B8]
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value of the pixels in the area selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the pixel value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A correction unit that calculates the fog amount based on the function estimated by the estimation unit, and corrects the luminance value measured by the measurement unit based on the fog amount and the correction target value determined by the determination unit. An image processing apparatus comprising:
[B9]
A fog amount is calculated based on the function estimated by the estimating means, and it is determined whether or not to perform correction by the correcting means based on a difference between the fog amount and a correction target value determined by the determining means. A judgment means for
The image processing apparatus according to [B8], wherein the correction unit performs correction when it is determined by the determination unit that correction is performed.
[B10]
Whether to calculate the fog amount based on the function estimated by the estimation unit, and to perform correction by the correction unit based on the difference between the fog amounts at each pixel in the image received by the reception unit Further comprising a judging means for judging
The image processing apparatus according to [B8], wherein the correction unit performs correction when it is determined by the determination unit that correction is performed.
[B11]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a luminance value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
An image processing program for functioning as a correction unit that corrects the luminance value measured by the measurement unit based on the fog amount estimated by the estimation unit and the correction target value determined by the determination unit.
[B12]
Computer
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value of the pixels in the area selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the pixel value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A correction unit that calculates the fog amount based on the function estimated by the estimation unit, and corrects the luminance value measured by the measurement unit based on the fog amount and the correction target value determined by the determination unit. Image processing program to make it function.

[C1]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means; An image processing apparatus comprising:
[C2]
The image processing apparatus according to [C1], wherein the correction unit corrects the saturation value to be reduced as the saturation value measured by the measurement unit becomes closer to an achromatic color.
[C3]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fogging amount estimated by the estimation means and the correction target value determined by the determination means;
The image processing apparatus according to [C1] or [C2], wherein the correction unit performs correction when the determination unit determines to perform correction.
[C4]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fog amounts estimated by the estimation means in each pixel in the image received by the reception means;
The image processing apparatus according to [C1] or [C2], wherein the correction unit performs correction when the determination unit determines to perform correction.
[C5]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
Image processing comprising: second correction means for correcting the brightness value measured by the measurement means based on the fog amount estimated by the estimation means and the correction target value determined by the determination means. apparatus.
[C6]
Judgment whether or not to perform correction by the first correction unit and the second correction unit based on the difference between the fog amount estimated by the estimation unit and the correction target value determined by the determination unit Further comprising means,
The image processing apparatus according to [C5], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction.
[C7]
Based on the difference between the fog amounts estimated by the estimation unit in each pixel in the image received by the reception unit, it is determined whether or not the correction by the first correction unit and the second correction unit is to be performed. A judgment means for
The image processing apparatus according to [C5], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction.
[C8]
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. An image processing apparatus comprising a correction unit.
[C9]
A fog amount is calculated based on the function estimated by the estimation means, and the first correction means and the second correction are calculated based on a difference between the fog amount and a correction target value determined by the determination means. A judgment means for judging whether or not to perform correction by means;
The image processing apparatus according to [C8], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction.
[C10]
A fog amount is calculated based on the function estimated by the estimation unit, and the first correction unit and the second correction unit are calculated based on a difference between the fog amounts at each pixel in the image received by the reception unit. A judgment means for judging whether or not correction by the correction means is performed,
The image processing apparatus according to [C8], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction.
[C11]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means Image processing program to make it function.
[C12]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
An image processing program for functioning as a second correction unit that corrects the brightness value measured by the measurement unit based on the fog amount estimated by the estimation unit and the correction target value determined by the determination unit.
[C13]
Computer
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. An image processing program for functioning as a correction means.

[D1]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means; Equipped,
The estimation means includes
Function selection means for calculating the information amount of a plurality of functions for estimating the fog amount for each scanning line in the image received by the receiving means, and selecting a function to be applied for each scanning line based on the information amount When,
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing apparatus comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[D2]
The function selection unit excludes a function that should not be selected based on a distribution of luminance values of pixels for each scanning line, and selects a function from the remaining functions. [D1] Image processing device.
[D3]
The correction unit corrects the saturation value so that the amount of correction of the saturation value decreases as the saturation value measured by the measurement unit becomes closer to an achromatic color. [D1] or [D2] Image processing device.
[D4]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fogging amount estimated by the estimation means and the correction target value determined by the determination means;
The image processing apparatus according to any one of [D1] to [D3], wherein the correction unit performs correction when the determination unit determines that correction is performed.
[D5]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fog amounts estimated by the estimation means in each pixel in the image received by the reception means;
The image processing apparatus according to any one of [D1] to [D3], wherein the correction unit performs correction when the determination unit determines that correction is performed.
[D6]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
Second correction means for correcting the brightness value measured by the measurement means based on the fog amount estimated by the estimation means and the correction target value determined by the determination means;
The estimation means includes
Function selection means for calculating the information amount of a plurality of functions for estimating the fog amount for each scanning line in the image received by the receiving means, and selecting a function to be applied for each scanning line based on the information amount When,
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing apparatus comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[D7]
The function selecting unit excludes a function that should not be selected based on a distribution of luminance values of pixels for each scanning line, and selects a function from the remaining functions. [D6] Image processing device.
[D8]
Judgment whether or not to perform correction by the first correction unit and the second correction unit based on the difference between the fog amount estimated by the estimation unit and the correction target value determined by the determination unit Further comprising means,
The image processing apparatus according to [D6] or [D7], wherein the first correction unit and the second correction unit perform correction when it is determined that the correction is performed by the determination unit. .
[D9]
Based on the difference between the fog amounts estimated by the estimation unit in each pixel in the image received by the reception unit, it is determined whether or not the correction by the first correction unit and the second correction unit is to be performed. A judgment means for
The image processing apparatus according to [D6] or [D7], wherein the first correction unit and the second correction unit perform correction when it is determined that the correction is performed by the determination unit. .
[D10]
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. A correction means,
The estimation means includes
Function selection means for calculating the information amount of a plurality of functions for estimating the fog amount for each scanning line in the image received by the receiving means, and selecting a function to be applied for each scanning line based on the information amount When,
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing apparatus comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[D11]
The function selecting unit excludes a function that should not be selected based on a distribution of luminance values of pixels for each scanning line, and selects a function from the remaining functions. [D10] Image processing device.
[D12]
A fog amount is calculated based on the function estimated by the estimation means, and the first correction means and the second correction are calculated based on a difference between the fog amount and a correction target value determined by the determination means. A judgment means for judging whether or not to perform correction by means;
The image processing apparatus according to [D10] or [D11], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction. .
[D13]
A fog amount is calculated based on the function estimated by the estimation unit, and the first correction unit and the second correction unit are calculated based on a difference between the fog amounts at each pixel in the image received by the reception unit. A judgment means for judging whether or not correction by the correction means is performed,
The image processing apparatus according to [D10] or [D11], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction. .
[D14]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means Make it work
The estimation means includes
Function selection means for calculating the information amount of a plurality of functions for estimating the fog amount for each scanning line in the image received by the receiving means, and selecting a function to be applied for each scanning line based on the information amount When,
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing program comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[D15]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
Based on the fog amount estimated by the estimation means and the correction target value determined by the determination means, function as a second correction means for correcting the luminance value measured by the measurement means,
The estimation means includes
Function selection means for calculating the information amount of a plurality of functions for estimating the fog amount for each scanning line in the image received by the receiving means, and selecting a function to be applied for each scanning line based on the information amount When,
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing program comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[D16]
Computer
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. Function as a correction means,
The estimation means includes
Function selection means for calculating the information amount of a plurality of functions for estimating the fog amount for each scanning line in the image received by the receiving means, and selecting a function to be applied for each scanning line based on the information amount When,
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing program comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.

[E1]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means; Equipped,
The estimation means includes
Calculating means for calculating a difference with an adjacent pixel for each scanning line in the image received by the receiving means;
A plurality of functions to be applied to the pixel difference calculated by the calculating means, and calculating an information amount of the plurality of functions for estimating a fog amount for each scanning line in the image received by the receiving means; Function selection means for selecting a function to be applied to each scanning line based on the amount of information;
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing apparatus comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[E2]
The function selecting unit excludes a function that should not be selected based on a distribution of luminance values of pixels for each scanning line, and selects a function from the remaining functions. [E1] Image processing device.
[E3]
The correction unit corrects the correction so that the amount of correction of the saturation value decreases as the saturation value measured by the measurement unit becomes closer to an achromatic color, according to [E1] or [E2] Image processing device.
[E4]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fogging amount estimated by the estimation means and the correction target value determined by the determination means;
The image processing apparatus according to any one of [E1] to [E3], wherein the correction unit performs correction when the determination unit determines that correction is performed.
[E5]
A judgment means for judging whether to perform correction by the correction means based on a difference between the fog amounts estimated by the estimation means in each pixel in the image received by the reception means;
The image processing apparatus according to any one of [E1] to [E3], wherein the correction unit performs correction when the determination unit determines that correction is performed.
[E6]
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
Second correction means for correcting the brightness value measured by the measurement means based on the fog amount estimated by the estimation means and the correction target value determined by the determination means;
The estimation means includes
Calculating means for calculating a difference with an adjacent pixel for each scanning line in the image received by the receiving means;
A plurality of functions to be applied to the pixel difference calculated by the calculating means, and calculating an information amount of the plurality of functions for estimating a fog amount for each scanning line in the image received by the receiving means; Function selection means for selecting a function to be applied to each scanning line based on the amount of information;
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing apparatus comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[E7]
The function selecting unit excludes a function that should not be selected based on a distribution of luminance values of pixels for each scanning line, and selects a function from the remaining functions. [E6] Image processing device.
[E8]
Judgment whether or not to perform correction by the first correction unit and the second correction unit based on the difference between the fog amount estimated by the estimation unit and the correction target value determined by the determination unit Further comprising means,
The image processing apparatus according to [E6] or [E7], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction. .
[E9]
Based on the difference between the fog amounts estimated by the estimation unit in each pixel in the image received by the reception unit, it is determined whether or not the correction by the first correction unit and the second correction unit is to be performed. A judgment means for
The image processing apparatus according to [E6] or [E7], wherein the first correction unit and the second correction unit perform correction when the determination unit determines to perform correction. .
[E10]
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. A correction means,
The estimation means includes
Calculating means for calculating a difference with an adjacent pixel for each scanning line in the image received by the receiving means;
A plurality of functions to be applied to the pixel difference calculated by the calculating means, and calculating an information amount of the plurality of functions for estimating a fog amount for each scanning line in the image received by the receiving means; Function selection means for selecting a function to be applied to each scanning line based on the amount of information;
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing apparatus comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[E11]
The calculation means, the function selection means, the function set selection means, and the fog amount estimation means perform processing for each area extracted by the extraction means,
The estimation means includes
An area constant estimating means for calculating an integral function of the function selected by the function set selecting means and estimating a constant term of the integral function;
The fog amount estimation unit estimates the constant term for each region estimated by the region constant estimation unit as a constant term that is a sum of a constant of the fog amount of the entire region and a specific amount of the background color of the region. The image processing apparatus according to [E10], wherein
[E12]
[E10] or [E11], wherein the function selecting means excludes a function that should not be selected based on a distribution of luminance values of pixels for each scanning line, and selects a function from the remaining functions. ] The image processing apparatus as described in.
[E13]
A fog amount is calculated based on the function estimated by the estimation means, and the first correction means and the second correction are calculated based on a difference between the fog amount and a correction target value determined by the determination means. A judgment means for judging whether or not to perform correction by means;
Any one of [E10] to [E12] is characterized in that the first correction unit and the second correction unit perform correction when the determination unit determines that the correction is performed. The image processing apparatus described.
[E14]
A fog amount is calculated based on the function estimated by the estimation unit, and the first correction unit and the second correction unit are calculated based on a difference between the fog amounts at each pixel in the image received by the reception unit. A judgment means for judging whether or not correction by the correction means is performed,
Any one of [E10] to [E12] is characterized in that the first correction unit and the second correction unit perform correction when the determination unit determines that the correction is performed. The image processing apparatus described.
[E15]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means Make it work
The estimation means includes
Calculating means for calculating a difference with an adjacent pixel for each scanning line in the image received by the receiving means;
A plurality of functions to be applied to the pixel difference calculated by the calculating means, and calculating an information amount of the plurality of functions for estimating a fog amount for each scanning line in the image received by the receiving means; Function selection means for selecting a function to be applied to each scanning line based on the amount of information;
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing program comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[E16]
Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
Based on the fog amount estimated by the estimation means and the correction target value determined by the determination means, function as a second correction means for correcting the luminance value measured by the measurement means,
The estimation means includes
Calculating means for calculating a difference with an adjacent pixel for each scanning line in the image received by the receiving means;
A plurality of functions to be applied to the pixel difference calculated by the calculating means, and calculating an information amount of the plurality of functions for estimating a fog amount for each scanning line in the image received by the receiving means; Function selection means for selecting a function to be applied to each scanning line based on the amount of information;
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing program comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.
[E17]
Computer
Receiving means for receiving images;
Extracting means for extracting a region from the image received by the receiving means;
Selecting means for selecting the region extracted by the extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. Function as a correction means,
The estimation means includes
Calculating means for calculating a difference with an adjacent pixel for each scanning line in the image received by the receiving means;
A plurality of functions to be applied to the pixel difference calculated by the calculating means, and calculating an information amount of the plurality of functions for estimating a fog amount for each scanning line in the image received by the receiving means; Function selection means for selecting a function to be applied to each scanning line based on the amount of information;
A function set selection unit that selects a combination of functions based on the information amount of the function for each scanning line selected by the function selection unit and a difference norm between a plurality of functions;
An image processing program comprising: a fog amount estimation unit that estimates a fog amount by applying a combination of functions selected by the function group selection unit for each scanning line.

DESCRIPTION OF SYMBOLS 110 ... Image reception module 120 ... Fog amount estimation module 130 ... Pixel value measurement module 140 ... Correction target value determination module 145 ... Saturation correction availability determination module 150 ... Saturation correction module 160 ... Output module 910 ... Correction module 950 ... Luminance correction Module 1445 ... Correctability determination module 1620 ... Non-edge region extraction module 1630 ... Inspection region selection module 1640 ... Pixel value measurement module 1650 ... Pixel value distribution estimation module 1910 ... Sobel filter module 1920 ... Binarization module 1930 ... Monochrome inversion module 1940 ... Labeling module 1950 ... Small size removal module 3110 ... Quasi-flat region extraction module 3112 ... Edge enhancement module 3114 ... Binarization module 3116 ... Inversion module 3118 ... Labeling module 3120 ... Sampling area selection module 3122 ... Color analysis module 3124 ... Discrimination function module 3126 ... Area selection module 3130 ... Fog amount estimation module 3132 ... Line sampling module 3134 ... Multi-model function estimation module 3136 ... Plane Luminance estimation module 3140 ... Luminance fog correction module 3142 ... Coordinate correction amount calculation module 3144 ... Luminance correction module 3146 ... Saturation correction module 3530 ... Fog amount estimation module 3532 ... Life sampling module 3534 ... Multi-model global differential function estimation module 3536 ... Local area constant estimation module 3538 ... Planar luminance estimation module 4310 ... Fog presence / absence processing module 4312 ... Document area extraction module 4314 ... Coverage determination module 4316 ... Control module

Claims (16)

Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Extraction means for extracting an area of a document image taken from an image received by the reception means;
A determination means for determining whether or not the fog is occurring only in the area by comparing the image inside the area extracted by the extraction means and the image outside the area;
Control means for controlling so as not to remove the amount of fog when the judgment means judges that the fog is occurring only in the region;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means; An image processing apparatus comprising: The region extracted by the extracting means is a quadrangle,
The judging means compares the outside and inside images of the sides of the quadrangle, and the inside image is within a predetermined distance from the sides of the quadrangle, and the amount of fogging is estimated by the estimating means. The image processing apparatus according to claim 1, wherein the image processing apparatus includes an estimated area. 3. The image processing according to claim 1, wherein the correction unit corrects the saturation value to be reduced as the saturation value measured by the measurement unit becomes closer to an achromatic color. apparatus. A judgment means for judging whether to perform correction by the correction means based on a difference between the fogging amount estimated by the estimation means and the correction target value determined by the determination means;
The image processing apparatus according to any one of claims 1 to 3, wherein the correction unit performs correction when the determination unit determines that the correction is performed. A judgment means for judging whether to perform correction by the correction means based on a difference between the fog amounts estimated by the estimation means in each pixel in the image received by the reception means;
The image processing apparatus according to any one of claims 1 to 3, wherein the correction unit performs correction when the determination unit determines that the correction is performed. Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Extraction means for extracting an area of a document image taken from an image received by the reception means;
A determination means for determining whether or not the fog is occurring only in the area by comparing the image inside the area extracted by the extraction means and the image outside the area;
Control means for controlling so as not to remove the amount of fog when the judgment means judges that the fog is occurring only in the region;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
Image processing comprising: second correction means for correcting the brightness value measured by the measurement means based on the fog amount estimated by the estimation means and the correction target value determined by the determination means. apparatus. The region extracted by the extracting means is a quadrangle,
The judging means compares the outside and inside images of the side of the quadrangle, the inside image is within a predetermined distance from the side of the quadrangle, and the amount of fogging by the estimating means The image processing apparatus according to claim 6, wherein the image processing apparatus includes an estimated area. Judgment whether or not to perform correction by the first correction unit and the second correction unit based on the difference between the fog amount estimated by the estimation unit and the correction target value determined by the determination unit Further comprising means,
The image processing apparatus according to claim 6, wherein the first correction unit and the second correction unit perform correction when it is determined that the correction is performed by the determination unit. Based on the difference between the fog amounts estimated by the estimation unit in each pixel in the image received by the reception unit, it is determined whether or not the correction by the first correction unit and the second correction unit is to be performed. A judgment means for
The image processing apparatus according to claim 6, wherein the first correction unit and the second correction unit perform correction when it is determined that the correction is performed by the determination unit. Receiving means for receiving images;
First extraction means for extracting a region from the image received by the reception means;
Selecting means for selecting an area extracted by the first extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Second extraction means for extracting an area of a document image taken from an image received by the reception means;
A determination means for determining whether or not the fog is generated only in the area by comparing the inside and outside of the area extracted by the second extraction means;
Control means for controlling so as not to remove the amount of fog when the judgment means judges that the fog is occurring only in the region;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. An image processing apparatus comprising a correction unit. The area extracted by the second extraction means is a quadrangle,
The judging means compares the outside and inside images of the sides of the quadrangle, and the inside image is within a predetermined distance from the sides of the quadrangle, and the amount of fogging is estimated by the estimating means. The image processing apparatus according to claim 10, wherein the image processing apparatus includes an estimated area. A fog amount is calculated based on the function estimated by the estimation means, and the first correction means and the second correction are calculated based on a difference between the fog amount and a correction target value determined by the determination means. A judgment means for judging whether or not to perform correction by means;
The image processing apparatus according to claim 10, wherein the first correction unit and the second correction unit perform correction when the determination unit determines that correction is performed. A fog amount is calculated based on the function estimated by the estimation unit, and the first correction unit and the second correction unit are calculated based on a difference between the fog amounts at each pixel in the image received by the reception unit. A judgment means for judging whether or not correction by the correction means is performed,
The image processing apparatus according to claim 10, wherein the first correction unit and the second correction unit perform correction when the determination unit determines that correction is performed. Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Extraction means for extracting an area of a document image taken from an image received by the reception means;
A determination means for determining whether or not the fog is occurring only in the area by comparing the image inside the area extracted by the extraction means and the image outside the area;
Control means for controlling so as not to remove the amount of fog when the judgment means judges that the fog is occurring only in the region;
Measuring means for measuring a saturation value of a pixel in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
Correction means for correcting the saturation value based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means An image processing program characterized by causing it to function. Computer
Receiving means for receiving images;
Estimating means for estimating a fog amount that is a difference between a luminance value of an original pixel and a luminance value of a pixel of the image based on the image received by the receiving means;
Extraction means for extracting an area of a document image taken from an image received by the reception means;
A determination means for determining whether or not the fog is occurring only in the area by comparing the image inside the area extracted by the extraction means and the image outside the area;
Control means for controlling so as not to remove the amount of fog when the judgment means judges that the fog is occurring only in the region;
Measuring means for measuring the luminance value and saturation value of the pixels in the image received by the receiving means;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
The first saturation value is corrected based on the fog amount estimated by the estimation means, the correction target value determined by the determination means, and the distance from the achromatic color of the saturation value measured by the measurement means. Correction means;
An image processing functioning as a second correction unit that corrects the luminance value measured by the measurement unit based on the fog amount estimated by the estimation unit and the correction target value determined by the determination unit. program. Computer
Receiving means for receiving images;
First extraction means for extracting a region from the image received by the reception means;
Selecting means for selecting an area extracted by the first extracting means based on a predetermined rule;
Measuring means for measuring the luminance value and saturation value of the pixels in the region selected by the selecting means;
Estimating means for estimating a function indicating the degree of fogging in the image received by the receiving means from the luminance value measured by the measuring means;
Second extraction means for extracting an area of a document image taken from an image received by the reception means;
A determination means for determining whether or not the fog is generated only in the area by comparing the inside and outside of the area extracted by the second extraction means;
Control means for controlling so as not to remove the amount of fog when the judgment means judges that the fog is occurring only in the region;
Determining means for determining a correction target value of the luminance value in the background in the image received by the receiving means;
A fog amount is calculated based on the function estimated by the estimating means, and based on a distance from the achromatic color of the fog amount, a correction target value determined by the determining means, and a saturation value measured by the measuring means. First correcting means for correcting the saturation value;
A fog amount is calculated based on the function estimated by the estimating means, and a brightness value measured by the measuring means is corrected based on the fog amount and a correction target value determined by the determining means. An image processing program which functions as a correction means.

Images