JP3515019B2 - Image correction method and apparatus, recording medium recording the method, image photographing apparatus incorporating the apparatus, and image display apparatus incorporating the apparatus - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-gradation image such as a color image or a monochrome image obtained by digitizing a scene image, in particular, an aerial photograph, by increasing the brightness and contrast of a shadow area. The present invention relates to an image correction method and device for improving image quality, and an image capturing device and an image display device incorporating the device.

[0002]

2. Description of the Related Art As a conventional technique for easily correcting the shadow portion of a digitized aerial photograph, information on a plurality of images of the same object taken at the same place at different dates and times is integrated, and There has been a technique for synthesizing an image without a shadow region by integrating only regions (Japanese Patent Laid-Open No. 10-269347).
issue).

In addition to aerial photography, as a technique for making the shadow portion of a general image easy to see, the brightness of the image is converted by a logarithmic function to increase the dynamic range of the dark portion, and then the image There was a technology for applying a high-frequency emphasis filter that emphasizes high-frequency components (TG Stockh
am, Jr. , Image Processingin
the Context of a visual
model, Proc. IEEE60, 1972, 82
8-842).

[0004]

The first prior art technique has a problem that the shadow portion cannot be corrected from one image.

In the second conventional technique, the dynamic range of the dark portion is increased because the entire image is first transformed by the logarithmic function, but conversely, the bright portion, that is, the sun-exposed portion outside the shadow is increased. The dynamic range has dropped,
There was a problem that the image quality deteriorates. Also, when dealing with color aerial photographs, the sunlit area receives the white light of the sun and the scattered light from the blue sky, while the shadow area receives only the scattered light from the blue sky, so the shadow area Is bluish compared to the Hyuga area. This is not noticeable when the brightness of the shadow area is low, but when this technology raises the brightness, there is a problem that the shadow area becomes bluish and looks unnatural. .

An object of the present invention is to improve the image quality by correcting a shadow area with only one image, and it is possible to prevent the sunlit area from adversely affecting the image quality. It is an object of the present invention to provide a technique capable of preventing a bright blue color when a shadow region is corrected when a color image is handled.

[0007]

The above-mentioned problems of the present invention are as follows.
It is solved by the means listed below.

As one of the means, in an image correction method for converting a multi-tone original image to improve legibility, an edge is extracted from the original image and the original image is smoothed to generate a smoothed image, An edge-preserving smoothed image creating step of adding the edge and the smoothed image to create an edge-preserving smoothed image, and a domain of brightness for the edge-preserving smoothed image is classified into a plurality of classes, An edge-preserving smoothed image brightness gradation conversion step of applying brightness gradation conversion for each class, a high frequency component extraction step of extracting high frequency components of a single or a plurality of frequencies of the original image, and a high frequency component of each extracted frequency. On the other hand, using the result of classifying the edge-preserving smoothed image into a class, a high-frequency component conversion step of performing a conversion for amplifying a high-frequency component for each class, and the luminance gradation conversion And an image synthesizing step of generating an image by integrating di preserving smoothing image conversion results of the high frequency component conversion step, an image correction method characterized in that it has.

Alternatively, in the above image correction method,
The image correction method is characterized in that in the image synthesizing step, the edge-preserving smoothed image subjected to the luminance gradation conversion and the individual element values of the conversion result in the high frequency component converting step are added to integrate the two.

Alternatively, in the above image correction method,
In the edge-preserving smoothed image creating step, the edge-preserving smoothed image is processed by a method of performing a process of combining the morphology opening and closing using a structuring element having a size corresponding to the smoothing frequency with respect to the original image. It is an image correction method characterized by creating.

Alternatively, in the above image correction method,
In the high-frequency component extraction step, for the high-frequency component of the highest frequency, by obtaining the difference between the original image and the image in which the highest-frequency component of the highest frequency is deleted from the original image, the high-frequency component of other frequencies is obtained. On the other hand, by extracting the difference between an image in which all high-frequency components higher than the frequency are deleted from the original image and an image in which the high-frequency components of the frequency are deleted from the image, the high-frequency components of individual frequencies are extracted. Is an image correction method.

Alternatively, in the above image correction method,
In the high frequency component extraction step, as a method of deleting high frequency components of individual frequencies from the image, the morphological op is used by using a structuring element having a size corresponding to the frequency for the image.
This is an image correction method characterized by using a method of performing a process in which ening and closing are combined.

Alternatively, in the above image correction method,
In the edge-preserving smoothed image brightness gradation conversion step, each pixel of the edge-preserving smoothed image is classified into at least two classes of a dark pixel class and a bright pixel class according to the magnitude of the brightness value. This is an image correction method characterized in that brightness gradation conversion is added by separate functions.

Alternatively, in the above image correction method,
In the edge-preserving smoothed image brightness gradation conversion step, as a method of determining the parameter value of the brightness gradation conversion function, the parameter value is changed within a predetermined domain and the edge when each parameter value is used An evaluation value represented by a function whose variable is a total value of luminance change values of pixels belonging to the bright pixel class in the stored smoothed image and a total value of luminance change values of pixels belonging to the dark pixel class is stored. The image correction method is characterized by using a method of calculating and using a parameter value for which a maximum evaluation value is obtained.

Alternatively, in the above image correction method,
In the high-frequency component conversion step, conversion is performed using a function, and the function is changed according to the result of classifying the corresponding pixels of the edge-preserved smoothed image into classes in the edge-preserved smoothed image luminance gradation conversion step. The image correction method is as follows.

Alternatively, in an image correction method for converting a color multi-gradation original image to improve legibility, the brightness is extracted from the color display information of each pixel of the original image to create a monochrome gray-scale image. A grayscale image creating step, a grayscale image correcting step of correcting the monochrome grayscale image created in the grayscale image creating step by the image correction method, and a corrected grayscale image corrected in the grayscale image correcting step are formed. For each pixel, the saturation and the hue are extracted from the information of the color display of the pixel of the corresponding original image, and combined with the luminance of the pixel in the corrected grayscale image to obtain the corrected display color information. After the color correction stage to create a color image later,
An image correction method characterized by having.

Alternatively, in an image correction method for converting a color multi-gradation original image to improve legibility, the brightness is extracted from the color display information of each pixel of the original image to create a monochrome gray-scale image. A grayscale image creating step, a grayscale image correcting step of correcting the monochrome grayscale image created in the grayscale image creating step by the image correction method, and a corrected grayscale image corrected in the grayscale image correcting step are formed. For each pixel, it is checked whether or not it is determined to be a dark pixel class in the processing of the edge-preserving smoothed image brightness gradation conversion step in the image correction method used in the grayscale image correction step, If judged,
After the saturation and hue are extracted from the information of the color display of the pixels of the corresponding original image, the extracted saturation is reduced, and then the extracted hue and the luminance of the pixel in the post-correction grayscale image are combined and corrected. If it is determined that the pixel is a bright pixel class, the saturation and hue are extracted from the information of the color display of the pixel of the corresponding original image and combined with the luminance of the pixel in the corrected grayscale image. And a color correction step of creating a color image after correction by using the information as the display color after correction.

Alternatively, in an image correction apparatus for converting a multi-gradation original image to improve legibility, an edge is extracted from the original image and the original image is smoothed to generate a smoothed image, An edge-preserving smoothed image creating means for adding the smoothed images to create an edge-preserving smoothed image, and a domain of brightness for the edge-preserving smoothed image is classified into a plurality of classes, and brightness is classified for each class. Edge-preserving smoothed image brightness gradation conversion means for applying gradation conversion,
High frequency component extraction means for extracting high frequency components of a single or a plurality of frequencies of the original image, and for each of the extracted high frequency components of each frequency, each class using the result of classifying into the class of the edge-preserving smoothed image Separately, high frequency component conversion means for performing conversion for amplifying high frequency components, and image synthesizing means for creating an image by integrating the edge preserving smoothed image subjected to the luminance gradation conversion and the conversion result of the high frequency component conversion means. The image correction device is characterized in that

Alternatively, in the above image correction device,
The image synthesizing unit integrates the edge-preserving smoothed image subjected to the luminance gradation conversion and the individual element values of the conversion result of the high-frequency component converting unit to integrate them. is there.

Alternatively, in the above image correction device,
The edge-preserving smoothed image creating means uses a means for performing a process of combining morphology opening and closing using a structuring element having a size corresponding to the smoothing frequency with respect to the original image to generate the edge-preserving smoothed image. An image correction device characterized by being created.

Alternatively, in the above image correction device,
The high-frequency component extraction means obtains the difference between the original image and the image obtained by deleting the high-frequency component of the highest frequency from the original image, for the high-frequency component of the highest frequency, to obtain the high-frequency component of other frequencies. On the other hand, by extracting the difference between an image in which all high-frequency components higher than the frequency are deleted from the original image and an image in which the high-frequency components of the frequency are deleted from the image, high-frequency components of individual frequencies are extracted. Is an image correction device.

Alternatively, in the above image correction device,
The high-frequency component extraction means is a means for deleting high-frequency components of individual frequencies from an image, and uses a structuring element having a size corresponding to the frequency for the image to operate the morphology.
The image correction apparatus is characterized by using a means for performing a process that combines a "ning" and a "closing".

Alternatively, in the above image correction device,
The edge-preserving smoothed image luminance gradation converting means classifies each pixel of the edge-preserving smoothed image into at least two classes, that is, a dark pixel class and a bright pixel class, according to the magnitude of the brightness value. The image correction device is characterized in that the brightness gradation conversion is added by different functions.

Alternatively, in the above image correction device,
The edge-preserving smoothed image brightness gradation conversion means is a means for determining the parameter value of the function of the brightness gradation conversion, in which the parameter value is changed within a predetermined domain,
A variable is a total value of luminance change values of pixels belonging to a bright pixel class in the edge-preserving smoothed image when each parameter value is used, and a total value of luminance change values of pixels belonging to a dark pixel class. The image correction apparatus is characterized by using means for calculating an evaluation value represented by a function and adopting a parameter value for which the maximum evaluation value is obtained.

Alternatively, in the above image correction device,
The high-frequency component conversion means performs conversion using a function, and changes the function according to the result of classifying the corresponding pixels of the edge-preserving smoothed image into the class in the edge-preserving smoothed image luminance gradation conversion means. The image correction device is characterized.

Alternatively, in an image correction apparatus for converting a multi-color original image of color to improve the readability, the brightness is extracted from the color display information of each pixel of the original image to create a monochrome grayscale image. A grayscale image creating means, a grayscale image correcting means for correcting the monochrome grayscale image created by the grayscale image creating means by the image correction device, and a corrected grayscale image corrected by the grayscale image correcting means. For each pixel, the saturation and the hue are extracted from the information of the color display of the pixel of the corresponding original image, and combined with the luminance of the pixel in the corrected grayscale image to obtain the corrected display color information. Color correction means to create the later color image,
An image correction apparatus characterized by being provided.

Alternatively, in an image correction apparatus for converting an original image of color multi-gradation to improve the legibility, the brightness is extracted from the color display information of each pixel of the original image to create a monochrome grayscale image. A grayscale image creating means, a grayscale image correcting means for correcting the monochrome grayscale image created by the grayscale image creating means by the image correction device, and a corrected grayscale image corrected by the grayscale image correcting means. For each pixel, it is checked whether or not it has been determined to be a dark pixel class in the processing of the edge-preserving smoothed image luminance gradation conversion means in the image correction device used in the grayscale image correction means. If judged,
After the saturation and hue are extracted from the information of the color display of the pixels of the corresponding original image, the extracted saturation is reduced, and then the extracted hue and the luminance of the pixel in the post-correction grayscale image are combined and corrected. If it is determined that the pixel is a bright pixel class, the saturation and hue are extracted from the information of the color display of the pixel of the corresponding original image and combined with the luminance of the pixel in the corrected grayscale image. And a color correction unit that creates a corrected color image by using the corrected display color information.

Alternatively, in an image correction method for converting a multi-tone original image to improve legibility, an edge is extracted from the original image and the original image is smoothed to generate a smoothed image, An edge-preserving smoothed image creating step of adding the smoothed images to create an edge-preserving smoothed image; a high-frequency component extracting step of extracting high-frequency components of a single or a plurality of frequencies of the original image; A probability calculation step of calculating the probability that each pixel belongs to the sunlit area and the probability that the pixel belongs to the shadow area from the information of the pixel of the edge-preserving smoothed image corresponding to the pixel, and the edge-preserving smoothed image On the other hand, assuming that each pixel belongs to the shadow area, the brightness gradation conversion is performed and then output, and assuming that each pixel belongs to the sunlit area, the brightness gradation conversion may be performed and output. Edge preserving smoothing image luminance gradation conversion step for outputting a raw edge preserving smoothing image,
With respect to the high frequency components of each of the extracted frequencies, output by performing conversion for amplification assuming that they all belong to the shadow region,
And using the probability of belonging to each region calculated in the probability calculation step, and the high frequency component conversion step of performing the conversion to be amplified and outputting or outputting the high frequency component as it is, assuming that it belongs to the sunlit area, 2 for each hypothetical region output from the edge-preserving smoothed image luminance gradation conversion step
The image correction method is characterized by including an output of each type and an image composition step of integrating two types of output for each hypothetical region output from the high frequency component conversion step to create an image.

Alternatively, in an image correction method for converting a color multi-gradation original image to improve legibility, a saturation / hue extraction step of extracting saturation and hue from information of each pixel of the original image, An edge-preserving smoothed image creating step of creating an edge-preserving smoothing image by adding edges to the smoothed image to generate a smoothed image by smoothing the original image while extracting edges from the original image, The high frequency component extraction step of extracting high frequency components of a single or a plurality of frequencies of the original image, the probability that each pixel of the original image belongs to the sunlit area, and the probability that the pixel belongs to the shadowed area A probability calculation step of calculating from the pixel information of the edge-preserving smoothed image, and a conversion for reducing the saturation extracted from each pixel of the original image using the calculated probability of belonging to each region. Saturation conversion step to be applied, and for the edge-preserving smoothed image, it is assumed that each pixel belongs to a shadow area, and luminance gradation conversion is performed and output, and each pixel is assumed to belong to a sunlit area. Then, an edge-preserving smoothed image luminance gradation conversion step of outputting the luminance-preserving smoothed image as it is, or outputting the same as that of the edge-preserving smoothed image. A high-frequency component conversion stage that outputs by performing a conversion that amplifies assuming that it belongs to a region, and outputs by performing a conversion that performs amplification by assuming that it belongs to a sunlit region, or outputs a high-frequency component as it is, Using the probabilities belonging to each region calculated in the probability calculation step, the two types of output from the edge-preserving smoothed image luminance gradation conversion step and the two types of output from the high frequency component conversion step are integrated to generate an image. Image synthesis for creating an image by combining the information of the hue of the original image output from the saturation / hue extraction step and the information of the converted saturation output from the saturation conversion step And an image correction method.

Alternatively, in the above image correction method,
In the saturation conversion step, as a method of integrating using the probabilities belonging to each area calculated in the probability calculation step, with respect to the saturation extracted from each pixel of the original image, the corresponding pixel of the original image becomes a shadow area. Saturation reduction processing is performed assuming that the pixel belongs, and the result is multiplied by the probability that the pixel belongs to the shadow area to calculate a value, and the pixel belongs to the sun for the value and the original saturation. It is an image correction method characterized by adding a value multiplied by a probability.

Alternatively, in the above image correction method,
In the image synthesizing stage, the two types of outputs from the edge-preserving smoothed image luminance gradation converting stage and the two types of output from the high frequency component converting stage are integrated by using the probabilities belonging to each region calculated in the probability calculating stage. As a process for creating an image, the brightness gradation conversion is performed on the edge-preserving smoothed image brightness gradation converting step assuming that each pixel belongs to a shadow area. A value is calculated by multiplying the output by the probability that each pixel belongs to a shadow area, and from the edge-preserving smoothed image brightness gradation conversion step, each pixel is set to a sunlit area for the edge-preserving smoothed image. Assuming that the output is subjected to luminance gradation conversion or the output of the edge-preserving smoothed image as it is, a value is calculated by multiplying the probability that each pixel belongs to the sunlit area, and the value from the high frequency component conversion step is calculated. High for each frequency The output obtained by converting the wave components assuming that they all belong to the shadow area is multiplied by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the shadow area, and the high frequency is calculated. For the high-frequency components of each frequency from the component conversion stage, assuming that all the high-frequency components belong to the sunlit area, the output that has been transformed is multiplied by the probability that the corresponding edge-preserving smoothed image pixel belongs to the sunlit area. Is calculated, and the calculated four types of values are added, which is an image correction method.

Alternatively, in the above image correction method,
In the image synthesizing stage, the two types of outputs from the edge-preserving smoothed image luminance gradation converting stage and the two types of output from the high frequency component converting stage are integrated by using the probabilities belonging to each region calculated in the probability calculating stage. As a process for creating an image, the brightness gradation conversion is performed on the edge-preserving smoothed image brightness gradation converting step assuming that each pixel belongs to a shadow area. The output and the output from the high-frequency component conversion step, which has been converted on the assumption that all high-frequency components of each frequency belong to the shadow region, are added for each corresponding element, and the value is added. A value is calculated by multiplying the probability that the pixel of the edge-preserving smoothed image belongs to the shadow area, and each pixel is in the sun for the edge-preserving smoothed image from the edge-preserving smoothed image luminance gradation conversion step. Assuming it belongs to a region The output was subjected to the luminance gradation conversion or the output of the edge-preserving smoothed image as it was, and the conversion was performed assuming that all the high frequency components of each frequency from the high frequency component conversion stage belong to the sunlit area. The output or the output as it is is added for each corresponding element, and a value is calculated by multiplying the value by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the sunlit area. The image correction method is characterized by adding the value of

Alternatively, in the above image correction method,
In the probability calculation step, as a process of calculating the probability that each pixel of the original image belongs to the sunlit area, the probability that the information of the corresponding pixel in the edge-preserving smoothed image will occur from the sunlit area, and the value thereof, Multiply by the value input in advance, which is the ratio of the number of pixels in the average sunny area in the edge-preserving smoothed image, and calculate the probability that each pixel in the original image belongs to the shadow area. As a method, the probability that the information of the corresponding pixel in the edge-preserving smoothed image is generated from the shadow area is calculated, and that value and the pre-entered value, which is the average shadow of the edge-preserving smoothed image, are calculated. The image correction method is characterized by performing a process of multiplying by a ratio of the number of pixels in the area.

Alternatively, in the above image correction method,
Information for calculating the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the sunlit area, and the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the shadow area The image correction method is characterized by comprising a learning step of learning in advance information for

Alternatively, in the above image correction method,
It is an image correction method characterized by using a probability density, a value obtained by converting a probability, or a value obtained by converting a probability density, instead of the probability.

Alternatively, in the above image correction method,
In the probability calculation step, when using probability density instead of probability, or using a combination of probability density and transformation, the probability density calculation processing in which the pixel information in the edge-preserved smoothed image is generated from the sunlit area, and the edge preservation To calculate the probability density of the pixel information in the smoothed image generated from the shadow area, calculate the probability density using the probability density function of a one-dimensional or multidimensional normal distribution having the number of dimensions representing the pixel information. Is an image correction method.

Alternatively, in the above image correction method,
In the high-frequency component extraction step, as a process of extracting high-frequency components of individual frequencies, for the high-frequency component of the highest frequency, the difference between the original image and the image obtained by deleting the high-frequency component of the highest frequency from the original image. By extracting
For high frequency components of other frequencies, extraction is performed by obtaining the difference between an image in which all high frequency components higher than the frequency are deleted from the original image and an image in which the high frequency components of the frequency are deleted from the image. The image correction method is characterized by performing the processing.

Alternatively, in the above image correction method,
In the edge-preserving smoothed image creation stage, an image correction method characterized by using a method of performing a process of combining morphology opening and closing using a structuring element having a size corresponding to a smoothing frequency for the image. is there.

Alternatively, in the above image correction method,
In the high-frequency component extraction step, as a method of removing high-frequency components of individual frequencies from the image in the process of extracting high-frequency components of individual frequencies, morphology is used for the image by using a structural element having a size corresponding to the frequency. The op
This is an image correction method characterized by using a method of performing a process in which ening and closing are combined.

Alternatively, in the above image correction method,
In the edge-preserving smoothed image creation step, not only an image in which high-frequency components of each frequency are deleted is created from the image, but also an edge-unsmoothed smoothed image is created by performing smoothing that does not save edges in the original image. Then, the corresponding pixels of the edge-preserving smoothed image and the edge-non-preserving smoothed image are compared, and the luminance of the latter is converted so that the luminance of the latter matches the luminance of the former to obtain a new edge-preserving smoothed image. It is an image correction method characterized by using a method of outputting.

Alternatively, in an image correction apparatus for converting a multi-gradation original image to improve legibility, an edge is extracted from the original image and the original image is smoothed to generate a smoothed image. Edge-preserving smoothed image creating means for adding the smoothed images to create an edge-preserving smoothed image, high-frequency component extracting means for extracting high-frequency components of one or more frequencies of the original image, and the original image Probability calculating means for calculating the probability that each pixel belongs to the sunlit area and the probability that the pixel belongs to the shadow area from the information of the pixel of the edge-preserving smoothed image corresponding to the pixel, and the edge-preserving smoothed image On the other hand, assuming that each pixel belongs to the shadow area, the brightness gradation conversion is performed and then output, and assuming that each pixel belongs to the sunlit area, the brightness gradation conversion may be performed and output. Edge preserving smoothing image luminance gradation conversion means for outputting a raw edge preserving smoothing image,
With respect to the high frequency components of each of the extracted frequencies, output by performing conversion for amplification assuming that they all belong to the shadow region,
And using the high frequency component conversion means that outputs by performing conversion for amplification assuming that they all belong to the sunlit area or outputs the high frequency component as it is, and the probability of belonging to each area calculated by the probability calculation means, 2 for each hypothetical region output from the edge-preserving smoothed image luminance gradation converting means
The image correction apparatus is characterized by comprising: an output of a type and an image synthesizing unit that creates an image by integrating two types of output for each assumed region output from the high frequency component converting unit.

Alternatively, in an image correction device for converting an original image of color multi-gradation to improve legibility, a saturation / hue extraction means for extracting a saturation and a hue from information of each pixel of the original image, Edge-preserving smoothed image creating means for creating an edge-preserving smoothed image by extracting edges from the original image and smoothing the original image to generate a smoothed image, and adding the edge and the smoothed image, High frequency component extraction means for extracting high frequency components of a single frequency or a plurality of frequencies of the original image, the probability that each pixel of the original image belongs to the sunlit area and the probability that the pixel belongs to the shadowed area A probability calculating unit that calculates the pixel information of the edge-preserving smoothed image, and a conversion that reduces the saturation extracted from each pixel of the original image using the calculated probability of belonging to each region. Saturation conversion means for adding, and for the edge-preserving smoothed image, it is assumed that each pixel belongs to a shadow area, and luminance and brightness conversion is performed and output, and each pixel belongs to a sunlit area. Then, an edge-preserving smoothed image luminance gradation converting means for performing the luminance gradation conversion for output or outputting the edge-preserving smoothed image as it is, and for all the extracted high frequency components of each frequency, shadows A high-frequency component conversion unit that outputs by performing conversion for amplification assuming that it belongs to a region, and outputs by performing conversion that performs amplification by assuming that it all belongs to the sunlit region, or outputs the high-frequency component as it is, Using the probabilities belonging to the respective areas calculated by the probability calculating means, two kinds of outputs from the edge-preserving smoothed image luminance gradation converting means and two kinds of outputs from the high frequency component converting means are integrated to generate an image. Image synthesis for creating an image by combining the information of the hue of the original image output from the saturation / hue extraction means and the information of the converted saturation output from the saturation conversion means And an image correction device.

Alternatively, in the above image correction device,
The saturation conversion means is a means for integrating using the probabilities belonging to the respective areas calculated by the probability calculation means, with respect to the saturation extracted from each pixel of the original image, a pixel of the corresponding original image is in a shadow area. Saturation reduction processing is performed assuming that the pixel belongs, and the result is multiplied by the probability that the pixel belongs to the shadow area to calculate a value, and the pixel belongs to the sun for the value and the original saturation. The image correction device is characterized in that a value obtained by multiplying a probability is added.

Alternatively, in the above image correction device,
The image synthesizing means uses the probabilities belonging to the respective areas calculated by the probability calculating means, and outputs two kinds of outputs from the edge-preserving smoothed image luminance gradation converting means and two kinds of outputs from the high frequency component converting means.
As a process for creating an image by integrating outputs of various types, it is assumed that each pixel of the edge-preserving smoothed image brightness gradation conversion unit belongs to a shadow area, assuming that each pixel belongs to a shadow area. A value is calculated by multiplying the output obtained by the tone conversion by the probability that each pixel belongs to the shadow area, and each pixel is added to the edge-preserving smoothed image from the edge-preserving smoothed image luminance gradation converting means. Is calculated as a value obtained by multiplying the output obtained by performing the luminance gradation conversion or the output of the edge-preserving smoothed image as it is, by multiplying the probability that each pixel belongs to the sunny area, The high-frequency components of each frequency from the component conversion means are converted by assuming that they belong to the shadow area, and the output is multiplied by the probability that the corresponding edge-preserving smoothed image pixel belongs to the shadow area. Calculated the value The probability that the pixels of the corresponding edge-preserving smoothed image belong to the sunlit area with respect to the output obtained by performing the conversion on the assumption that all the high frequency components of each frequency from the frequency component conversion means belong to the sunlit area. The image correction apparatus is characterized by calculating a multiplied value and adding the calculated four types of values.

Alternatively, in the above image correction device,
The image synthesizing means uses the probabilities belonging to the respective areas calculated by the probability calculating means, and outputs two kinds of outputs from the edge-preserving smoothed image luminance gradation converting means and two kinds of outputs from the high frequency component converting means.
As a process for creating an image by integrating outputs of various types, it is assumed that each pixel of the edge-preserving smoothed image brightness gradation conversion unit belongs to a shadow area, assuming that each pixel belongs to a shadow area. The output that has been subjected to the key conversion and the output that has been converted assuming that all the high frequency components of each frequency from the high frequency component conversion means belong to the shadow region, and add for each corresponding element, A value is calculated by multiplying the value by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the shadow region, and the value is calculated for the edge-preserving smoothed image from the edge-preserving smoothed image luminance gradation conversion means. Assuming that each pixel belongs to the sunlit area, the output that has been subjected to the luminance gradation conversion or the output of the edge-preserving smoothed image as it is, and all the sunlit parts for the high frequency components of each frequency from the high frequency component conversion means. Belongs to the realm of Assuming that the converted output or the output as it is, is added for each corresponding element, and the value is multiplied by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the sunlit area. However, the image correction apparatus is characterized by adding the calculated four types of values.

Alternatively, in the above image correction device,
Probability calculation means, as a process of calculating the probability that each pixel of the original image belongs to the sunlit area, calculates the probability that the information of the corresponding pixel in the edge-preserving smoothed image will occur from the sunlit area, and its value, Multiply by the value input in advance, which is the ratio of the number of pixels in the average sunny area in the edge-preserving smoothed image, and calculate the probability that each pixel in the original image belongs to the shadow area. As a means, the probability of the information of the corresponding pixel in the edge-preserving smoothed image occurring from the shadow area is calculated, and that value and the pre-populated value of the average shadow of the edge-preserving smoothed image are calculated. The image correction device is characterized by performing a process of multiplying by a ratio of the number of pixels in the area.

Alternatively, in the above image correction device,
Information for calculating the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the sunlit area, and the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the shadow area The image correction apparatus is provided with a learning unit that learns information for the above in advance.

Alternatively, in the above image correction device,
An image correction device characterized by using a probability density, a value obtained by converting a probability, or a value obtained by converting a probability density instead of the probability.

Alternatively, in the above image correction device,
When the probability calculating means uses the probability density or the transformed probability density instead of the probability, the probability density calculating process in which the pixel information in the edge-preserving smoothed image is generated from the sunlit area, and the edge preserving As a calculation process of the probability density in which the pixel information in the smoothed image is generated from the shadow area, the probability density is calculated by the one-dimensional or multidimensional normal distribution probability density function having the number of dimensions representing the pixel information. Is an image correction device.

Alternatively, in the above image correction device,
The high-frequency component extraction means, as a process of extracting high-frequency components of individual frequencies, for the high-frequency component of the highest frequency, the difference between the original image and the image obtained by deleting the high-frequency component of the highest frequency from the original image. By extracting, for the high frequency components of other frequencies, an image in which all high frequency components higher than the frequency have been deleted from the original image,
The image correction apparatus is characterized in that extraction processing is performed by obtaining a difference from an image in which a high frequency component of the frequency is deleted from the image.

Alternatively, in the above image correction device,
The edge-preserving smoothed image creating means uses means for performing a process of combining morphological opening and closing using a structuring element having a size corresponding to a smoothing frequency for the image. It is a correction device.

Alternatively, in the above image correction device,
The high-frequency component extraction means is a means for deleting high-frequency components of individual frequencies from an image in a process of extracting high-frequency components of individual frequencies, using a morphology for the image by using a structural element having a size corresponding to the frequency. Ope
The image correction apparatus is characterized by using a means for performing a process in which the "ning" and the "closing" are combined.

Alternatively, in the above image correction device,
The edge-preserving smoothed image creating means not only creates an image in which high-frequency components of each frequency are deleted from the image,
An edge-preserving smoothed image is created by performing edge-preserving smoothing on the original image, and the pixels of the corresponding edges of the edge-preserving smoothing image and the edge-preserving smoothing image are compared to determine the luminance of the latter. The image correction apparatus is characterized by using a means for converting the latter luminance so as to match the luminance of and outputting it as a new edge-preserving smoothed image.

Alternatively, it is a recording medium on which the image correcting method is recorded, in which the program for causing the computer to execute the steps in the image correcting method is recorded on the computer-readable recording medium.

Alternatively, there is provided an image taking device characterized by incorporating the above-mentioned image correcting device.

Alternatively, an image display device is characterized by incorporating the above image correction device.

According to the present invention, a single image is divided into a shadow area and a sunlit area depending on the brightness, so that it is necessary to integrate a plurality of photographed images of the same subject at the same place at different dates and times. It is possible to correct the shadow area with only one image, without affecting the quality of the image in the sunlit area. In addition, by performing a process of reducing the saturation of the shadow area when handling a color image, it is possible to prevent a bright blue color when the shadow area is corrected.

[0058]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a processing block diagram according to the first embodiment of the present invention. In FIG. 1, 1 is a grayscale image creating unit, 2 is a density correcting unit, 3 is an edge-preserving smoothed image producing unit, 4 is a high frequency component extracting unit, 5 is an edge-preserving smoothed image luminance gradation converting unit, and 6 is High frequency component converting means, 7 is an image synthesizing means, and 8 is a color correcting means.

The detailed operation of the embodiment having the block configuration shown in FIG. 1 will be described below.

The original image in this embodiment is a color multi-gradation image, and the color specification information of each pixel is represented by RGB values. Further, the frequency of the high frequency component is one for the sake of simplification of description.

First, the grayscale image creating means 1 converts the RGB values of each pixel of the original image into three values of luminance, saturation and hue. As a concrete method for mutual conversion of RGB values and luminance / saturation / hue, for example, there is a conversion method using the HSI 6 pyramid color model (see “Image Analysis Handbook (edited by Mikio Aoki, Yoshihisa Shimoda: The University of Tokyo Press)). Four
See pages 86-489). Then, the brightness values of each pixel are collected to create an image (this is called a grayscale image), and the image is sent to the density correction means 2.

In the density correction means 2, for the grayscale image,
Gamma correction is performed so that the intensity of the incident light and the brightness of the image have a proportional relationship, and the resulting image is sent to the high-frequency component extraction means 4 (For a specific method of gamma correction, see “Basics of Image Processing Engineering (Keiji Taniguchi). Author: Kyoritsu Shuppan Co., Ltd.) ", pp. 39-40).

The edge-preserving smoothed image creating means 3 smoothes the image using morphological crossing and opening. In the following description, an image is represented by a one-dimensional explanatory diagram for simplification of description. An example of a grayscale image is shown in FIG. 2A, which is represented by a function I (x).

Here, the closing and opening will be briefly described. Both of them are combinations of dilution and erosion, and closing is done after erosion.
opening is dila after performing erosion
to execute the action. There are various possible structural elements to be used, but in the present embodiment example, for simplicity of explanation,
A function of 0 within the domain, which is indicated by the bold line in FIG. 3, is used. In this case, the dilation is a process of obtaining the maximum value of I (x) within the domain when the domain of the structural element is continuously moved, and the erosion continuously moves the domain of the structural element. I (x) in the domain when
(See pages 75 to 77 of "Morphology (Hidefumi Obata: Corona Publishing Co., Ltd.)"). The smoothing frequency is set by adjusting the size of the domain of this structural element. That is, if the domain is increased, the frequency is lowered, and if the domain is reduced, the frequency is increased.

FIG. 2B shows a result of carrying out the dilution on the data of FIG. 2A using the structural element of FIG. 3, and FIG. 2 shows a result of performing the erosion on the result.
It shows in (c). That is, the data in FIG.
The result of performing the singing is shown in FIG. This erases the narrow recess.

Next, the result of performing erosion on the data of FIG. 2 (c) using the structural element of FIG. 3 is shown in FIG. 2 (d).
FIG. 2 (e) shows the result of performing the dilation on the result. That is, the data in FIG.
The result of the pening is shown in FIG. As a result, the narrow convex portion is erased, and an edge-preserving smoothed image is obtained.

Next, the operation of the high frequency component extracting means 4 will be described. In this case as well, similar to the edge-preserving smoothed image creating means 3, processing that combines closing and opening is performed. In the present invention, the frequency component to be extracted is single or plural, but in the present embodiment example, one frequency is used for simplification of the description, and the edge-preserving smoothed image creating means 3 is used as the structuring element to be used. The same zero function in the domain is used as shown in the thick line in FIG.

In the high frequency component extraction means 4, first, the image output from the density correction means 2 (FIG. 2A) is c
Losing is performed and then opening is performed to create an image that is smoothed while preserving edges. However, the structuring element used in this embodiment is the same as the function used in the description of the edge-preserving smoothed image creating means 3. Therefore, the progress of the processing and the image smoothed while preserving the edges are the same as in the case of the edge-preserving smoothed image creating means 3. Therefore,
The progress of the process is shown in FIGS. 2 (a) to 2 (e), and the results are shown in FIG.
(E). Next, a value obtained by subtracting the brightness value of the corresponding pixel in the image output from the edge-preserving smoothed image creating unit 3 from the brightness value of each pixel of the image output from the density correcting unit 2 is a two-dimensional array. (Luminance difference value array). That is, the difference between the image of FIG. 2 (a) and the image of FIG. 2 (e) is shown in FIG. 2 (f), which is the high frequency component.

As described above, the high frequency component extracting means 4
If one type of high-frequency component is extracted and the structuring element used is the same as that used in the edge-preserving smoothed image creating means 3, only one of the two processes is performed until the image is smoothed. The result can be diverted to the other.

Next, edge-preserving smoothed image luminance gradation conversion
The operation of the means 5 will be described. Here, an edge-preserving smoothed image
The brightness of each individual pixel of the
It There are various possible brightness conversion functions.
However, in the present embodiment example, the function shown in FIG. 4 is used. this
Is the threshold value I for the dark domain and the bright domain in the brightness domain.
_sDivided by, and performs different processing in each section
Is. This dark section corresponds to the shadow area in the image
However, the bright section corresponds to the sunlit section in the image.

First, the dark section will be described. The section smaller than I _s is named section 0, and the conversion for the luminance I in this section is expressed as f (I). In this embodiment example,
Use f (I) = aI. Here, a is a constant larger than 1, and this conversion expands the dynamic range in the interval 0. As a result, the converted luminance I ′ is
I '= aI. As f (I), in addition to aI, other functions such as a logarithmic function or a straight line f (I) that does not pass through the origin
= AI + b (a and b are constants) or the like can also be used.

Next, the bright section will be described. The conversion for the luminance I in this is expressed as g (I). Since this section corresponds to the sunlit area in the image, it is desirable not to change the luminance in all sections, that is, g
(I) = I is desirable, but if this is used in all sections, there arises a problem that a discontinuity in luminance occurs between the shadow area and the sunlit area in the image. Therefore, as g (I), a function that is continuous with f (I) at the point of I = aI _s and monotonically increases as I increases to approach the function g (I) = I is used. There are various possible such functions, but in the present embodiment, a straight line I ′ parallel to the I axis is used.
= AI _s and a straight line I '= I passing through the origin and having a slope of 1
When the value is I _m , the dark section is divided into a dark side section (section 1) and a bright side section (section 2) with I _m as a boundary. I '
I ′ = aI _s is set, and in the section 2, the luminance I ′ after conversion is set to the luminance I, and I ′ = I is set so that the luminance is not changed.
That is, g (I) is represented by formula (1).

[0074] In the section 1, a deviation from the desired conversion of g (I) = I occurs, which causes image quality deterioration in the sunlit area. In addition to the above method, the straight line I '=
It is also possible to use aI _s and another function that passes through the origin and the intersection of a straight line I ′ = I having a slope of 1.

I _s is obtained as follows. Since this value is the boundary between the brightness values of the shadow area and the sunlit area in the image, a brightness histogram of the edge-preserving smoothed image shown in FIG. A valley between the higher mountain (luminance distribution in the shadow area) and the higher mountain (luminance distribution in the sunlit area) is searched for, and is defined as I _s (FIG. 5).
(B)). As a method for finding a valley from this histogram, for example, there is a threshold value determination method using discriminant analysis (Otsu: “Automatic threshold value selection method based on discriminant and least squares criterion”, IEICE Transactions Vol. J63-
D, No. 4, pp 349-356 (1980.
4)).

Here, a method of setting parameters of f (I) and g (I) will be described. The degree of expansion of the dynamic range in the shadow region and the deterioration of the image quality in the sun change depending on the parameters of these functions. In order to set the parameter to an appropriate value, in the present embodiment example, f
The effect of image quality improvement in the shadow area and image quality deterioration in the sunlit area due to the luminance conversion in the edge-preserving smoothed image by (I) and g (I) is represented by the total I change amount in each area, and A method is used in which the effect of image quality improvement is evaluated by the evaluation scale F of the equation (2) expressed by the weighted sum of the two, and the parameter that maximizes F is obtained.

F = αΣ _s0 {f (I) −I} · h (I) −Σ _sa {g (I) −I} · h (I) (2) where s0 is the interval 0 and sa is It represents a section other than the section 0.

In equation (2), h (I) is the number of pixels whose luminance is equal to I in the edge-preserving smoothed image, and the coefficient α.
Is a weight that indicates the degree to which the improvement of the image quality of the shadow area is emphasized, and is set by the user. As described above, in the present embodiment, f (I) = aI is used in the section 0, and g (I) of the equation (1) is used in the sections other than the section 0.
The parameters of both are only a, and F is a function of only a as represented by the equation (3).

F (a) = αΣ _s0 {aI−I} · h (I) −Σ _s1 {aI _s −I} · h (I) (3) where s0 is the interval 0 and s1 is the interval 1 Represents

The above brightness conversion processing will be described with reference to FIGS. 6 (a) and 6 (b). FIG. 6A shows an example of the edge-preserving smoothed image created by the edge-preserving smoothed image creating means 3. For this image, using thresholds I _s and I _m to check in which section the luminance of each pixel is located, FIG.
It is assumed that it is found that all the pixels in the section indicated by section 0 in (a) belong to section 0, and all the pixels in section 2 belong to section 2.

In this case, the luminance of the pixels belonging to the section 2 is not changed, and the pixels of the section 0 are multiplied by a. As an example, FIG. 6B shows how the luminance is converted when a = 2 is calculated. In this way, the brightness in the shadow area in the image is increased on average and becomes brighter, and the dynamic range is expanded, so that the contrast is increased. Both of these are factors that make it easier to see. On the other hand, the sunlit area in the image is not affected.

Next, the operation of the high frequency component converting means 6 will be described. Here, a conversion process for amplifying the absolute value and obtaining a new brightness value i ′ is performed on the value i of each element of the brightness difference value array. Although there are various conversion functions, the conversion of the following formula is used in this embodiment.

[0083] In Expression (4), β is a weight to be multiplied in order to enhance the contrast in a dark area. I is the brightness of the edge-preserving smoothed image corresponding to each element of the brightness difference value array. In addition, section 1 in the edge-preserving smoothed image
In, the image quality has deteriorated, and to compensate for it, Can also be In Expression (5), γ is a weight for increasing the contrast in order to compensate the image quality deterioration in the section 1 of the edge-preserving smoothed image.

FIGS. 6A and 6C show examples of how the processing is performed by using the equation (4) on the luminance difference value array of FIG. 2F. In this example, a = 2 and β = 1 are used. In this way, the amplification result of FIG. 6 (c) is obtained,
The amplitude does not change in the sunlit area, but the amplitude is amplified in the shadow area, and the contrast in the shadow area increases.

Next, in the image synthesizing means 7, the image created by the edge-preserving smoothed image brightness / gradation converting means 5 (an example is shown in FIG. 7A) and the brightness created by the high frequency component converting means 6. The values of the individual elements of the difference value array (an example is shown in FIG. 7B) are added. An example of the addition result is shown in FIG. In this case, as a result, an image in which the sunlit areas in the image are not affected (that is, the image quality is not deteriorated) and the shadow areas have improved brightness and contrast can be obtained.

Next, the operation of the color correction means 8 will be described. Here, the saturation and hue information about each pixel of the original image is sent from the grayscale image creating unit 1, and whether each pixel belongs to the section 0 from the edge-preserving smoothed image brightness gradation converting unit 5. Information is sent, and a grayscale image whose brightness is corrected is sent from the image synthesizing means 7. The color correction means 8
For each pixel of the brightness-corrected grayscale image, it is checked whether or not the pixel of the corresponding coordinate sent from the edge-preserving smoothed image brightness / gradation converting means 5 belongs to the section 0, and if it belongs to the section 0. In order to reduce the saturation, the reduced saturation is combined with the luminance of the pixel of interest in the grayscale image whose hue and luminance have been corrected, and the RGB value is converted.
As a result, a final corrected image whose saturation is lower than that of the original image is obtained in the dark area. The saturation reduction can be performed by multiplying by a constant number smaller than 1, for example. As a result, the result of correcting the shadow area becomes less vivid, and the color becomes blue, which is closer to gray.

The second embodiment of the present invention will be described below with reference to the drawings.

FIG. 8 is a processing block diagram according to the present embodiment. In FIG. 8, 11 is a color information conversion means, 1
Reference numeral 2 is a density correction unit, 13 is an edge-preserving smoothed image creating unit, 14 is a high-frequency component extracting unit, 15 is an edge-preserving smoothed image luminance gradation converting unit, 16 is a high-frequency component converting unit,
Reference numeral 17 is an image synthesizing means, 18 is a prior learning means, 19 is a probability calculating means, and 20 is a saturation converting means.

The detailed operation of the block shown in FIG. 8 will be described below. The original image in this embodiment is a color multi-gradation image, and the color specification information of each pixel is represented by RGB values. Further, the frequency of the high frequency component is one for the sake of simplification of description.

The operation of this embodiment has two modes, a pre-learning mode and an image correction mode. First, the pre-learning mode will be described.

First, an image for pre-learning is input to the color information conversion means 11. The color information conversion unit 11 converts the RGB value of each pixel of the original image into three values of luminance, saturation, and hue, as in the process in the grayscale image creation unit 1. Then, the result is subjected to gamma correction by the density correction means 12 (this is the same as the operation of the density correction means 2). A specific method of gamma correction is described in "Basics of Image Processing Engineering (Kyoritsu Shuppan Co., Ltd., Keiji Taniguchi)" pp65-67. Then, the edge-preserving smoothed image creating means 13 smoothes the image using the morphological closing and opening (this is the same as the operation of the edge-preserving smoothed image creating means 3). Then, the edge-preserving smoothed image is sent to the pre-learning means 18.

The pre-learning means 18 has a display means for displaying an image, and an area designating means such as an electronic pen, a mouse or a keyboard for the operator to designate the area of the image displayed on the display means. The prior learning unit 18 first displays the received edge-preserving smoothed image on a display unit such as a display. On the other hand, the operator specifies the sunlit area and the shadow area. Then, the pre-learning means 18 determines, for each of the sunlit area and the shadow area designated by the operator,
Information for calculating the occurrence probability or occurrence probability density of pixel color information is obtained. In the present exemplary embodiment, the occurrence probability density of pixel color information is adopted. Then, the color information in this case is represented by a three-dimensional value of RGB values, and the occurrence probability density thereof is calculated by a three-dimensional normal distribution function. Then, in the pre-learning means 18, three variables (R, G, B) which are parameters necessary for calculation of a three-dimensional normal distribution function are calculated from a set of pixel RGB values for each of the sunlit area and the shadow area. ) Each mean value, 3 variables (R, G, B)
Then, the standard deviation of each of the three variables and the correlation coefficient between the combinations of all two variables of three variables (R, G, B) are calculated and sent to the probability calculating means 19. Further, the operator checks the ratio of pixels in the shadow area and the sunlit area by means such as visual measurement of an image displayed on the screen, inputs the value to the pre-learning means 18, and the pre-learning means 18 It is sent to the probability calculation means 19. This is the end of the operation in the pre-learning mode.

Next, the operation of the image correction mode will be described.

First, the image to be corrected is input to the color information conversion means 11. The color information conversion means 11 uses the RG of each pixel of the original image, similar to the processing in the pre-learning mode.
The B value is converted into three values of brightness, saturation and hue. And
The result is sent to the saturation converting means 20, the image synthesizing means 17, and the density correcting means 12. The processing contents of the saturation converting means 20 and the image synthesizing means 17 will be described later. The density correction means 12 performs gamma correction (this is the same as the operation of the density correction means 2). The density correction means 12 sends the result to the edge-preserving smoothed image creating means 13 and the high frequency component extracting means 14.

Next, the edge-preserving smoothed image creating means 13
Works. The processing content is the same as that of the edge-preserving smoothed image creating means 3. Then, the resulting edge-preserving smoothed image is sent to the probability calculating means 19 and the edge-preserving smoothed image luminance gradation converting means 15. The edge-preserving smoothed image creating unit 13 not only creates an image in which high-frequency components of respective frequencies are deleted from the original image, but also performs edge-unsave smoothing on the original image to perform edge-unsave smoothing. Create an image, compare the corresponding pixels of the edge-preserving smoothed image and the edge-non-preserving smoothed image, and convert the latter luminance so that the latter luminance matches the former luminance. What is output as a stored smoothed image can be used. This new edge-preserving smoothed image has information on the luminance of the original edge-preserving smoothed image and information on the saturation and hue of the edge-non-preserving smoothed image.
The edge-preserving smoothed image is a local average filter (“Image analysis handbook (edited by
Aoki Mikio, Shimoda Y .: Tokyo University Press), page 595).

Next, the probability calculating means 19 operates. The probability calculation means 19 is the average value of each of the three variables (R, G, B) sent from the pre-learning means 18 in the pre-learning mode, and the standard deviation of each of the three variables (R, G, B), Using the correlation coefficient between all combinations of two variables of three variables (R, G, B) and the ratio of pixels in the shadow area and the sunlit area, for the RGB value of each pixel of the edge-preserving smoothed image, The probability density that they belong to the sunlit area and the probability density that they belong to the shadow area are calculated as follows.

A vector X represents a set of RGB values of one pixel. An event whose pixel belongs to the shadow area is represented by A, and an event which belongs to the sunlit area is represented by B, and the pixel ratios of the shadow area and the sunlit area are P (A) and P (B), respectively, and the vector X The probability density belonging to the shadow area is represented by P (A | X), and the probability density belonging to the vector X belonging to the sunlit area is represented by P (B | X). Then, P (A | X) and P (B | X) are calculated by the following formula.

P (A | X) = P (A) P (X | A) (6) P (B | X) = P (B) P (X | B) (7) Equation (6) In equation (7), P (X | A) and P (X |
B) is calculated by a three-dimensional normal distribution function for R, G, and B. For the calculation method of the three-dimensional normal distribution function,
"Probability / Statistics Handbook (Morikita Publishing Co. R.
S. Burlington, D.A. C. By Mei) ”, pp. 130
~ 131. Then, the values of P (A | X) and P (B | X) for all pixels are used as the image synthesizing unit 1.
7. Send to the saturation conversion means 20.

Next, the high frequency component extracting means 14 operates. The processing content is the same as that of the high frequency component extracting means 4, and the result is sent to the high frequency component converting means 16. Here, the processing steps up to this point after the processing by the color information conversion means 11 and the density correction means 12 will be described with reference to the drawings.

FIG. 9 is a schematic diagram of a one-dimensional luminance value distribution of the original image, FIG. 10 is a schematic diagram of a one-dimensional saturation luminance value distribution of the original image, and FIG. 11 is compared with FIG. FIG. 12 is a schematic diagram of the distribution of luminance values as a result of the processing performed by the edge-preserving smoothed image creating unit 13, FIG. 12 is a one-dimensional schematic diagram of the high-frequency components extracted by the high-frequency component extracting unit 14, and FIG. Using the RGB values of each pixel of the stored smoothed image, each P (X
It is a schematic diagram of the calculation result of | A) and P (X | B).

Next, the operation of the edge-preserving smoothed image luminance gradation converting means 15 is started. In the edge-preserving smoothed image luminance gradation converting means 15, first, each pixel of the edge-preserving smoothed image is processed on the assumption that they belong to the shadow area, and outputs it to the image synthesizing means 17. . When the conversion for the brightness I is expressed as f (I), f (I) = aI is used in this embodiment. Here, a is a constant larger than 1. Then, if it is premised that the image belongs to the Hyuga area, no conversion is performed in the present embodiment, and I
Is output to the image synthesizing means 17 as it is. This is performed for all pixels of the edge-preserving smoothed image.

Next, the operation of the high frequency component converting means 16 will be described. Here, first, the value i of each element of the brightness difference value array is
On the other hand, on the assumption that they belong to the shadow area, the conversion processing is performed to amplify the absolute value and obtain a new luminance value i ′. Although there are various conversion functions, the conversion of the following formula is used in this embodiment.

I ′ = aβ · i (8) In equation (8), β is a weight to be multiplied in order to enhance the contrast in the dark region. If it is presumed that the image belongs to the sunlit area, the conversion is not performed in the present embodiment, and i is output to the image synthesizing unit 17 as it is. This is performed for all the elements of the brightness difference value array.

Next, the operation of the saturation conversion means 20 will be described. Here, with respect to the saturation S of each pixel of the original image, conversion processing for obtaining a new saturation S ′ by reducing the absolute value is performed on the assumption that they belong to the shadow area. As the reduction processing, for example, a constant ratio K (K is 1
It can be done by multiplying by a smaller number).

S ′ = KS (9) Then, as the final new saturation S ″, as shown in Expression (10), P (X | A) and S after conversion when the shadow area is assumed. Calculate the sum of the product of 'and the product of P (X | B) and the original S.

S ″ = P (X | A) · KS + P (X | B) · S (10) This is performed for the saturations of all the pixels of the original image to create a new set of saturations, Send to the synthesizing means 17.

Next, the operation of the image synthesizing means 17 will be described.

First, the two types of data sent from the edge-preserving smoothed image luminance gradation converting means 15 are integrated.
This is a new brightness I ″, and as shown in Expression (11), P
(X | A) and the luminance aI after conversion when assuming a shadow area
And the product of P (X | B) and the original luminance I are calculated.

I ″ = P (X | A) · aI + P (X | B) · I (11) This is performed for all pixels of the edge-preserving smoothed image to create a new edge-preserving smoothed image. 14 to 16 are schematic diagrams of the process of performing the present process on the image of No. 11. The process of creating P (X | A) · aI is shown in FIG.
Is the process of creating P (X | B) · I, and FIG. 16 is P (X | A)
-The process of adding aI and P (X | B) -I is shown.

Next, the two types of data sent from the high frequency component conversion means 16 are integrated. This is a new high-frequency component i ″, and as shown in equation (12), P (X | A) and the product of i ′ after conversion assuming the shadow area and P (X |
This is done by calculating the sum of B) and the original product of i.

I ″ = P (X | A) · aβ · i + P (X | B) · i (12) This is performed for all the elements of the brightness difference value array to create a new brightness difference value array. A schematic diagram of the result of performing this processing on the data of Fig. 12 is shown in Fig. 17. The concept is the same as the creation process according to the equation (11) shown in Figs.

Next, the I "calculated by the equation (11) and the i" calculated by the equation (12) are added to create an image having a new brightness. FIG. 18 shows a schematic diagram of a result of performing this processing on the data of FIGS. 14 to 16 and FIG. Further, for each pixel of the image, the hue value of each pixel of the original image sent from the color information conversion unit 11 and the new saturation calculated by the expression (10) sent from the saturation conversion unit 20. The final image is created by combining S "and converting to RGB values.

Further, in the saturation conversion means 20, as shown in FIG.
FIG. 19 shows a schematic diagram of the result of processing the data of 1. The idea is the same as the example of the processing steps of the image synthesizing means 17 shown in FIGS. 14 to 16.

If the image correction apparatus according to the present invention as described above is incorporated into an image photographing apparatus and a photographed image by the image photographing apparatus is output through the image correcting apparatus, a shadow image is corrected to improve the image quality. Can be output. Further, when the image correction device according to the present invention as described above is incorporated into an image display device and an input image is input to the image display device through the image correction device, a corrected image in which the shadow area is corrected to improve the image quality is obtained. Can be displayed.

It is needless to say that a part or all of the functions of each block of the apparatus shown in FIGS. 1 and 8 can be realized by using a computer in order to realize the function of each block by the computer. Computer program readable storage medium such as FD (floppy disk), MO, ROM,
It can be recorded, provided, and distributed on a memory card, a CD, a DVD, a removable disk, or the like.

[0116]

As described above in detail, according to the present invention, it is not necessary to provide a plurality of images of the same object at the same place at different dates and times. Correction is possible.

Further, it is possible to prevent the sunlit area from having a bad influence on the image quality and to prevent a bright blue color when the shadow area is corrected when a color image is handled. It has a remarkable effect.

[Brief description of drawings]

FIG. 1 is a block diagram illustrating a first exemplary embodiment of the present invention.

FIG. 2 is a cl of morphology in the first embodiment example.
It is a figure which shows the process of making the smoothing of the image using ossing and opening and the array of the difference value with an original image.

FIG. 3 is a cl of morphology in the first embodiment example.
It is a figure which shows the example of the structuring element used by ossing and opening.

FIG. 4 is an explanatory diagram of a luminance conversion function of each pixel of an edge-preserving smoothed image in the first embodiment example.

5A and 5B are explanatory diagrams of a process of obtaining threshold values of a shadow section and a sunlit section in edge-preserving smoothed image luminance gradation conversion in the first embodiment.

FIG. 6 is a diagram showing operations of an edge-preserving smoothed image luminance gradation converting unit and a high frequency component converting unit in the first embodiment example.

7A, 7B and 7C are diagrams showing the operation of the image synthesizing means in the first embodiment.

FIG. 8 is a block diagram illustrating a second exemplary embodiment of the present invention.

FIG. 9 is a schematic diagram of a one-dimensional distribution of luminance values of an original image in the second embodiment example.

FIG. 10 is a schematic diagram of a distribution of luminance values of one-dimensional saturation of an original image in the second embodiment example.

FIG. 11 is a schematic diagram of a distribution of luminance values as a result of the processing performed by the edge-preserving smoothed image creating means in the second embodiment example with respect to FIG. 9;

FIG. 12 is a one-dimensional schematic diagram of high-frequency components extracted by a high-frequency component extracting unit according to the second embodiment.

13 (a), (b), and (c) are P (X | A) and P (X) using RGB values of each pixel of an edge-preserving smoothed image in the second embodiment. It is a schematic diagram of the calculation result of | B).

14 (a), (b), and (c) are process steps (creation of P (X | A) · aI) in the edge-preserving smoothed image luminance gradation converting means in the second embodiment. It is a schematic diagram which shows the example of.

15 (a), (b), and (c) are processes (generation of P (X | B) .I) in the edge-preserving smoothed image luminance gradation converting means in the second embodiment. It is a schematic diagram which shows the example of.

16 (a), (b), and (c) are processing (P (X | A) .aI and P (X) in the edge-preserving smoothed image luminance gradation converting means in the second embodiment. | B) ・ I addition)
It is a schematic diagram which shows the example of a process.

FIG. 17 is a schematic diagram showing an example of the processing result of the high frequency component conversion means in the second embodiment.

FIG. 18 shows the individual elements of the image created by the edge-preserving smoothed image brightness gradation conversion means and the individual elements of the brightness difference value array created by the high frequency component conversion means in the image composition means in the second embodiment. It is a schematic diagram which shows the process of the process which adds a value.

FIG. 19 is a schematic diagram showing an example of the processing result of the chroma conversion means in the second embodiment.

[Explanation of symbols]

1 ... Gray image creating means 2. Density correction means 3 ... Edge-preserving smoothed image creating means 4. High frequency component extraction means 5 ... Edge-preserving smoothed image brightness gradation conversion means 6 ... High-frequency component converting means 7 ... Image synthesizing means 8 ... Color correction means 11 ... Color information conversion means 12 ... Density correction means 13 ... Edge-preserving smoothed image creating means 14 ... High-frequency component extracting means 15 ... Edge-preserving smoothed image luminance gradation converting means 16 ... High-frequency component conversion means 17 ... Image synthesizing means 18 ... Advance learning means 19 ... Probability calculation means 20 ... Saturation conversion means

─────────────────────────────────────────────────── ─── Continuation of front page (56) Reference JP-A-10-243238 (JP, A) JP-A-10-49670 (JP, A) JP-A-11-27533 (JP, A) JP-A-11- 69150 (JP, A) (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06T 5/00-5/50 G09G 5/36

Claims (49)

(57) [Claims] 1. An image correction method for converting a multi-gradation original image to improve legibility, wherein an edge is extracted from the original image and the original image is smoothed to generate a smoothed image. An edge-preserving smoothed image creating step of adding the smoothed images to create an edge-preserving smoothed image, and classifying the domain of luminance for the edge-preserving smoothed image into a plurality of classes, and luminance for each class An edge-preserving smoothed image luminance gradation conversion step that applies gradation conversion, a high-frequency component extraction step that extracts high-frequency components of a single frequency or a plurality of frequencies of the original image, and a high-frequency component of each extracted frequency, A high-frequency component conversion step of performing a conversion for amplifying a high-frequency component for each class using the result of classifying the edge-preserving smoothed image, Image correction method characterized by and an image synthesis step, to create an image by integrating the conversion result of the reduction image the high-frequency component conversion step. 2. The image correction method according to claim 1, wherein, in the image synthesizing step, the edge-preserving smoothed image subjected to the luminance gradation conversion and the individual element values of the conversion result in the high frequency component converting step are added, An image correction method characterized by integrating both. 3. The image correction method according to claim 1, wherein in the edge-preserving smoothed image creating step, a morphological opening is performed using a structuring element having a size corresponding to a smoothing frequency with respect to the original image. An image correction method characterized in that an edge-preserving smoothed image is created by using a method of performing a process that combines a closing and a closing. 4. Any one of claims 1 to 3
In the image correction method according to the item, in the high frequency component extraction step, for the high frequency component of the highest frequency, by obtaining the difference between the original image and the image in which the high frequency component of the highest frequency is deleted from the original image, For the high frequency components of other frequencies, the difference between the image in which all the high frequency components higher than the frequency are deleted from the original image and the image in which the high frequency components of the frequency are deleted from the image are calculated. An image correction method comprising extracting a high frequency component of a frequency. 5. The image correction method according to claim 4, wherein in the high frequency component extraction step, a structuring element having a size corresponding to the frequency is used for the image as a method of deleting high frequency components of individual frequencies from the image. Te morphology op
An image correction method characterized by using a method of performing a process in which ening and closing are combined. 6. Any one of claims 1 to 5
In the image correction method according to the item, in the edge-preserving smoothed image luminance gradation conversion step, each pixel of the edge-preserving smoothed image is classified into at least two classes, a dark pixel class and a bright pixel class, according to the magnitude of the brightness value. Then, the image correction method is characterized in that the luminance gradation conversion is applied to each of the pixels of each class by a separate function. 7. The image correction method according to claim 6, wherein, in the edge-preserving smoothed image luminance gradation conversion step, the parameter value is defined in a predetermined domain as a method of determining the parameter value of the luminance gradation conversion function. Of the brightness change values of pixels belonging to the bright pixel class and the brightness change values of pixels belonging to the dark pixel class in the edge-preserving smoothed image when each parameter value is used. An image correction method characterized by using a method of calculating an evaluation value represented by a function having a total value and a variable and adopting a parameter value for which a maximum evaluation value is obtained. 8. Any one of claims 1 to 7.
In the image correction method described in the item 1, in the high-frequency component conversion step, conversion is performed using a function, and the function is classified into classes of corresponding pixels of the edge-preserved smoothed image in the edge-preserved smoothed image luminance gradation conversion step. An image correction method, characterized in that it is changed according to the result. 9. An image correction method for converting a multi-tone original image of color to enhance legibility, and extracts brightness from color display information of each pixel of the original image to create a monochrome gray-scale image. A grayscale image creation step; a grayscale image correction step of correcting the monochrome grayscale image created in the grayscale image creation step by the image correction method according to any one of claims 1 to 8; For each pixel constituting the corrected grayscale image corrected in the correction stage, the saturation and hue are extracted from the information of the color display of the pixel of the corresponding original image, and combined with the luminance of the pixel in the corrected grayscale image. And a color correction step of creating a corrected color image by using the corrected display color information. 10. An image correction method for improving legibility by converting a color multi-gradation original image to create a monochrome grayscale image by extracting luminance from color display information of each pixel of the original image. A grayscale image creating step; a grayscale image correcting step of correcting the monochrome grayscale image created in the grayscale image creating step by the image correction method according to any one of claims 6 to 8; For each pixel constituting the corrected grayscale image corrected in the correction stage, it is determined as a class of a dark pixel in the processing of the edge-preserving smoothed image brightness gradation conversion stage in the image correction method used in the grayscale image correction stage. If it is determined that it is a class of dark pixels, the saturation and hue are extracted from the information of the color display of the pixels of the corresponding original image, and the extracted saturation is reduced. From the above, the extracted hue and the luminance of the pixel in the corrected grayscale image are combined to obtain the corrected display color information, and when it is determined that the pixel is a bright pixel class, the color display of the pixel of the corresponding original image is performed. A color correction step of creating a corrected color image by extracting saturation and hue from the information and combining the information with the luminance of the pixel in the corrected grayscale image to obtain corrected display color information. Image correction method characterized by. 11. An image correction apparatus for converting a multi-gradation original image to improve legibility, extracts an edge from the original image, and smoothes the original image to generate a smoothed image. An edge-preserving smoothed image creating means for adding the smoothed images to create an edge-preserving smoothed image, and a domain of brightness for the edge-preserving smoothed image is classified into a plurality of classes, and brightness is classified for each class. An edge-preserving smoothed image luminance gradation converting means for applying gradation conversion, a high-frequency component extracting means for extracting high-frequency components of a single or a plurality of frequencies of the original image, and a high-frequency component of each extracted frequency, High-frequency component conversion means for performing conversion for amplifying high-frequency components for each class using the result of classifying the edge-preserved smoothed image, and edge-preserving the luminance gradation conversion Wherein the smoothed image and an image synthesizing means for creating an image by integrating the conversion result of the high-frequency component transform means, an image correction apparatus characterized by comprising. 12. The image correction device according to claim 11, wherein the image synthesizing unit adds the values of the individual elements of the conversion result of the high-frequency component conversion unit and the edge-preserving smoothed image that has been subjected to the luminance gradation conversion, An image correction device characterized by integrating both. 13. The image correction apparatus according to claim 11 or 12, wherein the edge-preserving smoothed image creating means uses a structuring element having a size corresponding to a smoothing frequency with respect to the original image to open the morphology. An image correction apparatus, wherein an edge-preserving smoothed image is created by using a means that performs a process that combines a closing and a closing. 14. The image correction device according to claim 11, wherein the high-frequency component extraction means selects the original image and the original image from the original image for the high-frequency component of the highest frequency. By obtaining the difference from the image in which the high-frequency components of high frequency are deleted, for high-frequency components of other frequencies, the image in which all high-frequency components higher than the relevant frequency are deleted from the original image and the relevant frequency from the image The image correction apparatus is characterized in that a high frequency component of an individual frequency is extracted by obtaining a difference from the image in which the high frequency component of is deleted. 15. The image correction apparatus according to claim 14, wherein the high frequency component extraction means uses a structural element having a size corresponding to the frequency for the image as means for deleting high frequency components of individual frequencies from the image. Te of morphology
An image correction apparatus characterized by using a means for performing a process in which Ning and closing are combined. 16. The image correction apparatus according to claim 11, wherein the edge-preserving smoothed image luminance gradation conversion means sets each pixel of the edge-preserving smoothed image to a large or small luminance value. The image correction apparatus is characterized in that it is classified into at least two classes of a dark pixel class and a bright pixel class, and the brightness gradation conversion is applied to each pixel of each class by a separate function. 17. The image correction apparatus according to claim 16, wherein the edge-preserving smoothed image brightness gradation conversion means is a means for determining a parameter value of a function of the brightness gradation conversion, and the parameter value is defined in advance. Change in the area,
A variable is a total value of luminance change values of pixels belonging to a bright pixel class in the edge-preserving smoothed image when each parameter value is used, and a total value of luminance change values of pixels belonging to a dark pixel class. An image correction apparatus characterized by using a means for calculating an evaluation value represented by a function and adopting a parameter value for which a maximum evaluation value is obtained. 18. The image correction device according to claim 11, wherein the high-frequency component conversion means performs conversion using a function, and the function is converted into the edge-preserving smoothed image brightness level. An image correction apparatus, characterized in that the corresponding pixel of the edge-preserving smoothed image is changed by the tone conversion means according to the result of classifying the pixel. 19. An image correction device for enhancing legibility by converting a color multi-gradation original image to extract a luminance from color display information of each pixel of the original image to create a monochrome gray-scale image. A grayscale image creating means, a grayscale image correcting means for correcting the monochrome grayscale image created by the grayscale image creating means by the image correction device according to any one of claims 11 to 18, and the grayscale image For each pixel constituting the corrected grayscale image corrected by the correction means, the saturation and hue are extracted from the information of the color display of the pixel of the corresponding original image, and combined with the luminance of the pixel in the corrected grayscale image. And a color correction unit that creates a color image after correction by using the information of the display color after correction as an image correction apparatus. 20. An image correction apparatus for converting a multi-tone original image of color to improve legibility, extracts luminance from color display information of each pixel of the original image to create a monochrome gray-scale image. A grayscale image creating means, a grayscale image correcting means for correcting the monochrome grayscale image created by the grayscale image creating means by the image correction device according to any one of claims 16 to 18, and the grayscale image Each pixel constituting the corrected grayscale image corrected by the correction means is determined to be a class of dark pixels in the processing of the edge-preserving smoothed image brightness gradation conversion means in the image correction device used in the grayscale image correction means. If it is determined that the pixel has a dark pixel class, the saturation and hue are extracted from the color display information of the pixel of the corresponding original image, and the extracted saturation is reduced. Then, the extracted hue and the luminance of the pixel in the corrected grayscale image are combined to obtain the corrected display color information, and when it is determined that the pixel is a bright pixel class, the color of the pixel of the corresponding original image is determined. By extracting the saturation and hue from the display information, and by using the corrected display color information in combination with the luminance of the pixels in the corrected grayscale image, a color correction unit that creates a corrected color image, An image correction device comprising. 21. An image correction method for converting a multi-gradation original image to improve legibility, wherein an edge is extracted from the original image and the original image is smoothed to generate a smoothed image. An edge-preserving smoothed image creating step of adding the smoothed images to create an edge-preserving smoothed image; a high-frequency component extracting step of extracting high-frequency components of a single or a plurality of frequencies of the original image; A probability calculation step of calculating a probability that each pixel belongs to a sunlit area and a probability that the pixel belongs to a shadow area from pixel information of the edge-preserving smoothed image corresponding to the pixel; In contrast, assuming that each pixel belongs to the shadow area, the brightness gradation conversion is performed and output, and assuming that each pixel belongs to the sunlit area, the brightness gradation conversion is performed and output. An edge-preserving smoothed image luminance gradation conversion step for outputting an edge-preserving smoothed image as it is, and a conversion for amplifying the extracted high-frequency components of each frequency assuming that they all belong to the shadow region. Output
And using a probability of belonging to each region calculated in the probability calculation step, and a high-frequency component conversion step of performing a conversion to output by amplifying or outputting the high-frequency component as it is, assuming that it belongs to the sunlit area, An image is obtained by integrating the two types of output for each hypothetical region output from the edge-preserving smoothed image luminance gradation conversion step and the two types of output for each hypothetical region output from the high frequency component conversion step. An image correction method comprising: an image combining step to be created. 22. An image correction method for enhancing legibility by converting a color multi-gradation original image, wherein a saturation / hue extraction step of extracting saturation and hue from information of each pixel of the original image, An edge-preserving smoothed image creating step of creating an edge-preserving smoothing image by adding edges to the smoothed image to generate a smoothed image by smoothing the original image while extracting edges from the original image, A high frequency component extraction step of extracting high frequency components of a single or a plurality of frequencies of the original image; a probability that each pixel of the original image belongs to a sunlit region and a probability that the pixel belongs to a shadow region, A probability calculation step of calculating from the pixel information of the edge-preserving smoothed image, and a conversion for reducing the saturation extracted from each pixel of the original image using the calculated probability of belonging to each region. Saturation conversion step, and for the edge-preserving smoothed image, it is assumed that each pixel belongs to a shadow area, and luminance and brightness conversion is performed and output, and each pixel belongs to a sunlit area. Then, an edge-preserving smoothed image luminance gradation conversion step of outputting the luminance-preserving smoothed image as it is, or outputting the same as that of the edge-preserving smoothed image. Assuming that it belongs to the region, it performs amplification conversion and outputs,
And using a probability of belonging to each region calculated in the probability calculation step, and a high-frequency component conversion step of performing a conversion to output by amplifying or outputting the high-frequency component as it is, assuming that it belongs to the sunlit area, The two types of output from the edge-preserving smoothed image luminance gradation conversion step and the two types of output from the high frequency component conversion step are integrated to create an image, and the original image output from the saturation / hue extraction step is created therein. An image correction method comprising: an image synthesizing step of creating an image by integrating the information of the hue of the image and the information of the converted saturation output from the saturation converting step. 23. The image correction method according to claim 22, wherein in the saturation conversion step, the saturation extracted from each pixel of the original image is used as a method of integration using the probabilities belonging to each region calculated in the probability calculation step. , The saturation reduction processing is performed assuming that the corresponding pixel of the original image belongs to the shadow area, and the result is multiplied by the probability that the pixel belongs to the shadow area to calculate the value. And an image correction method which adds a value obtained by multiplying the original saturation by the probability that the pixel belongs to the sun. 24. The image correction method according to claim 21, wherein in the image combining step, the edge-preserving smoothed image is calculated by using the probabilities belonging to each region calculated in the probability calculating step. Edge preservation from the edge-preserving smoothed image luminance gradation conversion step is performed as a process of creating an image by integrating the two types of output from the luminance gradation conversion step and the two types of output from the high frequency component conversion step. It is assumed that each pixel belongs to the shadow area for the smoothed image, and the luminance gradation conversion is performed, and the output is multiplied by the probability that each pixel belongs to the shadow area. For the output that has undergone the luminance gradation conversion on the assumption that each pixel belongs to the sunlit area from the digitized image brightness gradation conversion stage or the output of the edge-preserved smoothed image as it is. Each picture Is calculated by multiplying by the probability of belonging to the sunlit area, and the high-frequency components of each frequency from the high-frequency component conversion step are converted assuming that they all belong to the shadowed area. A value is calculated by multiplying the probability that the pixels of the edge-preserving smoothed image belong to the shadow area, and the conversion is performed assuming that all the high frequency components of each frequency from the high frequency component conversion step belong to the sunlit area. An image correction method characterized by calculating a value obtained by multiplying the output obtained by a probability that a pixel of a corresponding edge-preserving smoothed image belongs to a sunlit area, and adding the calculated four types of values. 25. The image correction method according to claim 21, wherein in the image combining step, the edge-preserving smoothed image is calculated by using the probabilities belonging to each region calculated in the probability calculating step. Edge preservation from the edge-preserving smoothed image luminance gradation conversion step is performed as a process of creating an image by integrating the two types of output from the luminance gradation conversion step and the two types of output from the high frequency component conversion step. An output obtained by performing luminance gradation conversion on the assumption that each pixel belongs to a shadow area in the smoothed image, and all high frequency components of each frequency from the high frequency component conversion step belong to the shadow area. The output obtained by the conversion is added for each corresponding element, and the value is multiplied by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the shadow area, and the edge is calculated. Save smoothed image Output from the gradation-to-gradation conversion step, which has been subjected to luminance gradation conversion on the assumption that each pixel belongs to the sunlit area, or the output of the edge-preserved smoothed image as it is, and the high frequency From the component conversion stage, the output obtained by converting the high frequency components of each frequency, assuming that they all belong to the sunlit area, or the output as it is, is added for each corresponding element, and the value is corresponded. An image correction method, wherein a value obtained by multiplying a probability that a pixel of an edge-preserving smoothed image belongs to a sunlit area is added, and the calculated four types of values are added. 26. The image correction method according to claim 21, wherein in the probability calculating step, edge preservation is performed as a process of calculating a probability that each pixel of the original image belongs to a sunlit region. Calculate the probability that the information of the corresponding pixel in the smoothed image will occur from the sunlit area, and that value and the value that was input in advance, which is the average number of pixels in the sunlit area in the edge-preserved smoothed image. As a method of performing the process of multiplying by the ratio and calculating the probability that each pixel of the original image belongs to the shadow area, the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the shadow area is calculated. The image correction method is characterized by performing a process of multiplying the value by a ratio of the number of pixels of an average shadow region in the edge-preserving smoothed image, which is a value input in advance. 27. The image correction method according to claim 26, wherein the information for calculating the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the sunlit area, and the corresponding information in the edge-preserving smoothed image. An image correction method comprising: a learning step of learning in advance information for calculating the probability that the information of the pixel to be generated from the shadow area. 28. The image correction method according to any one of claims 21 to 27, wherein the probability density is replaced by a probability density, or a value obtained by transforming the probability, or a value obtained by transforming the probability density. An image correction method characterized by using. 29. The image correction method according to claim 28, wherein in the probability calculating step, when the probability density or a value obtained by adding a transform to the probability density is used instead of the probability, information of pixels in the edge-preserving smoothed image is obtained. One-dimensional or multi-dimensional calculation with the number of dimensions representing pixel information is performed as the calculation processing of the probability density generated from the sunlit area and the calculation processing of the probability density generated from the shadow area of the pixel information in the edge-preserving smoothed image. An image correction method characterized in that the probability density is calculated by using the probability density function of the normal distribution. 30. The image correction method according to any one of claims 21 to 29, wherein, in the high frequency component extraction step, a high frequency component of the highest frequency is selected as a process of extracting high frequency components of individual frequencies. On the other hand, by extracting the difference between the original image and the image in which the high frequency component of the highest frequency is deleted from the original image,
For high frequency components of other frequencies, extraction is performed by obtaining the difference between an image in which all high frequency components higher than the frequency are deleted from the original image and an image in which the high frequency components of the frequency are deleted from the image. An image correction method, characterized by performing the processing. 31. The image correction method according to claim 21, wherein in the edge-preserving smoothed image creating step, a structuring element having a size corresponding to a smoothing frequency is added to the image. An image correction method characterized by using a method of performing a process that combines opening and closing of morphology. 32. The image correction method according to claim 21, wherein in the high frequency component extraction step, a high frequency of an individual frequency is extracted from the image in a process of extracting a high frequency component of an individual frequency. As a method of deleting the component, a morphological op is used for the image by using a structuring element having a size corresponding to the frequency.
An image correction method characterized by using a method of performing a process in which ening and closing are combined. 33. The image correction method according to claim 31 or 32, wherein in the edge-preserving smoothed image creating step, not only an image in which high frequency components of respective frequencies are deleted is created, but an original image is created. On the other hand, an edge-preserving smoothed image is created by performing smoothing that does not preserve edges, and the corresponding luminance of the edge-preserving smoothed image and the edge-preserving smoothed image is compared and the latter luminance is set to the former luminance. An image correction method characterized by using the latter method of converting the luminance so as to match and outputting it as a new edge-preserving smoothed image. 34. An image correction device for converting a multi-gradation original image to improve legibility, extracts an edge from the original image, and smoothes the original image to generate a smoothed image. An edge-preserving smoothed image creating unit that adds the smoothed images to create an edge-preserving smoothed image; a high-frequency component extracting unit that extracts high-frequency components of a single or a plurality of frequencies of the original image; Probability calculating means for calculating the probability that each pixel belongs to the sunlit area and the probability that the pixel belongs to the shadow area from pixel information of the edge-preserving smoothed image corresponding to the pixel, and the edge-preserving smoothed image. In contrast, assuming that each pixel belongs to the shadow area, the brightness gradation conversion is performed and output, and assuming that each pixel belongs to the sunlit area, the brightness gradation conversion is performed and output. An edge-preserving smoothed image luminance gradation converting unit that outputs an edge-preserving smoothed image as it is, and a conversion that amplifies the extracted high-frequency components of each frequency assuming that they all belong to the shadow region. Output
And using a high-frequency component conversion unit that outputs by performing a conversion that amplifies and outputs the high-frequency component as it is, assuming that it belongs to the sunlit region, and the probability of belonging to each region calculated by the probability calculation unit, An image is obtained by integrating the two types of output for each hypothetical region output from the edge-preserving smoothed image luminance gradation conversion unit and the two types of output for each hypothetical region output from the high frequency component conversion unit. An image correction apparatus comprising: an image synthesizing unit to be created. 35. An image correction apparatus for converting a color multi-gradation original image to improve legibility, and a saturation / hue extraction means for extracting saturation and hue from information of each pixel of the original image, Edge-preserving smoothed image creating means for creating an edge-preserving smoothed image by extracting edges from the original image and smoothing the original image to generate a smoothed image, and adding the edge and the smoothed image, A high-frequency component extracting means for extracting high-frequency components of a single or a plurality of frequencies of the original image, a probability that each pixel of the original image belongs to a sunlit region, and a probability that the pixel belongs to a shadowed region corresponding to the pixel A probability calculating unit that calculates the pixel information of the edge-preserving smoothed image, and a conversion that reduces the saturation extracted from each pixel of the original image using the calculated probability of belonging to each region. Saturation conversion means for performing the luminance gradation conversion on the edge-preserving smoothed image, assuming that each pixel belongs to a shadow area, and outputting, and assuming that each pixel belongs to a sunlit area Then, an edge-preserving smoothed image luminance gradation converting means for performing the luminance gradation conversion and outputting or outputting the edge-preserving smoothed image as it is, and for all the extracted high-frequency components of each frequency, Assuming that it belongs to the region, it performs amplification conversion and outputs,
And using a high-frequency component conversion unit that outputs by performing a conversion that amplifies and outputs the high-frequency component as it is, assuming that it belongs to the sunlit region, and the probability of belonging to each region calculated by the probability calculation unit, The two types of outputs from the edge-preserving smoothed image luminance gradation converting means and the two types of outputs from the high frequency component converting means are integrated to create an image, and the original output from the saturation / hue extracting means is created therein. An image correction apparatus comprising: an image synthesizing unit that integrates the information on the hue of the image and the information on the converted saturation output from the saturation converting unit to create an image. 36. The image correction device according to claim 35, wherein the saturation conversion means is a means for integrating using the probabilities belonging to each region calculated by the probability calculation means, and the saturation extracted from each pixel of the original image. , The saturation reduction processing is performed assuming that the corresponding pixel of the original image belongs to the shadow area, and the result is multiplied by the probability that the pixel belongs to the shadow area to calculate the value. And an image saturation correction device that adds a value obtained by multiplying the original saturation by the probability that the pixel belongs to the sun. 37. The image correction apparatus according to claim 34, wherein the image synthesizing unit uses the probabilities belonging to the respective regions calculated by the probability calculating unit, and the edge-preserving smoothed image. Two types of output from the brightness gradation conversion means and two from the high frequency component conversion means
As a process for creating an image by integrating outputs of various types, it is assumed that each pixel of the edge-preserving smoothed image brightness gradation conversion unit belongs to a shadow area, assuming that each pixel belongs to a shadow area. A value is calculated by multiplying the output obtained by the tone conversion by the probability that each pixel belongs to the shadow area, and each pixel is added to the edge-preserving smoothed image from the edge-preserving smoothed image luminance gradation converting means. Is calculated as a value obtained by multiplying the output obtained by performing the luminance gradation conversion or the output of the edge-preserving smoothed image as it is, by multiplying the probability that each pixel belongs to the sunny area, The high-frequency components of each frequency from the component conversion means are converted by assuming that they belong to the shadow area, and the output is multiplied by the probability that the corresponding edge-preserving smoothed image pixel belongs to the shadow area. Calculated the value The probability that the pixels of the corresponding edge-preserving smoothed image belong to the sunlit area with respect to the output obtained by performing the conversion on the assumption that all the high frequency components of each frequency from the frequency component conversion means belong to the sunlit area. An image correction apparatus, which calculates a multiplied value and adds the calculated four types of values. 38. The image correction apparatus according to claim 34, wherein the image synthesizing unit uses the probabilities belonging to each region calculated by the probability calculating unit, and the edge-preserving smoothed image. Two types of output from the brightness gradation conversion means and two from the high frequency component conversion means
As a process for creating an image by integrating outputs of various types, it is assumed that each pixel of the edge-preserving smoothed image brightness gradation conversion unit belongs to a shadow area, assuming that each pixel belongs to a shadow area. The output that has been subjected to the key conversion and the output that has been converted assuming that all the high frequency components of each frequency from the high frequency component conversion means belong to the shadow region, and add for each corresponding element, A value is calculated by multiplying the value by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the shadow region, and the value is calculated for the edge-preserving smoothed image from the edge-preserving smoothed image luminance gradation conversion means. Assuming that each pixel belongs to the sunlit area, the output that has been subjected to the luminance gradation conversion or the output of the edge-preserving smoothed image as it is, and all the sunlit parts for the high frequency components of each frequency from the high frequency component conversion means. Belongs to the realm of Assuming that the converted output or the output as it is, is added for each corresponding element, and the value is multiplied by the probability that the pixel of the corresponding edge-preserving smoothed image belongs to the sunlit area. Then, the image correction apparatus is characterized by adding the calculated four types of values. 39. The image correction apparatus according to claim 34, wherein the probability calculation means calculates the probability that each pixel of the original image belongs to a sunlit area, edge preservation. Calculate the probability that the information of the corresponding pixel in the smoothed image will occur from the sunlit area, and that value and the value that was input in advance, which is the average number of pixels in the sunlit area in the edge-preserved smoothed image. As a means of performing the process of multiplying by the ratio and calculating the probability that each pixel of the original image belongs to the shadow area, the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the shadow area is calculated. The image correction apparatus is characterized by performing a process of multiplying the value by a ratio of the number of pixels of an average shadow area in the edge-preserving smoothed image, which is a value input in advance. 40. The image correction apparatus according to claim 39, wherein the information for calculating the probability that the information of the corresponding pixel in the edge-preserving smoothed image occurs from the sunlit area, and the corresponding information in the edge-preserving smoothed image. An image correction apparatus comprising: learning means for learning in advance information for calculating the probability that the information of the pixel to be generated from the shadow area. 41. The image correction device according to claim 34, wherein the probability density is replaced by a probability density, or a value obtained by transforming the probability, or a value obtained by transforming the probability density. An image correction device characterized by using. 42. The image correction apparatus according to claim 41, wherein when the probability calculating means uses a probability density instead of the probability, or the probability density is converted, the information of the pixel in the edge-preserving smoothed image is One-dimensional or multi-dimensional calculation with the number of dimensions representing pixel information is performed as the calculation processing of the probability density generated from the sunlit area and the calculation processing of the probability density generated from the shadow area of the pixel information in the edge-preserving smoothed image. An image correction apparatus, which calculates a probability density using a probability density function of the normal distribution. 43. The image correction apparatus according to claim 34, wherein the high-frequency component extracting means extracts high-frequency components of individual frequencies as high-frequency components of the highest frequency. On the other hand, it is extracted by obtaining the difference between the original image and the image in which the high frequency component of the highest frequency is deleted from the original image, and for the high frequency components of other frequencies, the high frequency higher than this frequency is extracted. An image with all components removed from the original image,
An image correction apparatus for performing extraction processing by obtaining a difference from an image obtained by deleting a high frequency component of the frequency from the image. 44. The image correction apparatus according to any one of claims 34 to 43, wherein the edge-preserving smoothed image creating means includes a structuring element having a size corresponding to a smoothing frequency for the image. An image correction apparatus characterized by using a means for performing a process in which morphological opening and closing are combined. 45. The image correction apparatus according to any one of claims 34 to 44, wherein the high-frequency component extracting means extracts high-frequency components of individual frequencies from the image in the process of extracting high-frequency components of individual frequencies. As a means for deleting the component, a structuring element having a size corresponding to the frequency is used for the image to operate the morphology.
An image correction apparatus using a means for performing a process in which Ning and closing are combined. 46. The image correction apparatus according to claim 44 or claim 45, wherein the edge-preserving smoothed image creating means not only creates an image in which high-frequency components of respective frequencies are deleted from the image,
An edge-preserving smoothed image is created by performing edge-preserving smoothing on the original image, and the pixels of the corresponding edges of the edge-preserving smoothing image and the edge-preserving smoothing image are compared to determine the luminance of the latter. An image correction apparatus using means for converting the latter luminance so as to match the luminance of and outputting it as a new edge-preserving smoothed image. 47. Claims 1 to 10 and 2.
An image correction method, characterized in that a program for causing a computer to execute the steps of the image correction method according to any one of claims 1 to 33 is recorded on a computer-readable recording medium. recoding media. 48. An image photographing device, characterized in that the image correction device according to any one of claims 11 to 20 and 34 to 46 is incorporated. 49. An image display device comprising the image correction device according to any one of claims 11 to 20 and 34 to 46.