JP2006254487A - Image processing method and apparatus, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a high-quality output image with a little processing load and further to prevent, as much as possible, troubles caused by image processing of a special image. <P>SOLUTION: The image processing method includes creating a histogram based on pixel data of a source image, detecting pixel data corresponding to a predetermined frequency cumulatively from a predetermined pixel value, and performing image correction processing based on the detected pixel data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method, apparatus, and recording medium that perform image correction processing according to an image histogram.

  With the recent increase in the speed of personal computers and the acceleration of memory capacity, and the spread of digital cameras and photo scanners in particular, it has become easy for the general public to digitize photo images. There are increasing opportunities to handle photographic images as digital data. In addition, output devices such as ink jet printers are also increasing in demand for high-quality images and low prices, so that ordinary users can easily output photos at home.

  However, there are many cases in which the photographic-tone input image is accompanied by image deterioration factors such as color cast for the following reasons. For example, taking a digital camera as an example, an image captured by a CCD camera may sometimes capture wavelengths such as infrared light that cannot be felt by human eyes. Of course, processing such as an infrared cut filter is also performed, but it is not always perfect, and there are restrictions on the CPU and the real-time processing that are equipped with color balance correction that is performed inside the camera. As a result, the overall color balance is destroyed. Sometimes it ends up. It is also known that humans correct (adapt) the difference in light source color and see the object, but the camera records the light source difference on the film as it is, so even if the camera accurately reproduces the color measurement. Sometimes it looks like it's covered. The same phenomenon may occur with photo scanners and flat head scanners, so even if it is the best with negatives and reversal film, color balance may occur when converting this to digital data. obtain.

  In addition, not only digital cameras but also general cameras, for example, when a scene where a clear blue sky is included in the majority of the background is automatically shot, AE will work, the exposure will be underexposed and it will become dark overall. In some cases, the subject is not always in the best state. In order to improve such a state at the time of output, various color correction processes have been proposed in various fields, particularly in the field of photographic printing. That is, when a film photographed with a silver salt camera is printed in a laboratory, a function of analyzing a scene by image analysis of a photographed image and performing automatic correction is generally included in a stage of printing on photographic paper.

  On the other hand, when the RGB signal on the input device side is printed by an output device such as an ink jet printer via a personal computer, for example, Apple's Macintosh is a color sink, and Microsoft's Windows (registered trademark) is ICM. Color matching between the input device side and the output device side via the CIE XYZ color space has been attempted. However, it is very difficult to make these adjustments strictly. This is because the color reproduction range is naturally different between the input device side and the output device side, and the input device side is a light emission signal of R, G, B, and the output device side is a reflection original of C, M, Y, K. This is because there is a fundamental difference. In addition, if the image from the input device side is not satisfactory in the first place, there is a problem that the user is not satisfied even if it can be faithfully reproduced as a printed matter. In other words, it is necessary to correct the input image data itself in order to obtain a good output result for images that are not well-exposed or for images where the color balance of the entire image is lost due to color cast. In addition, there is a need for a simple method that does not bother the user and is sufficiently acceptable as a processing speed. In any case, the problem is that a certain algorithm for adjusting the color balance has not yet been developed.

  On the other hand, some images should not be corrected. For example, when a photograph is taken with a color filter representing an LB filter, this is nothing but the intentional color cast caused by the photographer. In addition, it is necessary to avoid correcting the color of sepia-tone image data that has recently caused a big boom. Therefore, image correction that removes the color cast should not be applied to an image that aims at such a special effect.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to enable highly accurate nonlinear color balance correction with a simple configuration.

  It is another object of the present invention to make a simple determination on an image that the photographer intentionally aims for a special effect and not to perform image correction processing according to the histogram.

  The present invention is an image processing method for performing image correction processing based on highlight points and shadow points of an original image, creating a histogram relating to the hue of the original image, and based on the shape of the histogram relating to the created hue, It is determined whether or not to perform image correction processing on the original image, and the image correction processing is controlled according to the determination result.

  According to the present invention, highly accurate nonlinear color balance correction can be performed with a simple configuration.

  In addition, it is possible to easily determine an image for which a photographer intentionally aims for a special effect and not to perform image correction processing according to the histogram.

(Embodiment 1)
In the image processing method according to the present embodiment, RGB original image data is converted into data indicating luminance and color difference (chromaticity) indicating brightness for each pixel. Then, a luminance histogram is created, and the luminance position (luminance value) at which the cumulative frequency values from the high luminance side and the low luminance side respectively reach a prescribed frequency is determined as a highlight point (white position) and a shadow point (black position). At this time, in order to improve the accuracy of highlight point and shadow point detection, processing that does not include high saturation pixels in the luminance histogram may be performed. Further, when the data size of the original image exceeds a predetermined value, the histogram may be created by selecting pixels while thinning them out.

  In this embodiment, the color balance is corrected by paying attention to the highlight point and the shadow point in the image. That is, there is a possibility that an image with the correct color balance has a color difference of 0 at the highlight point and a maximum brightness, that is, “white”, and a color difference of 0 at the shadow point and a brightness of 0, that is, “black”. high.

Therefore, the outline of the color balance correction of the color image processing apparatus of the present embodiment is as follows: (1) The average value of the color difference amounts at the highlight points in the obtained image in a color space with the color difference signal and the luminance as three axes. The color solid axis (described later) of the image is formed using the average value of the color difference amounts at the shadow points and the respective luminances, and correction is performed so that the color solid axis coincides with the luminance axis by rotation and translation. .
(2) In this case, the contrast and saturation of the image are adjusted by enlarging or reducing the color solid representing the image.

  Hereinafter, this embodiment will be described in detail with reference to the drawings.

  An example of the outline of the system in this embodiment is shown in FIG. For example, a printer 105 such as an ink jet printer and a monitor 106 are connected to the host computer 100. The host computer 100 includes application software 101 such as a word processor, spreadsheet, and Internet browser, an OS (Operating System) 102, and various drawing command groups (image drawing commands and text drawing) indicating output images issued to the OS 102 by the application. The printer driver 103 that generates print data by processing commands and graphics drawing commands) and the monitor driver 104 that processes various drawing commands issued by the application and displays them on the monitor 106 are provided as software.

  The host computer 100 includes a central processing unit CPU 108, a hard disk driver HD 107, a random access memory 109, a read-only memory ROM 110, and the like as various hardware capable of operating these software.

  As an embodiment shown in FIG. 1, for example, a Windows (registered trademark) 95 of Microsoft Corporation is used as an OS on a personal computer of IBM AT compatible which is widely used, and an arbitrary printable application is installed, and a monitor is installed. A form in which a printer is connected can be considered as one embodiment.

  In the host computer 100, based on the display image displayed on the monitor, the application 101 classifies text data classified into text such as characters, graphics data classified into graphics such as graphics, and natural images. Output image data is created using image image data or the like. When the output image data is printed out, the application 101 issues a print output request to the OS 102, the graphics data portion is a graphics rendering command, and the image image data portion is a rendering command indicating an output image composed of an image rendering command. The group is issued to the OS 102. The OS 102 receives an application output request and issues a drawing command group to the printer driver 103 corresponding to the output printer. The printer driver 103 processes the print request and drawing command group input from the OS 102, creates print data that can be printed by the printer 105, and transfers the print data to the printer 105. When the printer 105 is a raster printer, the printer driver 103 sequentially performs image correction processing on the rendering command group from the OS 102, sequentially rasterizes it into the RGB 24-bit page memory, and rasterizes all the rendering commands before the RGB 24 A raster image is generated on the bit page memory. Then, CMYK data depending on the color reproducibility of the printer is generated for each pixel and transferred to the printer 105.

  Processing performed by the printer driver 103 will be described with reference to FIG.

  The printer driver 103 performs an image correction process, which will be described later, on the image correction processing unit 120 on the color information included in the drawing command group input from the OS 102. First, the printer correction processing unit 121 rasterizes a rendering command based on the color information subjected to the image correction process, and generates a raster image on the RGB 24-bit page memory. Then, CMYK data depending on the color reproducibility of the printer is generated for each pixel and transferred to the printer 105.

  Processing performed by the image correction processing unit of the present embodiment will be specifically described with reference to FIG. For the original image indicated by the image drawing command, the color balance deviation is corrected by the following processing. The following processing is not performed on the original image indicated by the drawing command other than the image drawing command.

  First, in step S10, a histogram of the original image is created, and highlight points and shadow points are determined based on the created histogram.

(Determination of Highlight Point and Shadow Point) In order to determine the highlight point and shadow point of the image, the luminance Y and the color difference signals (C1, C2) are obtained for each pixel belonging to the image. In the present embodiment, luminance and color difference signals are obtained by the following equations.
Y (luminance) = 0.30R + 0.59G + 0.11B
C1 (color difference) = R−Y
C2 (color difference) = BY
Here, since the pixel data RGB handled in this embodiment is 8-bit (256 gradations) data, the luminance Y is also converted to a depth of 256. The luminance histogram is created by calculating how many pixels each have 256 luminance values from 0 to 255.

  In addition, since the calculated values of C1 and C2 are used as data for calculating the average color difference amount at pixels belonging to the luminance values of the highlight point and the shadow point at the time of the subsequent color fog correction, in this embodiment, as follows: Hold data. Three members of frequency, C1 accumulated value, and C2 accumulated value are set in the structure array variable format from 0 to 255, and the calculation result for each pixel is reflected (incremented) in each member.

  A luminance histogram is created for the image to be corrected in this way, and a highlight point and a shadow point are determined from the luminance histogram. In the present embodiment, for image data of 350,000 pixels, for example, 3500 corresponding to about 1% is set as a threshold value, and the accumulated luminance value numerical value from each end of luminance value 0 and luminance value 255 to the center direction. Are defined as shadow points and highlight points, respectively.

Specifically, when the frequency of a pixel with luminance n is set to Yn, the cumulative frequency is obtained as Y0 + Y1 +. k). Next, an average color difference amount at the pixel having the luminance of Yk is obtained. As described above, since the accumulated value of the color difference signal of each luminance value is calculated at the time of creating the luminance histogram (the accumulated color difference of the pixel of luminance n is C1total, C2total), the average color difference amount at the shadow point luminance value Yk C1k and C2k are obtained as follows.
C1k = C1ktotal / Yk
C2k = C2ktotal / Yk
Similarly, the cumulative frequency is obtained as Y255 + Y254 +..., The luminance value (Yw) when the cumulative frequency exceeds 3500 is set as the luminance value (w) of the highlight point, and the average color difference amount of the pixel of the luminance value Yw C1w and C2w are obtained.
C1w = C1wttotal / Yw
C2w = C2wttotal / Yw
As described above, the highlight point (C1w, C2w, Yw) and the shadow point (C1k, C2k, Yk) can be obtained in the “C1, C2, Y” color space.

  In the present embodiment, the cumulative frequency is obtained from the luminance positions of the luminance value 0 and the luminance value 255, but may have a predetermined offset such as obtaining from the luminance value 1 and the luminance value 254.

  The method for determining the highlight point and the shadow point is not limited to the above method.

  For example, as described in JP-A-60-57594, a cumulative frequency histogram is created for a luminance signal obtained by weighted addition of each color signal R, G, B of an input signal, and a predetermined cumulative value set in advance in the cumulative frequency histogram is described. For example, a method may be used in which the upper limit value of the luminance signal corresponding to the frequency, for example, 1% and 99%, is used as the highlight point and the lower limit value is used as the shadow point.

  Alternatively, a histogram may be created by generating a signal indicating brightness using another calculation instead of the luminance signal.

  Next, in S20, image correction processing based on the highlight point and shadow point determined in S10 is performed. In the present embodiment, as image correction processing, color cast correction for correcting the color cast of the original image, contrast correction for adjusting the luminance contrast of the original image, and saturation correction for improving the appearance of the image are performed.

(Image correction processing)
As described above, color fog correction and luminance contrast correction are performed based on highlight points and shadow points in the “C1, C2, Y” color space of the original image.

  If the original image has no color cast and is an ideal image, the achromatic color is R = G = B, and the calculated value of the color difference amount between the highlight point and the shadow point is 0. In other words, considering a color solid image diagram representing an image, it can be said that the image is ideal when it is in a state as shown in FIG.

However, if there is a color cast, a straight line (color solid) connecting (C1w, C2w, Yw) and (C1k, C2k, Yk) in proportion to the degree of fog in the direction of the hue as shown in FIG. A tilt occurs in the axis. Color fog correction can be achieved by converting the color solid representing the original image so that the color solid axis and the Y axis coincide. This can be achieved by rotating and translating the color solid, or by changing the coordinate system. In the present embodiment, first, in the color solid of the original image, the color solid axis is moved to a coordinate system whose origin is the shadow point of the color solid axis, and in that coordinate system, the color solid axis is parallel to the Y axis centering on the shadow point, that is, the origin. Rotate to become
Since a method for obtaining a rotation matrix for rotating a coordinate system at a desired angle in a system in which a rotation axis and a rotation angle are determined in a three-dimensional space is a known technique, detailed description thereof is omitted.
Further, the translation is performed so that the rotated color solid axis overlaps the Y axis.

  In this way, the color cast of the original image can be corrected by correcting the color solid of the original image.

  Next, as adjustment of the brightness contrast, in this embodiment, the brightness of the shadow point is “0” or a value close thereto (eg “10”), and the brightness of the highlight point is “255” or a value close thereto (eg “245”). ) To adjust. This corresponds to enlarging (reducing) the color solid caused by the rotation in the luminance axis direction.

  The saturation correction can be performed as follows. In the first place, the saturation is equivalent to the distance when the color difference signal is used. For example, if the saturation is to be uniformly increased by 20%, C1 and C2 are each 1.20 times (where 1.20 is the saturation). This is achieved by a degree coefficient). That is, the color solid is nothing but enlargement (reduction) in the C1, C2 plane direction.

  Therefore, in this embodiment, the enlargement (reduction) rate for adjusting the luminance contrast is applied to the entire color solid, and the saturation correction is automatically performed along with the luminance contrast correction.

  In this way, by changing the degree of saturation correction according to the degree of brightness contrast adjustment, brightness and saturation correction suitable for an image can be automatically performed. The enlargement ratio for adjusting the luminance contrast is obtained from the ratio of the luminance range before and after conversion.

  Of course, if the user desires a desired saturation adjustment, the saturation coefficient may be given and brightness correction and saturation correction may be performed as separate processes.

  As described above, by converting each pixel of the original image into luminance and color difference signals (C1, C2, Y) and rotating, translating and enlarging (reducing) the color solid in the three-dimensional color space, ( C1 ′, C2 ′, Y ′), and color fog correction, contrast correction, and saturation correction can be performed simultaneously or separately.

Finally, reverse conversion to RGB signals is performed again to complete the correction. The inverse transformation is performed according to the following equation.
R ′ = Y ′ + C1 ′
G ′ = Y ′ − (0.3 / 0.59) × C1 ′ − (0.11 / 0.59) × C2 ′
B '= Y' + C2 '
As described above, a histogram is created based on pixel data of an original image, pixel data corresponding to a predetermined frequency is detected by accumulating from predetermined pixel values, and a color in a three-dimensional color space is detected based on the detected pixel data. By correcting the balance, it is possible to provide an image processing method that can reliably correct the color cast, contrast, and saturation with a small processing load.

(Embodiment 2)
Next, a second embodiment according to the present invention considering the degree of correction as compared with the first embodiment will be described.

  When the inclination of the color solid axis of the original image described in the first embodiment is too large, a problem may occur in the image if this is corrected forcibly. For example, there may be a case where a color cast is intentionally caused using a color filter representing an LB filter or a case where a special scene such as a sunset is photographed.

  In such a case, it is determined that the original image is a special case and correction is not performed, or the rotation angle is appropriately adjusted to reduce the correction degree, thereby eliminating the problem.

  For example, for an image aiming at a special effect such as an LB filter, the orthogonal projection of the color solid axis onto the color difference signal plane is directed to the hue direction of red, for example. The determination of which hue color is covered in the original image can be easily made from the direction of the color solid axis.

  Therefore, paying attention to the angle formed by the direction vector of the color solid axis and the luminance axis, for example, when the direction vector indicates a red expression bidirectional, or when the angle is 40 degrees or more, for example, the original image is a special image. It is determined that this is the case, and the processing is not performed or the degree of processing is relaxed.

  For example, the characteristics of various color filters are examined in advance, the characteristic values (orthographic projection hue and angle) are stored, and the characteristic values are compared with the characteristic values of the original image, so that images of a plurality of color filters are obtained. Can be detected.

  The degree of correction may be specified manually by the user, or may be set in advance according to the size of the angle and the direction of the color solid axis. For example, in the case of 40 degrees, the degree of processing can be adjusted by rotating and rotating the color solid of the image by 20 degrees.

  In addition, by making it possible to register the characteristic value of the color filter, the detection accuracy can be improved. As a characteristic value registration method, for example, a registration mode is provided. When the registration mode is set, the characteristic values of a plurality of original images using color filters for registering characteristic values are obtained and averaged, and the color filter name and It is only necessary to register correspondingly.

  As described above, according to the present embodiment, at least two threshold values for the correction angle are set based on the inclination of the color solid axis, that is, the direction of the axis and the magnitude of the rotation angle, and the image should be corrected. It is possible to determine whether or not to adjust the correction degree, and it is possible to reject only the harmful effects associated with image correction for each special case very simply.

  By incorporating the special case detection process of this embodiment between step S10 and step S20 of FIG. 3, it is possible to prevent the image correction process described in the first embodiment from being performed on the special case.

(Embodiment 3)
Next, as compared with the first and second embodiments described above, a third embodiment having a configuration for determining whether or not correction should be performed will be described.

In analyzing the image, a histogram relating to the hue of each pixel is separately created from the color difference signal of each pixel as shown in FIG. Here, the processing load can be reduced by thinning out the pixels to be sampled to such an extent that the tendency of the entire image is understood. For example, if the number of pixels is about the same as that of a thumbnail image, it can be sufficiently recognized as a photo image. In the present embodiment, the hue H is obtained by the following equation.
H = atan (C1 / C2) [0. . 360 °]
For example, when an image is taken using an LB filter, the hue of the image is limited to some extent over the entire image region due to the nature of the filter. Further, when photographing is performed using various color filters that allow only a specific range of wavelengths to pass, the tendency becomes more prominent. In addition, for sepia-like photos that have recently caused a big boom, in some sense intentional color fogging has occurred, so such images should not be corrected for color fogging. .

  Accordingly, in this case, the dispersion value is naturally reduced, so that a photograph aiming at such a special effect can be determined by setting a threshold value for this value. This is not limited to a photograph, and an original image (object) composed of a certain hue such as graphics is determined, and these images are also inconvenient due to the correction as shown in the first embodiment. Can be avoided.

  Furthermore, in the second embodiment, the method of discriminating the special image based on the angle of orthographic projection on the hue plane of the luminance axis and the distance thereof has been shown. However, if the condition is entangled with this variance value, the accuracy can be further improved. it can.

  In other words, when the orthogonal projection of the luminance axis enters a specific hue and the distance is a certain distance or more, it is possible to see whether or not the entire image is covered by looking at this dispersion value. .

  For example, when a picture is taken using a filter that passes only a certain wavelength, a constant hue is obtained over the entire image, so that it can be distinguished from an image that has a large color cast in a highlight or shadow area.

  In the present embodiment, a hue is obtained from the value of the color difference obtained in step S10 in FIG. 3, and a hue histogram is created separately from the luminance histogram.

  Then, a special case detection process is performed based on the hue histogram.

  When implemented in combination with the second embodiment, a hue histogram is created in addition to the luminance histogram in step S10 of FIG. 3, and a special case determination process based on the hue histogram is performed between steps S20 and S10. good.

  According to the third embodiment, it is determined whether the color cast is intentionally made by looking at the hue of the entire image. In such a case, the intention of the creator is not performed without performing image correction. It is possible to output in the same state.

(Other embodiments)
In the above-described embodiment, color fog correction is first performed in the image correction process, and so-called contrast adjustment for correcting underexposure and overexposure is performed, but the order is not limited. In addition, the highlight / shadow point determination method and the image correction algorithm are not limited to the above methods, and various other methods can be used.

  The present invention provides a software program for realizing the functions of the above-described embodiment in an apparatus or a computer in the system connected to the various devices so as to operate the various devices so as to realize the functions of the above-described embodiments. Implementations by supplying a code and operating the various devices according to a program stored in a computer (CPU or MPU) of the system or apparatus are also included in the scope of the present invention.

  In this case, the software program code itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, the program code are stored. The storage medium constitutes the present invention.

  As a storage medium for storing the program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) in which the program code is running on the computer, or other application software, etc. It goes without saying that the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the embodiment.

  Further, after the supplied program code is stored in the memory of the function expansion board of the computer or the function expansion unit connected to the computer, the CPU of the function expansion board or function storage unit based on the instruction of the program code However, it is needless to say that the present invention also includes a case where the function of the above-described embodiment is realized by performing part or all of the actual processing.

  Although the present invention has been described with the preferred embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims.

FIG. 2 is a block diagram illustrating a configuration of a system that performs color cast correction processing. FIG. 4 is a diagram illustrating a flow of image processing performed by a printer driver. The figure explaining the flow of the image process performed in an image correction process process part. FIG. 4 is a diagram for explaining the principle of color balance correction in an embodiment. The example of the histogram regarding the hue of each pixel of an original image.

Claims (5)

An image processing method for performing image correction processing based on highlight points and shadow points of an original image,
Create a histogram about the hue of the original image,
Based on the shape of the histogram relating to the created hue, it is determined whether to perform image correction processing on the original image,
An image processing method, wherein the image correction process is controlled according to the determination result.   The image processing method according to claim 1, wherein a variance of the histogram relating to the hue is obtained, and it is determined whether or not the image correction processing is performed based on the variance. A drawing command indicating the original image is input,
The image processing method according to claim 1, wherein the drawing command is analyzed to determine whether or not the original image is an image. Image correction processing means for performing image correction processing based on highlight points and shadow points of the original image;
Creating means for creating a histogram about the hue of the original image;
Determination means for determining whether or not to perform image correction processing on the original image based on a shape of a histogram relating to the created hue;
An image processing apparatus comprising: control means for controlling the image correction process according to the determination result. A recording medium for recording a program readable by a computer,
Create a histogram about the hue of the original image,
Based on the shape of the histogram relating to the created hue, it is determined whether to perform image correction processing on the original image,
A recording medium for recording a program for controlling an image correction process based on a highlight point and a shadow point of an original image according to the determination result.

Images