JP4909308B2 - Image processing method and apparatus - Google Patents

Description

  The present invention relates to an image processing method and apparatus for performing image correction on an original image, and a storage medium.

  Conventionally, there are two methods for adjusting the color balance of a photographic image: (1): adjusting the color balance before taking a picture, and (2): correcting the image after taking a picture. As an example of the method (1), there is, for example, color balance adjustment using a white balance switch of a video camera. In this method, white paper or the like is preliminarily photographed before the actual photographing is started, and the white color balance of the photographed image is adjusted.

  Method (2) is widely used in the field of printing, etc., but most of it depends on the intuition and experience of craftsmen.

  On the other hand, with the recent spread of digital cameras and photo scanners in particular, digitization of photo images has become easy for ordinary people. In addition, output devices such as inkjet printers have also been improved in image quality and price, so that ordinary users can easily output photos at home.

  However, when printing a digitized photo image, there are still many problems with the image quality of the output image.

  For example, when the RGB signal on the input device side is printed by an output device typified by an ink jet printer via a so-called personal computer or the like, for example, the color sync in the Macintosh (registered trademark) of Apple Inc., the Microsoft Inc. Color matching between the input device side and the output device side through the CIE XYZ color space has been attempted by CMS or the like in Windows (registered trademark). However, it is very difficult to make these adjustments strictly. This is because the color reproduction range is naturally different between the input device side and the output device side, and the input device side is a light emission signal of R, G, B, and the output device side is a reflection original of C, M, Y, K. This is because there is a fundamental difference.

  Even if the color matching between the input device and the output device was achieved by overcoming such difficulties, if the image from the input device was not satisfactory in the first place, the image could be reproduced exactly as a printed matter. However, an image that satisfies the user cannot be obtained. This is considered to be caused mainly by an overexposed or underexposed state of the image or a color balance of the entire image due to the phenomenon of “color cast”.

  For example, since AE (automatic exposure) is applied when shooting automatically with a camera, if the background contains a large amount of blue sky, it becomes dark as a whole, resulting in a so-called underexposed state. If the background is mostly dark, it may be overexposed, and the important subject may not always be in the best condition.

  Taking a digital camera as an example, an image is taken by a CCD camera, and therefore, a color having a wavelength that cannot be sensed by human eyes also enters as an image signal. If the signal is processed as a part of the RGB signal, the color that should not be visible becomes obvious and the color balance is lost. Of course, processing such as an infrared cut filter is performed, but it is not always perfect, and color balance correction is also limited by real-time processing. As a result, it cannot be processed and a phenomenon called “color cast” occurs, resulting in an overall color balance. May go wrong.

  The same phenomenon may occur with photo scanners and flat head scanners, so even if it is the best with negatives and reversal film, the color balance may be distorted when digitized. obtain.

  As described above, in order to obtain a good output result, it is necessary to correct the input image data itself to image data with an appropriate exposure in which color balance is achieved, and for that purpose, it does not bother the user. There is a need for a simple method that is sufficiently acceptable in terms of processing speed.

  The present invention has been made in view of the above-described conventional example, and an object thereof is to enable correction of color cast of an original image with a simple configuration.

In order to achieve the above object, the present invention has the following configuration. That is, the present invention detects a highlight point and a shadow point of the original image using an image obtained by reducing the number of pixels from the original image taken by the digital camera ,
Find the highlight point and shadow point color,
Based on the highlight point, the shadow point, and the color , calculate a matrix that at least rotates the highlight point, the shadow point toward an axis indicating brightness in brightness and color space ,
In the image processing method, color fog correction is performed on each pixel of the original image according to the matrix .

  According to the present invention, it is possible to correct a color cast of an original image with a simple configuration.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.

[First Embodiment]
In the image processing method according to the present embodiment, RGB original image data is converted into luminance data indicating brightness and chromaticity data indicating color for each pixel. At this time, if the saturation of the original image data exceeds a predetermined value, the pixels are sequentially selected while thinning out the pixels, and a luminance histogram is created. Then, the luminance positions (luminance values) at which the cumulative frequency values from the high luminance side and the low luminance side respectively reach predetermined frequencies are obtained as white positions (highlight points) and black positions (shadow points). At this time, in order to improve the accuracy of the white position and black position detection, a process of not including the high saturation pixel in the luminance histogram creation may be performed.

  Thus, the average value of the chromaticity of the pixel data having the luminance value determined as the white position and the average value of the chromaticity of the pixel data having the luminance value determined as the black position are calculated. A straight line connecting points is determined as the color solid axis (gray line) of the original image. The inclination of the color solid axis with respect to the luminance axis indicates that the color solid axis, which should be a change only in luminance, is accompanied by a change in chromaticity, that is, a color cast is generated. The color cast is corrected by performing conversion processing (rotation matrix calculation) on all the pixels so that the color solid axis is a normal position in the color space. Hereinafter, this embodiment will be described in detail with reference to the drawings.

<Configuration of image processing system>
An example of the system in this embodiment is shown in FIG. For example, a printer 105 such as an ink jet printer and a monitor 106 are connected to the host computer 100. The host computer 100 includes application software 101 such as a word processor, spreadsheet, Internet browser, OS (Operating System) 102, and various drawing command groups (image drawing commands, text drawing) indicating output images issued to the OS 102 by the application 101. A printer driver 103 that processes print commands and graphics drawing commands) and a monitor driver 104 that processes various drawing command groups issued by the application 101 and displays them on the monitor 106 are provided as software.

  The host computer 100 includes a central processing unit CPU 108, a hard disk driver HD 107, a random access memory RAM 109, a read-only memory ROM 110, and the like as various hardware capable of operating these software.

  As an example of the image processing system shown in FIG. 1, for example, a Windows-registered trademark 95 of Microsoft Corporation can be used as an OS for a PC-AT compatible personal computer manufactured by IBM, which can be used for printing. A form in which a desired application is installed and a monitor and a printer are connected is conceivable.

  In the host computer 100, based on the display image displayed on the monitor, the application 101 classifies the data into text data classified into text such as characters, graphics data classified into graphics such as graphics, and natural images. Output image data is created using image image data or the like. When the output image data is printed out, the application 101 issues a print output request to the OS 102, the graphics data portion is a graphics rendering command, and the image image data portion is a rendering command indicating an output image composed of an image rendering command. The group is issued to the OS 102. The OS 102 receives an application output request and issues a drawing command group to the printer driver 103 corresponding to the output printer. The printer driver 103 processes the print request and drawing command group input from the OS 102, creates print data that can be printed by the printer 105, and transfers the print data to the printer 105. When the printer 105 is a raster printer, the printer driver 103 sequentially performs image correction processing on drawing commands from the OS, sequentially rasterizes them into an RGB 24-bit page memory, and rasterizes all the drawing commands and then RGB 24-bits. The contents of the page memory are converted into a data format printable by the printer 105, for example, CMYK data, and transferred to the printer.

  Processing performed by the printer driver 103 will be described with reference to FIG.

  The printer driver 103 performs an image correction process (to be described later) in the image correction processing unit 120 on the color information of the image drawing command included in the drawing command group input from the OS 102. First, the printer correction processing unit 121 rasterizes a drawing command based on the color information subjected to the image correction process, and generates dot image data on the RGB 24-bit page memory. Then, masking processing, gamma correction processing, quantization processing, and the like corresponding to the color reproducibility of the printer are performed on each pixel, and CMYK data depending on the printer characteristics is generated and transferred to the printer 105.

<Image correction processing unit>
Next, processing performed by the image correction processing unit 120 on the original image indicated by the image drawing command will be described with reference to FIGS. The following processing is not performed on the original image indicated by the graphics drawing command or the text drawing command.

  As shown in FIG. 3, the image correction processing unit 120 of the present embodiment performs a histogram creation process (FIG. 3-S20) and an image correction process corresponding to the histogram (FIG. 3-S30). In step S20, a histogram is created by a process as shown in FIG. Then, the highlight point and shadow point of the image are determined based on the created histogram.

(Create brightness histogram)
FIG. 4 is a flowchart for creating a luminance histogram in the present embodiment.

In FIG. 4, when the routine for creating the luminance histogram of the original image is entered in S1, the selection ratio of the pixels used for creating the luminance histogram is determined from the pixels of the original image in S2. In the present embodiment, when the image data to be processed is 350,000 pixels, a luminance histogram is created for all pixels (selection ratio 1 (or 100%)). When image data having 350,000 or more pixels is input, pixel selection (sampling) is performed according to the ratio of the total number of pixels to 350,000 pixels. For example, when image data of 3.5 million pixels is input, the selection ratio is 3.5 million / 350,000 = 10, and a luminance histogram is displayed at a ratio of one pixel to 10 pixels (selection ratio 10 (or 10%)). create. In this embodiment, the selection ratio n is obtained by the following equation.
n = int (total number of pixels of target image data / reference number of pixels 350,000) (however, when n <1, n = 1 and n is a positive number).

  Subsequently, in S3, the counter for managing the line number is reset or set to a predetermined initial value, and in S4, the counter is incremented to be the line number of the target line.

In this embodiment, pixel thinning (sampling) is performed in units of lines. Therefore, in the case of a selection ratio n, if the remainder when the line number is divided by n is 0, pixels belonging to that line are processed. Select (S5-YES). For example, in the case of a selection ratio of 10, when the remainder when the line number is divided by 10 is 0, the line of interest for selecting the pixel belonging to that line as the processing target is thinned out, that is, it is not the processing target. In the case of a line, the process returns to S4. In the case of the processing target line, the process proceeds to S6, where attention is sequentially paid to pixels belonging to the target line, and luminance conversion and chromaticity conversion are performed on the target pixel. Luminance conversion and chromaticity conversion in this embodiment are performed by the following equations. The luminance and chromaticity conversion is not limited to the following formulas, and various formulas can be used.
Y (luminance) = int (0.30R + 0.59G + 0.11B) (Y is a positive number),
C1 (chromaticity) = RY, C2 (chromaticity) = BY.

In this embodiment, in order to improve the detection accuracy of the white position (highlight point) and the black position (shadow point), the saturation S of the target pixel is calculated by the following equation, and the saturation value (Sconst) is determined in advance. It is determined whether or not the pixel is large (S7). If the pixel is large, the pixel information is not reflected in the luminance histogram.
Saturation S = sqrt (C1 ^ 2 + C2 ^ 2).
Here, sqrt (x) is a function that gives the square root of x, and x ^ y represents x to the power of y.

  That is, if (S> Sconst), the process returns to S6, and the data of the pixel of interest is not reflected in the subsequent processing. This is because, as will be described later, the saturation at the white position is given by the average saturation of the high-luminance pixel group, and the value of the saturation is an error caused by the color cast. This is because it is better to exclude from highlight point calculation. The effect of this processing will be described with a specific example. For example, a yellow pixel (R = G = 255, B = 0) has a luminance Y of 226 and a saturation S of 227 from the above equation. That is, it can be seen that this pixel has extremely high luminance and has a sufficiently saturated color. Such a pixel is more likely to be mistaken in many cases if it is determined that it is originally a yellow pixel than it is determined that an achromatic pixel is colored yellow. If such high luminance and high saturation pixels are included in the luminance histogram, an error occurs in the detected white position. Therefore, in the present embodiment, a predetermined saturation (Sconst) is determined, and pixels having a saturation exceeding the predetermined saturation are not included in the luminance histogram. By doing so, it is possible to prevent an error from occurring in the white position detected by the high-saturation pixel and improve the accuracy of the white position.

  As described above, after the determination in S7, a luminance histogram is created for pixels that satisfy the condition (S ≦ Sconst) (S8). Here, since the pixel data RGB handled in the present embodiment is 8-bit (256 gradations) data, the luminance Y is also converted to a depth of 256. Therefore, the luminance histogram is obtained by counting how many pixels of each luminance value in 256 stages from 0 to 255 are counted.

  The calculated values of the chromaticities C1 and C2 are used as data for calculating the average chromaticity of the pixels having the respective luminance values at the time of the subsequent color fog correction. In this embodiment, the data is held as follows. . That is, three members of frequency, C1 accumulated value, and C2 accumulated value are set in the form of a structure array variable with an index range of 0 to 255, and the calculation result for each pixel is set to the luminance value of that pixel as an index. It will be reflected in each member.

  When the process for the target pixel is completed, it is determined whether or not the process for all the pixels on the target line has been completed (S9). If unprocessed pixels remain in the target line, the process returns to S6, and the processes after S6 are performed. repeat. When the processing of all the pixels in the target line is completed, it is determined in S10 whether an unprocessed line remains. If all lines are completed, the process ends in S11. If there is an unprocessed line, the process returns to S4. Move the line of interest to the next line and repeat the above process.

  By creating a luminance histogram while selecting the pixels of the original image data as described above, create a luminance histogram with the minimum number of pixels and taking into account improvements in accuracy when detecting the white and black positions later. can do.

(Determine white position (highlight point), black position (shadow point))
When the luminance histogram is completed, a white position (white point) and a black position (shadow point) are determined from the histogram. In the present embodiment, the points at which the cumulative luminance value is 1750 in the central direction from both ends of the luminance value 0 and the luminance value 255 in the luminance histogram are determined as the black position and the white position, respectively.

That is, if the frequency of the pixel having the luminance value Y is PY, the cumulative frequency is obtained as P0 + P1 +..., And the luminance value when the cumulative frequency exceeds 1750 is set as the luminance value YSD at the black position. Next, the average chromaticity of pixels with luminance YSD is obtained. As described above, when the luminance histogram is created, the cumulative value of the chromaticity of each luminance value is calculated (the cumulative chromaticity of the pixel of luminance N is C1Ntotal and C2Ntotal), so the pixel of the luminance value YSD that is the black position Average chromaticities C1SD and C2SD are obtained.
C1SD = C1YSDtotal / PYSD,
C2SD = C2YSDtotal / PYSD.

Similarly, the white position is determined. The cumulative frequency is obtained as P255 + P254 +... And the luminance value when the cumulative frequency exceeds 1750 is set as the luminance value YHL at the black position. Next, average chromaticities C1HL and C2HL of pixels with luminance YHL are obtained.
C1HL = C1YHLtotal / PYHL,
C2HL = C2YHLtotal / PYHL.

  By performing the above calculation, the white position (C1HL, C2HL, YHL) and the black position (C1SD, C2SD, YSD) can be obtained in the [C1, C2, Y] color space.

  In the present embodiment, the cumulative frequency is obtained from the luminance positions of the luminance value 0 and the luminance value 255. However, a predetermined offset may be given, for example, from the luminance value 1 and the luminance value 254.

  As described above, the white position (highlight point) / black position (shadow point) is determined in step S20 of FIG.

  Next, in S30 of FIG. 3, image correction processing based on the white position and black position determined in S20 is performed. In the present embodiment, as image correction processing, color fog correction for correcting the color cast of the original image, exposure correction for correcting the contrast of brightness to optimize the exposure of the original image, and improvement of the color of the output image. Perform saturation correction.

  FIG. 9 shows the flow of the correction process performed by the image correction processing unit 120 in step S30 of FIG. That is, first, a rotation matrix for color fog correction is obtained, and then the color balance (color fog) is corrected using the rotation matrix, and finally gamma conversion of the luminance signal is performed according to the state of image exposure. . Each of these processes will be described step by step.

(Color cast correction)
When the white position and the black position in the (C1, C2, Y) color space of the original image are obtained as described above, the color cast is subsequently corrected.

  If the original image has no color cast and is an ideal image, the achromatic color is R = G = B, and the calculated value of the chromaticity at the white position and the black position is “C1HL = C2HL = C1SD = C2SD = 0 ". However, when there is a color cast, it is inclined in a straight line (color solid axis) connecting (C1HL, C2HL, YHL) and (C1SD, C2SD, YSD) in proportion to the degree of the hue. Occurs. Color fog correction can be achieved by converting the color solid axis and the Y axis (luminance axis) to coincide. The conversion can be achieved by rotating and translating the color solid or by converting the coordinate system.

  In this embodiment, first, in the color solid of the original image, the color solid axis is rotated so that it is parallel to the Y axis, with the lowest luminance point (lower end point) of the color solid axis as the rotation center. Next, the coordinate system is converted so that the position of the lowest luminance point is the origin of the (C1, C2, Y) space. FIG. 5C shows the result of color fog correction performed on the color solid of FIG. 5B. With the above processing, a conversion result is obtained in which the lowest luminance point is the origin and the color solid axis coincides with the Y axis. FIG. 5A is a color solid showing the color distribution of ideal image data without color cast. By the above-described conversion, the converted color solid (FIG. 5C) is brought close to the ideal color solid (FIG. 5A).

  In the rotation conversion in which the color solid axis is rotated in parallel with the Y axis, the rotation axis and rotation angle of the rotation conversion can be easily determined from the coordinate values of the shadow point and the highlight point. Since a technique for obtaining a rotation matrix for rotating a solid at a desired angle around a desired rotation axis in a three-dimensional space is a known technique, this detailed description is omitted.

  As described above, each pixel of the original image is converted into pixel data (C1, C2, Y) in a three-dimensional color space with chromaticity and luminance as axes, and the image data is converted into black positions and white positions. By rotating and translating to a pixel data (C1 ′, C2 ′, Y ′) in which the solid color axis (gray line) connecting the lines coincides with the Y axis and the minimum luminance is the coordinate origin, Can be corrected.

(Adjustment of contrast and saturation)
Next, in order to realize further high quality of the image by adjusting the contrast and saturation, a method of simply determining whether the image is overexposed or underexposed and applying gamma correction to the luminance signal in accordance therewith will be described.

  The contrast is adjusted by adjusting the luminance at the black position (shadow point) to “0” or a value close thereto (for example, “10”), and the luminance at the white position (highlight point) to “255” or a value close thereto (for example, This is done by adjusting to “245”).

  Next, an embodiment in which overexposure / underexposure of an image is simply determined and gamma correction corresponding to the image data is performed will be described.

  First, the points where the color solid axis to be corrected and the luminance (Y) axis are the minimum distance, that is, T and T ′ in FIG. This can be easily obtained from the geometric relationship.

  Then, the contrast is adjusted so that the luminance component YT ′ after the color cast correction at the point T ′ becomes the luminance component YT at the point T. That is, as shown in FIG. 6, when (YT, YT ′) is a refraction point and the luminance Y ′ after color cast correction is smaller than YT ′, the luminance is corrected to Y ″ by a function given as a straight line a. If it is greater than ', it is corrected to Y "by a function given as a straight line b.

  Of course, the correction as given by the straight line l2 in FIG. 6 may be performed without using T and T '. When the color solid axis is parallel to the luminance axis, the points T and T ′ are not a pair, and when T and T ′ are outside the luminance range [0, 255], the point (YT , YT ′) cannot be the refraction point. In such a special case, correction may be made according to the straight line l2.

  The effect of correction using the two straight line closest points T and T 'particularly affects an overexposed or underexposed image. The overexposure is because the entire image is pulled to a bright place such as the sky. At this time, in an input device typified by a digital camera, high luminance color suppression is performed, and saturation of the high luminance portion is reduced. That is, when the color solid axis of an image subjected to high luminance color suppression is considered as a two-dimensional plane having saturation and luminance as axes as shown in FIG. Pixels close to. On the contrary, since the low luminance color suppression is applied to the underexposed image, the saturation is lowered in the low luminance portion as shown in FIG. 7B.

  Considering the luminance axis of the color solid in the luminance-saturation plane in an actual image, for example, an overexposed image is as shown in FIG. On the contrary, the under image is, for example, as shown in FIG. If the actual color solid is shifted from the original (ideal state) color solid by the influence of some shooting situation or input (A / D conversion), the positions of T and T ′ are It is considered to be the place with the smallest gap. Therefore, the present invention simply corrects the gray, that is, the overall brightness correction by returning this.

  Of course, this T is simply used as a means for simply determining whether the image is overexposed or underexposed, and an LUT (look-up table) for over and a LUT for over are prepared in advance, depending on the luminance component of the point T or T ′. Then, gamma adjustment of the luminance signal may be performed. For example, the contrast may be adjusted by a curve having the point (YT, YT ′) in FIG. 6 as an inflection point. Therefore, it is possible to easily determine whether the image is overexposed or underexposed based on the values of T and T ′. That is, if the luminance component of the point T ′, which is the point with the lowest saturation on the color solid axis, is higher than the luminance, the relationship between the luminance and saturation of the image shows a tendency as shown in FIG. On the contrary, if the luminance component at the point T ′ is lower than the low luminance, the luminance-saturation relationship of the image shows a tendency as shown in FIG. Therefore, in a high-luminance color-suppressed and low-luminance color-suppressed image, if the luminance component at the point T ′ is higher than the high luminance, the image tends to be overexposed, and the luminance component at the point T ′ is lower than the low luminance. If so, the image is considered underexposed.

On the other hand, the saturation adjustment can be easily performed by multiplying the color differences C1 and C2 by a saturation correction coefficient. For example, when the saturation is increased by 20%, the saturation after the correction is 120% before the correction, and therefore the saturation correction coefficient is calculated as 1.2. That is,
C1 ″ = 1.2 × C1 ′,
C2 ″ = 1.2 × C2 ′
Saturation correction can be performed as follows. This is (saturation) = sqrt (C1 ^ 2 + C2 ^ 2)
By being defined in

(Inverse conversion to RGB space)
The various corrections in this embodiment are thus completed. At this time, each pixel of the original image is in a state converted from the color signal data of (R, G, B) to the color space data of (C1 ″, C2 ″, Y ″), so again (R ′, G ', B') is converted back into color signal data, which is performed by the following equation.
R ′ = Y ″ + C1 ″,
G ′ = Y ″ − (0.3 / 0.59) * C1 ″ − (0.11 / 0.59) * C2 ″,
B ′ = Y ″ + C2 ″.

  In this way, RGB data in which the color cast, contrast, and saturation are corrected for the original image can be obtained.

  As described above, according to the present embodiment, color cast correction can be reliably performed with a small processing load.

  Further, according to the present embodiment, since the sampling condition is set according to the image data size of the original image, the histogram total frequency, the cumulative frequency for determining the white position and the black position, regardless of the input image, The relationship can be made almost constant. Therefore, it is possible to realize a good color cast correction.

  Furthermore, the luminance that is considered to be closest to the value of the original image is obtained by nonlinearly gamma-correcting the entire image so as to maintain the luminance at the point where the distance between the color solid axis and the luminance axis of the correction target image is the shortest. The contrast can be corrected while maintaining.

  Furthermore, it is possible to easily obtain an exposure state in which the image is overexposed or underexposed. Furthermore, a different table can be selected according to the exposure state and gamma correction can be performed.

  Note that sampling may be performed not on a line basis but on a column basis.

[Second Embodiment]
Next, a second embodiment in consideration of the degree of correction will be described with respect to the first embodiment described above.

  As described in the first embodiment, when the color solid axis of an image is obtained, if the inclination of the axis is too large, a problem may occur in the image if this is corrected forcibly. For example, a case where a color cast is intentionally caused using a color filter or a case where a sunset scene is photographed can be considered.

  In such a case, it is possible to eliminate the problem by judging that the obtained highlight point and shadow point are wrong and not performing correction, or by appropriately adjusting the rotation angle to weaken the correction degree. Such a determination that the highlight point and the shadow point are wrong can be made according to the direction of the color solid axis. Since it is possible to easily determine which color is covered from the inclination of the color solid axis, it is possible to determine that the color cast is not corrected for an image photographed using, for example, a color filter in order to obtain a special effect.

  In such a case, pay attention to the angle formed between the direction vector of the color solid axis and the luminance axis, and it is determined that inconvenience will occur if color fog correction is performed, or processing is not performed, or the degree of processing Loosen For example, when the color solid axis is oriented in the hue direction of red and the angle is a predetermined angle, for example, 40 degrees or more, it is determined that the image is originally a color cast image. When the degree of processing is loosened, the color cast is corrected by raising the color solid axis by a predetermined angle, for example, 20 degrees, or up to a predetermined angle, for example, until the inclination with respect to the Y axis becomes 20 degrees. The rotation matrix for this conversion can be easily obtained from the rotation axis and the rotation angle.

  The degree of correction may be manually specified by the user, or may be set in advance according to the size of the angle and the direction of the color solid axis. For example, in the case of 40 degrees, the color solid of the image is rotated by 20 degrees, but the degree of correction may be specified by the user. The subsequent processing is the same as in the first embodiment.

  In addition, since it can be easily determined which color is covered by the direction of the color solid axis, for example, for an image photographed using a color filter in order to obtain a special effect, the color direction of the filter is not corrected. Such a determination can also be made. That is, in such a case, a color that does not correct the color cast is designated, a rotation axis of rotation conversion is set along the direction of the color that is not corrected on the C1-C2 plane, and the rotation axis and the color solid are set. The color solid is rotated and converted until the plane including the axis l is parallel to the Y axis. In this way, the color cast can be corrected only for a specific color component.

  FIG. 10 shows a processing procedure for correcting an image so that the color solid is rotated to a separately designated angle when the color solid axis is inclined by a predetermined angle or more. First, a highlight / shadow point is determined in step S101, and it is determined in step S102 whether the inclination of the color solid axis is equal to or greater than a predetermined angle. If the predetermined angle has not been reached, the color cast is corrected in steps S104 and S105 by converting the color solid axis so as to coincide with the luminance axis in the same manner as in the first embodiment.

  On the other hand, if the color solid axis is inclined by a predetermined angle or more, a rotation matrix that rotates the color solid axis by 20 degrees toward the luminance axis is obtained in step S103, and in step S105, the color matrix is used by using the rotation matrix. Rotate the solid to correct the color cast. In this case, the angle of 20 degrees used in step S103 may be specified in any way.

  If the threshold of the color solid axis is larger than the first threshold (for example, 40 degrees), the color solid axis is raised to the extent that the color solid axis is not a complete gray line. (For example, 20 degrees), the rotation conversion is performed, and if it is between the first threshold value and the second threshold value (for example, 20 degrees), the rotation conversion is not performed and the second threshold value is used. If the inclination is small, rotation conversion can be performed so that the color solid axis coincides with the luminance axis. By doing so, an inconvenient correction is not performed on an image in which a color cast is intentionally generated.

  In this way, at least two threshold values for whether or not to correct are set from two amounts of the inclination of the axis of the color solid composed of pixels of the image data, that is, the direction of the axis and the angle of inclination. It is possible to determine whether the correction should be corrected or not, and the correction degree should be adjusted, so that only the harmful effects of a special case can be rejected very simply.

  Further, since it is possible to easily determine which color is covered according to the direction of the color solid axis, it is possible not to correct the color cast according to the color direction.

[Third Embodiment]
In the first and second embodiments, the color balance correction based on the highlight point and the shadow point of the image has been described. Examples of correction based on other reference points will be described below.

First, an average color difference amount ΔE at a highlight point and a shadow point is obtained.
ΔE = sqrt ((C1HL−C1SD) ^ 2 + (C2HL−C2SD) ^ 2).
If the color solid axis is parallel to the luminance axis, ΔE = 0 should be obtained, but if the color solid axis is inclined, the value is larger than 0. That is, FIG. 8 shows the E (saturation) -Y (luminance) plane.

  Next, prepare several sample luminances between highlight and shadow. Then, an average color difference amount is obtained as shown in FIG. 8 using pixels having a saturation smaller than ΔEn from pixels having luminance Yn in the image, for example. As ΔEn, a preset constant saturation may be used, or an average color difference amount limited to the inclination direction of the color solid axis may be obtained.

  For these several luminance points, a straight line is obtained in the least-squares manner with an average color difference, and processing based on the first embodiment is performed using this as a color solid axis.

  Alternatively, an approximate curve is obtained by using a B-Spline or the like with the obtained color difference and luminance as elements, and this curve is used as a luminance axis, that is, nonlinear color balance correction is performed so that the color difference is “0”. Also good.

  Although the present invention has been described with reference to the preferred embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims.

  By doing in this way, the line which should become a brightness | luminance axis | shaft can be obtained from the pixel sampled uniformly from not only the highlight and the shadow but the whole image. By converting the image data so that the obtained line coincides with the luminance axis, color cast correction reflecting the characteristics of the entire image can be performed.

[Other Embodiments]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), or a device (for example, a copier, a facsimile device, etc.) including a single device You may apply to.

  In addition, an object of the present invention is to supply a storage medium in which a program code of the procedure shown in FIGS. 3, 4, 9, and 10 for realizing the functions of the above-described embodiments is supplied to a system or apparatus. This can also be achieved by a computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on the instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

  Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes a case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram of an image processing device. 2 is a block diagram of a printer driver. FIG. It is a flowchart of the process sequence of an image correction process part. It is a flowchart of a luminance histogram creation procedure. It is a schematic diagram of a color solid showing luminance and chromaticity distribution. It is a graph which shows a nonlinear gamma conversion function. It is a figure which shows the characteristic of the overexposure and underexposure seen in the luminance-saturation plane. An example is shown. It is a figure which shows a mode that the color solid axis in 3rd Embodiment is determined. It is a flowchart of the control procedure of image correction. 10 is a flowchart of a control procedure for color fog correction according to the second embodiment.

Claims (27)

Using an image obtained by reducing the number of pixels from an original image taken by a digital camera, a highlight point and a shadow point of the original image are detected,
Find the highlight point and shadow point color,
Based on the highlight point, the shadow point, and the color , calculate a matrix that at least rotates the highlight point, the shadow point toward an axis indicating brightness in brightness and color space ,
An image processing method , wherein color fog correction is performed on each pixel of the original image according to the matrix . The color cast correction is not performed or the color cast correction is performed at a predetermined level according to a result of comparing an inclination of an axis connecting the highlight point and shadow point of the original image with a predetermined value. The image processing method according to claim 1. 2. The image processing method according to claim 1, wherein the color fog correction is not performed or the degree of color fog correction is reduced based on an angle of an axis connecting the highlight point and the shadow point of the original image.   2. The image processing method according to claim 1, wherein a hue for which the color cast correction is not performed is designated. In addition to the color fog correction, further image processing method according to claim 1, wherein said and performs hand adjustments to component indicating brightness of the original image. The image processing method according to claim 1, wherein, in addition to the color cast correction, a saturation of the original image is adjusted. An image with a reduced number of said pixels, claim, characterized in that the image was subjected to any of the processes except the high chroma pixels or predetermined chroma of at least the original image from the pixel thinning or the original image The image processing method according to claim 1. In addition to rotating the highlight point and the shadow point toward the axis indicating the brightness in the brightness and color space, the axis indicating the brightness point and the brightness in the color space. The image processing method according to claim 1, wherein the image processing method is moved toward the screen. Means for detecting highlight points and shadow points of the original image using an image obtained by reducing the number of pixels from the original image taken by the digital camera ;
Means for determining the color of the highlight point and shadow point;
Means for calculating a matrix that at least rotates the highlight point, the shadow point toward an axis indicating brightness in brightness and tint space based on the highlight point, the shadow point, and the tint ;
Means for correcting color fog according to the matrix on each pixel of the original image;
An image processing apparatus comprising: The color cast correction is not performed or the color cast correction is performed at a predetermined level according to a result of comparing an inclination of an axis connecting the highlight point and shadow point of the original image with a predetermined value. The image processing apparatus according to claim 9. The image processing apparatus according to claim 9, wherein the color fog correction is not performed or the degree of color fog correction is reduced based on an angle of an axis connecting the highlight point and the shadow point of the original image.   The image processing apparatus according to claim 9, wherein a hue for which the color cast correction is not performed is designated. The color fog correction in addition, further said original image The image processing apparatus according to claim 9, wherein the performing hand adjustments to component indicating brightness of. The image processing apparatus according to claim 9, wherein a saturation of the original image is adjusted in addition to the color fog correction. An image with a reduced number of said pixels, claim, characterized in that the image was subjected to any of the processes except the high chroma pixels or predetermined chroma of at least the original image from the pixel thinning or the original image The image processing apparatus according to claim 9. In addition to rotating the highlight point and the shadow point toward the axis indicating the brightness in the brightness and color space, the axis indicating the brightness point and the brightness in the color space. The image processing apparatus according to claim 9, wherein the image processing apparatus is moved toward the screen.   A computer program for causing a computer to execute the image processing method according to claim 1. Using an image obtained by reducing the number of pixels from an original image taken by a digital camera, a highlight point and a shadow point of the original image are detected,
Find the highlight point and shadow point color,
Based on the highlight point, the shadow point, and the color, calculate a matrix that at least rotates the highlight point, the shadow point toward an axis indicating brightness in brightness and color space,
Each pixel of the original image is color cast corrected according to the matrix,
An image processing method, wherein image signal conversion for a printer is performed on the corrected image data . In addition to the color fog correction, further image processing method according to claim 18, wherein said and performs hand adjustments to component indicating brightness of the original image. The image processing method according to claim 18, wherein the saturation of the original image is adjusted in addition to the color cast correction. An image with a reduced number of said pixels, claim, characterized in that the image was subjected to any of the processes except the high chroma pixels or predetermined chroma of at least the original image from the pixel thinning or the original image Item 19. The image processing method according to Item 18. In addition to rotating the highlight point and the shadow point toward the axis indicating the brightness in the brightness and color space, the axis indicating the brightness point and the brightness in the color space. The image processing method according to claim 18, wherein the image processing method is moved toward the image. Means for detecting highlight points and shadow points of the original image using an image obtained by reducing the number of pixels from the original image taken by the digital camera ;
Means for determining the color of the highlight point and shadow point;
Means for calculating a matrix that at least rotates the highlight point, the shadow point toward an axis indicating brightness in brightness and tint space based on the highlight point, the shadow point, and the tint;
Means for correcting color cast according to the matrix for each pixel of the original image;
Means for performing image signal conversion for a printer on the corrected image data;
An image processing apparatus comprising: The color fog correction in addition, further said original image The image processing apparatus according to claim 23, wherein the performing hand adjustments to component indicating brightness of. 24. The image processing apparatus according to claim 23, wherein a saturation of the original image is adjusted in addition to the color cast correction. An image with a reduced number of said pixels, claim, characterized in that the image was subjected to any of the processes except the high chroma pixels or predetermined chroma of at least the original image from the pixel thinning or the original image 24. An image processing apparatus according to item 23. In addition to rotating the highlight point and the shadow point toward the axis indicating the brightness in the brightness and color space, the axis indicating the brightness point and the brightness in the color space. 24. The image processing apparatus according to claim 23, wherein the image processing apparatus is moved toward the head.

Images