JP2003274427A - Image processing apparatus, image processing system, image processing method, storage medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of providing an excellent after-processing image at all times. <P>SOLUTION: An analysis means 110 analyzes conditional information 106 (conditional information at photographing of a photographed image 107) at acquisition of the image 107 included in image information and decides an algorithm for image correction processing to apply the image 107 on the basis of the result of analysis. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus used in, for example, an apparatus or system for executing image correction processing for printing out digital image data captured by a digital still camera and acquired by a printer. The present invention relates to an image processing system, an image processing method, a computer-readable storage medium storing a program for implementing the same, and the program.

[0002]

2. Description of the Related Art In recent years, for example, the spread of digital still cameras has facilitated the digitization of photographic images, and in particular, photographic images are digitalized on personal computers (hereinafter simply referred to as "personal computers"). Opportunities to handle as image data have increased. Furthermore, by using various application software on a personal computer, it is possible to easily perform processing and editing processing on digital image data which is a photographic image.

On the other hand, full-color hard copy technology is also rapidly developing, and in particular, in the ink jet printing technology, the image quality of the print output result is improved by improving the technology for reducing the graininess due to ink dots. It is becoming widespread due to the fact that it is becoming the same as the image quality in the above, and it is a relatively simple printing technology.

From the above technical background, it is required to easily print out digital image data obtained by photographing with a digital still camera. Also, regarding image correction processing at the time of print output, rather than manual image correction processing using complicated functions, by using application software on a personal computer, an automatic image that always obtains a good image corrected image can be obtained. The need for correction processing is increasing.

Therefore, in order to obtain a good print output result, as a method of performing image processing such as image correction processing and outputting when performing print output, for example, a scene of a photographed image is analyzed and the analysis result is obtained. Various methods have been proposed, such as a method for automatically performing image correction based on the above.

Further, for example, it relates to a so-called "density correction" which is an image correction for preventing a photographed image from being too bright (too light) or too dark (too dark) when printed out. A method has been proposed. Also,
A so-called "image correction" method has also been proposed for correcting an undesired image such as a color cast or poor exposure (brightness or contrast) in a captured image or poor saturation, and an image having a color balance failure due to a color cast or the like. There is.

In any of the "density correction" and "image correction" image processing methods, the target brightness is set for each brightness value of the brightness signal in the image (original image) to be processed as a structure for automatic image correction. A configuration is used in which an original image is analyzed by using a histogram in which the number of value pixels is accumulated, and the original image is corrected based on the analysis result.

As a function on the digital still camera side, not only is an image obtained by photographing taken recorded as digital image data in a storage medium such as a memory card, but also additional information indicating photographing conditions at the time of photographing is stored. It is possible to record together with digital image data on the recording medium.

[0009]

A scene of a target image is analyzed by performing image analysis of an image to be processed (target image) obtained by photographing with a digital still camera, and the target is analyzed based on the analysis result. When performing automatic image correction of images, basically, for all target images, automatic image correction is performed so that theoretically optimal images (standard images) are printed out. Become.

By the way, in the color correction and the brightness correction as described above, many of them are statistically image-processed by using the histogram of the image, and it is not necessarily optimum for all the images by only analyzing the image data. It is difficult to correct. More precise determination can be performed using white balance information for color correction and exposure information for brightness correction.

The above-mentioned automatic image correction is performed by the user.
Unlike manual image correction, which uses application software on a computer to manually correct the image while checking the target image displayed on the computer's monitor, automatic image correction to a standard image acquires the target image. There is a case where a correction contrary to the user's intention at the time of shooting (at the time of shooting) is performed.

For example, even for an image obtained by intentionally setting manually by a user and shooting under an exposure condition that makes the image brighter or darker, if the image is bright, the image is slightly darkened, and If a dark image is slightly brightened, the image is output as an image having appropriate brightness. In other words, whatever the target image is, the image is corrected to an image having the same brightness and is output.

In addition, even if the user intentionally changes the white balance of the digital still camera to obtain an image with a special effect, image correction is similarly performed so as to obtain the optimum color balance. .

On the other hand, when a plurality of image processes such as the exposure correction, the color balance correction, and the image optimization process based on the shooting scene information are performed independently, the same process is repeated. Sometimes, the points that should be emphasized are weakened in later processing. In other words, when performing a plurality of processes, it is necessary to avoid duplication of similar processes, and to consider which process is important and which process is weakened.

Similarly to the above, as digital still cameras have become more sophisticated in recent years, for example, special effect functions such as increasing the saturation or varying the contrast and brightness of an image obtained by photographing are provided. Although there is a camera provided, if the special effect function is used and the image correction process is executed, the special effect due to the special effect function is lost due to double correction, or the special effect correction is performed. Is too strong,
The image after processing was sometimes deteriorated.

Therefore, the present invention has been made to eliminate the above-mentioned drawbacks, and is capable of always providing a good processed image, an image processing apparatus, an image processing system, an image processing method, and to carry out the same. It is an object of the present invention to provide a computer-readable storage medium that stores the program, and the program.

[0017]

[Means for Solving the Problems] Under such a purpose,
A first aspect of the present invention is an image processing apparatus that performs image processing on image information including condition information at the time of image acquisition, and based on an analysis unit that analyzes the condition information and an analysis result of the analysis unit, The brightness correction amount of the image is calculated based on the value indicating the degree of brightness, the color correction amount of the image is calculated based on the value indicating the degree of color balance, and the smoothing correction amount of the image is calculated according to the value related to the zoom. And a processing determining means.

The second invention acquires a plurality of image information,
When performing a plurality of processes corresponding to them, it is characterized by setting a priority order for performing the plurality of processes.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

This embodiment is applied to, for example, an image printing system 100 as shown in FIG.

The image printing system 100 of the present embodiment
Is based on the analysis result of the information (additional information) indicating the shooting condition added to the image data to be processed obtained by shooting with the digital still camera 101, and automatically corrects the image of the image data. By implementing the method, it is possible to realize highly accurate automatic image correction and provide a high-quality photographic image print result that further reflects the user's intention at the time of shooting.

Hereinafter, the image printing system 1 of the present embodiment
00 will be specifically described.

<Structure of Image Printing System 100> As shown in FIG. 1, the image printing system 100 includes a digital still camera 101, an image processing device 108, and a printer 115.

The digital still camera 101 acquires the photographed image data 107 by the photographing operation, and at the same time, transmits the image data 105 including the photographing condition data 106 (additional information) and the photographed image data 107 in the photographing operation to the image processing device 108. The shooting condition setting unit 102, the shooting condition recording unit 103, and the shot image recording unit 104 are provided.

The shooting condition setting unit 102 sets various shooting conditions necessary for a shooting operation.

The photographing condition recording unit 103 converts the photographing condition data 106 set by the photographing condition setting unit 102 into image data 105 (data for output to the image processing apparatus 108).
Record inside.

The photographed image recording unit 104 records the photographed image data 107 acquired by the photographing operation according to the photographing condition set by the photographing condition setting unit 102 in the image data 105.

The image processing apparatus 10 for the image data 105
As a method of supplying the data to the recording medium 8, for example, a method of transferring data via a communication line, a method of recording on an arbitrary recording medium or a storage medium, and the like can be applied.

The image processing device 108 is composed of, for example, a personal computer, and by activating a predetermined application software, the digital still camera 101 is activated.
The image correction processing is performed on the captured image data 107 of the image data 105 from the above, and the printer 115 prints out.

Therefore, the image processing apparatus 108 includes the reader unit 109, the photographing condition analyzing unit 111, and the photographed image analyzing unit 1.
A data analysis unit 110 including 12 and an image correction processing unit 11
3 and the print data conversion unit 114, and the respective functions of these constituent units 109 to 114 are realized by starting the above-mentioned predetermined application software.

The reader unit 109 reads the image data 105 from the digital still camera 101. The data analysis unit 110 analyzes the shooting condition data 106 included in the image data 105 obtained by the reader unit 109 by the shooting condition analysis unit 111, and at the same time, the shot image analysis unit 112.
Thus, the captured image data 107 included in the image data 105 obtained by the reader unit 109 is analyzed, and the image correction algorithm is selected based on the analysis result.

The image correction processing unit 113 includes a data analysis unit 1
Image correction processing is performed on the captured image data 107 by the image correction algorithm selected in 10.

Regarding the selection (determination) of the image correction algorithm, specifically, for example, when the photographed image analysis unit 112 analyzes the luminance value and the luminance distribution from the signal value of the photographed image data 107, the photographed image is analyzed according to the analysis result. Image data 10
The characteristic of No. 7 is recognized, the optimum correction condition corresponding to this result is determined, and the image correction algorithm based on the determined condition is selected.

That is, the final image correction algorithm is selected by the shooting condition data 1 by the shooting condition analysis unit 111.
It is determined by the algorithm determined by the analysis result of 06 and the algorithm determined by the characteristic recognition of the captured image data 107 by the captured image analysis unit 112.

The image correction processing includes brightness correction processing, contrast correction processing, color correction processing, saturation correction processing, smoothing processing, contour enhancement processing, noise reduction processing and the like.

The print data conversion unit 114 outputs the photographed image data 107 after the correction by the image correction processing unit 113 in an appropriate format that can be printed by the printer 115 (for example, C
MYK data) and transfer the converted data to the printer 115 via a predetermined interface.

Therefore, the printer 115 prints out the data transferred from the print data conversion unit 114 of the image processing apparatus 108.

As the printer 115, for example, a serial scan type ink jet printer or the like can be applied.

In the present embodiment, the image processing unit 108
The constituent units 109 to 114 included in are implemented by activating application software running on a personal computer, but the configuration is not limited to this, and may be implemented by hardware, for example. More specifically, it may be realized in the form of a driver for the printer 115.

Further, for example, when a personal computer is used as the image processing unit 108, the image data 105
Is stored in a storage medium such as a hard disk of the image processing unit 108 or in a storage medium of another personal computer (including a server or the like) connected to the image processing unit 108, and this is stored in the image processing unit 108. You may make it process.

The transfer of the image data 105 from the digital still camera 101 to the image processing unit 108 (in the case where a personal computer is used as the image processing unit 108, the internal storage medium or the like) is as described above. A method using an arbitrary communication line, a recording medium, or a storage medium can be applied, and more specifically, for example, a card reader, a cable connection, infrared communication, or wireless communication can be applied. In this case, for example, the digital still camera 101 and the image processing unit 108
May be connected by cable, infrared communication, or wireless communication, and the image processing unit 108 may directly read the image data 105 from a memory card, a built-in memory, or the like held by the digital still camera 101.

As a system configuration of the image printing system 100, for example, with respect to the inside of the printer 115,
The function of the image processing apparatus 108 may be provided. In this case, it is not necessary to use a personal computer or the like as the image processing device 108. In this case, for example, in the printer 115, the image data 10
5 by a data reading means (corresponding to the function of the reader unit 109) such as a card reader provided in the printer 115.
You may make it read via recording media or memory media, such as a memory card. Alternatively, the digital still camera 101 and the printer 115 are connected by a wired cable, infrared communication, or wireless communication, and the printer 115
However, the image data 105 may be read from a memory card, a built-in memory, or the like held by the digital still camera 101.

FIG. 2 shows a flowchart for converting an RGB image signal into a Lab signal most suitable for image correction. 202
After performing the image correction processing in step 1, the L'a'b 'signal is returned to R'G'B' again, so that simple and effective correction can be performed. Alternatively, a simple YCbCr signal may be used instead of the Lab signal. In this embodiment, the YcbCr signal is used.

Next, the processing contents of the image correction processing unit 113 will be described. First, the information about white balance in the shooting condition data 106 is read, and if it is automatic shooting (auto mode), color balance correction (color correction)
I do.

On the other hand, the exposure information in the shooting condition data 106 is read, and if it is automatic shooting (auto mode), brightness correction (print density correction) is performed.

Further, the information regarding the digital zoom in the photographing condition data 106 is read, and if it is executed (ON), smoothing correction is performed on the image.

<Color Balance Correction> [First Embodiment] First, the processing contents of color balance correction will be described with reference to FIGS. 3 to 10.

As shown in FIG. 3, the image correction processing unit 113 of this embodiment has a histogram creating process (FIG. 3-S).
1) and image correction processing according to the histogram (Fig. 3-S
Perform 2). In step FIG. 3-S1, a histogram is created by the processing shown in FIG. Then, the highlight point and the shadow point of the image are determined based on the created histogram.

(Creation of Brightness Histogram) FIG. 4 is a flowchart for creating a brightness histogram in the present embodiment.

In FIG. 4, when the routine for creating the luminance histogram of the original image is entered in FIG. 4-S1, the selection ratio of the pixels used for creating the luminance histogram from the pixels of the original image is determined in FIG. 4-S2. In the present embodiment, when the image data to be processed is 350,000 pixels, all pixels are targeted (selection ratio 1
(Or 100%)), a luminance histogram is created. When the image data having the number of pixels of 350,000 pixels or more is input, pixel selection (sampling) is performed according to the ratio of the total number of pixels to 350,000 pixels. For example, when 3.5 million pixel image data is input,
The selection ratio is 3.5 million / 350,000 = 10, and the luminance histogram is created at a ratio of 1 pixel to 10 pixels (selection ratio 10 (or 10%)). In this embodiment, the selection ratio n
Is calculated by the following formula. n = int (total pixel number of target image data / reference pixel number 3
50,000) (However, when n <1, n = 1, n is a positive number.) Then, in FIG. 4-S3, the counter for managing the line number is reset or set to a predetermined initial value, and in FIG. 4-S4. The counter is incremented to obtain the line number of the line of interest.

In the present embodiment, thinning out (sampling) of pixels is performed in line units. Therefore, in the case of the selection ratio n,
When the remainder when dividing the line number by n is 0, the pixel belonging to that line is selected as the processing target (FIG. 4).
S5-YES). For example, if the selection ratio is 10,
When the remainder when the line number is divided by 10 is 0, the line of interest for selecting the pixels belonging to that line as the processing target is thinned out, that is, when the line is not the processing target, it is shown in FIG. Return. In the case of the processing target line, the process proceeds to FIG. 4-S6, the pixels belonging to the target line are sequentially focused, and the luminance conversion and the chromaticity conversion are performed on the target pixel. Luminance conversion and chromaticity conversion in this embodiment are performed by the following formulas. Note that the luminance and chromaticity conversions are not limited to the following equations, and various equations can be used. Y (luminance) = int (0.30R + 0.59G + 0.
11B) (Y is a positive number) C1 (chromaticity) = RY C2 (chromaticity) = BY In this embodiment, the detection accuracy of the white position (highlight point) and the black position (shadow point) is improved. In order to do so, the saturation S of the pixel of interest is calculated by the following equation, and it is judged whether or not it is larger than a predetermined saturation value (Sconst) (FIG. 4-S7). Is not reflected in the brightness histogram. Saturation S = sqrt (C1 ^ 2 + C2 ^ 2) Here, sqrt (x) is a function that gives the square root of x, and x ^ y represents x to the power of y.

That is, in the case of (S> Sconst), the processing returns to FIG. 4-S6, and the data of the target pixel is not reflected in the subsequent processing. This is because, as will be described later, the saturation at the white position is given by the average saturation of the high-luminance pixel group, and the value of the saturation is an error caused by the color cast, so pixels that are originally considered to have high saturation. Is because it is better to exclude from the calculation of highlight points. The effect of this processing will be described with a specific example. For example, yellow pixels (R = G = 2
55, B = 0), the luminance Y is 226 and the saturation S is 227 from the above equation. That is, it can be seen that this pixel has extremely high brightness and has a sufficiently saturated color. In many cases, it is less erroneous to judge that such a pixel is originally a yellow pixel than it is to judge that the achromatic pixel has changed its color to yellow. Including such a high-luminance / high-saturation pixel in the luminance histogram causes an error in the detected white position. Therefore, in the present embodiment, a predetermined saturation (Sconst) is set, and pixels having a saturation exceeding the predetermined saturation are not included in the luminance histogram. By doing so, it is possible to prevent an error from occurring in the white position detected by the high-saturation pixel and improve the accuracy of the white position.

As described above, after the determination in FIG. 4-S7, a luminance histogram is created for pixels satisfying the condition (S ≦ Sconst) (FIG. 4-S8). Here, each of the pixel data RGB handled in this embodiment has 8 bits (2
Since it is data of 56 gradations, the luminance Y is also converted to a depth of 256. Therefore, the luminance histogram is from 0 to 255
It is possible to obtain by counting how many times there are 256 pixels of each luminance value up to.

Further, the calculated values of the chromaticities C1 and C2 are used as data for calculating the average chromaticity of the pixels having the respective brightness values at the time of the subsequent color fog correction. Therefore, in the present embodiment, the following data are obtained. Hold. That is, in the format of the structure array variable whose index range is 0 to 255, the frequency, C
Three members, one cumulative value and C2 cumulative value, are set, and the calculation result for each pixel is reflected in each member having the brightness value of that pixel as an index.

When the processing for the target pixel is completed, it is judged whether or not the processing for all the pixels on the target line has been completed (S9 in FIG. 4). Returning to S6, the processes of FIG. 4-S6 and subsequent steps are repeated. When all pixels in the line of interest have been processed,
In S10, it is determined whether or not an unprocessed line remains. If all lines are completed, the process ends in FIG. 4-S11. If unprocessed lines remain, the process returns to FIG. 4-S4, and the line of interest is set to the next line. And repeat the above process.

As described above, the luminance histogram is created while selecting the pixels of the original image data, so that the luminance can be minimized with the minimum number of pixels taken into consideration and the accuracy of the subsequent white and black positions can be improved. A histogram can be created.

(Determination of White Position (Highlight Point) and Black Position (Shadow Point)) When the luminance histogram is completed, the white position (white point) and black position (shadow point) are determined from the histogram. In the present embodiment, the cumulative luminance value is 175 from the both ends of the luminance value 0 and the luminance value 255 in the luminance histogram toward the center.
Points that become 0 are defined as a black position and a white position, respectively.

That is, assuming that the frequency of the pixel having the brightness value Y is PY, the cumulative frequency is calculated as P0 + P1 +.
The brightness value when the cumulative frequency exceeds 1750 is defined as the brightness value YSD at the black position. Next, the average chromaticity of the pixel having the luminance YSD is obtained. As described above, the cumulative value of the chromaticity of each brightness value is calculated when the brightness histogram is created (the cumulative chromaticity of the pixel of brightness N is C1Ntotal and C2Ntotal).
Therefore, the average chromaticity C1 of the pixel with the luminance value YSD at the black position
Obtain SD and C2SD. C1SD = C1YSDtotal / PYSD C2SD = C2YSDtotal / PYSD Similarly, the white position is determined. P255 + P254 + ...
Then, the cumulative frequency is calculated, and the brightness value when the cumulative frequency exceeds 1750 is defined as the brightness value YHL at the black position. Next, the average chromaticity C1HL, C2HL of the pixel having the luminance YHL is obtained. C1HL = C1YHLtotal / PYHL C2HL = C2YHLtotal / PYHL By performing the above calculation, the white position (C1HL, C2HL, YHL) and the black position (C1SD, C2SD, YSD) are obtained in the [C1, C2, Y] color space. be able to.

In this embodiment, the brightness value 0 and the brightness value 25
Although the cumulative frequency is obtained from the luminance position of 5, a predetermined offset may be given, for example, from the luminance value 1 and the luminance value 254.

As described above, the white position (highlight point) / black position (shadow point) is determined in step S1 of FIG.

Next, in S2 of FIG. 3, image correction processing is performed based on the white position and the black position determined in FIG. 3-S1. In the present embodiment, as the image correction processing, color cast correction for correcting the color cast of the original image, exposure correction for correcting the brightness contrast to optimize the exposure of the original image, and for improving the color appearance of the output image are performed. Perform saturation correction.

FIG. 9 is a block diagram of the image correction processing unit 113 shown in FIG.
7 shows the flow of the correction process performed in step S2. That is, first, a rotation matrix for color cast correction is obtained, and then the rotation matrix is used to perform color balance (color cast).
Is corrected, and finally gamma conversion of the luminance signal is performed according to the exposure state of the image. Each of these processes will be described in order.

(Color cast correction) As described above, (C
If the white position and the black position in the (1, C2, Y) color space are obtained, the color cast is subsequently corrected.

If the original image is an ideal image with no color cast, the achromatic color is R = G = B and the white position is
The calculated value of the chromaticity at the black position is “C1HL = C2HL = C1S
D = C2SD = 0 ”. However, if there is a color cast, (C1HL, C2HL, YHL) and (C1S
A straight line (color solid axis) connecting D, C2SD, and YSD) is tilted. The color cast correction can be achieved by performing conversion so that the color solid axis and the Y axis (luminance axis) match. The conversion can be achieved by rotating or translating the color solid, or by converting the coordinate system.

In the present embodiment, first, in the color solid of the original image, the color solid axis is rotated so as to be parallel to the Y axis with the lowest luminance point (lower end point) of the color solid axis as the center of rotation. Next, the coordinate system is transformed so that the position of the lowest luminance point becomes the origin of the (C1, C2, Y) space. By the above processing,
A conversion result in which the lowest luminance point is the origin and the color solid axis matches the Y axis is obtained. Note that FIG. 5A is a color solid showing an ideal color distribution of image data without color cast. By the above conversion, the color solid after conversion is an ideal color solid (see FIG.
(A)).

In the rotation conversion for rotating the color solid axis parallel to the Y axis, the rotation axis and the rotation angle of the rotation conversion can be easily determined from the coordinate values of the shadow point and the highlight point. Since a method of obtaining a rotation matrix for rotating a solid body around a desired rotation axis at a desired angle in a three-dimensional space is a known technique, its detailed description is omitted.

As described above, each pixel of the original image has pixel data (C1, C1) in a three-dimensional color space with chromaticity and luminance as axes.
2, Y), and the image data is converted into pixel data (C) in which the color solid axis (gray line) connecting the black position and the white position coincides with the Y axis and the lowest luminance is the coordinate origin.
1 ′, C2 ′, Y ′) can be rotated and translated to convert the color cast.

(Adjustment of Contrast and Saturation) Next, in order to realize higher image quality by adjusting the contrast and saturation, overexposure / underexposure of the image is simply determined, and the luminance signal is determined accordingly. A method of applying gamma correction will be described.

Note that this processing is mainly a part of color balance correction, and the exposure data included in the above-mentioned photographing condition data 106 is not taken into consideration, and the brightness correction considering the value of the exposure data described later is performed. Is different from.

To adjust the contrast, the brightness at the black position (shadow point) is adjusted to "0" or a value close to it (for example, "10"), and the white position (highlight point) is adjusted.
Brightness of "255" or a value close to it (for example, "2"
45 ").

Next, an example will be shown in which overexposure / underexposure of an image is simply determined and gamma correction is applied to the image data accordingly.

First, the point where the color solid axis to be corrected and the luminance (Y) axis have the minimum distance, that is, T in FIG.
Find T '. This can be easily obtained from the geometrical relationship.

Then, the contrast is adjusted so that the luminance component YT ′ after the color cast correction at the point T ′ becomes the luminance component YT at the point T. That is, as shown in FIG. 6, (YT ', Y
T) is a refraction point, and the luminance Y ′ after color cast correction is YT ′.
When it is smaller than YT ', the brightness is corrected to Y "by a function given as a line a, and when it is larger than YT', a line b
Is corrected to Y ″ by the function given as

Of course, without using these T and T ', FIG.
The correction as given by the straight line 12 may be performed. If the color solid axis is parallel to the luminance axis, points T, T '
Are not a pair, and T and T'are the luminance range [0,
255], the point (YT ′, YT) cannot be a refraction point. In such a special case, the correction may be made according to the straight line l2.

The effect of the correction using the two straight line closest points T, T'especially works on an image which is overexposed or underexposed. The overexposure is because the entire image is pulled in a bright place such as the sky. At this time, in an input device representing a digital camera, high-luminance color suppression is performed, and the saturation of the high-luminance portion is suppressed. That is,
The color solid axis of the image subjected to high-luminance color suppression is shown in FIG.
If a two-dimensional plane with the saturation and the luminance as axes is considered as shown in, the pixel having the closest achromatic color appears in the high-luminance portion. Conversely, low-luminance color suppression is applied to an underexposed image, so that the saturation is low in the low-luminance portion as shown in FIG. 7B.

Considering the luminance axis of the color solid in the actual image on the luminance-saturation plane, for an overexposed image, for example,
It becomes like FIG.7 (c). On the other hand, an under image is, for example, as shown in FIG. If you think that the actual color solid will be displaced from the original (ideal) color solid due to some shooting conditions or input (A / D conversion), the positions of T and T ' It is considered to be the place with the smallest deviation. Therefore, in an embodiment of the present invention, by returning this, a proper gray color can be easily obtained.
That is, the overall brightness is corrected.

Of course, this T is simply used as a means for simply determining whether the image is overexposed or underexposed, and an under-use LUT (look-up table) and an over-use LUT are prepared in advance, and the brightness of the point T or T'is adjusted. Gamma adjustment of the luminance signal may be performed according to the component. For example, the contrast may be adjusted by a curve having the point (YT ′, YT) in FIG. 6 as an inflection point. Therefore, it is possible to easily determine whether the image is overexposed or underexposed based on the values of T and T '. That is, if the luminance component of the point T ′, which is the point with the lowest saturation on the color solid axis, is higher than the luminance, the luminance-saturation relationship of the image shows a tendency as shown in FIG. Conversely, if the luminance component of the point T'is lower than the luminance, the luminance-saturation relationship of the image shows a tendency as shown in FIG. 7 (b). Therefore,
In an image in which high-luminance color suppression and low-luminance color suppression are performed, if the luminance component at point T ′ is higher than high luminance, the image tends to be overexposed, and if the luminance component at point T ′ is lower than luminance, , The image is considered to be underexposed.

On the other hand, the saturation adjustment can be easily performed by multiplying the color differences C1 and C2 by the saturation correction coefficient. For example, when the saturation is increased by 20%, the corrected saturation is 120% before the correction, so the saturation correction coefficient is calculated as 1.2. That is, the saturation correction can be performed with C1 ″ = 1.2 × C1 ′ C2 ″ = 1.2 × C2 ′. This is because (saturation) = sqrt (C1 ^ 2 + C2 ^ 2) is defined.

(Inverse Conversion to RGB Space) With the above, various corrections in this embodiment are completed. At this point, each pixel of the original image is (C1 '', C from the color signal data of (R, G, B).
Since it has been converted into the color space data of 2 ", Y"), it is converted back into the color signal data of (R ', G', B ') again. Inverse conversion is performed by the following formula. R '= Y "+ C1"G' = Y "-(0.3 / 0.59) * C1"-(0.1
1 / 0.59) * C2 ″ B ′ = Y ″ + C2 ″ In this way, it is possible to obtain RGB data in which the color cast, the contrast, and the saturation have been corrected for the original image.

As described above, according to this embodiment, it is possible to surely correct the color cast with a small processing load. Also,
According to the present embodiment, since the sampling condition is set according to the image data size of the original image, the relationship between the histogram total frequency and the cumulative frequency for determining the white position and the black position regardless of the input image. Can be kept almost constant. Therefore, good color cast correction can be realized.

Furthermore, it is considered that the image is closest to the value of the original image by performing non-linear gamma correction on the entire image so as to maintain the luminance at the point where the distance between the color solid axis of the image to be corrected and the luminance axis is the shortest. The contrast can be corrected while maintaining the brightness that is obtained.

Further, it is possible to easily obtain the exposure state of whether the image is overexposed or underexposed. Further, a different table can be selected according to the exposure state and gamma correction can be performed. However, this means for checking the exposure is simply performed by analyzing the histogram of the image data, and is different from the method for checking the exposure state based on the shooting condition data 106.

The sampling may be performed in column units instead of line units.

[Second Embodiment] Next, the above-mentioned first embodiment
A second embodiment in which the degree of correction is taken into consideration will be described with respect to the embodiment.

As described in the first embodiment, when the color solid axis of the image is obtained, if the inclination of this axis is too large, the image may be deficient if it is forcibly corrected. This may be, for example, a case where color cast is intentionally caused by using a color filter or a case where a sunset scene is photographed.

In such a case, it is judged that the obtained highlight point and shadow point are wrong and no correction is made, or the rotation angle is appropriately adjusted to weaken the correction degree to eliminate the problem. it can. It is possible to judge that the highlight point and the shadow point are wrong by the direction of the color solid axis. Since it is possible to easily determine which color is covered by the inclination of the color solid axis, it is possible to make a determination not to correct the color cast for an image captured using a color filter in order to obtain a special effect.

In such a case, paying attention to the angle formed by the direction vector of the color solid axis and the luminance axis, it is determined that the inconvenience will occur conversely by performing the color cast correction, or the processing is not performed, or Loosen the degree of processing. For example, the color solid axis is oriented in the hue direction of red, and the angle is a predetermined angle,
For example, when the angle is 40 degrees or more, it is determined that the image is originally an image with a color cast. When the degree of processing is loosened, the color cast is corrected by raising the color solid axis by a predetermined angle, for example, 20 degrees, or by raising it up to a predetermined angle, for example, the inclination with respect to the Y axis becomes 20 degrees. The rotation matrix for this conversion can be easily obtained from the rotation axis and the rotation angle.

The degree of correction may be manually specified by the user, or may be set in advance according to the angle magnitude and the direction of the color solid axis. For example, in the case of 40 degrees, the color solid of the image is supposed to be rotated by 20 degrees, but the degree of the correction may be designated by the user. The subsequent processing is the same as in the first embodiment.

Further, since it is possible to easily determine which color is covered by the direction of the color solid axis, for an image photographed using a color filter for obtaining a special effect, the color direction of the filter is It is also possible to make a judgment such as no correction. That is, in such a case, a color for which the color cast is not corrected is designated, and a rotation axis for rotation conversion is set along the direction of the color that is not corrected on the C1-C2 plane. Rotate the color solid until the plane containing the axis l is parallel to the Y axis. By doing so, the color cast can be corrected only for a specific color component.

FIG. 10 shows a processing procedure for correcting an image so that, when the color solid axis is tilted by a predetermined angle or more, the color solid is rotated to a separately designated angle. First, a highlight / shadow point is determined in step S101, and it is determined in step S102 whether the inclination of the color solid axis is equal to or larger than a predetermined angle. If the predetermined angle is not reached, in steps S104 and S105, the color solid axis is converted so as to coincide with the luminance axis and the color cast is corrected, as in the first embodiment.

On the other hand, if the color solid axis is tilted by a predetermined angle or more, a rotation matrix for rotating the color solid axis by 20 degrees toward the luminance axis is obtained in step S103, and the rotation matrix is calculated in step S105. Use to rotate the color solid to correct the color cast. In this case, the angle of 20 degrees used in step S103 may be specified in any way.

If the threshold value is set in two steps and the inclination of the color solid axis is larger than the first threshold value (for example, 40 degrees), the color solid axis does not become a complete gray line. The rotation conversion is performed so as to raise the axis (for example, 20 degrees), and when it is between the first threshold value and the second threshold value (for example, 20 degrees), the rotation conversion is not performed and the second stroke is performed. When the inclination is smaller than the threshold value, rotation conversion may be performed so that the color solid axis matches the luminance axis. By doing so, inconvenient correction is not performed for an image in which color cast is intentionally generated.

As described above, from at least two inclinations of the axis of the color solid formed by the pixels of the image data, that is, the direction of the axis and the angle of the inclination, at least two or more threshold values for correction are required. It is possible to determine whether the image should be set and corrected, or not, and the degree of correction should be adjusted, and it is possible to very simply repel the harmful effects of the special case.

Further, since it is possible to easily determine which color is covered by the direction of the color solid axis, it is possible not to correct the color cast depending on the direction of the covered color.

[Third Embodiment] In the first and second embodiments, the color balance correction based on the highlight point / shadow point of the image has been described, but the correction based on other reference points is performed. An example will be described below.

First, the average color difference amount ΔE at the highlight point and the shadow point is obtained. ΔE = sqrt ((C1HL-C1SD) ^ 2 + (C2
HL-C2SD) ^ 2) If the color solid axis is parallel to the luminance axis, ΔE = 0 should be obtained, but if it is tilted, it becomes a value larger than 0. That is, when considered on the E (saturation) -Y (luminance) plane, the result is as shown in FIG.

Next, several sample luminances between the highlight and the shadow are prepared. Then, in the image, for example, the luminance Yn
From the pixels of, the average color difference amount is obtained as shown in FIG. 8 by using pixels of saturation smaller than ΔEn. As ΔEn, a predetermined saturation may be used, or an average color difference amount limited to the tilt direction of the color solid axis may be obtained.

With respect to the luminance at these several points, a straight line is obtained by the least-squares method with the average color difference, and the processing based on the first embodiment is performed with this as a color solid axis.

Alternatively, an approximate curve is obtained by a B spline or the like at the points having the obtained color difference and luminance as elements, and this curve is used as the luminance axis, that is, non-linear color balance correction is performed so that the color difference becomes "0". You may go.

Although the present invention has been described with reference to the preferred embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims.

By doing so, the line to be the luminance axis can be obtained not only from highlight and shadow, but also from pixels uniformly sampled from the entire image. By converting the image data so that the obtained line coincides with the luminance axis, it is possible to perform color cast correction that reflects the characteristics of the entire image.

By the way, when the white balance is auto, the correction amount of the color balance may be variable depending on the deviation amount from the reference value. For example, in addition to the information that the white balance is auto, if you get the information that the white balance is largely deviated from the reference value in the blue direction, the color balance correction will change the color difference signal to yellow, which is the opposite of blue. A threshold value may be set so that the inclination is easy.

<Density Correction> Next, the density correction in the image correction processing unit 113 of this embodiment will be described. This is to read information regarding exposure or strobe issuing ON / OFF information from the image condition data 106 and perform optimum brightness correction (print density correction).

(Gradation Curve Judgment) In step S2 of FIG. 11, correction condition setting processing (gradation curve judgment processing) is performed based on the histogram obtained in S20 of FIG. That is, a gradation curve is selected from a plurality of gradation curves for brightness correction prepared in advance based on the analysis result of the image.

In the gradation curve judgment of this embodiment, the brightness of the image is judged based on three parameters (highlight point, shadow point, and the number of pixels in a certain luminance area), and the gradation curve is determined based on this. select.

FIG. 12 is a flow chart showing details of the gradation curve judgment processing. The gradation curve determination processing of this embodiment will be described with reference to this.

Highlight Point Determining Section In the highlight point determining process in step S21 in FIG. 12, the highlight point in the image to be processed is calculated from the histogram.

In this embodiment, in the histogram of the luminance signal Y, the frequencies of the respective luminance values are accumulated from the highest luminance value (luminance value 255) in the luminance range in order toward the lower luminance side, and the cumulative frequency obtained here is , For example, a brightness value that matches 1.0% of the total number of pixels of the image data to be processed, or a brightness value that first exceeds 1.0% of the total number of pixels,
This point is a highlight point (hereinafter referred to as "HLP").

Next, the HLP is set to a plurality of predetermined thresholds Th.
_H1, Th_H2, ... (Th_H1 <Th_H2
<...) and analyze the distribution in the high brightness area of the histogram.

For example, in the present embodiment, as shown in FIG. 13, a case will be described in which two threshold values in which the values of 200 and 230 are set in order from the bottom are used.

Then, when HLP <Th_H1,
The image is judged as a dark image with few high-brightness areas, and Th
When _H1 ≦ HLP <Th_H2, it is determined that the image is a dark image as a whole even though there is a distribution in the high luminance region, and Th_H
When 2 ≦ HLP, the image is determined to be a bright image with a large distribution in the high-luminance region.

For example, in the bright image histogram shown in FIG. 14, HLP exceeds the threshold Th_H2 (HL
P> Th_H2), and it is determined that the image has a large distribution in the high luminance region. In the histogram of FIG. 14, the distribution is biased to the high brightness side as a whole, and as a result, the HLP is also located on the high brightness side. Generally, many images showing such distribution are bright images.

In the histogram shown in FIG. 15, HLP is lower than the threshold value Th_H2 and higher than Th_H1 (Th_H1).
H1 <HLP ≦ Th_H2), and there is some distribution in the high-luminance region, but it can be regarded as an image that is not bright.
The histogram of FIG. 15 shows a roughly intermediate distribution of luminance, and the HLP is also located on the relatively lower luminance side than that of FIG. 14, so this determination can be made.

In the histogram shown in FIG. 16, HLP is lower than the threshold value Th_H1, and the image has no distribution in the high luminance area. In this case, the histogram is biased toward the low brightness side,
It can be seen that the image is dark overall. Further, the low HLP means that the gradation level is narrow. For such an image, it is necessary to correct the brightness by γ correction or to extend the brightness value to the high brightness side to make it brighter.

The calculation of the HLP does not necessarily have to be obtained by the above-mentioned method, and a conventionally known method may be appropriately used.

When the automatic gradation correction processing of the present embodiment is performed in combination with other image correction processing, for example, the so-called color cast correction, contrast correction, and saturation correction described above, it is used in advance in this image processing. HLP can also be used. In this case, instead of using the highlight point, it is possible to determine the brightness (darkness) of the image using the shadow point which is also used in the color cast correction and the like, and based on this, the following processing is performed. It is obvious from the following description that the above can be performed.

Histogram Balance Determination In step S22 of FIG. 12, step S1 of FIG.
Balance determination of the histogram is performed using the histogram obtained in.

In the histogram balance determination process, in step S22, Slow, which is the ratio of the cumulative frequency of the predetermined area to the total number of pixels of the image to be processed, is calculated. That is, for example, for a 256-gradation image, the brightness values from 0 to 128
The ratio of the cumulative number of pixels up to (half of the histogram) to the total number of pixels is calculated, and the overall balance of the histogram of the image is analyzed.

First, the cumulative frequency S of a certain brightness area (0 to 128) is calculated. This cumulative frequency S is obtained as a cumulative frequency from the lowest luminance value (luminance value 0) of the luminance range in the histogram to a predetermined luminance value toward the higher luminance side. In this embodiment, the maximum brightness value (brightness value 255) is 1
Although the cumulative frequency up to the luminance value (luminance value 128) of / 2 is obtained as the cumulative frequency S of the low luminance region, other values may be used as a matter of course.

Next, the ratio Slow of the cumulative frequency S to the total number of pixels is calculated using the following formula. Slow = (cumulative frequency S of a certain luminance area) / (total number of pixels) (%) In the above histogram aggregation, when pixels are thinned to create a thinned histogram, the above Sl
The denominator in the definition formula of ow is set to the number of pixels for which the histogram is created.

Next, threshold value determination is performed again using the Slow obtained above. This is to examine the overall luminance balance of the image by calculating the ratio of the lower half of the histogram to the whole. In the above-described highlight point determination, the images are classified into several types according to the distribution state of the high-brightness area of the histogram. As shown in FIG. judge.

For example, in the case of the bright image shown in FIG. 14, the ratio of the area up to the brightness value 128 to the total number of pixels is Slow. In this example, Slow is 2 which is the total number of pixels.
It is 0%. Therefore, it is determined that the HLP determination is a bright image, and Slow is determined to be in the range of 16 to 50.

On the other hand, in the example of the dark image shown in FIG.
The area Slow up to the brightness value 128 is 60% of the total number of pixels.
Therefore, the HLP determination determines that the image is dark and Slow is determined to be in the range of 50 to 80.

In the method of judging the balance of the histogram by using only the intermediate value and the average value of the histogram without using the ratio of the cumulative frequency in a certain brightness area, the actual distribution state of the histogram is not properly reflected in the image. The brightness will be determined. For example, although the intermediate value and the average value themselves show relatively high luminance values, in reality, there are few peaks in the luminance distribution around the intermediate value and the average value, and the frequency distribution itself in the low luminance region is small. For images,
In some cases, the brightness correction that increases the density by mistakenly determining that the image is a bright image is selected, and as a result, a dark portion occupying a relatively large portion on the image may be crushed.

On the other hand, as in the present embodiment, by obtaining the cumulative frequency in the region of the luminance values 0 to 128 which is the lower half of the histogram and using the ratio Slow of this cumulative frequency to the total number of pixels, It is possible to determine the brightness distribution of the image in which the actual balance of the histogram is reflected, and it is possible to perform the appropriate gradation correction for the dark image as described above.

In the above embodiment, the range of brightness values 0 to 128 is equally divided in the range of Slow.
To obtain the information of the low-luminance region in more detail, the low-luminance region may be divided into some and the cases may be divided for each, and when Slow is 0 to 64, it is doubled.
The weights of 65 to 128 may be multiplied by 1 and added.

In such cases, the histogram balance can be determined more accurately.

In the shadow point determination processing in step S23, first, the shadow point in the image to be processed is calculated from the histogram.

In the present embodiment, the frequencies of the respective brightness values are accumulated in the histogram from the lowest brightness value (brightness value 0) in the brightness range toward the higher brightness side in order, and the cumulative frequency obtained here is, for example, A luminance value that matches 1.0% of the total number of pixels of the target image data, or a luminance value that first exceeds 1.0% of the total number of pixels, is obtained, and this point is calculated as a shadow point (hereinafter referred to as “SDP”). Also called).

Next, a plurality of thresholds Th_S1, Th_S2 ,.
1 <Th_S2 <...) and analyze the distribution of the histogram in the low luminance region.

Since the shadow point determination used in the present embodiment is performed after the highlight point determination and the histogram balance determination, as shown in FIG. 13, the threshold of the shadow point is the highlight point or the histogram. Different values are set according to the result of the balance judgment of.

When SDP ≧ Th_S2, it is determined that the image is a bright image with few low-luminance regions, and Th
When _S1 ≦ SDP <Th_S2, it is determined that the image is a bright image as a whole even though there is a distribution of low-luminance regions, and SDP
When <Th_S1, it is determined that the image is a dark image with a large distribution in the low luminance region.

For example, in the histogram of the bright image shown in FIG. 14, SDP exceeds the threshold Th_S2 (SD
P> Th_S2), and therefore, it is determined that the image has no distribution in the low luminance region. In this case, the distribution of the histogram is biased to the high-luminance side as a whole as described above, and as a result, the SDP is also located on the relatively high-luminance side. Also, SD
High P also means that the gradation level is narrow. For such an image, it is necessary to perform correction such that it is darkened by γ correction or the brightness value is extended to the low brightness side to darken it.

On the other hand, in the histogram shown in FIG.
When DP is lower than the threshold Th_S2 and higher than Th_S1 (Th_S1 <SDP ≦ Th_S2), it is determined that the image has a certain distribution in the low luminance region but is not dark. In this case, the luminance has an intermediate distribution, and the SDP is also located on the relatively low luminance side, so such a determination is performed.

Next, in the histogram shown in FIG. 16, SD
P is lower than the threshold value Th_S1, and the image has a large distribution in the low luminance region. In this case, it can be seen that the histogram is biased toward the low luminance side and the image is dark as a whole.

Determination of Corrected Gradation Curve Details of distribution of high-brightness area in histogram above (highlight point), histogram balance S
Details of distribution in low and low brightness areas (shadow points)
As shown in FIG. 13, the processing target image is classified into a plurality of types according to the three parameters. Then, in the next step S24, the correction gradation curve is determined using the table shown in FIG.

As is clear from the correction table shown in FIG. 13, the gradation curve of this embodiment makes a comprehensive determination of the image type according to three parameters. For example, if the highlight point is relatively low, A gradation curve that includes processing for cutting the high-brightness region and extending the histogram to the high-brightness side is selected. Further, for an image for which the balance of the histogram is desired to be adjusted, a gradation curve for γ correction is selected. If the balance Slow of the histogram is biased toward the low-luminance region, a gradation curve including a process for brightening the entire image by converting the γ value is selected. Further, if the shadow point is relatively high, a gradation curve including processing for cutting the low-luminance region and extending the histogram to the low-luminance side is selected. The determination table shown in FIG. 13 is used to determine the gradation curve for the images classified into the above plurality.

For example, the HLP is as high as 245, and Slo
In the case of an image in which w is 20% and SDP is 60, which is relatively high,
A gradation curve that cuts the low luminance area 20 and below is selected.

In the case of the bright image shown in FIG. 14, HLP
Is larger than the threshold Th_H2 and Slow is 20%, and SDP is larger than the threshold Th_S2. Therefore, from the table shown in FIG. 13, it is determined that this image is a bright image.
The γ value is set to 1.1. By this γ value determination, correction is performed to darken the comparatively high luminance area (increase the print density), and a print image having an optimum density is obtained as a whole. In addition, since the ratio of pixels in the low-luminance region is small, the image is less likely to be crushed.

Next, in the image having the intermediate balance of the histogram shown in FIG. 15, HLP is larger than Th_H1 and smaller than Th_H2. And Slow is 40%,
Since SDP is larger than Th_S1 and smaller than Th_S2, the correction gradation curve is S based on the table shown in FIG.
The contrast can be emphasized by making it a letter. In this way, the entire printed image is crisp and the printed image looks good.

On the other hand, in the dark image shown in FIG.
Is smaller than Th_H1 and Slow is 60%,
Since SDP is smaller than Th_S1, a straight line that cuts 200 or more in the high brightness area is selected according to the table shown in FIG. As a result, the entire image to be printed becomes brighter, and in particular, the histogram of the image is expanded to the high-luminance side, resulting in a printed image in which density with contrast is balanced.

In the above description, in the highlight point determination, the brightness of the high luminance area of the image is determined in three steps, but the number of branches is further increased in order to obtain a more optimal gradation curve. More detailed determination may be performed by classifying into four or more stages. In addition, the number of branches may be increased in order to make more detailed determinations in histogram balance determination and shadow point determination.

(LUT Creation) Upon completion of the gradation curve determination processing (step S2 in FIG. 11) described above, LUT creation is performed in step S3 shown in FIG. A lookup table (LUT) for brightness correction is created based on the parameters for creating the gradation curve obtained by the gradation curve determination processing.

The LUT of this embodiment records the gradation curve obtained as described above as an exponential function and a quintic function. In other words, when simply performing γ correction, the exponential function (Fig. 1
7) is used to correct with a straight line that cuts a high-luminance or low-luminance region and a more complicated curve (see FIG. 1).
8) is used.

That is, if the input luminance signal is Y and the output luminance signal is Y ′, then LUT L [Y] is an exponential function that performs only γ correction. Y ′ = 255 × [(Y / 255) 1 / γ] is used for conversion, and γ
A value is given.

On the other hand, in the case of a quintic curve, Y ′ = A + B × Y + C × Y ^ 2 + D × Y ^ 3 + E × Y ^
4 + F × Y ^ 5 is converted and the coefficient A,
A quintic curve is completed by giving B, C, D, E and F. Also, they are created dynamically. That is, it is created for each processing of the target image. By dynamically creating the correction table in this way, the required memory amount can be reduced.

It is needless to say that the LUT may be statically prepared in advance in the memory for each gradation curve instead of being dynamically created.

(Correction) Next, step S4 shown in FIG.
At, the luminance signal Y is corrected. That is, the created LUT L [Y] is used to convert the brightness value Y of the input image as Y ′ = L [Y] to perform brightness correction.

Further, the luminance-corrected luminance signal Y'and the color-difference signals Cr, Cb of the input image are returned to the R, G, B signals to create a corrected image signal R'G'B '.

According to the present embodiment, in the histogram, the component value showing the predetermined value of the cumulative frequency from the maximum value or the minimum value in the range of the component value related to the brightness of the image data is obtained, so that the entire image It is possible to know the brightness of the image, and the ratio of the cumulative frequency from the minimum value or the maximum value to the predetermined component value to the total number of pixels in the histogram can be obtained, so that the distribution of the brightness of the image can be known. . Then, the brightness distribution is determined based on these component values and ratios, and the gradation curve for correction is determined based on the determination. Therefore, the brightness distribution and correction are performed for each overall brightness of the image. The correspondence with the gradation curve can be different.

That is, one correction gradation curve most suitable for the image is finally selected from the plurality of correction gradation curves using the three parameters of the highlight point, the balance degree of the histogram, and the shadow point. be able to.

As a result, for example, in a dark image as a whole, the distribution of the brightness corresponding to the correction of making the image brighter (the density of the print image is made lower) can be made small, and the distribution showing the dark range can be made small. Thereby, the balance of brightness in the printed image can be made more preferable. On the other hand, by making the overall bright image darker, the density of the printed image can be made higher, and thus the density output characteristic that the printing device can inherently achieve a relatively low density is compensated for, and the overall density is high. It is possible to perform density printing. Further, in the case of an image having a narrow dynamic range originally, by widening the width, it is possible to print an image with contrast and having a sharp and crisp appearance.

Although the determination is made in the order of FIG. 12 in this embodiment, the order may be changed.

Further, the calculation of HLP and SDP does not necessarily have to be obtained by the above-mentioned method, and a conventionally known method may be appropriately used.

By the way, as described above, the highlight point was used to check the brightness distribution in the high brightness region, and the shadow point was used to check the brightness distribution in the low brightness region. However, for example, if the details of the brightness distribution in the high-luminance region are known with another parameter that changes the highlight point, it may be used. That is, the ratio of the cumulative frequency from the maximum value of the histogram to a certain brightness value (for example, the brightness value 220) to the total number of pixels may be obtained, and the distribution distribution of the brightness of the high brightness region may be determined from the value. In that case, similar to the highlight point, a plurality of thresholds are set for the parameter, and the case determination is performed. Of course, also in the low luminance area, instead of the shadow point, the ratio of the cumulative frequency from the minimum value of the histogram to a certain luminance value (for example, the luminance value 30) to the total number of pixels may be obtained, and the same determination may be performed.

Further, in the above-mentioned embodiment, the correction relating to the luminance value Y has been described, but the same correction may be directly applied to each of the R, G and B signals. At this time, the above L
UT, using R, G, instead of Y in the LUT
The correction can be performed by using R ', G', B'instead of B, Y '. The correction for the R, G, B signals is RG
Since B-YCrCb conversion is unnecessary, the processing speed can be improved.

When the exposure setting is automatic, the above-mentioned processing is automatically added. However, if the exposure is automatic and the amount of change from the reference value is known, the above-mentioned processing contents are changed accordingly. You may.

That is, the HLP and S12 shown in FIG.
8. The SDP threshold value is set using the exposure value of the shooting condition data 106. For example, when the exposure value is smaller than the reference value, the threshold value in FIG. The captured image can be corrected to have an appropriate brightness. On the other hand, if the exposure value is large, there is a possibility of overexposure. Therefore, the threshold value in FIG. 13 may be changed accordingly so that the brightness is lowered as a whole.

<Digital Zoom ON / OFF Information> By the way, recent digital still cameras can perform image enlargement processing by adding digital processing in addition to optically zooming in / out an object. In many cases, the digital zoom simply multiplies the image size by an advantage, and the outline tends to rattling (jaggies). Further, methods such as the nearest neighbor method, the bilinear method, and the bicubic method are generally used as the expansion method.

When the digital zoom magnification and enlargement processing method are written in the photographing condition data 106, smoothing processing for smoothing the contour of the image may be performed based on them. For example, there is known a method of enlarging a low-resolution image to obtain a high-resolution image as proposed in Japanese Patent Laid-Open No. 11-331565. On the other hand, it is desirable that the coefficient (processing degree) of such smoothing processing is variably changed according to the magnification of the digital zoom and the enlargement processing method. That is, the lower the digital zoom magnification, the weaker the degree of smoothing, or the nearest neighbor method, which involves coarse enlargement processing, the degree of smoothing may be increased. This series of processing is shown in FIG.

<Strobe light emission ON / OFF information>
When the information that the flash fired is obtained from the shooting condition data 106, the content of the density correction processing may be changed as described above. For example, in addition to the table shown in FIG. 13, another table for strobe light emission is provided so that the strobe light is
In the case of FF, the conventional table of FIG. 13 is used, but when the strobe is ON, a new new table is used, so that more accurate image correction can be performed.

In the case of stroboscopic photography, the entire image tends to be photographed darkly, such as in a night scene photography or dark room photography. In this case, it is necessary to change the threshold value of the table so that the brightness can be easily corrected. Also, the subject reflected by the strobe tends to appear pale. In this case, it is preferable to add a correction that suppresses the brightness of the subject to be yellowish. On the other hand, the strobe light does not reach the background of the subject,
The image tends to be dark and the gradation is lost. In other words, it is necessary to increase the brightness of the low gradation portion to generate gradation.

By adding such processing, optimum processing can be performed on an image which is stroboscopically emitted at the time of photographing.

<Shooting scene information> Shooting condition data 1 when shooting modes (landscape mode, portrait mode, macro shooting mode, etc.) are switched depending on the subject.
Consider the case where shooting scene information is added to 06. In that case, the optimum memory color reproduction correction can be performed based on the information. For example, in the landscape mode, the sky and plants can be the main subjects. Therefore, the image correction processing unit in S30 of FIG. 3 may perform processing of increasing the saturation of blue or green and increasing the contrast. Further, by applying the sharpness filter, the landscape image can be finished into a more attractive image. When increasing the saturation, a saturation expansion coefficient is obtained for each hue, and the original saturation value is multiplied by this coefficient, whereby the effect can be obtained with a small memory amount when the saturation correction is performed.

On the other hand, in the case of portrait mode photography, FIG.
An image correction processing unit of 0 may add processing for correcting saturation and luminance of the skin color region to a more preferable color. Further, blurring the entire image using a blurring filter is also one of the methods for making a person image look more beautiful, and automatic processing can be performed by using shooting condition data.

In the above-described embodiment, the correction method when the photographing condition data such as white balance, exposure, flash, photographing scene and digital zoom is read has been described. However, when a plurality of image correction processes are performed, the same process may be duplicated, or the portion emphasized in the previous correction may be weakened in the next correction.

For example, the above-described color balance correction and density correction both include contrast enhancement processing using highlight points and shadow points. In such a case, a sufficient degree of contrast processing is performed twice, resulting in an image that is contrary to the user's intention.

Even if the image is smoothed by the digital zoom information, if the shooting scene information is the landscape mode, if the sharpness filter is applied after that,
It negates the effect of the previous smoothing process.

In order to prevent the harmful effect of applying such a plurality of processes to one image, it is necessary to restrict the process. In this embodiment, the order of each image correction process is prioritized.

First, as shown in FIG. 20, which judgment is made for each of the color balance information, exposure information, strobe information, photographic scene information, and digital zoom information is stored in the memory section.

Next, in FIG. 20S1, smoothing processing is performed using the digital zoom information. This process of converting a low-resolution image into a high-resolution image is considered to have no effect on the tint and brightness of the entire image, and is therefore preferably performed first.

Next, in FIG. 20S2, when it is determined that the contrast is emphasized based on the degree of the brightness correction determined from the exposure information, the contrast correction of the color balance correction that first performs the correction processing is not performed. . on the other hand,
When it is determined that only the gamma correction is to be performed in the brightness correction (when the contrast enhancement is not performed), the contrast correction of the color balance correction is performed. By doing so, it is possible to prevent the contrast correction from being duplicated in both the brightness correction and the color balance correction.

Next, brightness correction is performed in FIG. 20S3. It is checked whether strobe information is ON or OFF. If ON, refer to the strobe table for the brightness correction table. OF
If it is F, the conventional table shown in FIG. 13 is referred to. By doing so, the optimum brightness correction obtained from the exposure and flash information can be applied to the image.

Finally, in step S4 of FIG. 20, memory color reproduction correction is performed based on the shooting scene information. Here, when the saturation adjustment is performed in the color balance correction, the saturation expansion coefficient may be set to be small accordingly. Further, when the degree of smoothing is large in the digital zoom correction, the degree of blurring in the portrait mode should also be set small.

When performing a plurality of image correction processes in this way,
By prioritizing the order in which each processing is performed, weighting the processing under various conditions, and providing a constraint that if a certain correction processing is performed, another processing is not performed,
Finally, it is possible to correct the image without any waste.

<Other Examples> In the above embodiment, the photographing condition information is displayed on the screen. However, the present invention is not limited to this, and the photographing condition information can be displayed, for example, by turning on / off a predetermined lamp. You may make it show that it is correctly recognized and utilized.

The present invention can be variously modified and implemented without departing from the scope of the invention.

In the above embodiment, the digital still camera is used as an example of the image input means (acquisition means), but the present invention is not limited to this. For example, a digital video camera, An input device such as an image scanner or a film scanner can be applied.

Further, according to the present invention, a storage medium storing a program code of software for realizing the functions of a host and a terminal is supplied to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can be achieved by reading out and executing the program code stored in.

In this case, the program code itself read from the storage medium realizes the functions of the present invention, and the storage medium storing the program code and the program code constitute the present invention.

As a storage medium for supplying the program code, ROM, flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM,
A CD-R, a magnetic tape, a non-volatile memory card, etc. can be used.

Moreover, not only the functions of the present invention are realized by executing the program code read by the computer, but also the OS or the like running on the computer is actually executed based on the instruction of the program code. It goes without saying that a case where a part or all of the processing is performed and the functions of the present invention are realized by the processing is also included.

Further, after the program code read from the storage medium is written in the memory provided in the extended function board inserted in the computer or the extended function unit connected to the computer, based on the instruction of the program code, A CPU provided in the function expansion board or function expansion unit performs a part or all of the actual processing,
It goes without saying that the processing includes the case where the function of the present invention is realized.

FIG. 21 shows the function 600 of the computer.
Is shown.

As shown in FIG. 21, the computer function 600 includes a CPU 601, a ROM 602, and a RAM.
603, a keyboard controller (KBC) 605 of a keyboard (KB) 609, and a CRT controller (C of a CRT display (CRT) 610 as a display unit).
An RTC) 606, a disk controller (DKC) 607 of a hard disk (HD) 611 and a flexible disk (FD) 612, and a network interface controller (NIC) 608 for connection with a network 620 via a system bus 604. It is configured so that they can communicate with each other.

The CPU 601 is a ROM 602 or HD.
By executing the software stored in 611 or the software supplied from FD612, each component connected to the system bus 604 is comprehensively controlled.

That is, the CPU 601 performs control for realizing the operation in the present embodiment by reading a processing program according to a predetermined processing sequence from the ROM 602, the HD 611, or the FD 612 and executing it.

The RAM 603 functions as the main memory or work area of the CPU 601.

The KBC 605 controls instruction input from the KB 609 or a pointing device (not shown).

The CRTC 606 controls the display of the CRT 610.

The DKC 607 includes a boot program, various applications, edit files, user files,
The HD 611 and F that store the network management program and the predetermined processing program and the like according to the present embodiment
Control access to D612.

The NIC 608 bidirectionally exchanges data with a device or system on the network 620.

[0193]

As described above, according to the present invention, condition information (imaging condition information, etc.) added to image information to be processed (image information obtained by photographing with a digital still camera) is analyzed. Based on the result, image processing (image correction processing, etc.) is performed on the image information. Accordingly, it is possible to perform image processing according to the situation at the time of image acquisition, and it is possible to provide a good processed image intended by the user.

For example, in a digital still camera, color balance correction is performed based on white balance information, and brightness correction is performed based on exposure information. Also, the brightness correction is performed by using the correction table corresponding to the ON / OFF of the strobe emission, and the image smoothing process is performed based on the digital zoom information. Then, based on the photographed scene information, a memory color reproduction process according to, for example, a landscape or a person mode is performed. In this way, it is possible to perform the optimum correction on the image based on more detailed information, as compared with the conventional correction method that only analyzes the histogram based on the image data.

Further, when performing a plurality of processes, in order to eliminate the adverse effects such as duplication of similar processes and weakening of the previously performed emphasis process, the priority is set for each of the plurality of processes. By weighting the processing of, it is possible to perform optimal image correction processing without waste.

As described above, according to the present invention, highly accurate automatic image correction can be realized by the information indicating the condition at the time of acquisition of the image information, and the high quality which more reflects the intention at the time of photographing. The processed image can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image printing system to which the present invention has been applied in a first embodiment.

FIG. 2 is a diagram in which an RGB image signal is converted into a YCC signal, image processing is performed, and the RGB signal is converted into an RGB signal again.

FIG. 3 is a flowchart of a processing procedure of an image correction processing unit.

FIG. 4 is a flowchart of a luminance histogram creation procedure.

5A and 5B are color stereograms for explaining the principle of color balance correction in the embodiment, where FIG. 5A is an ideal color stereogram and FIG. 5B is a color stereogram before color balance correction of the image.

FIG. 6 is a graph showing a non-linear gamma conversion function.

FIG. 7 is a diagram showing the characteristics of overexposure / underexposure viewed from the luminance-saturation plane, (a) is a conceptual diagram of overexposure,
(B) is a conceptual diagram of underexposure, (c) is an example of overexposure, (d) is an example of underexposure

FIG. 8 is a diagram showing a manner of determining a color solid axis in the third embodiment.

FIG. 9 is a flowchart of a control procedure of image correction.

FIG. 10 is a flowchart of a control procedure for color cast correction according to the second embodiment.

FIG. 11 is a flowchart showing a procedure of the automatic gradation correction processing.

12 is a flowchart showing a processing procedure of gradation curve determination in the automatic gradation correction processing shown in FIG.

FIG. 13 is a diagram showing the contents of a table used in the gradation curve determination processing and explaining how to define a gradation curve according to the type of image.

FIG. 14 is a diagram showing a histogram when an image to be processed by the automatic gradation correction process is a bright image.

FIG. 15 is a diagram showing a histogram in the case where the brightness of the image that is the processing target of the automatic gradation correction processing is an intermediate image.

FIG. 16 is a diagram showing a histogram when an image to be processed by the automatic gradation correction process is a dark image.

FIG. 17 is a diagram showing a conversion characteristic curve (exponential function) of a brightness correction table according to a γ value.

FIG. 18 is a diagram showing a conversion characteristic curve (quintic curve) of a brightness correction table other than simple γ correction.

FIG. 19 is a flowchart for performing smoothing processing based on digital zoom information.

FIG. 20 is a flowchart showing restrictions on performing a plurality of image correction processes based on shooting condition data.

FIG. 21 is a diagram showing functions of a computer.

[Explanation of symbols]

101 digital still camera 102 Shooting condition setting section 103 Shooting condition recording section 104 Captured image recording section 105 image data 106 Shooting condition data 107 Captured image data 108 image processing device 109 Reader unit 110 Data Analysis Department 111 Imaging condition analysis unit 112 Captured image analysis unit 113 Image correction processing unit 114 print data converter 115 Printer

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 1/60 H04N 1/46 Z 9/73 1/40 101D 9/79 9/79 H (72) Invention Gaku Yamazoe 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc. (72) Inventor Takashi Fujita 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc. (72) Inventor Shin Torikoshi Tokyo 3-30-2 Shimomaruko, Ota-ku, Canon Inc. (72) Inventor Shigeru Mizoguchi 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc. F-term (reference) 5B057 AA20 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE03 CE05 CE11 CE17 DA17 DB02 DB06 DB09 DC23 5C055 BA06 BA08 CA16 EA02 EA03 EA04 EA05 EA06 HA16 HA17 HA37 5C066 AA01 AA05 CA17 EA14 EC02 EE03 GA01 GA02 GA05 KM01 5C077 MP08 15 PP32 PP37 PP52 PP53 PQ08 PQ12 PQ19 PQ22 5C079 HB01 LA02 LA12 LA14 LA23 LA31 MA01 MA11

Claims (34)

[Claims] 1. An image processing apparatus for performing image processing on image information including condition information at the time of image acquisition, comprising: analyzing means for analyzing the condition information; and color analysis based on an analysis result of the analyzing means. The color correction amount of the image is obtained according to the value indicating the degree of balance, and the brightness correction amount of the image is obtained according to the value indicating the degree of brightness.
An image processing apparatus, comprising: a process determining unit that obtains a smoothing correction amount of an image according to a value related to zoom. 2. A processing means for subjecting the image information to the image processing determined by the processing determining means, and an output means for outputting the image information processed by the processing means. The image processing device described. 3. The image processing apparatus according to claim 2, wherein the output unit includes a print output function. 4. The process of obtaining the correction amount of the color of the image is
The brightness of the highlight point and the shadow point of the original image is detected, and from the plurality of pixels having the brightness,
2. The image according to claim 1, wherein the color tones of the highlight point and the shadow point are obtained, and color correction processing is performed on the original image based on the highlight point, the shadow point and the color tone. Processing equipment. 5. The color correction process is characterized in that a color solid axis of the original image is adjusted to an axis indicating brightness, and contrast adjustment is performed on a color component indicating brightness of the original image. The image processing apparatus according to claim 4. 6. The color correction process according to claim 4, wherein the color component indicating the tint of the original image is corrected to adjust the saturation of the original image. Image processing device. 7. The image processing apparatus according to claim 4, wherein a histogram is created based on a color component indicating the brightness of the original image, and the highlight point and the shadow point are detected based on the histogram. . 8. The color solid axis of the original image is detected, the exposure state of the original image is judged from the positional relationship between the color solid axis and the axis showing the brightness in the color space showing the color solid, and the judgment is made. An image processing method characterized by setting an image correction condition according to a result. 9. The image processing method according to claim 8, wherein the image correction condition is a condition for performing contrast adjustment on a component indicating the brightness of the original image. 10. An image processing method for performing image correction processing on an original image according to a color distribution of the original image, comprising detecting a color solid axis of the original image in a predetermined color space, Based on the positional relationship of the color solid axis,
An image processing method comprising controlling the image correction process. 11. The process determining means for determining the brightness correction amount of the image determines the distribution of the brightness of the image from a histogram concerning the number of pixels of the value of the component concerning the brightness of the image indicated by the image data, Based on the above, one of a plurality of gradation correction conditions is automatically selected, and the component relating to the brightness is corrected using the selected gradation correction condition. . 12. The image processing method according to claim 11, wherein the distribution of the brightness of the image is determined based on the cumulative frequency of a predetermined range in the histogram. 13. The brightness distribution of the image is determined based on a ratio of the cumulative frequency of the predetermined range to the total number of pixels of the histogram.
The described image processing method. 14. The image processing method according to claim 11, wherein the gradation correction condition is selected based on a combination of a highlight point and a shadow point of the image and a distribution of brightness of the image. 15. The density is increased when the ratio of highlight areas in the histogram is large, the density is decreased when the ratio of shadow areas is large, and the dynamic range is expanded when the histogram width is narrow. The image processing method according to claim 11. 16. When the proportion of the highlight area is large, the degree of density increase is large as compared with the case where the proportion of the highlight area is small, and when the proportion of the shadow area is large, the proportion of the shadow area 16. The image processing according to claim 15, wherein the degree of lowering the density is larger than that when the histogram width is small, and the degree of widening the dynamic range is larger when the histogram width is smaller than when the histogram width is large. Method. 17. The image processing apparatus according to claim 1, wherein the image information includes image information obtained by a digital still camera. 18. The image information includes photographed image information, and the condition information includes at least one of exposure, strobe, white balance, photographing mode, and digital zoom when the photographed image information is acquired. The image processing apparatus according to claim 1. 19. The image processing apparatus according to claim 1, further comprising a presentation unit that presents information regarding the condition information. 20. The presenting means presents information about the condition information based on a processing timing of the image information to which the condition information is added.
The image processing device described. 21. The image processing apparatus according to claim 1, wherein the analysis means has a function of analyzing the image information. 22. The process determining means, when acquiring a plurality of image information and performing a plurality of processes corresponding thereto, sets a priority order for performing the plurality of processes. Image processing device. 23. An image processing system comprising a plurality of devices communicably connected to each other, wherein at least one device of the plurality of devices is an image processing device according to any one of claims 1 to 22. An image processing system having the function of. 24. An image processing method for performing image processing on image information including condition information at the time of image acquisition, comprising: an analysis step of analyzing the condition information; and an analysis result of the analysis step. , The color correction amount of the image is calculated according to the value indicating the degree of color balance, the brightness correction amount of the image is calculated according to the value indicating the degree of brightness, and the smoothing correction amount of the image is calculated according to the value indicating zoom. And a process determining step for obtaining 25. An image processing method for performing image correction processing and print processing on image information including condition information at the time of image acquisition, comprising an analysis step of analyzing the condition information, and an analysis result by the analysis step. Based on, the color correction amount of the image is obtained according to the value indicating the degree of color balance, the brightness correction amount of the image is obtained according to the value indicating the degree of brightness, and the image correction amount is determined according to the value during the zoom. An image correction condition determination step for determining an image correction condition for obtaining a smoothing correction amount of, and an image correction step for performing image correction processing on the image information based on the image correction condition determined in the image correction condition determination step, A printing step of printing the image information processed by the image correction step. 26. An image processing method for performing image correction processing and printing processing on image information including condition information at the time of image acquisition, comprising an analysis step of analyzing the condition information, and an analysis result by the analysis step. Based on, the amount of color correction of the image is obtained according to the value indicating the degree of color balance, the amount of brightness correction of the image is obtained according to the value indicating the degree of brightness, and image smoothing is performed according to the value related to zoom. An image correction condition determination step for determining an image correction condition for obtaining a correction amount; an image correction step for performing an image correction process on the image information based on the image correction condition determined in the image correction condition determination step; A printing step for printing the image information after the processing by the correction step, and the processing execution timing of the image correction step and the printing step Based, the image processing method characterized by including a display step of displaying the processing mode information corresponding to the condition information. 27. An image processing method for performing image correction processing and printing processing on image information including condition information at the time of image acquisition, comprising a first analysis step of analyzing the condition information, and the image information. Second analysis step for analyzing the characteristics of the above, a first image correction condition determination step for determining the image correction condition based on the analysis result by the first analysis step, and an analysis by the second analysis step. A second image correction condition determining step for determining the image correction condition based on the result, and an image correction condition determined by the first image correction condition determining step and the second image correction condition determining step. And an image correction step for performing image correction processing on the image information, and a printing step for printing the image information processed by the image correction step. An image processing method characterized by: 28. An image processing method for performing image correction processing and printing processing on image information including condition information at the time of image acquisition, comprising a first analysis step of analyzing the condition information, and the image information. Second analysis step for analyzing the characteristics of the above, a first image correction condition determination step for determining the image correction condition based on the analysis result by the first analysis step, and an analysis by the second analysis step. A second image correction condition determining step for determining the image correction condition based on the result, and an image correction condition determined by the first image correction condition determining step and the second image correction condition determining step. And an image correction step of performing image correction processing on the image information, a printing step of printing the image information processed by the image correction step, and Based on the processing execution timing of the correction step and the printing step, the image processing method characterized by including a display step of displaying the processing mode information corresponding to the condition information. 29. The image processing method according to claim 24, further comprising an image acquisition step of acquiring the image information by photographing with a digital still camera. 30. The image information includes photographed image information, and the condition information includes at least one of exposure, strobe, white balance, and photographing mode at the time of photographing. 29. The image processing method according to any one of to 28. 31. A computer-readable storage medium recording a program for causing a computer to realize the function of the image processing apparatus according to claim 1 or the function of the image processing system according to claim 23. . 32. A computer-readable storage medium recording a program for causing a computer to execute the processing steps of the image processing method according to claim 24. 33. A program for causing a computer to realize the function of the image processing apparatus according to claim 1 or the function of the image processing system according to claim 10. 34. A program for causing a computer to execute the processing steps of the information processing method according to claim 24.