JP5056006B2 - Imaging apparatus and program - Google Patents

Description

The present invention relates to a beauty program Oyo imaging device.

  Conventionally, in general image processing of an electronic camera, color correction processing has been performed in order to realize color reproduction intended by a user after performing white balance correction in accordance with a photographing light source. Since such color correction processing is premised on being performed on image data that has been subjected to appropriate white balance correction, if the white balance correction is insufficient, the color tone of the image after color correction processing will be It becomes unnatural.

For this reason, various methods for appropriately performing white balance correction and color correction on a captured image have been proposed. As an example, Patent Document 1 discloses an image correction apparatus that calculates reliability for each of a plurality of color correction values and calculates a new color correction value reflecting each reliability.
JP 2006-50559 A

However, while the imaging apparatus captures various scenes, it is actually difficult to perform complete white balance correction in any environment. Therefore, depending on the scene to be imaged, there is a possibility that the final image tone may be unnatural due to insufficient white balance correction.
The present invention solves the above-mentioned problems of the prior art. An object of the present invention is to provide means for making the color tone of an image after color correction processing more appropriate in accordance with a photographed scene.

An imaging apparatus which is an example of the present invention includes an imaging element, a white balance calculation unit, an image processing unit, and a control unit. The imaging device captures a subject image and generates image data. The white balance calculation unit executes an auto white balance calculation for obtaining a white balance adjustment amount for the image data, and calculates a reliability of the auto white balance calculation. The image processing unit performs white balance correction and color correction processing on the image data after the color interpolation processing. The control unit changes the correction distribution ratio for white balance correction based on the white balance adjustment amount and the correction distribution ratio for color correction processing based on the reliability value .

  In the present invention, the color tone of the image is adjusted to a more appropriate color tone by changing the white balance adjustment amount or the content of the color correction processing based on the reliability of the auto white balance calculation.

  FIG. 1 is a block diagram showing the configuration of the electronic camera of this embodiment. The electronic camera has an image pickup device 11, an analog signal processing unit 12, an image processing unit 13, a buffer memory 14, a CPU 15, a ROM 16, a recording I / F 17, a release button 18 and an operation unit 19. ing. The image sensor 11, the analog signal processing unit 12, the image processing unit 13, the ROM 16, the recording I / F 17, the release button 18, and the operation unit 19 are connected to the CPU 15, respectively.

The image sensor 11 is disposed on the image space side of a photographing optical system (not shown), and captures a subject at the time of release to generate data for a recording image (main image). The output of the image sensor 11 is connected to the analog signal processing unit 12.
The imaging element 11 has a light receiving surface on which light receiving elements are two-dimensionally arranged, and generates an image signal by photoelectrically converting a subject image formed on the light receiving surface. In each light receiving element of the image sensor 11, any one of red (R), green (G), and blue (B) color filters (not shown) is arranged according to a known Bayer array. Therefore, each light receiving element of the image sensor 11 outputs an image signal corresponding to the color of the color filter by color separation in the color filter.

The analog signal processing unit 12 is an analog front end circuit that performs analog signal processing on the output of the image sensor 11. The analog signal processing unit 12 performs correlated double sampling, image signal gain adjustment, and A / D conversion of the image signal. The output of the analog signal processing unit 12 is connected to the image processing unit 13.
The image processing unit 13 is an ASIC that performs various types of image processing on the digital image signal. The image processing unit 13 includes a white balance correction unit 21, an interpolation processing unit 22, a first color correction unit 23, a gradation conversion unit 24, a color space conversion unit 25, and a second color correction unit 26. Have. The image processing unit 13 is connected to the recording I / F 17.

The white balance correction unit 21 performs white balance correction on the output signal from the analog signal processing unit 12 based on the white balance adjustment amount calculated by the CPU 15.
The interpolation processing unit 22 performs color interpolation processing on the digital image signal. Since the image sensor 11 can obtain only one color for each pixel corresponding to the light receiving element, the RAW data of the image sensor 11 is a mosaic image with respect to color. The interpolation processing unit 22 performs interpolation using color information obtained from surrounding pixels, thereby generating image data in which all colors (RGB) are aligned in all pixels. For example, the interpolation processing unit 22 generates green data corresponding to red pixels by interpolating from surrounding green pixels. Similarly, the interpolation processing unit 22 generates blue data corresponding to red pixels by interpolating from surrounding blue pixels.

The first color correction unit 23 performs a color conversion process on the image data after the interpolation process using a color conversion matrix corresponding to the color mode setting and also performs a memory color correction. The memory color correction is a correction that brings an image area of a color close to a color (skin color, sky blue, leaf green, etc.) that a human has a specific fixed concept closer to a color corresponding to the fixed concept.
The gradation conversion unit 24 executes gradation conversion processing of image data based on a gradation conversion curve indicating a correspondence relationship between input gradation values and output gradation values.

The color space conversion unit 25 converts color data expressed by RGB color space coordinate values to YCrCb color space coordinate values.
The second color correction unit 26 performs color correction processing for each hue, saturation adjustment, and the like on the image data after the interpolation processing.
The buffer memory 14 is connected to the image processing unit 13. The buffer memory 14 temporarily records image data in a pre-process and post-process of image processing by the image processing unit 13.

The CPU 15 is a processor that performs overall control of the electronic camera. The CPU 15 controls the operation of each part of the electronic camera according to the sequence program, and executes various calculations (AF, AE, etc.) necessary for shooting. The CPU 15 also functions as a white balance calculation unit 27 and a color correction calculation unit 28.
The white balance calculation unit 27 outputs the white balance adjustment amount to the white balance correction unit 21. This white balance adjustment amount corresponds to the gain (Gr) for the signal of the R pixel, the gain (Ggr) for the signal of the Gr pixel (G pixel aligned with the R pixel), and the signal of the Gb pixel (G pixel aligned with the B pixel). It is composed of a gain (Ggb) and a gain (Gr) for the B pixel signal.

  Here, when the auto white balance is set to ON, the white balance calculation unit 27 calculates the white balance adjustment amount based on the data of the main image. As an example, the white balance calculation unit 27 converts RGB data into TC-Duv coordinate data according to “Measurement method of light source distribution temperature and color temperature / correlated color temperature” of JISZ8725, and determines an achromatic color portion in image data. To do. When it is determined that the data after coordinate conversion is an achromatic color, the white balance calculation unit 27 applies the correlated color temperature obtained from the TC-Duv coordinates to a predetermined color temperature-white balance gain relational expression. The above white balance adjustment amount is calculated.

At this time, the white balance calculating unit 27 compares the color temperature information given in advance for each light source with the correlated color temperature, thereby obtaining the light source type Lk (sunny, cloudy, sunny, shade, light bulb) , Fluorescent lights, etc.).
Further, the white balance calculation unit 27 obtains a reliability P related to the calculation of auto white balance. As an example, the white balance calculation unit 27 divides the image into a plurality of regions and obtains a reliability value based on the number of regions determined to be achromatic in the divided regions. At this time, the white balance calculation unit 27 sets the reliability higher as the number of regions determined to be achromatic is larger.

  The white balance calculation unit 27 calculates auto white balance from the analysis data of the object scene (data indicating the luminance distribution and color distribution of the object scene) acquired from the image sensor 11 and the photometric sensor (not shown). The reliability may be adjusted. Specifically, the white balance calculation unit 27 compares the teacher data indicating scenes that are statistically frequently misjudged in the auto white balance calculation with the analysis data of the object scene, and uses a method such as pattern matching. Assess similarity. Then, the white balance calculation unit 27 sets the reliability of the auto white balance calculation to be low according to the high degree of similarity between the analysis data of the object scene and the teacher data.

  Here, the following examples are given as examples of scenes that are frequently erroneously determined by white balance computation (scenes in which teacher data is generated in advance). For example, a scene in which a lawn or a vegetation is photographed is often erroneously determined as a scene in which a fluorescent lamp or a mercury lamp is the light source. Further, a scene in which autumn leaves or a person's face is photographed is often mistakenly determined as a scene in which a light bulb is a light source. Furthermore, a scene where a shade is photographed and a scene where a blue sky is photographed are often misjudged.

The color correction calculation unit 28 executes various calculations based on the calculation result of the white balance calculation. For example, the color correction calculation unit 28 calculates the correction distribution ratio of the white balance correction unit 21 and the second color correction unit 26 in the image processing for the image data.
Further, the color correction calculation unit 28 sets parameters for color correction processing in the second color correction unit 26 (color correction amount θi for each hue, low saturation suppression amount D). In addition, the description regarding the calculation process of the color correction calculating part 28 is mentioned later.

The ROM 16 stores data of the first table and the second table.
FIG. 2 is a diagram schematically showing the contents of the first table. In the first table, data on the correspondence relationship between the value of the reliability P of the auto white balance calculation, the correction distribution ratio C, and the low saturation suppression amount D is recorded. The correction distribution ratio C in the second table indicates the value of the color correction sharing ratio occupied by the white balance correction unit 21.

FIG. 3 is a diagram schematically showing the contents of the second table. In the second table, data on the correspondence relationship between the light source type Lk of the image and the color correction amount θi for each hue is recorded. In the second table, the correspondence relationship between the light source type and the color correction amount for each hue is set to a different value when the reliability P exceeds the threshold Tp and when the reliability P is less than or equal to the threshold Tp. Yes.
In the example of FIG. 3, the light source type Lk is classified into natural light, a light bulb, and a fluorescent lamp. Further, the hue-specific color correction amount θi is a coordinate value in an arbitrary color system (for example, L ^* a ^* b ^* color system), which is a conversion amount when a color belonging to a predetermined hue region (i) is color-corrected. This is indicated by the amount of rotation. For example, when the value of the color correction amount in FIG. 3 is +1, it means that the color coordinate belonging to the hue is corrected to a color obtained by rotating the coordinate in the counterclockwise direction once. Similarly, when the value of the color correction amount in FIG. 3 is −1, it means that the color coordinates belonging to the hue are corrected to a color obtained by rotating the coordinate in the clockwise direction once. As the color correction amount θi for each hue, values in seven hues of red, flesh color, orange, yellow, green, blue, and purple are set.

  A connector for connecting the recording medium 29 is formed in the recording I / F 17. The recording I / F 17 executes data writing / reading with respect to the recording medium 29 connected to the connector. The recording medium 29 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the recording medium 29.

The release button 18 receives from the user an instruction input such as AF and AE calculation by a half-press operation and an input of a release instruction (exposure start instruction) by a full-press operation.
The operation unit 19 includes, for example, a command dial and a cross-shaped cursor key, and receives various setting input operations from the user. For example, the user can input (1) setting of shooting conditions, (2) setting of recording mode, and the like to the CPU 15 via the operation unit 19.

  Here, in the setting of the photographing condition in (1) above, the user can switch the white balance mode. In the recording mode setting of (2), the first recording mode for recording image data subjected to image processing and RAW data (image data in a state where no image processing is performed after A / D conversion). The user can switch to the second recording mode for recording the image.

Next, the photographing operation of the electronic camera in this embodiment will be described with reference to the flowchart of FIG. In the example of FIG. 4, the description will be made on the assumption that the white balance is set to the auto mode and the first recording mode is selected.
Step 101: The CPU 15 executes a photographing process for the main image in response to a full pressing operation of the release button 18. First, the CPU 15 drives the image sensor 11 to capture the main image. The data of the main image passes through the analog signal processing unit 12 in a pipeline manner and is input to the image processing unit 13.

Step 102: The white balance calculation unit 27 of the CPU 15 executes auto white balance calculation based on the data of the main image. Then, the white balance calculation unit 27 outputs white balance adjustment amounts (gain values of Gr, Ggr, Ggb, and Gb), light source type Lk, reliability P, correlated color temperature values, and the like as white balance information.
Step 103: The color correction calculation unit 28 of the CPU 15 refers to the first table and acquires the value of the correction distribution ratio C from the value of the reliability P (S102).

  Here, the value of the correction distribution ratio C in the first table is set so that the share ratio of the color correction in the white balance correction unit 21 increases in proportion to the reliability P. That is, the electronic camera of the present embodiment controls the color correction by the white balance correction unit 21 when the reliability P is high, and suppresses the color correction processing by the second color correction unit 26. In the electronic camera of this embodiment, when the reliability P is low, the color correction in the white balance correction unit 21 is suppressed, and the correction amount of the color correction process in the second color correction unit 26 is set large.

  For example, consider a case where the white balance calculation unit 27 determines that the light source type of the main image is a light bulb. In the above, when the value of the reliability P is high, it is considered that there is a high possibility that the scene of the bulb light source has been shot. Therefore, the electronic camera in this case can obtain an image having a natural color tone by performing white balance correction for the scene of the light bulb light source. Therefore, the necessity for color correction in the second color correction unit 26 is relatively low.

  On the other hand, when the value of the reliability P is low in the above, there is a possibility that it is erroneously determined as a light bulb light source scene even though an autumnal scene or the like is photographed. When the electronic camera performs white balance correction for a scene with a light bulb light source in the case of such an erroneous determination, the image is cast in blue and has an unnatural color tone. Therefore, in this case, it is preferable to correct the color cast by the color correction process of the second color correction unit 26 while suppressing the color correction in the white balance correction unit 21.

Step 104: The CPU 15 corrects the white balance adjustment amount (S102) by the following equation (1) based on the correction distribution ratio C (S103).
WBG _out = C × WBG + (1−C) × WBG _default (1)
Here, each symbol in the above equation (1) will be described. “WBG _out ” means a white balance adjustment amount output to the white balance correction unit 21. “C” means the value of the correction distribution ratio acquired in S103. “WBG” means the white balance adjustment amount acquired in S102. “WBG _default ” means the default value of the white balance adjustment amount used when the light source is unknown.

  Step 105: The white balance correction unit 21 of the image processing unit 13 performs white balance correction on the data of the main image based on the white balance adjustment amount (obtained in S104) output from the CPU 15. The data of the main image is subjected to color interpolation by the interpolation processing unit 22, and then passes through the first color correction unit 23, the gradation conversion unit 24, and the color space conversion unit 25 in a pipeline manner to perform various image processing. Will be. Thereafter, the data of the main image is input to the second color correction unit 26.

Step 106: The color correction calculation unit 28 determines whether or not the value of the reliability P (S102) exceeds the threshold value Tp. When the reliability P exceeds the threshold value Tp (YES side), the CPU 15 proceeds to S107. On the other hand, when the reliability P is equal to or less than the threshold value Tp (NO side), the CPU 15 proceeds to S108.
Step 107: The color correction calculation unit 28 refers to the data when the reliability P exceeds the threshold value Tp (P> Tp) in the second table, and determines the hue-specific color correction amount θi corresponding to the light source type Lk (S102). get.

  The hue-specific color correction amount θi in S107 is for adjusting the hue of an image that cannot be corrected by white balance correction when appropriate white balance correction is performed on the main image. In the above case, the tendency of the hue region where the color cast is conspicuous after the white balance correction is determined for each light source type. Therefore, by determining in advance the hue region for correcting the color cast and the amount of correction for each light source type, more accurate color cast correction can be realized.

  For example, when appropriate white balance correction is performed on a scene image of a fluorescent light source, purple is easily reproduced in the blue direction, and the skin color is easily reproduced in the yellow direction. Therefore, when the reliability P is high and the light source is determined to be a fluorescent light source, a correction amount for correcting the purple, flesh and orange hue regions is given in S107 (see FIG. 3). Note that the CPU 15 proceeds to S109 after the process of S107 is completed.

Step 108: The color correction calculation unit 28 refers to the data when the reliability P is equal to or less than the threshold value Tp (P ≦ Tp) in the second table, and determines the hue-specific color correction amount θi corresponding to the light source type Lk (S102). get.
The hue-specific color correction amount θi in S108 corrects the color of a specific hue region in order to reduce the influence of the erroneous determination when there is a high possibility that the light source type is erroneously determined.

For example, in the case where a natural light source scene is used and a white balance correction is performed due to an erroneous determination as a fluorescent light source, a magenta color cast becomes conspicuous in the skin color area of the image. Therefore, when the reliability P is low and the light source is determined to be a fluorescent light source, a correction amount for correcting the flesh color and green hue regions is given in S108 (see FIG. 3).
Step 109: The color correction calculation unit 28 corrects the hue-specific color correction amount θi (obtained in S107 or S108) based on the correction distribution ratio C (S103) by the following equation (2).

θi _out = (1−C) × θi _table (2)
Here, each symbol in the above equation (2) will be described. “Θi _out ” means the hue-specific color correction amount output to the second correction unit. “C” means the value of the correction distribution ratio acquired in S103. “Θi _table ” means the hue-specific color correction amount obtained from the second table in S107 or S108.

That is, when the share of color correction in the white balance correction unit 21 is high, the hue-specific color correction amount obtained in S109 is a small value. On the other hand, when the share of color correction in the second correction unit is high, the hue-specific color correction amount obtained in S109 is a large value.
Step 110: The second color correction unit 26 of the image processing unit 13 performs color correction processing for each hue on the data of the main image based on the color correction amount for each hue output from the CPU 15 (obtained in S109). Execute.

Step 111: The color correction calculation unit 28 refers to the first table, and acquires the value of the low saturation suppression amount D from the value of the reliability P (S102). Note that the value of the low saturation suppression amount D is set so that the value decreases according to the level of the reliability P (the level of the share of color correction in the white balance correction unit 21). .
Step 112: The second color correction unit 26 executes saturation adjustment processing on the data of the main image based on the low saturation suppression amount D (S111). In general, when a color cast remains in an image, the color cast at a portion to be reproduced with an achromatic color gives a greater impression to human vision than the color cast at a high saturation portion. For this reason, in S112, a color cast feeling of the main image is obtained by applying correction for suppressing saturation to a portion where saturation is slightly high due to low accuracy of white balance correction, which is originally an achromatic color. Improve.

Specifically, the saturation adjustment process in S112 is performed as follows. First, when the input image is expressed by saturation Sat = (Cr × Cr + Cb × Cb) ^0.5 , the second color correction unit 26 sets a range (0 ≦ Sat ≦ Tsat) where the saturation Sat is equal to or less than the threshold value Tsat. Set the degree adjustment target. Next, the second color correction unit 26 multiplies the saturation value in the above-described saturation adjustment target (the low saturation portion in the image) by the low saturation suppression amount D. Correction for suppressing the degree value is performed (that is, output saturation after correction = input saturation × D).

Note that the data of the main image subjected to the saturation adjustment processing is finally recorded on the recording medium 29 after being JPEG compressed by a compression processing unit (not shown). This is the end of the description of the series of operations in FIG.
Hereinafter, effects of the electronic camera of the present embodiment will be described. In the electronic camera of this embodiment, the correction distribution ratio between the white balance correction unit 21 and the second color correction unit 26 is changed according to the reliability related to auto white balance. In particular, when the reliability is low, by reducing the white balance correction level and increasing the color correction level, it is possible to effectively suppress the color cast of the image even in a scene where white balance is erroneously determined. it can.

In the electronic camera of this embodiment, color correction processing for each hue is executed according to the reliability of the white balance and the light source type. Thereby, since appropriate color correction processing corresponding to each scene is performed based on the result of white balance, it is possible to further suppress the color cast of the image.
Furthermore, in the electronic camera of the present embodiment, the degree of saturation adjustment for the low saturation portion of the image is varied according to the reliability of white balance. As a result, when white balance correction is insufficient, the low-saturation portion, which is greatly affected by color cast, approaches an achromatic color, so that the color cast feeling of the image can be greatly improved.

(Supplementary items of the embodiment)
(1) When shooting in the second recording mode of the above embodiment, the CPU 15 records the RAW data of the main image on the recording medium 29 as it is. Thereafter, when the RAW data is read from the recording medium 29 and development processing is performed, the electronic camera can perform the processing from S102 to S112 to perform image processing substantially similar to that described in the above embodiment. is there.

  Note that the white balance correction process and the color correction process shown in the above-described embodiment are not necessarily executed by the electronic camera. For example, a computer that reads the RAW data and performs development processing may execute an image processing program including the processing from S102 to S112 (illustration of the configuration of the computer is omitted).

(2) The various parameters of the above embodiment are merely examples. For example, the second table may be subdivided into three or more by providing a plurality of threshold values. Moreover, you may set the color correction amount according to hue with respect to another light source type about a 2nd table. Further, for the second table, the color correction amount for each hue may be selected based on the correlated color temperature instead of the light source type.
(3) In S104 of the above embodiment, the CPU 15 may use the product of the white balance adjustment amount (S102) and the correction distribution ratio C (S103) as an output value to the image processing unit 13 without considering the default value. (Ie, WBG _out = C × WBG).

(4) In S109 of the above embodiment, the color correction calculation unit 28 corrects the hue-specific color correction amount θi (S107) when the reliability P is high by the following equation (3). Also good.
θi _out = C × θi _table (3)
(5) In S112 of the above embodiment, the second color correction section 26 the saturation of other indicators ^{(e.g., L * a * b * a} * b * color space) may be determined by. Further, in the saturation adjustment in S112, the same processing is performed for all hues, but different saturation adjustment may be performed for each hue. Further, the second color correction unit 26 may perform a process of increasing the saturation in the saturation adjustment.

  It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

The block diagram which shows the structure of the electronic camera of this embodiment The figure which shows the content of the 1st table typically The figure which shows the content of the 2nd table typically Flowchart explaining photographing operation of electronic camera in this embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Image sensor, 13 ... Image processing part, 15 ... CPU, 21 ... White balance correction part, 22 ... Interpolation processing part, 23 ... 1st color correction part, 26 ... 2nd color correction part, 27 ... White balance calculation part , 28 ... Color correction calculation unit

Claims (7)

An image sensor that captures a subject image and generates image data;
An auto white balance calculation for obtaining a white balance adjustment amount for the image data, and a white balance calculation unit for obtaining a reliability of the auto white balance calculation;
An image processing unit that performs white balance correction and color correction processing on the image data after color interpolation processing;
A control unit that changes the correction distribution ratio of the white balance correction based on the white balance adjustment amount and the correction distribution ratio of the color correction processing based on the reliability value;
An imaging apparatus comprising: The imaging device according to claim 1,
The control unit increases the correction distribution ratio of the white balance correction based on the white balance adjustment amount and decreases the correction distribution ratio of the color correction processing according to the high reliability value. An imaging device. In the imaging device according to claim 1 or 2 ,
The image processing unit performs the color correction processing after the white balance correction based on the white balance adjustment amount . In the imaging device according to any one of claims 1 to 3 ,
The control unit changes the content of the color correction processing by further using white balance information other than the reliability acquired by the auto white balance calculation . In the imaging device according to any one of claims 1 to 4 ,
The image processing unit performs saturation control during the color correction process. In the imaging device according to any one of claims 1 to 5 ,
The image processing unit performs color correction for each hue at the time of the color correction processing. A first step of reading data of an image captured by an image sensor;
A second step of performing an auto white balance calculation for determining a white balance adjustment amount for the image data, and determining a reliability of the auto white balance calculation;
A third step of changing a correction distribution ratio of white balance correction based on the white balance adjustment amount and a correction distribution ratio of color correction processing based on the reliability value;
A fourth step of executing the white balance correction and color correction processing on the image data after the color interpolation processing;
A program that causes a computer to execute.

Images