JP2009022044A - Image processing apparatus and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and image processing program with which adjustment can be performed, in such a way as to obtain an image of optimal saturation, after gradation correction. <P>SOLUTION: The present invention relates to an image processing apparatus comprising a photographic situation estimating section 18 for estimating a photographic situation, on the bais of photometric information of a photometric evaluating section 7 and focusing information of a focal point detecting section 8; a Y/C-separating section 12 for separating a long-time exposure image and a short-time exposure image into a luminance signal and a color difference signal, respectively; a proper exposure extracting section 13 for extracting a proper exposure area, on the basis of a luminance signal level of the long-time exposure image; a luminance-correcting section 32 for performing gradation correction, by weighting the luminance signal in the proper exposure area, according to the photographic situation; a color difference correcting section 33 for correcting the color difference signal in the proper exposure area, on the basis of the luminance signal, before and after the gradation correction and theoretical limit characteristics of color reproduction; a Y/C-combining section 34 for combining the luminance signal and the color difference signal, after the correction; and an image combining section 15 for combining the long-time exposure image and the short-time exposure image after Y/C-combining, to produce a wide dynamic range image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing program, and more particularly to an image processing apparatus and an image processing program that generate one wide dynamic range image from a plurality of images captured under different exposure conditions.

  2. Description of the Related Art Conventionally, there has been proposed an image processing apparatus that generates a single wide dynamic range image by combining a plurality of images captured under different exposure conditions.

  As an example of such a technique, Japanese Patent Application No. 11-338551 (Japanese Patent Laid-Open No. 2000-228747) divides each image into a proper exposure area and an improper exposure area, and a gradation for each appropriate exposure area. An image processing apparatus is described that performs correction and synthesizes a proper exposure area for each image that has been subjected to gradation correction to generate one wide dynamic range image, and further, an example of an apparatus to which this image processing apparatus is applied Describes a super latitude digital camera that can image a subject in a wider dynamic range.

  The gradation conversion processing in this super latitude digital camera is performed based on the histogram flattening of the edge portion, and this histogram flattening indicates that the main subject has many edges and the non-major portion such as the background has few edges. This is a prerequisite technology.

On the other hand, in a conventional digital camera, a color difference signal is also converted based on a coefficient when a luminance signal is subjected to gradation conversion. That is, if the luminance Yorg is converted by the gradation conversion characteristic F as Ytra = F (Yorg), conventionally, the luminance signal conversion coefficient gain is
gain = Ytra / Yorg
Using this conversion coefficient as it is,
Cbtra = gain ・ Cborg
Crtra = gain ・ Crorg
The color difference signal was converted as shown in FIG.
JP 2000-228747 A

  Histogram flattening technology based on the premise that the main subject has many edges as described above can deal with a relatively wide range of subjects, but may not be able to cope with it. An example that may fall under such an exception is when a person is shown relatively small under a background having a complicated shape or outline. At this time, since many edges are detected from the background portion, it is determined that the background is the main subject, and the gradation width assigned to the person is reduced.

  Further, in the technique of converting the gradation of the color difference signal as described above using the same conversion coefficient as that of the luminance signal, an unnatural color may be generated in the high luminance portion. That is, there is a theoretical limit characteristic (see FIG. 11 showing the embodiment of the present invention) for color reproduction in a color space (for example, Y, Cb, Cr space). As Y increases, the color difference range in which color reproduction is possible widens, and when a certain luminance Y is exceeded, the color difference range in which color reproduction can be performed is narrowed. In other words, if the luminance is low, the entire image becomes blackish, so the color reproduction range is narrow, and with a suitable luminance, a wide range of colors can be reproduced. It is a characteristic that

  If gradation conversion is performed in the same way as luminance without considering such a color reproduction range, the color reproduction range may approach or exceed the limit, and the color after gradation conversion may become whitish. It was. In order to deal with these issues, processing to suppress the saturation of high-brightness areas has been added in the past, but this is not sufficient, and the realization of a technology that further improves color reproducibility is desired. ing.

  The present invention has been made in view of the above circumstances, and by considering the theoretical limit characteristics of color reproduction, it is possible to make adjustments so that an image with optimum saturation can be obtained after gradation correction. An object of the present invention is to provide an image processing apparatus and an image processing program.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention processes an image group made up of a plurality of images captured under different exposure conditions on the same subject to produce one wide dynamic range image. An image processing device to generate,
Luminance color difference separation means for separating an image signal into a luminance signal and a color difference signal for each image in the image group,
Extraction means for extracting an appropriate exposure area based on the signal level of the luminance signal;
Gradation correction means for performing gradation correction on the luminance signal in the appropriate exposure area;
The appropriate exposure based on the luminance signal before gradation correction output from the luminance / color difference separation means, the luminance signal after gradation correction output from the gradation correction means, and the theoretical limit characteristic of color reproduction. Color difference correction means for correcting the color difference signal of the area;
Luminance color difference synthesizing means for synthesizing the luminance signal after gradation correction and the color difference signal after correction into an original image signal;
A synthesizing means for generating one wide dynamic range image by synthesizing the synthesized image signals of the appropriate exposure area;
It has.

An image processing program according to an aspect of the present invention is an image processing program for processing an image group composed of a plurality of images captured with different exposure conditions on the same subject to generate one wide dynamic range image. And
A luminance color difference separation step for separating the image signal into a luminance signal and a color difference signal for each image in the image group;
An extraction step for extracting an appropriate exposure area based on the signal level of the luminance signal;
A gradation correction step for performing gradation correction on the luminance signal in the appropriate exposure area;
The appropriate exposure based on the luminance signal before gradation correction output by the luminance / chrominance separation step, the luminance signal after gradation correction output by the gradation correction step, and the theoretical limit characteristic of color reproduction. A color difference correction step for correcting the color difference signal of the area;
Luminance color difference synthesis step for synthesizing the luminance signal after gradation correction and the color difference signal after correction to the original image signal;
A synthesizing step for generating one wide dynamic range image by synthesizing the synthesized image signals of the appropriate exposure area;
Is executed by a computer.

  The image processing apparatus and the image processing program according to the present invention make it possible to perform adjustment so that an image having optimum saturation can be obtained after gradation correction by considering the theoretical limit characteristics of color reproduction.

Embodiments of the present invention will be described below with reference to the drawings.
1 to 7 show a first embodiment of the present invention, and FIG. 1 is a block diagram showing a basic configuration of an electronic camera.

  In the present embodiment, the image processing apparatus of the present invention is applied to an electronic camera. For simplicity, two images of a short-time exposure image and a long-time exposure image are combined to create a single wide dynamic range image. However, the present invention can be applied to a case where a larger number of images are synthesized.

  This electronic camera is composed of a single-plate color CCD having an electronic shutter function, a CCD 4 for photoelectrically converting a subject image and outputting it as an image signal, and a lens system for forming a subject image on the CCD 4 1, a diaphragm 2 for controlling the passage range of the light beam that has passed through the lens system 1, a low-pass filter 3 that is an optical filter for removing unnecessary high-frequency components from the light beam that has passed through the diaphragm 2, and An A / D converter 5 for converting an analog image signal which has been output from the CCD 4 and then subjected to removal of noise components by a correlated double sampling circuit (not shown) and then amplified to a digital signal, and this The image data for one screen digitized by the A / D converter 5 is accumulated, and an image by long exposure and an image by short exposure are stored. The image data is read out from the first image buffer 6a and the second image buffer 6b to be stored and the first image buffer 6a used for accumulating the photometry and focus detection data. Image data is read from the first image buffer 6a and the photometric evaluation section 7 for controlling the aperture diameter of the diaphragm 2 and the electronic shutter of the CCD 4 so as to obtain a brightness distribution and obtain an appropriate exposure at the time of imaging. An in-focus detection unit 8 that performs detection and controls an AF motor 9 to be described later based on the detection result, and an AF lens of the lens system 1 is driven by the in-focus detection unit 8 to drive an object on the CCD 4 A single plate image data read out from the AF motor 9 for forming an image and the first and second image buffers 6a and 6b is interpolated to be converted into three plate image data. Interpolating unit 10, working buffer 11 for accumulating the interpolated image data, and luminance / chrominance separation for separating the three-plate image data read from the working buffer 11 into luminance signal Y and color difference signals Cb and Cr. First, the luminance signal Y is read from the Y / C separation unit 12 and the Y / C separation unit 12 to determine whether or not each pixel constituting the entire screen has an appropriate exposure according to the signal level. The appropriate exposure extraction unit 13 serving as an extraction unit that discriminates and outputs divided image information based on the result, the photometric information output from the photometric evaluation unit 7 and the focus output from the in-focus detection unit 8. As will be described in detail later based on the information, a shooting situation estimation unit 18 serving as a shooting situation estimation unit that estimates a shooting situation, and an estimation result obtained by the shooting situation estimation unit 18 is referred to as a feature amount. A conversion characteristic calculation unit 14 which is a gradation correction unit that performs weight conversion when calculating the histogram of the image, calculates conversion characteristics, and performs gradation conversion of the appropriate exposure area output from the appropriate exposure extraction unit 13; A wide dynamic image obtained by synthesizing the image related to the long-time exposure and the image related to the short-time exposure after the gradation conversion output from the conversion characteristic calculation unit 14 while referring to the area information output from the exposure extraction unit 13. An image synthesis unit 15 that is a synthesis means for generating a range image, an output unit 16 that outputs a wide dynamic range image synthesized by the image synthesis unit 15 to, for example, a recording medium or a display device, and the photometric evaluation unit 7 While receiving the detection result of the focus detection part 8, the said interpolation part 10, the appropriate exposure extraction part 13, the imaging | photography condition estimation part 18, the conversion characteristic calculation part 14, the image composition part 15, and an output part It is configured to include a control unit 17 for controlling the entire electronic camera containing 6.

  Next, FIG. 2 is a block diagram showing a detailed configuration of the photographing situation estimation unit 18.

  Focus (AF) information output from the focus detection unit 8 and input via the control unit 17 is input to a focus position estimation unit 20 serving as a focus position estimation unit. It is classified into one of three types (see FIG. 5): landscape photography (5 m to ∞), person photography (1 m to 5 m), and close-up photography (1 m or less).

  Photometric (AE) information output from the photometric evaluation unit 7 and input via the control unit 17 is input to the subject distribution estimation unit 21 serving as subject distribution estimation means, and the luminance distribution is classified into several categories. Is done.

  Specifically, first, the photometric evaluation section 7 classifies the area on the CCD 4 into 13 as shown in FIG. 4, for example, and performs divided photometry. FIG. 4 is a diagram showing an example of a division pattern for evaluation photometry.

  That is, the middle region at the center is a1, the left neighbor is a2, and the right neighbor is a3.

  Further, the upper and lower sides of the region a1 in the inner peripheral portion surrounding the central portion are a4 and a5, the left and right of the region a4 are a6 and a7, and the left and right of the region a5 are a8 and a9. .

  The upper left of the outer periphery surrounding the inner periphery is a10, the upper right is a11, the lower left is a12, and the lower right is a13.

In the divided photometry in such a region, the subject distribution estimation unit 21 calculates the following evaluation parameters.
[Equation 1]
S1 = | a2 -a3 |
[Equation 2]
S2 = max (| a4 -a6 |, | a4 -a7 |)
[Equation 3]
S3 = max (a10, a11) -Av
Av = (Σai) / 13

  That is, the evaluation parameter S1 indicates the difference in luminance between the left and right of the centermost part, and the evaluation parameter S2 indicates the larger of the difference in luminance between the upper center of the inner peripheral part and the upper and left and right. The parameter S3 indicates the difference between the larger one of the upper left and right sides of the outer peripheral portion and the average luminance of the entire screen.

  Such an evaluation parameter is obtained from the subject distribution estimation unit 21, and the classification of the focus position is obtained from the focus position estimation unit 20, and the integration unit 22 as the integration means performs the integrated classification as shown in FIG. I do. FIG. 5 is a chart showing scene classification patterns from AF information and AE information.

  As shown in the figure, when the AF information is 5 m to ∞, it is determined that the scene photography is performed, and the evaluation parameter S3 is further compared with a predetermined value Th1. At this time, if the evaluation parameter S3 is larger than the predetermined value Th1, at least one of a10 and a11 has a brightness higher than the average brightness of the entire screen to some extent, so that the scenery is empty at the top. (Type 1). On the other hand, if the evaluation parameter S3 is smaller than the predetermined value Th1, on the contrary, it is determined that the scenery is small even if there is no sky in the upper part (Type 2).

  Next, when the AF information is 1 m to 5 m, it is determined that the image is a person image, and the evaluation parameter S2 is further compared with a predetermined value Th2. At this time, if the evaluation parameter S2 is larger than the predetermined value Th2, it is determined that the portrait is a single person (Type 3), whereas if it is smaller than the predetermined value Th2, it is determined that the portrait is a plurality of persons (Type 4). .

  Further, when the AF information is 1 m or less, it is determined that close-up photography is performed, and the evaluation parameter S1 is further compared with a predetermined value Th3. At this time, if the evaluation parameter S1 is larger than the predetermined value Th3, it is determined that the single object is close-up (Type 5), whereas if it is smaller than the predetermined value Th3, it is determined that the plurality of objects are close-up. (Type 6).

  The result of classification into these types is output from the integration unit 22 to the conversion characteristic calculation unit 14.

  FIG. 3 is a block diagram showing a detailed configuration of the conversion characteristic calculation unit.

  As described above, the appropriate exposure extraction unit 13 first reads the luminance signal Y of the long-time exposure image and compares the signal level of each pixel constituting the entire screen with a predetermined value, so that the pixel is determined appropriately. The set of pixels determined to be proper exposure becomes the proper exposure area for long exposure, and the part other than the proper exposure area for this long exposure is exposed for a short time. The appropriate exposure area is set.

  When the luminance signal Y of the appropriate exposure area for the long time exposure output from the appropriate exposure extraction unit 13 is input to the edge extraction unit 26 serving as a feature amount calculation unit, the edge extraction unit 26 controls the control unit 17. Edge detection is performed based on the above. Specifically, the edge extraction unit 26 is a general edge detection operator such as Laplacian or Sobel. If the strength by the edge detection operator is equal to or greater than a predetermined threshold, an edge exists at the reference position. If not, binary information indicating that it is not an edge is output.

  On the other hand, when the result of the type classification by the photographing situation estimation unit 18 is input to the weight pattern selection unit 24 as selection means, the weight pattern selection unit 24 is based on the control of the control unit 17 as shown in FIG. A weight pattern corresponding to the type is selected from the weight pattern ROM 25 in which a plurality of weight patterns are stored in advance. FIGS. 6A and 6B are diagrams showing weighting coefficients at the time of calculating a histogram based on the classification pattern as shown in FIG. 5. FIG. 6A shows the Type 1, FIG. 6B shows the Type 2, and FIG. 6 (C) shows the type 3 pattern, FIG. 6 (D) shows the type 4 type, FIG. 6 (E) shows the type 5 type, and FIG. 6 (F) shows the type 6 type pattern.

  Thus, the histogram creation unit 27 as a histogram creation unit calculates an edge histogram indicating the appearance frequency with respect to the luminance level for the pixels constituting the edge and its neighboring pixels based on the result output from the edge extraction unit 26. When creating this histogram, as shown in FIG. 6, the weighting is performed according to the pixel position in the image. Further, the histogram creation unit 27 converts the calculated edge histogram into a cumulative edge histogram by, for example, integrating it.

  The conversion curve calculation unit 28 serving as a gradation conversion curve calculation unit generates a target histogram by convolving the edge histogram using a Gaussian kernel or the like, and the target edge and the cumulative edge histogram output from the histogram creation unit 27. Are used to calculate the tone curve that is the tone correction characteristic.

  The conversion unit 29 serving as a conversion unit performs gradation correction on the image data input from the appropriate exposure extraction unit 13 based on the tone curve obtained from the conversion curve calculation unit 28, and the image data after the gradation correction is obtained. The image is output to the image composition unit 15. The conversion unit 29 first performs tone correction of the luminance signal Y related to the long exposure, and then sequentially performs tone correction of the color difference signals Cb and Cr related to the long exposure and outputs them to the image composition unit 15. Thereafter, the luminance signal Y and the color difference signals Cb and Cr relating to the short-time exposure are similarly subjected to gradation correction and output to the image composition unit 15.

  The image composition unit 15 first receives the luminance signal Y and the color difference signals Cb and Cr after gradation correction related to long exposure, generates, for example, an RGB signal related to the long exposure, and then short exposure. The luminance signal Y and the color difference signals Cb and Cr after the gradation correction according to the above are received, and the same RGB signal related to the short-time exposure is generated. These are then combined to generate and output one wide dynamic range image.

  Next, FIG. 7 is a flowchart showing image conversion processing.

  A subject image formed on the CCD 4 made of a single plate is captured a plurality of times under different exposure conditions. As described above, for example, imaging by long exposure and imaging by short exposure are performed in this manner. It is performed in order and is sequentially output as an image signal.

  These image signals are converted into digital signals by the A / D converter 5 and then stored in the first image buffer 6a and the second image buffer 6b, respectively.

  The photometry evaluation unit 7 and the in-focus detection unit 8 perform the AE information and the AF information as described above based on the long-exposure image data stored in the first image buffer 6a. It outputs to the control part 17 (step S1).

  On the other hand, the image data stored in the first image buffer 6a and the second image buffer 6b are sequentially sent to the interpolation unit 10 to perform interpolation for each of the R image data, G image data, and B image data. By doing so, it is converted into image data of three plates (step S2) and stored in the work buffer 11.

The Y / C separation unit 12 reads the RGB image data from the work buffer 11 and calculates the luminance signal Y and the color difference signals Cb and Cr as shown in the following Equation 4 (step S3).
[Equation 4]
Y = 0.29900R + 0.58700G + 0.14400B
Cb = −0.16874R−0.33126G + 0.50000B
Cr = 0.50000R-0.41869G-0.0811B

  The proper exposure extraction unit 13 compares the signal level of the luminance signal Y among these with a predetermined threshold value for each pixel, and determines whether or not it belongs to the proper exposure area, thereby extracting divided image information. And output (step S4).

  Thereafter, the edge extraction unit 26 in the conversion characteristic calculation unit 14 applies a known second-order differential filter such as Laplacian to the luminance signal Y to extract an edge component (step S5), and the extracted edge component has, for example, a standard deviation. Binarization processing is performed by setting a threshold value of about twice (step S6).

  On the other hand, the shooting situation estimation unit 18 estimates the shooting situation based on the AF information and the AE information as described above (step S7), and selects whether the weight pattern is Type 1 to Type 6 (step S7). S8). Then, the weighting coefficient as shown in FIG. 6 corresponding to the selected weighting pattern is read from the weighting pattern ROM 25 (step S9).

Thus, an edge histogram weighted by the histogram creating unit 27 is created based on the edge component binarized in step S6 and the weight pattern read in step S9 (step S10), and further from this edge histogram. A cumulative edge histogram is generated (step S11).
Based on the cumulative edge histogram thus obtained, the conversion curve calculation unit 28 calculates a gradation conversion curve (step S12).

  In the subsequent conversion unit 29, the luminance signal Y and the color difference signals Cb and Cr output from the appropriate exposure extraction unit 13 are converted by the gradation conversion curve obtained from the conversion curve calculation unit 28 (step S13). The converted image data is output (step S14).

  In the above description, both the photometric information and the focusing information are used to estimate the shooting situation. However, it is possible to estimate and change the weight using only one of them. Not only photometric information and focusing information, but also zoom position information, multi-spot photometric information, line-of-sight input information, flash emission information, information on detection sensors for detecting the vertical and horizontal positions of electronic cameras, white balance information, etc. By referring to one or more, it is possible to estimate the shooting situation in more detail.

  Furthermore, the gradation correction technique according to the shooting situation as described above is not only applied to a color image, but can also be applied to a monochrome image.

  In the first embodiment, gradation correction is performed according to the shooting situation by an image processing device configured as a circuit in the electronic camera. Such processing is performed by a computer processing program. Is also possible. In this case, shooting information such as photometry information and focus information is recorded in the header portion of the image file, for example, and the shooting situation is estimated on the basis of the shooting information in the computer. Tone correction may be performed.

  The image processing apparatus is not limited to application to an electronic camera, and can be widely applied to devices that handle images, such as a printer apparatus.

  According to the first embodiment, the shooting situation is determined based on shooting information such as focusing information and photometry information, and weighting according to the shooting situation is performed when an edge histogram is created. In addition, it is possible to perform gradation correction most suitable for the shooting scene in consideration of the main subject.

  8 to 12 show a second embodiment of the present invention. FIG. 8 is a block diagram showing a basic configuration of the electronic camera, and FIG. 9 is a block diagram showing a detailed configuration of a luminance correction unit. FIG. 10 is a block diagram showing the detailed configuration of the color difference correction unit, FIG. 11 is a diagram showing how color difference correction is performed in consideration of the theoretical limit characteristics of color reproduction, and FIG. 12 is a flowchart showing image conversion processing. is there.

  In the second embodiment, the same parts as those in the first embodiment described above are denoted by the same reference numerals, description thereof is omitted, and only different points will be mainly described.

  In the second embodiment, the luminance signal Y and the color difference signals Cb and Cr separated by the Y / C separation unit 12 pass through the appropriate exposure extraction unit 13 and the luminance correction unit 32 and the color difference correction unit as gradation correction units. The color difference correction unit 33 is input to each.

  As shown in FIG. 9, the luminance correction unit 32 receives the luminance signal Y of the appropriate exposure area output from the appropriate exposure extraction unit 13, and performs gradation correction for luminance, and features amount calculation means The edge extracting unit 40, a histogram creating unit 41 as a histogram creating unit, a conversion curve calculating unit 42 as a gradation conversion curve calculating unit, and a luminance converting unit 43 as a luminance converting unit are configured.

  The processing in the luminance correction unit 32 will be described with reference to FIG.

  The luminance correction unit 32 reads the luminance signal Y of the appropriate exposure area output from the appropriate exposure extraction unit 13 (step S21), and the edge extraction unit 40 applies a filter such as Laplacian to extract the edge component (step S21). S22) A binarization process is performed for determining whether each pixel is an edge by comparing with a predetermined threshold value (step S23).

  Based on the information output from the edge extraction unit 40, the histogram generation unit 41 generates an edge histogram indicating the appearance frequency of the edge with respect to the luminance (step S24), and further generates an accumulated edge histogram by integrating it. (Step S25).

  The conversion curve calculation unit 42 uses the accumulated edge histogram output from the histogram creation unit 41 and the like to calculate a tone curve that serves as a gradation correction characteristic as described above (step S26).

  Based on this conversion curve, the luminance conversion unit 43 performs gradation conversion of the luminance signal Y under the control of the control unit 17 (step S27), and outputs it to the color difference correction unit 33, and at the same time, Y / The data is output to the C composition unit 34 (step S28).

  Thus, if the luminance signal before gradation correction output from the appropriate exposure extraction unit 13 is Yorg, and the luminance signal after gradation correction by the luminance conversion unit 43 is Ytra, these luminance signals Yorg and Ytra is used when the color difference correction unit 33 corrects the gradation of the color difference as described below.

  As shown in FIG. 10, the color difference correction unit 33 receives the color difference signals Cb and Cr of the appropriate exposure area output from the appropriate exposure extraction unit 13, and performs gradation correction for the color difference. A first correction coefficient calculation unit 47 as a calculation means, a second correction coefficient calculation unit 45 as a second calculation means, a color reproduction limit characteristic ROM 46, and a color difference conversion unit 48 as a color difference conversion means. Has been.

In the color difference correction unit 33, the first correction coefficient calculation unit 47 receives the luminance signal Yorg before gradation correction from the appropriate exposure extraction unit 13, and then sets the color reproduction range borg corresponding to the luminance signal Yorg. Calculation is performed as shown in Equation 5 (step S31).
[Equation 5]
borg = B (Yorg)

  Here, the function B (Y) is a function indicating the theoretical limit characteristics of color reproduction in the color space (Y, Cb, Cr space). For example, as schematically shown in FIG. As a result, the color difference range capable of color reproduction is widened, and when a certain luminance Y is exceeded, the color difference range capable of color reproduction is narrowed.

  The calculation shown in Equation 5 is performed by, for example, referring to the table data stored in the color reproduction limit characteristic ROM 46 (step S30) and obtaining the color reproduction range borg corresponding to the luminance Yorg. .

  The color reproduction limit characteristic ROM 46 stores in advance the function B (Y) indicating the theoretical limit characteristic of color reproduction as table data. Here, the ROM 46 takes into consideration the load and processing speed due to the calculation. However, of course, the calculation may be actually performed.

Next, the second correction coefficient calculation unit 45 receives the luminance signal Ytra after gradation correction from the luminance correction unit 32, and the color reproduction range btra corresponding to the luminance signal Ytra is the same as that in the above-described equation 5. The calculation is performed as shown in the following equation 6 (step S32).
[Equation 6]
btra = B (Ytra)

  Similarly, the calculation shown in Equation 6 is performed by referring to the table data stored in the color reproduction limit characteristic ROM 46 (step S30) to obtain the color reproduction range btra corresponding to the luminance Ytra. Do.

The color difference conversion unit 48 calculates the conversion coefficient gainc for the color difference signal as shown in the following Expression 7, based on the borg that is the first correction coefficient and the btra that is the second correction coefficient.
[Equation 7]
gainc = btra / borg

  Thus, the conversion coefficient gainc for the color difference signal is a ratio between the theoretical limit characteristic borg of color reproduction with the luminance signal Yorg before gradation correction and the theoretical limit characteristic btra of color reproduction with the luminance signal Ytra after gradation correction. In this way, faithful color reproduction can be performed while maintaining saturation without causing overexposure, such as when using the same conversion coefficient as luminance.

When the conversion coefficient gain is obtained, the uncorrected color difference signals Cborg and Crorg are sequentially received from the appropriate exposure extraction unit 13 (step S29), and the corrected color difference signals Cbtra and Crtra are expressed by the following equation (8). (Step S33).
[Equation 8]
Cbtra = gainc ・ Cborg
Crtra = gainc ・ Crorg

  The color difference signals Cbtra and Crtra thus converted are output to the Y / C synthesis unit 34 (step S34).

  The Y / C synthesis unit 34 Y / C-combines the luminance signal Ytra after gradation conversion and the color-difference signals Cbtra and Crtra after conversion to convert them into RGB signals, for example, and outputs them to the image synthesis unit 15.

  In this image synthesizing unit 15, an appropriate exposure image portion after gradation conversion related to long-time exposure and an appropriate exposure image portion after gradation conversion related to short-time exposure are combined to generate one wide dynamic range image. After that, the data is output from the output unit 16.

  In the second embodiment, tone correction is performed in consideration of the theoretical limit characteristics of color reproduction by an image processing apparatus configured as a circuit in an electronic camera. Such processing is performed by a computer processing program. It is also possible to do this.

  The image processing apparatus is not limited to application to an electronic camera, and can be widely applied to devices that handle images, such as a printer apparatus.

  According to the second embodiment, since the tone correction is performed with respect to the color difference signal in consideration of the theoretical limit characteristic of color reproduction, an appropriate saturation is maintained even when the tone correction of the image is performed. can do.

  FIG. 13 shows a third embodiment of the present invention and is a block diagram showing a basic configuration of an electronic camera.

  In the third embodiment, portions that are the same as those in the first and second embodiments described above are denoted by the same reference numerals, description thereof is omitted, and only differences are mainly described.

  The third embodiment is configured to exhibit a function combining the functions of the first embodiment and the second embodiment described above, that is, to calculate the gradation conversion characteristic of the luminance signal. In this case, weighting is performed according to the shooting situation, and the theoretical limit characteristic of color reproduction is taken into consideration when calculating the conversion characteristic of the color difference signal.

  That is, the result of classification of the shooting situation as shown in FIG. 5 estimated by the shooting situation estimation unit 18 is input to the luminance correction unit 32, and the luminance correction unit 32 uses the edge of the luminance signal. When creating a histogram, weighting as shown in FIG. 6 is performed.

  When the luminance signal that has been tone-converted based on the characteristic curve thus obtained is input to the color difference correction unit 33, a table showing the theoretical limit characteristics of color reproduction is referred to, as in the second embodiment described above. Accordingly, the second correction coefficient is calculated, and similarly, the first correction coefficient is calculated based on the luminance signal before gradation conversion. A conversion coefficient related to the color difference signal is calculated based on the first correction coefficient and the second correction coefficient, and conversion suitable for the color difference is performed and output to the Y / C synthesis unit 34.

  The Y / C synthesis unit 34 Y / C-combines the luminance signal after gradation conversion output from the luminance correction unit 32 and the color difference signal after conversion output from the color difference correction unit 33 to perform image synthesis unit 15. The image composition unit 15 synthesizes the appropriate exposure image portion after gradation conversion related to long-time exposure and the appropriate exposure image portion after gradation conversion related to short-time exposure to produce a wide dynamic image. After the range image is generated, it is output from the output unit 16.

  In the third embodiment, tone correction is performed as a circuit in the electronic camera, but such processing can also be performed by a computer processing program.

  The image processing apparatus is not limited to application to an electronic camera, and can be widely applied to devices that handle images, such as a printer apparatus.

  According to such 3rd Embodiment, both the effect of 1st Embodiment mentioned above and the effect of 2nd Embodiment can be show | played.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit of the invention.

  Further, according to the image processing apparatus of the present invention according to claim 1, since the color difference signal is corrected based on the luminance signal before and after the gradation correction and the theoretical limit characteristic of the color reproduction, it is optimal after the gradation correction. An image with high saturation can be obtained.

  According to the image processing apparatus of the present invention according to claim 2, since the photographing situation is estimated and the gradation correction of the luminance signal is performed accordingly, it is possible to adaptively adjust the gradation width of the main subject. In addition, since the color difference signal is corrected based on the luminance signal before and after the gradation correction and the theoretical limit characteristic of color reproduction, an image with optimum saturation can be obtained after the gradation correction.

  According to the image processing apparatus of the present invention according to claim 3, the same effect as that of the invention according to claim 2 can be obtained, and the photographing situation can be appropriately estimated based on the focus information and the photometric information. It becomes.

  According to the image processing apparatus of the present invention of claim 4, the same effect as that of the invention of claim 2 is achieved, and the gradation correction unit creates a histogram by weighting the feature quantity of the appropriate exposure area. By calculating the tone conversion curve, appropriate gradation conversion can be performed in consideration of the main subject.

  According to the image processing apparatus of the present invention of claim 5, the same effect as that of the invention of claim 1 is achieved, and the gradation correcting means creates a histogram from the feature amount of the appropriate exposure area and performs gradation conversion. By calculating the curve, appropriate gradation conversion can be performed.

  According to the image processing apparatus of the present invention according to claim 6, the same effect as the invention according to claim 1 or claim 2 is obtained, and the color difference correcting means is a luminance signal before gradation correction and the theory of color reproduction. By using the first correction coefficient calculated from the limit characteristic and the second correction coefficient calculated from the luminance signal after gradation correction and the theoretical limit characteristic of color reproduction, the conversion of the color difference signal is appropriately performed. It can be carried out.

  According to the image processing program of the present invention according to claim 7, since the color difference signal is corrected based on the luminance signal before and after the gradation correction and the theoretical limit characteristic of the color reproduction, the optimum saturation after the gradation correction. Images can be obtained.

1 is a block diagram showing a basic configuration of an electronic camera according to a first embodiment of the present invention. The block diagram which shows the detailed structure of the imaging | photography condition estimation part of the said 1st Embodiment. The block diagram which shows the detailed structure of the conversion characteristic calculation part of the said 1st Embodiment. The figure which shows an example of the division | segmentation pattern for evaluation photometry in the said 1st Embodiment. The chart which shows the classification pattern of the scene from AF information and AE information in the said 1st Embodiment. The figure which shows the weighting coefficient at the time of the edge histogram calculation based on a classification pattern as shown in the said FIG. 5 in the said 1st Embodiment. 5 is a flowchart showing image conversion processing in the first embodiment. The block diagram which shows the basic composition of the electronic camera of the 2nd Embodiment of this invention. The block diagram which shows the detailed structure of the brightness | luminance correction | amendment part of the said 2nd Embodiment. The block diagram which shows the detailed structure of the color difference correction | amendment part of the said 2nd Embodiment. FIG. 6 is a diagram showing how color difference correction is performed in consideration of the theoretical limit characteristics of color reproduction in the second embodiment. 7 is a flowchart showing image conversion processing in the second embodiment. The block diagram which shows the basic composition of the electronic camera of the 3rd Embodiment of this invention.

Explanation of symbols

7: Photometric evaluation section 8: Focus detection section 12 ... Y / C separation section (luminance color difference separation means)
13 ... Proper exposure extraction unit (extraction means)
14 ... Conversion characteristic calculation section (tone correction means)
15: Image composition unit (composition means)
16 ... Output unit 17 ... Control unit 18 ... Shooting situation estimation unit (shooting situation estimation means)
20: Focus position estimation unit (focus position estimation means)
21 ... Subject distribution estimation unit (subject distribution estimation means)
22 ... Integration unit (integration means)
24 ... Weight pattern selection section (selection means)
25 ... ROM for weight pattern
26, 40... Edge extraction unit (feature amount calculation means)
27, 41 ... Histogram creation section (histogram creation means)
28, 42 ... conversion curve calculation unit (tone conversion curve calculation means)
29 ... Conversion unit (conversion means)
32. Brightness correction unit (gradation correction means)
33 ... Color difference correction unit (color difference correction means)
34... Y / C synthesis unit (luminance color difference synthesis means)
43 ... luminance conversion section (luminance conversion means)
45. Second correction coefficient calculation unit (second calculation means)
46 ... Color reproduction limit characteristic ROM
47. First correction coefficient calculation section (first calculation means)
48 .. Color difference conversion unit (color difference conversion means)

Claims (7)

An image processing apparatus that generates a single wide dynamic range image by processing an image group consisting of a plurality of images captured under different exposure conditions for the same subject,
Luminance color difference separation means for separating an image signal into a luminance signal and a color difference signal for each image in the image group,
Extraction means for extracting an appropriate exposure area based on the signal level of the luminance signal;
Gradation correction means for performing gradation correction on the luminance signal in the appropriate exposure area;
The appropriate exposure based on the luminance signal before gradation correction output from the luminance / color difference separation means, the luminance signal after gradation correction output from the gradation correction means, and the theoretical limit characteristic of color reproduction. Color difference correction means for correcting the color difference signal of the area;
Luminance color difference synthesizing means for synthesizing the luminance signal after gradation correction and the color difference signal after correction into an original image signal;
A synthesizing means for generating one wide dynamic range image by synthesizing the synthesized image signals of the appropriate exposure area;
An image processing apparatus comprising: It further comprises a shooting situation estimation means for estimating the shooting situation,
The image processing apparatus according to claim 1, wherein the gradation correction unit performs gradation correction on the luminance signal in the appropriate exposure area based on the shooting situation. The shooting situation estimation means is
Focusing position estimation means for estimating at least three types of focusing positions of landscape shooting, person shooting, and close-up shooting from focusing information;
Subject distribution estimation means for estimating at least three types of subject distributions of the entire screen, center weight, and center from photometric information;
An integration unit that collectively estimates a shooting situation by combining the in-focus position estimated by the in-focus position estimation unit and the subject distribution estimated by the subject distribution estimation unit;
The image processing apparatus according to claim 2, further comprising: The gradation correction means is
A selection means for selecting an arrangement of weighting factors based on the shooting situation;
Feature amount calculating means for calculating an edge with respect to the appropriate exposure area;
The pixel with respect to the appearance frequency of the pixel represented in the edge histogram indicating the correspondence relationship between the level of the luminance signal of the pixel constituting the edge and its neighboring pixels and the appearance frequency of the pixel having the luminance signal level. Histogram creating means for creating a weighted histogram for the edge by weighting the selected weighting factor assigned to the pixel position;
Gradation conversion curve calculating means for calculating a gradation conversion curve based on the weighted histogram;
Conversion means for performing gradation conversion using the gradation conversion curve;
The image processing apparatus according to claim 2, wherein the image processing apparatus comprises: The gradation correction means is
Feature amount calculating means for calculating an edge with respect to the appropriate exposure area;
Histogram creation means for creating a histogram showing a correspondence relationship between the level of the luminance signal of the pixels constituting the edge and its neighboring pixels and the appearance frequency of the pixel having the level of the luminance signal;
A gradation conversion curve calculating means for calculating a gradation conversion curve based on the histogram;
Luminance conversion means for converting a luminance signal using the gradation conversion curve;
The image processing apparatus according to claim 1, further comprising: The color difference correcting means is
First calculating means for calculating a first correction coefficient based on the luminance signal before gradation correction and the theoretical limit characteristic of color reproduction;
Second calculating means for calculating a second correction coefficient based on the luminance signal after the gradation correction and the theoretical limit characteristic of the color reproduction;
Color difference conversion means for converting a color difference signal using the first correction coefficient and the second correction coefficient;
The image processing apparatus according to claim 1, wherein the image processing apparatus comprises: An image processing program for processing an image group consisting of a plurality of images taken under different exposure conditions for the same subject to generate one wide dynamic range image,
A luminance color difference separation step for separating the image signal into a luminance signal and a color difference signal for each image in the image group;
An extraction step for extracting an appropriate exposure area based on the signal level of the luminance signal;
A gradation correction step for performing gradation correction on the luminance signal in the appropriate exposure area;
The appropriate exposure based on the luminance signal before gradation correction output by the luminance / chrominance separation step, the luminance signal after gradation correction output by the gradation correction step, and the theoretical limit characteristic of color reproduction. A color difference correction step for correcting the color difference signal of the area;
Luminance color difference synthesis step for synthesizing the luminance signal after gradation correction and the color difference signal after correction to the original image signal;
A synthesizing step for generating one wide dynamic range image by synthesizing the synthesized image signals of the appropriate exposure area;
An image processing program for causing a computer to execute.

Images