JP2001238128A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus which corrects gradation so that saturation becomes appropriate while seriously considering a main object and generates a wide dynamic range picture from a plurality of pictures. SOLUTION: An image processing apparatus is provided with a photographing situation estimation part 18 for estimating a photographing situation on the basis of the photometric information of a photometric evaluation part 7 and the focusing information of a focused point detection part 8, a Y/C separation part 12 for separating a long time exposure picture and a short time exposure picture into a luminance signal and a color different signal, an correct exposure extraction part 13 for extracting a correct exposure area on the basis of the luminance signal level of the long time exposure picture, a luminance correction part 32 for weighting the luminance signal of the correct exposure area in accordance with the photographing situation and correcting gradation, a color difference correction part 33 for correction color difference signal of the appropriate exposure area on the basis of the luminance signal before and after gradation correction and the theoretical limit characteristics of color reproduction, a Y/C synthesis part 34 for synthesizing the luminance signal and the color difference signal after correction and a picture synthesis part 15 for synthesizing the long time exposure picture and the short time exposure picture after Y/C synthesis to generate a wide dynamic range picture.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus for generating one wide dynamic range image from a plurality of images captured under different exposure conditions.

[0002]

2. Description of the Related Art Conventionally, an image processing apparatus has been proposed which combines a plurality of images captured under different exposure conditions to generate one wide dynamic range image.

[0003] As an example of such a device, Japanese Patent Application No.
No. 338551 discloses that each image is divided into a proper exposure area and an improper exposure area, gradation correction is performed for each proper exposure area, and a proper exposure area for each gradation-corrected image is synthesized. An image processing device that generates one wide dynamic range image is described, and further, as an example of a device to which the image processing device is applied, a super latitude digital camera capable of imaging a subject with a wider dynamic range is described. Have been.

[0004] The gradation conversion processing in this super latitude digital camera is performed based on histogram flattening of an edge portion.
This technique assumes that a main subject has many edges and a non-main part such as a background has few edges.

On the other hand, in a conventional digital camera, a color difference signal is also converted based on a coefficient for converting a luminance signal into a gradation. That is, assuming that the luminance Yorg is converted by the gradation conversion characteristic F as Ytra = F (Yorg), conventionally, the conversion coefficient of the luminance signal is
The gain is calculated as gain = Ytra / Yorg, and the conversion coefficient is used as it is to convert the color difference signal as Cbtra = gain · Cborg Crtra = gain · Crorg.

[0006]

The histogram flattening technique based on the premise that the main subject has many edges as described above can cope with a relatively wide range of subjects. It may not be possible. An example that may fall under such an exception is a case where a person is relatively small in a background having a complicated shape or contour. At this time, since many edges are detected from the background portion, the background is determined to be the main subject, and the gradation width assigned to the person is reduced.

Further, in the above-described technique of performing gradation conversion of a color difference signal using a conversion coefficient similar to that of a luminance signal, an unnatural color may be generated in a high luminance portion. That is,
Color reproduction in a color space (for example, Y, Cb, Cr space) has a theoretical limit characteristic (see FIG. 11 showing an embodiment of the present invention). The color difference range in which color reproducibility can be expanded as the value is increased, and the color difference range in which color reproducibility can be reduced after a certain luminance Y is exceeded. In other words, if the luminance is low, the color reproduction range is narrow because the whole becomes blackish, and a wide range of colors can be reproduced with appropriate luminance, and the color reproduction range is narrowed again because the whole becomes whitish when the luminance is high This is the characteristic of becoming

If gradation conversion is performed in the same way as luminance without considering such a color reproduction range, the color reproduction range approaches or exceeds the limit, and the color after gradation conversion becomes whitish. There was something. In order to cope with such a problem, processing for suppressing the saturation of the high-luminance part has been added in the past, but it is not sufficient, and it is desired to realize a technique for further improving color reproducibility. ing.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and has as its object to provide an image processing apparatus capable of adaptively adjusting the gradation width of a main subject in accordance with a shooting scene.

Another object of the present invention is to provide an image processing apparatus capable of adjusting the saturation more optimally in consideration of the theoretical limit characteristics of color reproduction.

[0011]

In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention processes an image group consisting of a plurality of images of the same subject captured under different exposure conditions. An image processing apparatus for generating one wide dynamic range image, comprising: a photographing state estimating means for estimating a photographing state; and an extracting means for extracting a proper exposure area based on an image signal level for each image in the image group. Generating a wide dynamic range image by synthesizing a proper exposure area whose gradation has been corrected by the gradation correcting means for performing gradation correction based on the photographing situation with respect to the proper exposure area. And synthesizing means.

An image processing apparatus according to a second aspect of the present invention includes:
An image processing apparatus that processes an image group consisting of a plurality of images captured under different exposure conditions for the same subject to generate one wide dynamic range image, wherein an image signal is output for each image in the image group. Luminance and color difference signals, a luminance and color difference separation means, an extraction means for extracting a proper exposure area based on the signal level of the luminance signal, and a gradation correction means for performing gradation correction on the luminance signal of the proper exposure area The appropriate exposure area based on the luminance signal before gradation correction output from the luminance / color difference separation unit, the luminance signal after gradation correction output from the gradation correction unit, and the theoretical limit characteristic of color reproduction. Color difference correction means for correcting the color difference signal, luminance and color difference synthesis means for synthesizing the luminance signal after gradation correction and the color difference signal after correction with the original image signal, and the synthesized proper exposure area Image signal Synthesizing means for generating a one wide dynamic range image by combining, those having a.

Further, an image processing apparatus according to a third aspect of the present invention processes an image group consisting of a plurality of images of the same subject captured under different exposure conditions to generate one wide dynamic range image. A photographing state estimating means for estimating a photographing state; a luminance / color difference separating means for separating an image signal into a luminance signal and a color difference signal for each image in the image group; Extraction means for extracting an exposure area, gradation correction means for performing gradation correction on the luminance signal of the proper exposure area based on the shooting conditions, and a luminance signal before gradation correction output from the luminance / color difference separation means. Color difference correction means for correcting the color difference signal of the appropriate exposure area based on the luminance signal after the tone correction output from the tone correction means and the theoretical limit characteristics of color reproduction,
A luminance / color difference synthesizing means for synthesizing the luminance signal after the gradation correction and the color difference signal after the correction with an original image signal; and a wide dynamic range by synthesizing the synthesized image signal of the appropriate exposure area. Combining means for generating a range image.

The image processing apparatus according to a fourth aspect of the present invention is the image processing apparatus according to the first or third aspect, wherein the photographing state estimating means determines at least three types of landscape photographing, person photographing, and close-up photographing from the focus information. The in-focus position estimating means for estimating the in-focus position, the subject distribution estimating means for estimating at least three types of subject distributions of the entire screen, the center weight, and the central portion from the photometric information, and the in-focus position estimating means Integrating means for integrating and estimating a shooting situation by combining the in-focus position and the object distribution estimated by the object distribution estimating means.

The image processing apparatus according to a fifth aspect of the present invention is the image processing apparatus according to the first or third aspect, wherein the gradation correction means selects an arrangement of weight coefficients based on the photographing situation; Feature value calculating means for calculating a feature value with respect to the proper exposure area; histogram creating means for creating a weighted histogram for the feature value based on the arrangement of the weighting coefficients; and a floor calculating section for calculating a gradation conversion curve based on the histogram. Tone conversion curve calculation means, and conversion means for performing tone conversion using the tone conversion curve.

The image processing apparatus according to a sixth aspect of the present invention is the image processing apparatus according to the second aspect, wherein the gradation correction means calculates a feature quantity for the proper exposure area; Histogram generation means for generating a histogram relating to the image, gradation conversion curve calculation means for calculating a gradation conversion curve based on the histogram, and luminance conversion means for converting a luminance signal using the gradation conversion curve. It is made.

The image processing apparatus according to a seventh aspect of the present invention is the image processing apparatus according to the second or third aspect, wherein the color-difference correcting means includes a luminance signal before gradation correction and a theoretical limit characteristic of color reproduction. First calculating means for calculating a first correction coefficient based on the above-described formula, and a second calculating means for calculating a second correction coefficient based on the luminance signal after the gradation correction and the theoretical limit characteristic of color reproduction. Means for converting a color difference signal using the first correction coefficient and the second correction coefficient.

[0018]

Embodiments of the present invention will be described below with reference to the drawings. 1 to 7 show a first embodiment of the present invention, and FIG. 1 is a block diagram showing a basic configuration of an electronic camera.

In the present embodiment, the image processing apparatus of the present invention is applied to an electronic camera. For simplicity, two images, that is, an image obtained by short-time exposure and an image obtained by long-time exposure, are combined to form one wide area. Although the case where a dynamic range image is obtained will be described, it is needless to say that the present invention can be applied to a case where a larger number of images are combined.

The electronic camera comprises a single-chip color CCD or the like having an electronic shutter function. The CCD 4 photoelectrically converts a subject image and outputs it as an image signal. The CCD 4 forms the subject image on the CCD 4. Lens system 1
A stop 2 for controlling a passing range of a light beam passing through the lens system 1, a low-pass filter 3 serving as an optical filter for removing unnecessary high-frequency components from the light beam passing through the stop 2, and an output from the CCD 4. After that, an A / D converter 5 for converting an analog image signal, which has been subjected to noise component removal by a correlated double sampling circuit (not shown) or the like and then amplified to a digital signal, into a digital signal, and the A / D converter A first image buffer 6a and a second image buffer 6b for storing image data of one screen digitized by the converter 5 and storing an image obtained by long-time exposure and an image obtained by short-time exposure, respectively. The image data is read out from the first image buffer 6a, which is also used to store the data for photometry and focus detection, and The exposure control unit 7 for controlling the electronic shutter of the opening diameter and the CCD4 of the diaphragm 2 so that the proper exposure when imaging seek, the first image buffer 6a
And a focusing point detecting unit 8 for detecting an in-focus position and controlling an AF motor 9 to be described later based on the detection result, and an AF of the lens system 1 controlled by the in-focus detecting unit 8. An AF motor 9 for driving an imaging lens to form a subject image on the CCD 4, and interpolating the single-plate image data read out from the first and second image buffers 6a and 6b by 3 An interpolating unit 10 for converting the image data of the plate, a working buffer 11 for storing the interpolated image data, and image data of the three plates read from the working buffer 11 for the luminance signal Y and the color difference signals Cb, Cr. A Y / C separation unit 12 serving as a luminance / color difference separation unit for separating into
First, the luminance signal Y is read from the Y / C separation unit 12, and for each pixel constituting the entire screen, it is determined whether or not the pixel has an appropriate exposure based on a signal level. This will be described in detail later on the basis of the appropriate exposure extracting unit 13 as extracting means for extracting and outputting the photometric information, the photometric information output from the photometric evaluation unit 7 and the focus information output from the focus detection unit 8. Condition estimating section 18 as a photographing condition estimating means for estimating the photographing condition
With reference to the estimation result by the photographing situation estimating unit 18, weighting is performed when calculating a histogram of an edge, which is a feature amount, a conversion characteristic is calculated, and the proper exposure extracting unit 1 is calculated.
A conversion characteristic calculation unit 14 as a gradation correction unit for performing gradation conversion of a proper exposure area output from the proper exposure region 3;
3 and the image related to the long-time exposure and the image related to the short-time exposure after gradation conversion output from the conversion characteristic calculating unit 14 with reference to the area information output from An image synthesizing unit 15 that is a synthesizing unit that generates, an output unit 16 that outputs a wide dynamic range image synthesized by the image synthesizing unit 15 to, for example, a recording medium or a display device, the photometric evaluation unit 7, and a focal point detection unit 8 and the interpolation unit 10,
The control unit 17 includes an appropriate exposure extracting unit 13, a photographing situation estimating unit 18, a conversion characteristic calculating unit 14, an image synthesizing unit 15, and an output unit 16 for controlling the entire electronic camera.

FIG. 2 is a block diagram showing a detailed configuration of the photographing situation estimating section 18. As shown in FIG.

The control unit 1 output from the in-focus point detection unit 8
The focus (AF) information input via the control unit 7 is input to a focus position estimating unit 20 serving as a focus position estimating unit, and according to the subject distance, for example, landscape shooting (5 m to ∞), person shooting ( 1m to 5m) and close-up photography (1m or less) (Fig. 5
Reference).

Also, photometric (AE) information output from the photometric evaluation unit 7 and input via the control unit 17 is input to a subject distribution estimating unit 21 serving as a subject distribution estimating unit, and the number of luminance distributions is determined. It is classified into crab.

Specifically, first, the photometric evaluation section 7
The area on the CCD 4 is classified into, for example, 13 as shown in FIG. 4, and divided photometry is performed. This figure 4
FIG. 4 is a diagram showing an example of a division pattern for evaluation photometry.

That is, the center area at the center is defined as a1
The left neighbor is a2 and the right neighbor is a3.

Further, upper and lower portions of the above-mentioned area a1 in the inner peripheral portion surrounding the centermost portion are defined as a4 and a5, respectively.
The left and right sides of 4 are a6 and a7, and the left and right sides of the area a5 are a8 and a9.

The upper left part of the outer peripheral part surrounding the inner peripheral part is a10, the upper right part is a11, the lower left part is a12, and the lower right part is a13.

In the photometry divided by such an area,
The subject distribution estimating unit 21 calculates the following evaluation parameters.

S1 = | a2-a3 |

S2 = max (| a4-a6 |, | a4-a7 |)

S3 = max (a10, a11) -Av Av = (Ａai) / 13

That is, the evaluation parameter S1 indicates the difference between the left and right luminances at the center and the evaluation parameter S2.
Indicates the larger luminance difference between the upper center and the upper right and left sides of the inner peripheral portion, and the evaluation parameter S3 indicates the difference between the larger upper right and left sides of the outer peripheral portion and the average luminance of the entire screen. It has become something.

Such evaluation parameters are obtained from the subject distribution estimating unit 21 and the in-focus position is obtained from the in-focus position estimating unit 20. The integrating unit 22 as an integrating means as shown in FIG. Classification. This figure 5
9 is a chart showing a classification pattern of a scene from AF information and AE information.

As shown in the figure, when the AF information is 5 m to と し て, it is determined that the image is landscape photography, and the evaluation parameter S3 is compared with a predetermined value Th1. At this time, if the evaluation parameter S3 is larger than the predetermined value Th1, at least one of a10 and a11 has a brightness higher than the average brightness of the entire screen to a certain extent or more, so that the scenery has an upper sky. (Type 1). On the other hand, when the evaluation parameter S3 is smaller than the predetermined value Th1, on the contrary, it is determined that there is no sky above the scenery or that there is little scenery (Type 2).

Next, when the AF information is 1 m to 5 m,
Assuming that a person is photographed, the evaluation parameter S
2 is compared with a predetermined value Th2. At this time, if the evaluation parameter S2 is larger than the predetermined value Th2, it is determined that the portrait is a portrait of one person (Type 3).
If it is smaller than h2, it is determined that the portrait is of a plurality of persons (Type 4).

Further, when the AF information is 1 m or less,
Assuming that the shooting is a close-up shooting, the evaluation parameter S
1 is compared with a predetermined value Th3. At this time, if the evaluation parameter S1 is larger than the predetermined value Th3, it is determined that the object is a close-up of a single object (Type 5).
If it is smaller than the predetermined value Th3, it is determined that the object is a close-up of a plurality of objects (Type 6).

The results classified into such types are output from the integration unit 22 to the conversion characteristic calculation unit 14.

FIG. 3 is a block diagram showing a detailed configuration of the conversion characteristic calculator.

As described above, the appropriate exposure extracting unit 13 first reads out the luminance signal Y of the long-time exposure image, compares the signal level of each pixel constituting the entire screen with a predetermined value, and Is to determine whether or not the exposure is appropriate, the set of pixels determined to be appropriate exposure is the appropriate exposure area for long-time exposure, the portion other than the appropriate exposure area for this long-time exposure An appropriate exposure area for short-time exposure is set.

When a luminance signal Y in a proper exposure area for long-time exposure output from the proper exposure extracting unit 13 is input to an edge extracting unit 26 serving as a feature amount calculating means, the edge extracting unit 26 Edge detection is performed based on the control of S17. Specifically, the edge extraction unit 26 is a general edge detection operator such as Laplacian or Sobel. If the strength of the edge detection operator is equal to or greater than a predetermined threshold, it is determined that an edge exists at the reference position. Otherwise, it outputs binary information that is not an edge.

On the other hand, the result of the type classification by the photographing situation estimating section 18 is used as a weighting pattern selecting section 2 as selecting means.
4, the weight pattern selecting unit 24, based on the control of the control unit 17, stores a plurality of weight patterns as shown in FIG.
A weight pattern corresponding to the type is selected from M25. FIG. 6 is a diagram showing weighting factors at the time of calculating a histogram based on the classification pattern as shown in FIG. 5. FIG. 6 (A) is for Type 1, FIG. 6 (B) is for Type 2, 6 (C) corresponds to Type 3 shown in FIG.
(D) shows the Type 4 and FIG. 6 (E) shows the Type 4.
FIG. 6F shows a weight pattern corresponding to Type 6 described above.

Thus, based on the result output from the edge extracting unit 26, the histogram creating unit 27, which is a histogram creating unit, calculates an edge histogram indicating the appearance frequency with respect to the luminance level for the pixels constituting the edge and the neighboring pixels. However, when the histogram is created, the histogram is calculated by performing weighting according to the pixel position in the image as shown in FIG. Further, the histogram creation unit 27 converts the calculated edge histogram into a cumulative edge histogram by integrating the calculated edge histogram.

The conversion curve calculation unit 28, which is a gradation conversion curve calculation means, generates a target histogram by convolving the edge histogram using a Gaussian kernel or the like, and outputs the target histogram and the target histogram. A tone curve serving as a gradation correction characteristic is calculated using the cumulative edge histogram.

The conversion unit 29, which is a conversion unit, performs gradation correction on the image data input from the appropriate exposure extraction unit 13 based on the tone curve obtained from the conversion curve calculation unit 28. Image data is stored in the image synthesizing unit 1
5 is output. The conversion unit 29 first performs gradation correction of the luminance signal Y related to long-time exposure, and then sequentially performs gradation correction of the color difference signals Cb and Cr related to the long-time exposure, and outputs the result to the image combining unit 15. After that, the gradation correction of the luminance signal Y and the color difference signals Cb and Cr related to the short-time exposure is performed in the same manner and output to the image synthesizing unit 15.

In the image synthesizing unit 15, first, the luminance signal Y and the color difference signals Cb and Cr after gradation correction for long-time exposure are performed.
To generate, for example, RGB signals related to the long-time exposure, and then receive the luminance signal Y and color difference signals Cb, Cr after gradation correction related to the short-time exposure, and Generate RGB signals. Then, by combining these, one wide dynamic range image is generated and output.

FIG. 7 is a flowchart showing the image conversion processing.

The subject image formed on the CCD 4 made of a single plate is imaged a plurality of times under different exposure conditions. Are performed in this order, and are sequentially output as image signals.

After these image signals are converted into digital signals by the A / D converter 5, they are stored in the first image buffer 6a and the second image buffer 6b, respectively.

The photometric evaluation section 7 and the focal point detection section 8 perform the above-described operations based on the long-time exposure image data stored in one of the first image buffers 6a, as described above.
The AE information and the AF information are output to the control unit 17 (Step S1).

On the other hand, the image data stored in the first image buffer 6a and the second image buffer 6b are sequentially sent to the interpolator 10, and each of the R image data, the G image data, and the B image data is Are converted into three-plate image data by performing interpolation (step S
2) It is stored in the work buffer 11.

The Y / C separation section 12 reads the RGB image data from the working buffer 11 and
And the color difference signals Cb and Cr are calculated as shown in the following Expression 4 (step S3).

Y = 0.29900R + 0.58700G + 0.14400B Cb = −0.16874R−0.33126G + 0.50000B Cr = 0.50000R−0.41869G−0.08131B

The appropriate exposure extracting section 13 compares the signal level of the luminance signal Y among these with a predetermined threshold value for each pixel to determine whether or not the pixel belongs to the appropriate exposure area, thereby obtaining a divided image. Extract and output information (Step S
4).

Thereafter, an edge component is extracted by applying a known secondary differential filter such as Laplacian to the luminance signal Y in an edge extraction unit 26 in the conversion characteristic calculation unit 14 (step S5). Deviation for example 2
The binarization process is performed by providing a threshold of about twice (step S6).

On the other hand, the photographing state estimating section 18 estimates the photographing state based on the AF information and the AE information as described above (step S7), and the weighting pattern is set to Ty.
It selects any one of pe1 to Type6 (step S8). Then, the weighting factor corresponding to the selected weighting pattern as shown in FIG.
(Step S9).

In this way, the histogram creating section 27 creates an edge histogram weighted based on the edge components binarized in step S6 and the weight pattern read in step S9 (step S1).
0), a cumulative edge histogram is further generated from the edge histogram (step S11). The conversion curve calculation unit 28 calculates a gradation conversion curve based on the cumulative edge histogram thus obtained (step S1).
2).

In the conversion section 29, the luminance signal Y and the color difference signals Cb and C output from the proper exposure extraction section 13 are output.
r is converted by the gradation conversion curve obtained from the conversion curve calculator 28 (step S13), and the converted image data is output (step S1).
4).

In the above description, both the photometric information and the focusing information are used for estimating the photographing situation, but it is also possible to use only one of them to estimate and change the weight. Not only the photometric information and focus information but also zoom position information, multi-spot photometric information, line-of-sight input information, flash emission information, information on the detection sensor that detects the vertical and horizontal position of the electronic camera, and white balance information. By referring to one or more, it is possible to estimate the shooting situation in more detail.

Further, the above-described technique of gradation correction according to a shooting situation is not limited to a color image but can be applied to a black and white image.

Then, in the first embodiment,
Although the image processing apparatus configured as a circuit in the electronic camera performs gradation correction according to the shooting situation, such processing can be performed by a processing program of a computer. In this case, photographing information such as photometric information and focusing information is recorded in, for example, a header portion of the image file, and a computer estimates a photographing state based on the photographing information, and is suitable for the photographing state. Tone correction may be performed.

The image processing apparatus is not limited to being applied to an electronic camera, but can be widely applied to, for example, an image processing apparatus such as a printer apparatus.

According to the first embodiment, the photographing state is determined based on photographing information such as focusing information and photometric information, and weighting is performed according to the photographing state when creating an edge histogram. As a result, it is possible to perform gradation correction most suitable for the shooting scene in consideration of the main subject.

FIGS. 8 to 12 show a second embodiment of the present invention. FIG. 8 is a block diagram showing a basic configuration of an electronic camera, and FIG. 9 shows a detailed configuration of a luminance correction unit. FIG. 10 is a block diagram showing a detailed configuration of a color difference correction unit, FIG. 11 is a diagram showing how color difference correction is performed in consideration of a theoretical limit characteristic of color reproduction, and FIG. 12 is a diagram showing image conversion processing. It is a flowchart shown.

In the second embodiment, the first
The same components as those of the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. Only different points will be mainly described.

In the second embodiment, the Y / C separation unit 1
2, the luminance signal Y and the color difference signals Cb, Cr
Are input through a proper exposure extraction unit 13 to a luminance correction unit 32 as a gradation correction unit and a color difference correction unit 33 as a color difference correction unit.

As shown in FIG. 9, the brightness correction section 32 receives the brightness signal Y in the proper exposure area output from the proper exposure extraction section 13 and performs gradation correction on the brightness. Edge extractor 40 serving as an amount calculator
And a histogram creating unit 41 as a histogram creating means
A conversion curve calculation unit 42 as a gradation conversion curve calculation unit;
And a brightness conversion unit 43 as a brightness conversion unit.

The processing in the brightness correction unit 32 is described in FIG.
This will be described with reference to FIG.

The brightness correction section 32 is provided with the appropriate exposure extraction section 1
The luminance signal Y in the proper exposure area output from the pixel 3 is read (step S21), and the edge extraction unit 40 extracts an edge component by applying a filter such as Laplacian (step S22), and compares it with a predetermined threshold value for each pixel. (Step S2)
3).

Based on the information output from the edge extracting section 40, the histogram creating section 41 generates an edge histogram indicating the frequency of appearance of the edge with respect to the luminance (step S24), and further integrates this to obtain a cumulative edge histogram. Is created (step S25).

The conversion curve calculation unit 42 calculates a tone curve serving as a gradation correction characteristic as described above using the cumulative edge histogram output from the histogram creation unit 41 (step S26).

The luminance conversion unit 43 performs gradation conversion of the luminance signal Y based on the conversion curve under the control of the control unit 17 (step S27), outputs the gradation to the color difference correction unit 33, and outputs the luminance / color difference combining means. The result is output to the Y / C combining section 34 (step S28).

As described above, if the luminance signal before gradation correction output from the proper exposure extraction unit 13 is Yorg, and the luminance signal after gradation correction by the luminance conversion unit 43 is Ytra, The signals Yorg and Ytra are used when the color difference correction section 33 corrects the gradation of the color difference, as described below.

As shown in FIG. 10, the color difference correction section 33 receives the color difference signals Cb and Cr in the proper exposure area output from the proper exposure extracting section 13 and performs gradation correction on the color difference. A first correction coefficient calculation unit 47 as a first calculation unit, a second correction coefficient calculation unit 45 as a second calculation unit, a ROM 46 for color reproduction limit characteristics, and a color difference conversion unit 48 as a color difference conversion unit. It is configured.

In the color difference correction section 33, the first
The correction coefficient calculating unit 47 receives the luminance signal Yorg before the gradation correction from the appropriate exposure extracting unit 13 and calculates a color reproduction range borg corresponding to the luminance signal Yorg as shown in the following Expression 5 (step) S31).

## EQU5 ## borg = B (Yorg)

Here, the function B (Y) is expressed in the color space (Y, C
(b, Cr space), which is a function indicating the theoretical limit characteristic of color reproduction. For example, as schematically shown in FIG. Is exceeded, the color difference range in which color reproduction is possible is narrowed.

The calculation as shown in Expression 5 is, for example,
Referring to the table data and the like stored in the color reproduction limit characteristic ROM 46 (step S30), the luminance Yor
This is performed by finding a color gamut borg corresponding to g.

The color reproduction limit characteristic ROM 46 stores the function B (Y) indicating the theoretical limit characteristic of color reproduction in advance as table data. Here, the load and the processing speed by calculation are taken into consideration. Then, the table data stored in the ROM is used, but of course, the calculation may be actually performed.

Next, the second correction coefficient calculation section 45 receives the luminance signal Ytra after the gradation correction from the luminance correction section 32 and receives the color reproduction range b corresponding to the luminance signal Ytra.
tra is calculated as shown in the following Expression 6 similar to Expression 5 described above (Step S32).

## EQU6 ## btra = B (Ytra)

The calculation as shown in Expression 6 is similarly performed
With reference to the table data and the like stored in the color reproduction limit characteristic ROM 46 (step S30), the luminance Ytr
This is performed by obtaining a color reproduction range btra corresponding to a.

The color difference conversion section 48 calculates a conversion coefficient gainc for the color difference signal based on the first correction coefficient borg and the second correction coefficient btra as shown in the following equation (7). I do.

[Mathematical formula-see original document] gain = btra / borg

Thus, the conversion coefficient ga for the color difference signal
inc is defined to be the ratio of the theoretical limit characteristic borg of color reproduction with the luminance signal Yorg before gradation correction and the theoretical limit characteristic btra of color reproduction with the luminance signal Ytra after gradation correction. Thus, it is possible to perform faithful color reproduction while maintaining saturation without causing overexposure and the like as in the case where the same conversion coefficient as luminance is used.

When the conversion coefficient gainc has been obtained, the uncorrected color difference signal C
borg and Crorg are sequentially received (step S2).
9) The corrected color difference signals Cbtra and Crtra are calculated as shown in the following Expression 8 (step S33).

## EQU8 ## Cbtra = gainc Cborg Crtra = gaincCrorg

The color difference signals Cbtra,
Crtra is output to the Y / C synthesis unit 34 (step S34).

The Y / C synthesizing section 34 Y / C-combines the luminance signal Ytra after the gradation conversion and the color difference signals Cbtra and Crtra after the conversion, for example, converts them into RGB signals, and outputs the RGB signals to the image synthesizing section 15. I do.

The image synthesizing section 15 synthesizes the properly exposed image portion after gradation conversion for long-time exposure and the properly exposed image portion after gradation conversion for short-time exposure, to form one wide dynamic range. After generating the image, the output unit 1
6 is output.

In the second embodiment, tone correction is performed in consideration of the theoretical limit characteristics of color reproduction by an image processing device configured as a circuit in an electronic camera. It is also possible to carry out by the processing program of the above.

The image processing apparatus is not limited to being applied to an electronic camera, but can be widely applied to, for example, an image processing apparatus such as a printer apparatus.

According to the second embodiment, the color difference signal is subjected to the gradation correction in consideration of the theoretical limit characteristics of the color reproduction. Degree can be maintained.

FIG. 13 shows a third embodiment of the present invention and is a block diagram showing a basic configuration of an electronic camera.

In the third embodiment, the same parts as those in the first and second embodiments are denoted by the same reference numerals, and description thereof will be omitted. Only different points will be mainly described.

The third embodiment is configured to exhibit a function that combines the functions of the first and second embodiments described above, that is, the gradation conversion characteristic of the luminance signal. Is calculated in accordance with the photographing situation, and the theoretical characteristics of color reproduction are considered when calculating the conversion characteristics of the color difference signal.

That is, the classification result of the photographing situation as shown in FIG. 5 estimated by the photographing situation estimating section 18 is input to the luminance correcting section 32. When creating an edge histogram of a signal, weighting is performed as shown in FIG.

When the luminance signal whose gradation has been converted based on the characteristic curve thus obtained is input to the color difference correction section 33,
Similarly to the above-described second embodiment, a second correction coefficient is calculated with reference to a table indicating a theoretical limit characteristic of color reproduction, and a first correction coefficient is similarly calculated based on a luminance signal before gradation conversion. Is calculated. Then, a conversion coefficient related to the color difference signal is calculated based on the first correction coefficient and the second correction coefficient, conversion suitable for the color difference is performed, and the result is output to the Y / C synthesis unit 34.

The Y / C synthesizing section 34 Y / C synthesizes the luminance signal after the gradation conversion output from the luminance correction section 32 and the converted color difference signal output from the color difference correction section 33 to form an image. The image is output to the synthesizing unit 15, and the image synthesizing unit 15 synthesizes the properly exposed image portion after gradation conversion for long-time exposure and the properly exposed image portion after gradation conversion for short-time exposure. After generating a wide dynamic range image, the output unit 16 outputs the image.

In the third embodiment, tone correction is performed as a circuit in the electronic camera. However, such processing can be performed by a processing program of a computer.

Further, the image processing apparatus is not limited to application to an electronic camera, but can be widely applied to devices that handle images, such as a printer.

According to the third embodiment, both the effects of the above-described first embodiment and the effects of the second embodiment can be obtained.

The present invention is not limited to the above-described embodiment, and it goes without saying that various modifications and applications are possible without departing from the gist of the invention.

[0095]

As described above, according to the image processing apparatus of the first aspect of the present invention, since the photographing state is estimated and the gradation correction is performed in accordance with it, the gradation width of the main subject is adapted. It becomes possible to adjust it.

Further, according to the image processing apparatus of the present invention, since the color difference signal is corrected based on the luminance signal before and after the gradation correction and the theoretical limit characteristic of color reproduction,
After the gradation correction, an image having an optimum saturation can be obtained.

Further, according to the image processing apparatus of the present invention, since the photographing state is estimated and the gradation correction of the luminance signal is performed in accordance with the estimation, the gradation width of the main object is adaptively adjusted. It is possible to adjust the color difference signal based on the luminance signal before and after gradation correction and the theoretical limit characteristics of color reproduction, so that an image with optimal saturation can be obtained after gradation correction. Can be.

According to the image processing apparatus of the fourth aspect of the present invention, the same effects as those of the first or third aspect of the present invention are obtained, and the photographing state is determined based on the focusing information and the photometric information. It is possible to make an appropriate estimation.

According to the image processing apparatus of the present invention, the same effect as that of the first or third aspect is obtained, and the gradation correction means weights the characteristic amount of the appropriate exposure area. By creating a histogram and calculating a gradation conversion curve, appropriate gradation conversion can be performed in consideration of the main subject.

According to the image processing apparatus of the sixth aspect of the present invention, the same effect as that of the second aspect of the present invention can be obtained, and the gradation correction means creates a histogram from the characteristic amount of the appropriate exposure area. By calculating the gradation conversion curve,
Appropriate gradation conversion can be performed.

According to the image processing apparatus of the present invention, the same effects as those of the second or third aspect can be obtained, and the color difference correcting means can control the luminance signal and the color signal before gradation correction. By using a first correction coefficient calculated from a theoretical limit characteristic of reproduction and a second correction coefficient calculated from a luminance signal after gradation correction and a theoretical limit characteristic of color reproduction, conversion of a color difference signal is performed. Can be performed appropriately.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of an electronic camera according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of a shooting situation estimating unit according to the first embodiment.

FIG. 3 is a block diagram illustrating a detailed configuration of a conversion characteristic calculation unit according to the first embodiment.

FIG. 4 is a diagram showing an example of a division pattern for evaluation photometry in the first embodiment.

FIG. 5 is a diagram showing AF information and AE in the first embodiment.
7 is a table showing a classification pattern of scenes from information.

FIG. 6 is a diagram showing weighting factors at the time of calculating an edge histogram based on the classification pattern as shown in FIG. 5 in the first embodiment.

FIG. 7 is a flowchart illustrating image conversion processing according to the first embodiment.

FIG. 8 is a block diagram showing a basic configuration of an electronic camera according to a second embodiment of the present invention.

FIG. 9 is a block diagram illustrating a detailed configuration of a luminance correction unit according to the second embodiment.

FIG. 10 is a block diagram illustrating a detailed configuration of a color difference correction unit according to the second embodiment.

FIG. 11 is a diagram showing a state in which color difference correction is performed in consideration of the theoretical limit characteristics of color reproduction in the second embodiment.

FIG. 12 is a flowchart illustrating image conversion processing according to the second embodiment.

FIG. 13 is a block diagram illustrating a basic configuration of an electronic camera according to a third embodiment of the present invention.

[Explanation of symbols]

7: Photometric evaluation unit 8: Focus point detection unit 12: Y / C separation unit (luminance / color difference separation unit) 13: Appropriate exposure extraction unit (extraction unit) 14: Conversion characteristic calculation unit (gradation correction unit) 15: Image synthesis Unit (synthesizing unit) 16 ... Output unit 17 ... Control unit 18 ... Shooting condition estimating unit (shooting condition estimating unit) 20 ... In-focus position estimating unit (in-focus position estimating unit) 21 ... Subject distribution estimating unit (subject distribution estimating unit) 22 ... integrating section (integrating means) 24 ... weight pattern selecting section (selecting means) 25 ... weight pattern ROM 26,40 ... edge extracting section (feature amount calculating means) 27,41 ... histogram creating section (histogram creating means) 28, 42: conversion curve calculation unit (gradation conversion curve calculation unit) 29: conversion unit (conversion unit) 32: luminance correction unit (gradation correction unit) 33: color difference correction unit (color difference correction unit) 34: Y / C Synthesizer ( 43: luminance conversion unit (luminance conversion unit) 45: second correction coefficient calculation unit (second calculation unit) 46: ROM for color reproduction limit characteristics 47: first correction coefficient calculation unit (first Calculation means) 48: color difference conversion unit (color difference conversion means)

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) // H04N 101: 00 H04N 101: 00 F term (Reference) 5C022 AA13 AB06 AB12 AB17 AB19 AB22 AB68 AC42 AC51 AC69 5C065 AA03 BB01 BB48 CC02 CC03 DD01 GG13 GG18 GG42 5C066 AA01 CA07 DC01 DD07 EA05 EA07 EE01 FA02 GA02 GA05 KE03

Claims (7)

[Claims] 1. An image processing apparatus for processing an image group consisting of a plurality of images of the same subject imaged under different exposure conditions to generate one wide dynamic range image, wherein an imaging state is estimated. Situation estimating means, extracting means for extracting a proper exposure area based on an image signal level for each image in the image group, and tone correcting means for performing tone correction based on the shooting conditions with respect to the proper exposure area, An image processing apparatus comprising: a synthesizing unit that generates one wide dynamic range image by synthesizing an appropriate exposure area whose gradation has been corrected by the gradation correcting unit. 2. An image processing apparatus for processing an image group consisting of a plurality of images of the same subject captured under different exposure conditions to generate one wide dynamic range image, comprising: A luminance / color difference separating unit that separates an image signal into a luminance signal and a color difference signal for each image; an extracting unit that extracts a proper exposure area based on a signal level of the luminance signal; and a gradation correction for the luminance signal in the proper exposure area. Gradation correction means for performing, a luminance signal before gradation correction output from the luminance / color difference separation means, a luminance signal after gradation correction output from the gradation correction means, and a theoretical limit characteristic of color reproduction. A color difference correction unit that corrects the color difference signal in the appropriate exposure area based on the above, and a luminance / color difference combination unit that combines the luminance signal after the gradation correction and the color difference signal after the correction with the original image signal. The above is synthesized And a synthesizing unit for generating one wide dynamic range image by synthesizing the image signals in the appropriate exposure area. 3. An image processing apparatus for processing an image group consisting of a plurality of images of the same subject imaged under different exposure conditions to generate one wide dynamic range image, wherein the image capturing apparatus estimates a shooting state. A situation estimating unit, a luminance / chrominance separating unit that separates an image signal into a luminance signal and a color difference signal for each image in the image group, an extracting unit that extracts a proper exposure area based on a signal level of the luminance signal, A gradation correction means for performing gradation correction on the luminance signal in the appropriate exposure area based on the photographing condition; a luminance signal before gradation correction output from the luminance / color difference separation means; and a luminance signal output from the gradation correction means. Color difference correction means for correcting the color difference signal in the appropriate exposure area based on the luminance signal after the gradation correction and the theoretical limit characteristic of color reproduction; and the luminance signal after the gradation correction and the color difference after the correction. Signal and And a combining means for combining the combined image signals in the appropriate exposure range to generate one wide dynamic range image. Image processing device. 4. The photographing situation estimating means includes: a focus position estimating means for estimating at least three types of focus positions of landscape photographing, portrait photographing, and close-up photographing from the focus information; A subject distribution estimating means for estimating three types of subject distributions of a focus and a central portion; The image processing apparatus according to claim 1, further comprising: an integrating unit configured to estimate the position of the image processing apparatus. 5. A gradation correction means, comprising: a selection means for selecting an arrangement of weighting factors based on the photographing situation; a feature amount calculation means for calculating a feature amount for the proper exposure area; Histogram creating means for creating a weighted histogram related to the feature amount, tone conversion curve calculating means for calculating a tone conversion curve based on the histogram, and conversion means for performing tone conversion using the tone conversion curve The image processing apparatus according to claim 1, wherein the image processing apparatus comprises: 6. The gradation correction means, wherein: a feature quantity calculation means for calculating a feature quantity with respect to the proper exposure area; a histogram creation means for creating a histogram related to the feature quantity; and a gradation conversion curve based on the histogram. 3. The image processing apparatus according to claim 2, comprising: a gradation conversion curve calculating unit for calculating; and a luminance conversion unit for converting a luminance signal using the gradation conversion curve. 4. apparatus. 7. The color difference correction unit includes: a first calculation unit that calculates a first correction coefficient based on the luminance signal before the gradation correction and the theoretical limit characteristic of color reproduction; A second calculating means for calculating a second correction coefficient based on the luminance signal after the tone correction and the theoretical limit characteristic of the color reproduction, and using the first correction coefficient and the second correction coefficient. The image processing apparatus according to claim 2, further comprising: a color difference conversion unit that converts a color difference signal.