JP2009071408A - Image processing apparatus, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that a level weight to other image corresponding to black decay portions of a high EV (exposure value) image or white saturation portions of a low EV image cannot be directly designated because the level weight is independently calculated in the high EV image and the low EV image, in a method of expanding a dynamic range of a digital image by adding two images obtained by multiplying the low EV image and the high EV image by the respective level weights so that the digital images can be compressed within an output gradation. <P>SOLUTION: This image processing apparatus is provided with: a luminance calculating means for each of two kinds of digital images, i.e., a high EV image and a low EV image; a means of converting the luminance and pixel data of the high EV image corresponding to the EV ratio of the two images; a means of adding the luminance of the low EV image to the luminance of the high EV image converted; a means of calculating a composite ratio for the two images from a result of addition; a means of multiplying the pixel data of the low EV image and the pixel data of the high EV image converted by the respective composite ratios of the two images; a means of adding the pixel data after multiplication by the respective composite ratios; and a gradation converting means of the pixel data after addition. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing method and an image processing apparatus for widening a dynamic range of a digital image for imaging a subject having a wide dynamic range without overexposure or blackout.

  One technique for expanding the dynamic range of a digital image is to synthesize two images with different exposure times. FIG. 3 is a configuration diagram of a conventional image processing apparatus described in, for example, Japanese Patent No. 3074967. A CCD (Charge Coupled Devices) 101 in FIG. 3 is an image sensor, and switches between exposure times to acquire two images, a low exposure value image with a long exposure time and a high exposure value image with a short exposure time, in a time-sharing manner. Can do. A memory 102 is a buffer memory for storing a high exposure value image, and is used for timing adjustment when two images acquired in a time division manner are combined.

The high exposure value image is multiplied by a constant by a multiplier 103 and then multiplied by a weight coefficient corresponding to a gradation by a level weight H multiplier 105. The low exposure value image is multiplied by a weight coefficient corresponding to the gradation by the level weight L multiplier 106. The two images multiplied by the weighting coefficient are added by the adder 107. After the addition, the output timing is adjusted by the speed conversion means 108, the level is compressed within the output gradation by the level compression means 109, and the compressed image data is output.
The timing control unit 104 controls the processing timing of the CCD 101, the multiplier 103, and the speed conversion unit 108.

Japanese Patent No. 3074967

  In the conventional image processing apparatus configured as described above, the level weight is calculated independently for each of the high exposure value image and the low exposure value image. There is a problem in that the level weight L of the low exposure value image corresponding to the location cannot be directly specified. Similarly, there is a problem in that the level weight H of the high exposure value image corresponding to the overexposed portion in the low exposure value image cannot be directly specified.

  The present invention solves the above-mentioned conventional problems, and calculates the combination ratio when combining two images of a high exposure value image and a low exposure value image in conjunction with the two images, thereby obtaining a black image. The purpose is to apply an appropriate composition ratio to the crushing part and the whiteout part.

An image processing apparatus according to the present invention includes:
In an image processing apparatus that synthesizes two types of digital images, a high exposure value image and a low exposure value image with the same subject and different exposure conditions,
Means for calculating luminance for each of the two images;
Means for converting the brightness and pixel data of the high exposure value image in accordance with the ratio of the exposure values of the two images;
Means for adding the brightness of the low exposure value image and the brightness of the high exposure value image after conversion;
Means for calculating a composition ratio for each of the two images from the addition result;
Means for multiplying the pixel data of the low exposure value image and the pixel data of the converted high exposure value image by the respective composition ratios;
Means for adding pixel data after multiplication of the respective synthesis ratios;
Means for converting the gradation of the pixel data after addition is provided.

  According to the image processing apparatus of the present invention, the means for converting the luminance and pixel data of the high exposure value image according to the exposure value ratio of the high exposure value image and the low exposure value image allows the gradation of the low exposure value image to be adjusted. After the conversion process that performs level adjustment, the composite ratio for each of the high exposure value image and the low exposure value image is calculated from the result of adding the luminance of the converted high exposure value image and the luminance of the low exposure value image. Therefore, since it is possible to calculate the composition ratio by simultaneously referring to both the high exposure value image and the low exposure value image, it is possible to apply an appropriate composition ratio to the blackened portion and the overexposed portion.

Embodiment 1 FIG.
FIG. 1 shows the configuration of the image processing apparatus according to the present embodiment. The image processing apparatus according to the present embodiment is configured to process pixel data of a high exposure value image and a low exposure value image pixel by pixel and output pixel data of a processed image. As the image format of the high exposure value image, the low exposure value image, and the processed image, RAW data output from the image sensor is assumed. However, RGB4: 4: 4 format other than RAW data, YCbCr4: 4: 4 format, A YCbCr4: 2: 2 format or the like may be used.

  The difference between the image processing apparatus of the present embodiment and the conventional image processing apparatus is that the luminance component of the high exposure value image and the low exposure value image is added, and a portion for calculating the composition ratio from the luminance addition result is provided. It is.

  The luminance calculation means 11 in FIG. 1 calculates the luminance of the high exposure value image. Similarly, the luminance calculation means 12 calculates the luminance of the low exposure value image. When the high exposure value image and the low exposure value image are in a data format in which luminance data is calculated in advance, such as YCbCr4: 4: 4, the luminance calculating unit 11 and the luminance calculating unit 12 are not necessary.

  The exposure value ratio conversion means 13 is applied to the high exposure value image to adjust the gradation level with the low exposure value image. The exposure value ratio conversion means 13 converts the input pixel data in a look-up table (LUT) format that outputs converted output data. The LUT format is used to correct a portion where the gradation characteristics are nonlinear in the high exposure value image. The exposure value ratio converting means 13 is applied independently to each of image data including luminance and luminance components. For example, in the case of RAW data, the exposure value ratio conversion means 13 includes five types of LUTs: LUT for luminance, LUT for R component, LUT for Gr component, LUT for Gb component, and LUT for B component. Further, the exposure value ratio conversion means 13 is not applied to image data that does not include a luminance component such as a Cb component and a Cr component in the YCbCr format.

  The adding means 15 adds the brightness of the high exposure value image after the exposure value ratio conversion by the exposure value ratio converting means 13 and the brightness of the low exposure value image calculated by the brightness calculating means 12. Thereafter, the synthesis ratio coefficient calculation means 16 calculates a synthesis ratio coefficient from the luminance addition result by the addition means 15. The composition ratio coefficient calculation means 16 is composed of an LUT that receives the luminance addition result and outputs the composition ratio coefficient. There are two types of composition ratio coefficients: a composition ratio coefficient for a high exposure value image and a composition ratio coefficient for a low exposure value image, each of which is in the range of 0.00 to 1.00. The sum of both composite ratio coefficients is 1.00.

  The calculated composite ratio coefficient is multiplied by the composite ratio multiplication means 17 to the high exposure value image pixel data after the exposure value ratio conversion. Similarly, the composition ratio multiplication means 18 multiplies the low exposure value image pixel data by a composition ratio coefficient. Two types of multiplication results are added by the adding means 19 to synthesize an image.

  Since the exposure value ratio is converted by the exposure value ratio conversion means 13 for the high exposure value image to expand the gradation, a gradation conversion process is required to keep the original gradation. The post-combination gradation conversion processing means 20 compresses the gradation, thereby compressing the spread gradation within a predetermined gradation. The post-combination gradation conversion processing unit 20 is configured by an LUT that receives the pixel data of the post-combination image output from the addition unit 19 and outputs the processed image pixel data after gradation compression.

  By configuring the image processing apparatus in this way, it is possible to calculate the composition ratio by simultaneously referring to both the high exposure value image and the low exposure value image, so that the composition ratio suitable for the blacked-out and overexposed areas is appropriate. Can be applied.

  In the description of the present embodiment, the method for the high exposure value image and the low exposure value image is described. However, there are methods for adjusting the exposure, such as an exposure time and an analog gain of the image sensor. The present embodiment can be applied even if the exposure is adjusted.

Embodiment 2. FIG.
FIG. 2 shows the configuration of the image processing apparatus according to the present embodiment. The image processing apparatus according to the present embodiment is different from the image processing apparatus according to the first embodiment in that exposure value ratio conversion means 14 for low exposure value images is provided.

  Similar to the high exposure value image, the exposure value ratio converting means 14 is applied to the low exposure value image to adjust the gradation level of the high exposure value image. The exposure value ratio conversion means 14 converts the input pixel data in a look-up table (LUT) format that outputs converted output data. The LUT format is used to correct a portion where the tone characteristics are nonlinear in the low exposure value image. The exposure value ratio conversion means 14 is also applied independently to each of image data including luminance and luminance components. Further, the exposure value ratio conversion means 14 is not applied to image data that does not include a luminance component.

  By configuring the image processing apparatus in this way, it is possible to correct a portion where the tone characteristic is nonlinear even for a low exposure value image, and thus it is possible to perform synthesis without a nonlinear characteristic. Become.

  The image processing technique of the present invention can be used, for example, in a digital still camera signal processing DSP (Digital Signal Processor).

1 is a block configuration diagram of an image processing apparatus according to Embodiment 1 of the present invention. It is a block block diagram of the image processing apparatus by Embodiment 2 of this invention. It is a block block diagram of the conventional image processing apparatus.

Explanation of symbols

  11: Luminance calculation means, 12; Brightness calculation means, 13; Exposure value ratio conversion means, 14; Exposure value ratio conversion means, 15; Addition means, 16; Composite ratio coefficient calculation means, 17; Synthesis ratio coefficient multiplication means 19; addition means 20; post-synthesis gradation conversion processing means 101; CCD 102; memory 103; multiplier 104; timing control means 105; level weight H multiplier 106 Level weight L multiplier 107; adder 108; speed conversion means 109; level compression means.

Claims (4)

In an image processing apparatus that synthesizes two types of digital images, a high exposure value image and a low exposure value image with the same subject and different exposure conditions,
Means for calculating luminance for each of the two images;
Means for converting luminance and pixel data of the high exposure value image in accordance with a ratio of the exposure values of the two images;
Means for adding the brightness of the low exposure value image and the brightness of the high exposure value image after conversion;
Means for calculating a composition ratio for each of the two images from the addition result;
Means for multiplying the pixel data of the low exposure value image and the pixel data of the converted high exposure value image by the respective composite ratios;
Means for adding pixel data after multiplication of the respective synthesis ratios;
An image processing apparatus comprising means for performing gradation conversion on pixel data after addition.   The pixel data of the low exposure value image multiplied by the luminance of the low exposure value image to be added to the luminance of the high exposure value image converted according to the ratio of the exposure value of the two images and the composite ratio are exposed to the two images. The image processing apparatus according to claim 1, further comprising means for converting according to a value ratio. In an image processing method for synthesizing two types of digital images, a high exposure value image and a low exposure value image with the same subject and different exposure conditions,
Calculating a luminance for each of the two images;
Converting the brightness and pixel data of the high exposure value image in accordance with the ratio of the exposure values of the two images;
Adding the brightness of the low exposure value image and the brightness of the converted high exposure value image;
Calculating a synthesis ratio for each of the two images from the addition result;
Multiplying the respective composition ratios by the pixel data of the low exposure value image and the pixel data of the converted high exposure value image;
Adding pixel data after multiplication of each combination ratio;
An image processing method comprising a step of performing gradation conversion on pixel data after addition.   Pixel data of the low exposure value image multiplied by the luminance of the low exposure value image added to the luminance of the high exposure value image converted according to the ratio of the exposure value of the two images and the composition ratio are added to the two images. The image processing method according to claim 3, further comprising a step of converting according to a ratio of exposure values.

Images