JP4118864B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus using a solid-state imaging device such as a CCD image sensor, and more specifically to an imaging apparatus that performs γ correction processing based on a calculation result of image data in units of pixels.

  Generally, the gradation characteristics of an image are not uniform for any subject, and it can be said that adjusting the gradation characteristics for each subject is desirable for improving the image quality. For example, in a cloudy outdoors or indoors, the luminance difference between the bright and dark parts of the subject is small, so it is desirable to increase the γ coefficient (to make the amplification factor close to 1) in order to enhance the contrast. On the contrary, since the contrast is originally strong for a subject in fine weather, it is desirable to reduce the γ coefficient so as to reduce the contrast, and as a result, it contributes to the expansion of the dynamic range.

  For example, Patent Document 1 below proposes to measure a plurality of locations of a subject and perform gradation conversion according to a light amount difference between a maximum value and a minimum value.

  Further, Patent Document 2 below divides an image into a plurality of regions and uses the average luminance of the area of interest as a variable, for example, obtains the expected luminance after γ correction by referring to a monotonic decreasing function such as an exponential function or a linear function. To γ correction.

Japanese Patent Laid-Open No. 3-104486 (6th page, FIG. 1) JP 2000-209437 A (page 6, FIG. 6)

  However, in the publication of the above-mentioned Patent Document 1, it is necessary to use a photometric element when determining the γ coefficient. If the number of photometric areas is increased in order to improve the detection accuracy, the photometric element is increased accordingly. There is a problem that the circuit scale becomes large. In addition, since only brightness values obtained by photometry are observed, for example, in an indoor image with a narrow dynamic range, the illumination light part is considered to be high brightness in the case of an object in which illumination light appears only in a part of the photometry area. By determining that the difference is larger than a certain value, there is a possibility that selection of γ correction is erroneous.

  Further, the publication of the above-mentioned Patent Document 2 has a problem that the expected luminance value becomes a quadratic function of the evaluation luminance value of the base image, and the amount of calculation of the CPU in the process until the γ coefficient is determined increases. is there. Moreover, since the above-mentioned Patent Document 2 uses only the average brightness value of the central area and the peripheral area as a parameter, when a high-luminance part such as illumination is present in a part of an image with a narrow dynamic range, as in the above-mentioned Patent Document 1, On the other hand, there is a possibility that the selection of correction will be wrong, and on the contrary, even if the bright and dark areas are distributed in equal proportions in each divided area of the screen in a subject in fine weather, the image is originally a wide dynamic range. However, when the average value is calculated, no difference occurs, and as a result, there is a possibility that selection of γ correction is wrong.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to realize an optimum γ correction without making a mistake in selecting a γ correction in an imaging apparatus, thereby improving contrast. In addition, it is possible to reduce the processing load of the CPU when selecting the γ coefficient and to enable the calculation processing in a short time.

  An imaging apparatus according to the subject of the present invention divides one screen into a predetermined number of regions, a charge storage time calculation unit that calculates a charge storage time based on image data of R, G, and B pixel units for one screen, An average luminance value calculating unit that calculates an average luminance value in each region based on the image data in R, G, and B pixel units; a feature value calculating unit that calculates a characteristic value based on the average luminance value in each region; Γ coefficient selection means for selecting a γ coefficient based on the calculation result of the feature value calculation unit and the charge accumulation time, and γ correction for performing γ correction on the image data based on the γ coefficient determined by the γ coefficient selection means And an exposure control unit that controls exposure according to the γ coefficient determined by the γ coefficient selecting means.

  Hereinafter, various embodiments of the subject of the present invention will be described in detail along with the effects and advantages thereof with reference to the accompanying drawings.

  According to the subject matter of the present invention, contrast can be improved by detecting the dynamic range of a subject with high accuracy and performing optimum γ correction, and reducing the calculation processing load of the CPU when selecting a γ coefficient. Thus, it is possible to provide an imaging apparatus that can perform arithmetic processing in a short time.

(Embodiment 1)
FIG. 1 is a block diagram illustrating a basic configuration of the imaging apparatus according to the present embodiment. In the figure, a solid-state imaging device 2 such as a CCD photoelectrically converts a subject image incident through a lens 1. The analog signal processing unit 3 performs correlated double sampling processing (CDS) and automatic signal amplification processing (AGC) on the output signal of the solid-state imaging device 2.

  The A / D conversion unit 4 converts the output signal of the analog signal processing unit 3 into digital data separated into three colors of R, G, and B, and then the dynamic range conversion unit 5 is necessary for γ correction. After selecting the γ coefficient, γ correction is executed on the image data. In particular, the CPU 7 of the dynamic range conversion unit 5 corresponds to a core part of the present embodiment, and among the plurality of γ coefficient candidate values stored (set) in the γ coefficient LUT unit 8, This is a calculation unit that selects a γ coefficient necessary for correction. The signal processing unit 6 of the dynamic range conversion unit 5 includes a portion that performs γ correction based on the determined γ coefficient (γ curve) selected by the CPU 7 and read from the γ coefficient LUT unit 8.

  FIG. 2 is a block diagram showing a detailed configuration example of the dynamic range converter 5 of FIG. In FIG. 2, an R, G, B pixel data reading unit 9 in the signal processing unit 6 reads out and reads 10-bit precision signal levels for all pixels of one image from the A / D converted image data. The outputted 10-bit precision image data is output to the respective units 10, 11, and 12 to be described later.

  Upon receiving this signal, the area-specific average luminance value calculation unit 11 in the CPU 7 calculates the average luminance value of each area (each divided area) when one image is divided into 16 as shown in FIG. Calculation is performed after conversion to an 8-bit precision signal.

  Based on the average luminance value for each region output by the region-specific average luminance value calculating unit 11, the feature value calculating unit 13 calculates (1) the maximum value and the minimum value, and (2) the maximum value and the minimum value. The difference value (corresponding to the second numerical value) and (3) the average value (m) of the average luminance values obtained by dividing the average luminance value of each divided region for all the divided regions by the number of divided regions An absolute value obtained by subtracting from the average luminance value of each area is integrated for all the divided areas, and then a first numerical value (σ) divided by the number of divided areas is calculated. Here, the calculated values of (1), (2), and (3) are referred to as “feature values”.

  Further, upon receiving the read 10-bit precision image data, the charge accumulation time calculation unit 12 calculates the charge accumulation time and outputs the calculation result to the γ coefficient selection unit 14.

  Thereafter, based on both the calculation result of the feature value calculation value 13 and the data of the charge accumulation time calculated by the charge accumulation time calculation unit 12, the γ coefficient selection unit 14 selects the γ coefficient that is optimal for improving the contrast of the subject. Make a selection. Here, a predetermined number of γ curves are prepared in advance in the γ coefficient LUT unit 8 (the look-up table (LUT) corresponds to table data stored in a storage device such as a memory). Therefore, the γ coefficient selection unit 14 reads the set value corresponding to the γ coefficient selected in the same unit 14 from the γ coefficient LUT unit 8 and reads the read set value (γ curve data) in the signal processing unit 6. By outputting to the unit 10, the γ correction unit 10 performs γ correction on the R, G, and B pixel data.

  Note that the constituent parts composed of the γ coefficient selecting unit 14 and the γ coefficient LUT unit 8 are collectively referred to as “γ coefficient selecting means”.

  Further, since the luminance of the entire image increases as the γ coefficient is closer to 0 (smaller) and the luminance of the entire image decreases as the γ coefficient is closer to 1 (larger), the exposure control unit 15 selects the γ coefficient with the γ coefficient selecting unit 14. Exposure control is dynamically performed according to the γ coefficient, and an average picture luminance level (APL: Average Picture Level) is changed as shown in FIG. 7 according to the γ coefficient. The change to the desired APL is realized by controlling the charge accumulation time by the number of drive pulses for discharging the charges output from the exposure control unit 15 to the timing generation unit 16. In order to increase APL, the number of drive pulses for discharging the charge is controlled so that the charge accumulation time is lengthened, and in order to decrease APL, the charge accumulation time is shortened.

  Here, the γ coefficient will be described. In the present embodiment, as shown in FIG. 3, three types of γ coefficients of γ = 0.45, γ = 0.60, and γ = 0.75 are prepared. Referring to FIG. 3, the x-axis direction is input (10 bits), the y-axis direction is output (8 bits), and the following equation (1) is used between the input x and the output y. The given relationship is established.

  In equation (1), the closer the γ coefficient is to 1 (larger), the stronger the contrast of the image. Conversely, the closer the γ coefficient is to 0 (smaller), the weaker the contrast. Therefore, for an image with a narrow dynamic range, such as when the subject is indoors, it is more preferable to set a larger value of the γ coefficient to increase the contrast. Conversely, for images with a wide dynamic range, such as when the subject is outdoors on a clear sky, by setting the γ coefficient to a small value, whiteout and blackout are reduced, and the wide dynamic range is maintained. An image is obtained, and a more preferable image quality is obtained.

  Next, details of the process up to selecting the γ coefficient will be described with reference to the flowchart of FIG.

  First, step S1 is processing in the area-specific average luminance value calculation unit 11 of FIG. In step S1, the same unit 11 divides one screen into 16 areas and calculates an average luminance value for each area. In calculating the average luminance value, a calculation formula according to human visual characteristics can be used as in the following formula (2). However, in terms of luminance, the G pixel is dominant and the CPU calculation is performed. From the viewpoint of reducing processing and shortening the average value calculation time, in the present embodiment, the same unit 11 calculates an average luminance value using only G pixels as shown in Equation (3).

  For example, in the image shown in FIG. 5, in order to calculate the average luminance value in the region 13, the same unit 11 integrates all the G pixel data in the region 13 and then calculates the integration result in the region 13. By dividing by the number of G pixels, the average luminance value of the divided region 13 is obtained.

  Similarly, the unit 11 sequentially calculates average luminance values for other regions. If FIG. 5 is regarded as an output image of an image, pixel data is read in the right direction from the G pixel at the upper left of the uppermost line, and when reading of the G pixel for one line is completed, the same applies to the G pixel at the next lower line. The operation of reading is repeated, and finally G pixel data up to the lowest line is read.

  In this way, by calculating the average luminance value using only the G pixel, the same unit 11 can reduce the arithmetic processing of the CPU and shorten the average value calculation time as described above.

  The timing of calculating the average luminance value of each area is as shown in FIG. In an arbitrary vertical synchronizing signal (VD) period, first, an average luminance value is calculated for the G pixel data in the regions 13 to 16, and then an average luminance value is calculated for the G pixel data in the regions 9 to 12, and the regions 5 to 8 are calculated. Also, the average luminance value of G pixels is sequentially calculated for the regions 1 to 4, and when the average luminance value is calculated for all regions, the feature value is calculated in the blanking period based on these average luminance values.

  In step S2 of FIG. 4, the feature value calculation unit 13 of FIG. 2 calculates the maximum value and the minimum value of these average luminance values based on the 16 average luminance values, and then calculates the maximum value and the A difference value (second numerical value) from the minimum value is calculated, the first numerical value described above is calculated, and these feature value data are output to the γ coefficient selection unit 14. In parallel with the calculation of the feature value data, the charge accumulation time calculation unit 12 in FIG. 2 calculates the charge accumulation time and outputs the result to the γ coefficient selection unit 14.

  Each step after step S3 in FIG. 4 is an operation in the γ coefficient selection unit 14 in FIG. First, in step S3, the part 14 compares the charge accumulation time with the first predetermined value A, and switches the subsequent flow according to the comparison result. For example, when the first predetermined value A is set to 1/60 seconds and the charge accumulation time is shorter than this, it is determined that the subject is outdoors or in the shade on a fine day, and the unit 14 performs step S4. Then, γ correction is performed so as to reduce overexposure and underexposure in the image.

  On the other hand, when the charge accumulation time is longer than 1/60 seconds or the same time, it is a case where the subject is indoors or in the shade when it is cloudy, and at this time, it is determined that the dynamic range is narrow, The unit 14 performs γ correction for improving the contrast feeling in the subsequent flow. Specifically, the part 14 changes the flow coefficient in step S8 according to the difference value (α) between the maximum value and the minimum value calculated in step S2, thereby changing the γ coefficient. decide. For example, under the condition H (when 100 <α <150), the part 14 applies γ = 0.60, and under the condition I (when 0 <α ≦ 100), the dynamic range is narrower than the condition H, and the contrast Judging that it is an image with a low feeling, the same part 14 is set so as to increase the contrast feeling by applying γ = 0.75.

  If the charge accumulation time is shorter than 1/60 seconds in step S3, the γ coefficient selection unit 14 proceeds to step S4. In step S4, the same part 14 selects whether the highest luminance part of the subject exists in the surrounding 12 areas or the central 4 areas for the 16 areas.

  For example, in an image in which the sky appears in the background, the highest luminance part often exists in 12 surrounding areas. Therefore, the part 14 switches the subsequent flow depending on whether or not the maximum luminance value area is in the center of the image.

  If NO in step S4, that is, if the highest luminance value area is not in the center of the image but in the surrounding area, it is determined that the image has the widest dynamic range, and the same section 14 proceeds to step S6. .

  On the other hand, if YES in step S4, that is, if the maximum luminance value area is in the center of the image, the sky is not reflected around the image, or only the sky is captured over the entire screen. May be in the shade, the sky is reflected, but a highly reflective subject is present in the center of the screen. Therefore, if YES in step S4, the part 14 proceeds to step S5.

  In step S <b> 5, the γ coefficient selection unit 14 compares the first numerical value σ calculated based on the average luminance value of the 16 regions with the second predetermined value B. Here, the case where the first numerical value σ is equal to the second predetermined value B or smaller than the second predetermined value B is a case where the dynamic range is a flat image having a narrow dynamic range. This corresponds to the case where the wall of a building is imaged on the entire screen even in a sunny day. On the other hand, when the first numerical value σ is larger than the second predetermined value B, the image has a wide dynamic range in which there are no bright objects such as the sky in the surroundings but there are both bright and dark areas. ,to decide.

  By using not only the difference value (second numerical value) calculated from the maximum value and the minimum value in the average luminance value of each area, but also a value that defines the deviation amount such as the first numerical value, the dynamic range of the subject Characteristics can be determined with high accuracy.

  In this embodiment, an average deviation (mean deviation: md) is used as the first numerical value that defines the deviation amount. The calculation formula of the average deviation md is as the following formula (4).

  The advantage of using the average deviation formula is that, unlike the standard deviation formula (not shown), it does not include the square term or square root of xi, and therefore, the calculation processing load is small.

  The second predetermined value B in step S5 is set to 20, for example, and the magnitude is compared with the calculated first numerical value σ (the same value as the average deviation md). If the first numerical value σ is greater than 20, the γ coefficient selection unit 14 proceeds to step 6. Conversely, if the first numerical value σ is less than or equal to 20, the part 14 proceeds to step 7.

  In step S6, for example, when the difference value (second numerical value α) between the maximum value and the minimum value calculated in step S2 is the condition C (150 <α <255), the subject having the widest dynamic range is selected. Since it is determined that there is, the same unit 14 applies γ = 0.45 (corresponding to γ1). When the difference value (α) is a condition D (100 <α ≦ 150), the same unit 14 applies γ = 0.60 (corresponding to γ2), and the difference value (α) is a condition E (0 <α ≦ 150). In the case of 100), the part 14 applies γ = 0.75 (corresponding to γ3).

  Similarly, in step S7, for example, when the difference value (α) between the maximum value and the minimum value calculated in step S2 is a condition F (100 <α <150), the same unit 14 applies γ = 0.60. When the difference value (α) is the condition G (0 <α ≦ 100), the same unit 14 applies γ = 0.75.

  In the above described example of the present embodiment, the number of tables prepared in the γ coefficient LUT unit 8 is three kinds of γ = 0.45, γ = 0.60, and γ = 0.75. However, more γ curves may be prepared (the number of tables is arbitrary).

  Furthermore, although the number of divided areas is set to 16 in one example of the present embodiment, the number of divided areas is not limited to 16, and it is obvious that there is no problem even if the number of divided areas is arbitrarily set.

  As described above, in the imaging apparatus according to the present embodiment, the dynamic range of the subject can be detected with high accuracy and the optimum γ correction can be performed, thereby improving the contrast and selecting the γ coefficient. It is possible to provide an imaging apparatus that can reduce the burden of arithmetic processing on the CPU and perform arithmetic processing in a short time.

(Modification)
In this embodiment, the average luminance value in each area is used. However, if the increase in the CPU processing load can be allowed, the median value (median) may be used instead of the average value. Is possible. Here, the “median value” is a median luminance value when image data of R, G, and B pixel units of each divided region is rearranged in descending order or ascending order.

  In this way, there is an advantage that the influence of the abnormal value of the defective pixel can be ignored by using the median value instead of the average luminance value.

(Appendix)
While the embodiments of the present invention have been disclosed and described in detail above, the above description exemplifies aspects to which the present invention can be applied, and the present invention is not limited thereto. In other words, various modifications and variations to the described aspects can be considered without departing from the scope of the present invention.

  The imaging apparatus according to the present invention can be applied to devices such as a mobile phone having a camera function, a digital still camera, a digital video camera, and a personal digital assistant (PDA).

It is a block diagram which shows the structural example of the imaging device which concerns on this Embodiment. It is a block diagram which shows the structural example of the dynamic range conversion part which concerns on this Embodiment. It is a figure which shows the structural example of the gamma curve which concerns on this Embodiment. It is a flowchart which shows an example of the process of (gamma) coefficient selection which concerns on this Embodiment. It is a figure which shows an example of the area | region division which concerns on this Embodiment. It is a timing chart which shows the calculation timing of the average luminance value which concerns on this Embodiment. It is a figure which shows APL relative ratio with respect to (gamma) coefficient.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens, 2 Solid-state image sensor (CCD), 3 Analog signal processing part, 4 A / D conversion part, 5 Dynamic range conversion part, 6 Signal processing part, 7 CPU, 8 gamma coefficient LUT part, 9 RGB pixel data reading part DESCRIPTION OF SYMBOLS 10 (gamma) correction | amendment part, 11 area | region average luminance value calculation part, 12 charge accumulation time calculation part, 13 feature value calculation part, 14 gamma coefficient selection part, 15 exposure control part, 16 timing generation part.

Claims (5)

A charge accumulation time calculation unit for calculating a charge accumulation time based on image data in units of R, G, and B pixels for one screen;
An average luminance value calculating unit that divides one screen into a predetermined number of regions and calculates an average luminance value in each region based on the image data in units of R, G, and B pixels;
A feature value calculation unit that calculates a feature value based on an average luminance value of each region;
Γ coefficient selection means for selecting a γ coefficient based on the calculation result of the feature value calculation unit and the charge accumulation time;
A γ correction unit that performs γ correction on the image data based on the γ coefficient determined by the γ coefficient selection unit;
An exposure control unit that controls exposure according to the γ coefficient determined by the γ coefficient selecting means,
Imaging device. The imaging apparatus according to claim 1,
The characteristic value is
The maximum value and the minimum value selected from the average luminance value of each of the divided areas with the number of divided areas as a parameter,
A difference value between the maximum value and the minimum value;
The absolute value is obtained by subtracting the average value of the average luminance values obtained by dividing the average luminance value of each divided region over all the divided regions by the number of divided regions from the average luminance value of each divided region. The first numerical value calculated by dividing the total number of divided areas and then dividing by the number of divided areas,
Imaging device. The imaging apparatus according to claim 2,
The γ coefficient selecting means has three types of γ1, γ2, and γ3 as values of the γ coefficient.
A relationship of 0 <γ1 <γ2 <γ3 <1 is established between γ1, γ2, and γ3,
When defining the difference value between the maximum value and the minimum value of the average luminance value in each of the divided areas calculated in the feature value calculation unit as a second numerical value,
The γ coefficient selection means includes:
The charge accumulation time is compared with a first predetermined value, and when the charge accumulation time ≧ the first predetermined value, either γ2 or γ3 is given to the γ correction unit according to the second numerical value. On the other hand, if the charge accumulation time is less than the first predetermined value, the maximum value of the average luminance value of each of the divided regions is located at the center of the image or at the periphery of the image When the maximum value of the average luminance value of each of the divided areas is located in the peripheral portion of the image, any one of γ1, γ2, and γ3 is selected according to the second numerical value. When the maximum value of the average luminance value of each divided region is located at the center of the image, the first numerical value is compared with a second predetermined value. If numerical value> the second predetermined value, the values γ1, γ2, and And γ3 is selected as the γ coefficient to be given to the γ correction unit, and when γ2 and γ3 are in accordance with the second numerical value when the first numerical value ≦ the second predetermined value, Any one is selected as the γ coefficient given to the γ correction unit,
Imaging device. The imaging apparatus according to any one of claims 1 to 3,
The average luminance value calculating unit calculates an average luminance value of each of the divided regions using only G pixels instead of all of R, G, and B types of pixels.
Imaging device. The imaging apparatus according to any one of claims 1 to 3,
The average luminance value calculation unit is not the average luminance value of each divided region, but the central luminance value when the image data of each R, G, B pixel unit of each divided region is rearranged in descending order or ascending order. It is characterized by using
Imaging device.

Images