JP2010183461A - Image capturing apparatus and method of controlling the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image capturing apparatus capable of obtaining an image with a wide dynamic range while suppressing an increase of noise, and to provide a method of controlling the same. <P>SOLUTION: The apparatus includes a means for determining a dynamic range expansion amount for decreasing the halation amount, based upon a halation amount of the image. The apparatus also captures the image with sensitivity reduced, based upon the dynamic range expansion amount, and corrects a decrease in lightness of the captured image due to the sensitivity reduction, thereby obtaining the image having an expanded dynamic range. When the reduction of the sensitivity based upon the dynamic range expansion amount is performed by giving priority to a sensitivity setup, and when the reduction can not be achieved only with the sensitivity setup, change of exposure conditions is combined. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a control method thereof.

  Conventionally, digital cameras and digital videos using image sensors such as CCD image sensors and CMOS image sensors have been widely used. However, these image sensors have a narrow dynamic range (latitude) compared to silver salt films. For this reason, when a high-contrast scene is imaged, it is easy to cause gradation deterioration (blackout) in a low luminance part and gradation deterioration (whiteout) in a high luminance part.

  In response to such a problem, a system capable of automatically controlling the dynamic range has been proposed.

  For example, in Patent Document 1, when a main subject is detected to be backlit or high-contrast from a captured image, a black saturation point and a white saturation point are specified from the histogram of the image, and the brightness of the main subject is determined. It has been proposed to perform gradation correction so as to be appropriate.

  Moreover, in patent document 2, using the imaging device which has arrange | positioned the high sensitivity light receiving element and the low sensitivity light receiving element, the same scene is imaged with the light receiving element of different sensitivity, and two types of images with different dynamic ranges are acquired, It has been proposed to synthesize them according to the scene analysis results.

Japanese Patent Application Laid-Open No. 2005-209012 JP 2004-186876 A

  However, in the method of Patent Document 1, although there is an effect for enhancing the contrast feeling, the point at which the sensor is saturated does not change, so the effect of expanding the dynamic range cannot be obtained. In addition, when tone correction is performed so that the output is larger than the input, there is a problem that noise components are also amplified simultaneously.

  In addition, the method of Patent Document 2 can obtain the effect of expanding the dynamic range, but in order to synthesize the output of the light receiving element with low sensitivity and the output of the light receiving element with high sensitivity, the output of the light receiving element with low sensitivity is added. In this case, noise components are also added. Therefore, there has been a problem that image quality deterioration occurs.

  The present invention has been made in view of such a problem of the prior art, and an object thereof is to provide an imaging apparatus capable of obtaining an image with a wide dynamic range while suppressing an increase in noise, and a control method thereof. One.

  The above-described object is to reduce the amount of whiteout based on the amount of whiteout amount calculation means for calculating the amount of whiteout from the image captured using the image sensor, and the amount of whiteout amount. D range expansion amount determining means for determining the dynamic range expansion amount, a reduction amount of the imaging sensitivity according to the dynamic range expansion amount, and a correction amount for correcting the brightness of an image captured with the reduced imaging sensitivity A correction amount determining means for determining the imaging sensitivity and a setting means for setting an imaging condition for realizing a reduction amount of the imaging sensitivity. If the reduction amount can be realized by setting the imaging sensitivity, the reduction amount is realized. If the imaging sensitivity is set as the imaging condition and the reduction amount cannot be realized by setting the imaging sensitivity, the imaging sensitivity reduced as much as possible and the exposure condition that realizes the reduction amount by combining this imaging sensitivity are set. Imaging strip A setting unit that sets the image, an imaging control unit that uses the image sensor and performs imaging according to the imaging conditions set by the setting unit, and a correction unit that corrects the brightness of the image captured by the imaging control unit according to the correction amount. It is achieved by an imaging device characterized by having.

  Further, the above-described object is to reduce the amount of whiteout based on the amount of whiteout amount calculation step for calculating the amount of whiteout of the image from the image captured using the image sensor and the amount of whiteout amount. A D range expansion amount determination step for determining a dynamic range expansion amount, an imaging sensitivity reduction amount corresponding to the dynamic range expansion amount, and brightness of an image captured with the reduced imaging sensitivity. A correction amount determining step for determining a correction amount, and a setting step for setting an imaging condition for realizing a reduction amount of the imaging sensitivity. If the reduction amount can be realized by setting the imaging sensitivity, the reduction amount If the imaging sensitivity is set as the imaging condition and the reduction amount cannot be realized by setting the imaging sensitivity, the reduction amount is realized by combining the imaging sensitivity with the imaging sensitivity reduced to the extent possible. A setting step for setting an exposure condition to be captured as an imaging condition, an imaging control step for performing imaging in accordance with the imaging condition set in the setting step using an imaging device, and correcting the brightness of the image captured in the imaging control step. It is also achieved by a method for controlling an imaging apparatus, comprising a correction step for correcting according to a quantity.

  With such a configuration, according to the present invention, it is possible to obtain an image with a wide dynamic range while suppressing an increase in noise.

It is a block diagram which shows the structural example of the imaging device which concerns on embodiment of this invention. It is the figure which showed typically the process in which a face is detected from an original image. It is a flowchart for demonstrating the face detection operation | movement in the face detection circuit of the imaging device which concerns on embodiment of this invention. It is a chromaticity diagram showing representative colors in the CIE-L * a * b * color space. It is a figure which shows the example of the coefficient of the two-dimensional high pass filter which the face detection circuit of the imaging device which concerns on embodiment of this invention uses. It is a block diagram showing an example of composition of an AFE circuit of an imaging device concerning an embodiment of the present invention. It is a figure which shows typically the example of the histogram which the histogram preparation circuit of the imaging device which concerns on embodiment of this invention produces. It is the block diagram which showed the flow of the signal processing at the time of imaging standby in the imaging device which concerns on embodiment of this invention. It is a flowchart for demonstrating the determination operation | movement of the dynamic range (D range) expansion amount in the imaging device which concerns on embodiment of this invention. In the imaging device concerning an embodiment of the present invention, it is a figure showing the example of the D range expansion amount finally determined according to the size of the D range expansion amount of the face area, and the D range expansion amount of the whole image. It is a figure showing the conceptual diagram of D range in the imaging device concerning an embodiment of the present invention. It is a figure which shows the example of a relationship between AE target value, a saturation signal value, and D range in the imaging device which concerns on embodiment of this invention. It is a figure which shows the example of a setting of the gamma correction characteristic in the signal processing circuit of the imaging device which concerns on embodiment of this invention. It is a flowchart explaining operation | movement of the main imaging process at the time of D range expansion in the imaging device which concerns on embodiment of this invention.

Hereinafter, the present invention will be described in detail based on exemplary embodiments with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to an embodiment of the present invention. The imaging apparatus according to the present embodiment includes any apparatus having a function of capturing an image using an imaging element. Such devices include not only digital still cameras and digital video cameras, but also mobile phones, PDAs, personal computers, etc. that have built-in or connected cameras.

  In FIG. 1, the operation unit 112 includes buttons, switches, and the like, and is used by a user to give an instruction to the imaging apparatus or make settings. A shutter button is also included in the operation unit 112, and in this embodiment, it is possible to detect a half-pressed state and a fully-pressed state of the shutter button.

  The system controller 107 recognizes the half-pressed state of the shutter button as an imaging preparation start instruction and the fully-pressed state as an imaging start instruction. The system controller 107 includes, for example, a CPU, a ROM, a RAM, and the like, and controls the overall operation of the imaging apparatus when the CPU executes a program stored in the ROM using the RAM.

  The lens device 200 includes a lens group including a focus lens, a drive device that drives the focus lens, a diaphragm, and a mechanical shutter, and operates based on control of the system controller 107.

  The imaging element 101 is a photoelectric conversion element such as a CCD image sensor or a CMOS image sensor. An analog front end (AFE) circuit 150 performs gain adjustment, A / D conversion, and the like on the analog image signal output from the image sensor 101 and outputs the digital image signal. Details of the AFE circuit 150 will be described later.

The buffer memory 103 temporarily stores the digital image signal output from the AFE circuit 150.
The compression / decompression circuit 104 encodes the captured image data into a recording image file (for example, JPEG file) format, or decodes the image file read from the recording medium 106.

The recording device 105 reads / writes data from / to a recording medium 106 such as a built-in memory or a removable memory card under the control of the system controller 107.
The display control circuit 108 controls the display operation on the display unit 110 having a display device such as an LCD in accordance with the control of the system controller 107.
The D / A converter 109 converts the display digital image signal output from the display control circuit 108 into an analog image signal that can be displayed by the display unit 110.

  The display unit 110 displays not only the captured image but also a GUI screen for the user to make various settings and instructions to the imaging device, and various information related to the imaging device. In addition, by sequentially displaying continuously captured images on the display unit 110, the display unit 110 can function as an electronic viewfinder (EVF). This sequential imaging / display operation for causing the display unit 110 to function as an EVF is also referred to as a through display or a live view display.

  The face detection circuit 120 performs face detection processing on image data in YUV format or RAW format stored in the buffer memory 103, and generates a face detection result including the size and position of the face area in the image as a histogram generation circuit. To 130.

  The face detection method used by the face detection circuit 120 is not particularly limited, and any known method can be applied. As a known face detection technique, a method based on learning using a neural network or the like, template matching is used to search a part having a characteristic shape of eyes, nose, mouth, etc. from an image, and if the degree of similarity is high, it is regarded as a face There are methods. In addition, many other methods have been proposed, such as a method that detects image feature amounts such as skin color and eye shape and uses statistical analysis. A plurality of these methods can be combined to improve the accuracy of face detection. Specific examples include a face detection method using wavelet transform and image feature amount described in JP-A-2002-251380.

Here, a specific face detection operation example of the face detection circuit 120 will be described with reference to FIGS.
FIG. 2 is a diagram schematically showing the process of detecting a face from an original image, and FIG. 2A shows the original image.

FIG. 3 is a flowchart for explaining the face detection operation of the face detection circuit 120.
In S101, the face detection circuit 120 extracts a skin color area from the original image. FIG. 4 is a chromaticity diagram showing representative colors in the CIE-L * a * b * color space, and an ellipse therein is an area that is highly likely to be a skin color.
The face detection circuit 120 converts the original image data in RGB format into the L * a * b * format by a known method, and extracts a skin color region composed of pixels having chromaticity within the region indicated by the ellipse in FIG. . FIG. 2B schematically shows a skin color area extracted from the original image.

  Next, in S102, the face detection circuit 120 extracts a high frequency component from the extracted skin color region. Specifically, the face detection circuit 120 applies a high-pass filter to the skin color area. FIG. 5 is a diagram illustrating an example of coefficients of a two-dimensional high-pass filter. An example of an image obtained by applying a high-pass filter to the image of FIG. 2B is shown in FIG.

In S103, the face detection circuit 120 performs template matching using an eye template as shown in FIG. 2D on the image after the high-pass filter is applied, and detects an eye in the image.
In S104, the face detection circuit 120 determines the face area based on the positional relationship of the eye area detected in S103, and obtains a face detection result including the position and size of the face area.

  Returning to FIG. 1, the histogram creation circuit 130 acquires the detection result of the face area from the face detection circuit 120, and creates a histogram related to the luminance value of the pixels included in the face area. The histogram creation circuit 130 can also create a histogram of the luminance values of the included pixels for each partial region obtained by dividing the image into a plurality of regions. The created histogram is stored in the buffer memory 103.

  The signal processing circuit 140 applies signal processing to the image data stored in the buffer memory 103 in accordance with signal processing parameters (white balance correction coefficient, gamma parameter, etc.) set by the system controller 107. Then, the signal processing circuit 140 generates YUV format image data and stores it again in the buffer memory 103.

  As will be described later, the imaging apparatus according to the present embodiment realizes expansion of the dynamic range by sensitivity adjustment (gain adjustment) in the AFE circuit 150 and gamma correction in the signal processing circuit 140.

FIG. 6 is a block diagram illustrating a configuration example of the AFE circuit 150.
The clamp circuit 151 clamps the signal output from the image sensor 101 to the reference black level so that the light shielding level of the sensor or the output value of the reference voltage region becomes zero.
The CDS gain circuit 152 applies a CDS gain (analog gain) to the clamped signal. The CDS gain applied in a general AFE circuit has discrete values such as 0, 3, 6 [dB].

  The signal to which the analog gain is applied is converted into digital data by the A / D converter 153. Next, a variable gain amplifier (VGA) gain circuit 154 applies a VGA gain to the digital data. In the present embodiment, it is assumed that the value of the VGA gain can be adjusted in units of 0.125 dB, for example, in the range of 0 to 36 dB. The signal to which the VGA gain is applied is clipped to a predetermined number of bits by the clip circuit 155 and output to the buffer memory 103.

  In this embodiment, the imaging sensitivity (ISO sensitivity) setting of the imaging apparatus is realized by controlling the values of the CDS gain and the VGA gain applied by the AFE circuit 150 by the system controller 107.

  Also, the sensitivity for correcting variations in the sensitivity characteristics of the image sensor 101 by irradiating the imaging device with a reference amount of light and controlling the CDS gain and VGA gain so that a constant luminance signal value is output from the imaging device. Adjustments can be made.

  In the present embodiment, sensitivity setting is performed by combining a CDS gain and a VGA gain. For example, when setting a gain of 6 dB as a whole at low sensitivity, for example, the CDS gain is set to 3 dB and the VGA gain is set to 3 dB. When a gain of 24 dB is set at high sensitivity, for example, the CDS gain is set to 6 dB and the VGA gain is set to 18 dB. As described above, since the CDS gain cannot generally be set finely, a rough gain is set by the preceding CDS gain, and the VGA gain is controlled in order to perform delicate sensitivity control such as a sensitivity adjustment portion.

  Generally, a gain circuit amplifies a noise component simultaneously with a signal component. Therefore, in order to suppress the amplification of the noise component superimposed in the analog circuit, it is preferable to set the preceding CDS gain as high as possible in the combination of the CDS gain and the VGA gain that can realize the total gain amount. Such a setting can also realize an effect that the quantization accuracy of the A / D converter 153 can be effectively utilized to the maximum extent.

Next, an operation at the time of imaging in the imaging apparatus having the above-described configuration will be described.
In the standby mode when operating in the imaging mode and no imaging preparation instruction or imaging start instruction is input, the imaging apparatus of the present embodiment causes the display unit 110 to function as an EVF. That is, the system controller 107 performs processing for continuously capturing images at a predetermined rate (for example, 30 frames / second), generating a display image from the captured images, and displaying the display image on the display unit 110.

  When face detection execution is set, the face detection circuit 120 detects a face for a display image (hereinafter also referred to as an EVF image), and outputs the detection result to the system controller 107. Then, the system controller 107 instructs the display control circuit 108 together with the position information of the face area to superimpose and display the face frame for presenting the detected face area to the user on the EVF image.

  The face detection result is also supplied to the histogram creation circuit 130, which creates a histogram from pixels included in the face area in the EVF image. The histogram creation circuit 130 can also create a histogram for each region obtained by dividing the entire image into a plurality of regions. The created histogram is stored in the buffer memory 103.

FIG. 7 is a diagram schematically showing an example of a histogram created by the histogram creation circuit 130 in the present embodiment.
FIG. 7 shows that a histogram 73 for each of the partial areas 71 obtained by dividing the entire image into four equal parts in the vertical and horizontal directions and a histogram 74 for the face area 72 are created. Note that the histograms 73 and 74 in FIG. 7 are cumulative histograms. Note that when creating a histogram for a partial region, the face region may be excluded. By doing so, it is possible to create a histogram of the face region and the rest (background).

  In the present embodiment, the luminance value YHi having a frequency of 80% in the cumulative histogram 73 of the partial region 71 and the luminance value YHiFace having a frequency of 90% in the cumulative histogram 74 of the face region 72 are as follows. It is used for evaluation of a whiteout area.

  When the user fully presses the shutter button and an imaging start instruction is input, the system controller 107 performs an imaging operation based on processing results such as automatic exposure control (AE) and automatic focus detection (AF). Specifically, the system controller 107 controls the focal position and aperture of the lens device 200, the mechanical shutter, the image sensor 101, and further, if necessary, a flash (not shown) to capture an image.

  The analog image signal output from the image sensor 101 is stored in the buffer memory 103 as digital image data through the AFE circuit 150 described above. The signal processing circuit 140 processes the digital image data in accordance with various signal processing parameters set by the system controller 107, generates image data in YUV format, and stores it again in the buffer memory 103.

  The image data processed by the signal processing circuit 140 is encoded into, for example, a JPEG format file by the compression / decompression circuit 104 and recorded on the recording medium 106 by the recording device 105.

  Further, the display control circuit 108 generates a display image from the YUV format image data stored in the buffer memory 103 and causes the display unit 110 to display it as a quick review image through the D / A converter 109.

  FIG. 8 is a block diagram illustrating the flow of signal processing during imaging standby in the imaging apparatus of the present embodiment. As described above, at the time of imaging standby, the imaging apparatus of the present embodiment performs continuous imaging and display for causing the display unit 110 to function as an EVF.

  The analog image signal output from the image sensor 101 is converted into gain data (sensitivity adjustment) and digital data by the AFE circuit 150. Then, so-called development processing such as pixel interpolation processing and white balance correction is performed on the RAW format image data by the signal processing circuit 140 to generate YUV format digital image data.

  The digital image data is stored in a display area (commonly referred to as VRAM) 103 a of the buffer memory 103 and is output to the display unit 110 through the display control circuit 108 and the D / A converter 109.

  On the other hand, the digital image data generated by the signal processing circuit 140 is also stored in the image analysis area (image analysis buffer) 103 b of the buffer memory 103. The image data stored in the image analysis buffer 103 b is used for face detection by the face detection circuit 120 and histogram creation by the histogram creation circuit 130. Note that it is not necessary to store all EVF image data in the image analysis buffer 103b, and a part of EVF image data corresponding to the period of face detection and histogram creation is stored.

  The face detection circuit 120 performs face detection on the image data stored in the image analysis buffer 103b. When a face is detected, information (for example, position and size) that can specify the face area is detected. The included face detection result is output.

  The histogram creation circuit 130 creates a histogram based on the image data stored in the image analysis buffer 103 b and the face detection result from the face detection circuit 120. As described above, the histogram creation circuit 130 can create the face area histogram 130a for the face area and the divided area histogram 130b for each area image obtained by dividing the entire image.

  The divided region histogram 130b can be created for each region image including the face region for the entire image, or can be obtained for each region image excluding the face region. The former is easy to process, but the latter is preferable in order to accurately detect whether a whiteout region is a face region or a background region.

  As described above, while the display unit 110 is functioning as an EVF, imaging is continuously repeated, and the VRAM is rewritten in a short time. In general, the time required for face detection and histogram creation is longer than the EVF image display period (for example, 1/30 second). Therefore, in this embodiment, an image analysis buffer 103b is provided in addition to the VRAM, and the image analysis buffer 103b is updated until face detection and histogram creation for the image data stored in the image analysis buffer 103b are completed. do not do.

  With this configuration, face detection and histogram creation can be performed on the same EVF image, and image analysis can be performed easily and with high accuracy. Of course, it may be performed if face detection and histogram creation are possible for each frame of the EVF image. However, since it is generally unlikely that the captured scene will change significantly from frame to frame, It is not necessary to execute.

FIG. 9 is a flowchart for explaining the determination operation of the dynamic range expansion amount (D range expansion amount) in the imaging apparatus of the present embodiment.
In this embodiment, the amount of whiteout in the captured image is obtained based on the face detection result and the histogram creation result for the EVF image, and the D range expansion amount is determined according to the amount of whiteout. Then, the exposure and sensitivity at the time of actual imaging are adjusted using the D range expansion amount that is determined in advance using the EVF image.

  In step S <b> 201, the system controller 107 continuously captures images as described above, generates an EVF image, and sequentially stores the EVF image from the signal processing circuit 140 in the display area 103 a of the buffer memory 103. The EVF image is also stored in the image analysis area 103 b of the buffer memory 103 at a predetermined cycle.

In S202, the face detection circuit 120 performs face detection on the EVF image stored in the image analysis buffer 103b.
When the face detection is successful (S203, Y), the histogram creation circuit 130 creates a face area histogram from the face area of the EVF image based on the face detection result from the face detection circuit 120 (S204).
In step S <b> 205, the system controller 107 serving as a whiteout amount calculation unit calculates a whiteout amount of the face area from the histogram of the face area.

  When face detection fails (S203, N), and after calculating the amount of whiteout in the face area, in S206, the histogram creation circuit 130 creates a divided area histogram for each area image obtained by dividing the entire EVF image.

In step S207, the system controller 107 calculates the amount of whiteout in the entire EVF image from the divided region histogram.
In S208, the system controller 107 as the D range expansion amount determination means determines the D range expansion amount based on at least the amount of whiteout in the entire image obtained in S207.

Next, a specific example of the whiteout amount calculation process performed by the system controller 107 in S205 and S207 of FIG. 9 will be described.
First, the calculation process of the amount of whiteout in the face area will be described.
In step S205, the system controller 107 calculates a luminance value YHiFace from which the cumulative frequency of the cumulative histogram becomes a predetermined value (90% in the example of FIG. 7) as the amount of whiteout in the face region from the face region histogram created in S204. To do.

  In step S <b> 208, the system controller 107 determines the D range expansion amount (D + (face)) of the face area according to the magnitude relationship between the value of the whiteout amount YHiFace of the face area and a predetermined threshold value. To do.

Specifically, for example, when the predetermined threshold is set in three stages of THHiFace, THMidFace, and THLowFace in descending order,
D + (face) = 1 correction level (when YHiFace> THHiFace)
D + (face) = correction level 2/3 stage (when THHiFace ≧ YHiFace> THMidFace)
D + (face) = correction level 1/3 step (when THMidFace ≧ YHiFace> THLowFace)
D + (face) = correction level 0 stage (when THLowFace ≧ YHiFace)
And decide.

  In S206, the system controller 107 determines the luminance value at which the cumulative frequency of the cumulative histogram is a predetermined value (80% in the example of FIG. 7) as the amount of whiteout in the partial region for each of the divided region histograms created in S206. YHi_n (n is 1 to the number of divisions, 16 in the example of FIG. 7) is calculated.

  In S207, the system controller 107 counts the number YH_BNum of regions where the luminance value YHi_n exceeds a predetermined threshold Y_B_Th. Then, the system controller 107 determines the D range expansion amount (D + (background)) of the entire image according to the magnitude relationship between the number of regions YH_BNum and a predetermined threshold value.

Specifically, for example, assuming that the predetermined threshold value is ThYH_BNumHi, ThYH_BNumMid, ThYH_BNNumlow in order from the largest value,
D + (background) = 1 correction level (when YH_BNum> ThYH_BNumHi)
D + (background) = correction level 2/3 stage (when ThYH_BNumHi ≧ YH_BNum> ThYH_BnumMid)
D + (background) = correction level 1/3 stage (when ThYH_BnumMid ≧ YH_BNum> ThYH_BNNumlow)
D + (background) = 0 correction level (when ThYH_BNNumLo ≧ YH_BNum)
And decide.

That is, the system controller 107 determines a larger D range expansion amount as the area of the whiteout region in the image is larger.
Note that the method for determining the whiteout region is not limited to the method using the cumulative histogram described here, and any other method can be used.

  In step S208, the system controller 107 determines a final D range expansion amount. If the face detection is successful, the system controller 107 as the D range expansion amount determination means compares the D range expansion amounts determined in S205 and S207 to determine the final D range expansion amount. . For example, the system controller 107 can determine the larger D range expansion amount as the final D range expansion amount among the D range expansion amount of the face area and the D range expansion amount of the entire image.

  Alternatively, one of the D range expansion amount of the face area and the D range expansion amount of the entire image is determined as the final D range expansion amount according to the imaging mode set by the mode dial included in the operation unit 112. Also good. For example, in the case of a person imaging mode (for example, portrait mode), the D range expansion amount of the face area is final. In the case of a landscape imaging mode (for example, landscape mode), the D range expansion amount of the entire image is final. It can be determined as a typical D range expansion amount.

  Further, not only the imaging mode preset with the mode dial or the like, but also the case where it is selected according to the imaging mode automatically selected according to the scene discrimination result can be determined similarly.

  In addition to selecting one of the D range expansion amount of the face area and the D range expansion amount of the entire image and determining it as the final D range expansion amount, the D range expansion amount of the face area and the D range of the entire image are also determined. The final D range expansion amount can also be determined according to the size of the expansion amount.

FIG. 10 is a diagram illustrating a specific example of the D range expansion amount that is finally determined according to the D range expansion amount of the face area and the D range expansion amount of the entire image.
In the example shown in FIG. 10, the value of the D range expansion amount finally determined is changed for each imaging mode. For example, when the D range expansion amount of the face area is 1/3 step and the D range expansion amount of the entire image is 0/3 step, the final D range expansion amount is the Auto mode (FIG. 10A) and the port. In the rate mode (FIG. 10C), 1/3 step, and in the landscape mode (FIG. 10B), 0/3 step.

  The system controller 107 stores the D range expansion amount determined as described above in the buffer memory 103 and refers to it during the main imaging. The determination operation of the D range expansion amount can be performed at the time of standby, for example, every fixed frame or every fixed time of the EVF image, and the buffer memory 103 stores the latest D range expansion amount.

FIG. 11 is a diagram illustrating a conceptual diagram of the D range in the present embodiment.
In the present embodiment, the D range is defined as the ratio of the saturation signal amount luminance of the image sensor to the appropriate luminance. The appropriate luminance is a luminance target value level when performing automatic exposure control (AE). For example, if the AE mode is the average photometry mode, it corresponds to the average value of the screen luminance.

Therefore,
D range = sensor saturation signal amount luminance / AE target value.
Here, the AE target value is an AE target value based on the output signal of the image sensor 101 before sensitivity adjustment is performed by the AFE circuit 150.

The AE target value may change according to the AE mode, and the AE target value in each mode can be used even in the evaluation photometry mode or the spot photometry mode.
FIG. 12 is a diagram illustrating a relationship example between the AE target value, the saturation signal value, and the D range.
FIG. 12 shows that the D range amount can be increased by lowering the AE target value.

FIG. 13 is a diagram illustrating a setting example of gamma correction characteristics in the signal processing circuit 140 of the present embodiment.
An example of setting the gamma characteristic (brightness correction amount) when the D range expansion amount is set to four levels of normal (0/3 step), +1/3 step, +2/3 step, and +3/3 step. ing.

  Here, the AE target value corresponding to each D range expansion amount is the same as that shown in FIG. As shown in FIG. 13, the AE target value after performing gamma correction on the AE target value at the time of each D range expansion becomes a normal AE target value that does not perform the D range expansion regardless of the D range expansion amount. Set the gamma correction characteristics to.

  As described with reference to FIGS. 11 and 12, the D range can be expanded by lowering the AE target value. However, simply lowering the AE target value results in underexposure and the captured image becomes dark. Therefore, the D range is expanded while the brightness (exposure) of the captured image is appropriately adjusted by performing gamma correction by the signal processing circuit 140 so that the image data after imaging is brightened according to the D range expansion amount. Can do. Therefore, the signal processing circuit 140 operates as correction means.

  In the present embodiment, the configuration in which the decrease in luminance of the captured image due to the decrease in the AE target value is compensated by gamma correction is described, but the same luminance correction is performed using another means such as a lookup table. Also good.

  Further, a gain such as a gain of a white balance coefficient for correcting white balance and a clipping amount for determining a saturation signal amount may be controlled. That is, after the A / D conversion is performed on the image signal whose gain has been reduced by reducing the exposure amount or the AFE gain, the gain is increased by the signal processing circuit in the subsequent stage, and the clipping amount is increased by the gain increase (saturation signal amount The same effect can be obtained.

FIG. 14 is a flowchart for explaining the operation of the main imaging process when the D range is expanded in the imaging apparatus of the present embodiment.
It is assumed that the D-range expansion amount is determined, for example, at a constant period from the EVF image by the above-described method during imaging standby. In this embodiment, it is assumed that the D range expansion amount (AE target value reduction amount) can be determined in four steps from 0/3 step to 3/3 step in units of 1/3 step. The range of the D range expansion amount and the size per stage can be set arbitrarily.

  Then, at the time of imaging standby, in response to the shutter button included in the operation unit 112 being fully pressed and an imaging start instruction being input, the system controller 107 as imaging control means starts the following processing.

In step S <b> 401, the system controller 107 acquires from the buffer memory 103 the D range expansion amount determined immediately before the imaging start instruction is input.
In S402, the system controller 107 determines whether or not the acquired D range expansion amount, that is, the reduction amount of the AE target value can be realized by sensitivity adjustment (control of the CDS gain circuit and the VGA gain circuit) in the AFE circuit 150. . This determination can be made by comparing the sensitivity range adjustable by the AFE circuit 150 with the D range expansion amount acquired in S401. If the sensitivity corresponding to the D range expansion amount cannot be reduced (gain reduction), the system controller 107 determines that the D range expansion amount cannot be realized only by sensitivity adjustment in the AFE circuit 150.

  When the D range expansion amount can be realized by sensitivity adjustment in the AFE circuit 150, the system controller 107 as the correction amount determination unit calculates the gain setting as an imaging condition in S404. Note that there is no particular limitation on how the CDS gain and the VGA gain are set in combination, and any combination can be set.

On the other hand, when it is determined that the D range expansion amount cannot be realized only by the sensitivity adjustment in the AFE circuit 150, the system controller 107 performs exposure based on the gain amount that is still insufficient even if the AFE circuit 150 performs the gain control that is possible. The condition is changed (S405). Specifically, the system controller 107 calculates an exposure correction amount for realizing an insufficient gain amount.
The exposure correction here is a minus correction, and can be realized by a general method such as reducing the aperture, increasing the shutter speed, or inserting a neutral density filter (ND filter).

  In step S406, the system controller 107 determines whether the exposure correction calculated in step S405 is possible. For example, in an imaging apparatus that does not have an ND filter, the exposure cannot be negatively corrected when the shutter speed is already set to the maximum speed and the aperture is set to the minimum (the aperture value is maximum) by automatic exposure control. Also, when the maximum shutter speed that can be tuned is set during flash imaging, the shutter speed cannot be increased. The same applies to the case where the upper limit of the shutter speed is determined. In the aperture priority AE mode, it is preferable that the aperture value set by the user is not changed. Therefore, if the shutter speed is already the highest speed, it may be determined that minus correction cannot be performed. The same applies to the shutter speed priority AE mode.

  If it is determined that the negative exposure correction corresponding to the insufficient gain adjustment cannot be performed, the system controller 107 corrects the D range expansion amount to the maximum value that can be realized by the sensitivity adjustment and the exposure correction in S407. . Then, the system controller 107 calculates a gain amount set in the AFE circuit 150 and an exposure correction amount as necessary.

  In step S <b> 408, the system controller 107 sets a gain amount as an imaging condition in the CDS gain circuit 152 and the VGA gain circuit 154 of the AFE circuit 150. When performing exposure correction, the exposure parameters (shutter speed, aperture value, use / non-use filter setting, etc.) corresponding to the AE result are changed according to the exposure correction amount, and the lens apparatus is also used as the imaging condition. Set to 200.

In step S409, the system controller 107 sets a gamma parameter corresponding to the D range expansion amount in the signal processing circuit 140.
In step S410, the system controller 107 performs still image shooting (main exposure).

Here, the relationship between sensitivity setting (gain setting of the AFE circuit 150) and saturation of the image sensor 101 will be described.
Generally, the gain for the output signal of the image sensor is controlled so that the output satisfies a predetermined relationship with respect to the light amount and the exposure value (input) of the camera.

  However, there is an upper limit on the amount of charge that can be accumulated in the photodiode of the image sensor. Therefore, if the gain applied by the AFE circuit 150 is reduced for the purpose of reducing the imaging sensitivity of the image sensor, the maximum signal amount after gain application is also reduced. Therefore, the saturation signal amount decreases as the gain decreases.

  As described above, in the direction of increasing the sensitivity of the image sensor, it is possible to set a desired sensitivity if the amplification of noise is ignored, whereas in the direction of decreasing the sensitivity, the limit caused by the saturation signal amount. Value exists.

  In S408 of FIG. 14, when the sensitivity cannot be lowered, the minimum sensitivity that can be set by the imaging apparatus is often set. This means that the gain for the output signal of the image sensor has already been reduced to a value corresponding to the minimum sensitivity. Therefore, the sensitivity cannot be further reduced by controlling the gain of the AFE circuit 150. Therefore, in this embodiment, when the target AE target value (sensitivity) reduction amount cannot be realized even if the gain of the AFE circuit 150 is controlled, further sensitivity reduction is realized by exposure correction.

  The operation of performing imaging with reduced sensitivity by exposure correction, and correcting the brightness by performing gamma correction on the obtained dark image by the signal processing circuit 140 is the same as increasing the sensitivity after all. Noise is also amplified during correction, resulting in degradation of image quality.

  However, in the present embodiment, when the imaging sensitivity reduction corresponding to the D range expansion amount can be realized by the gain control of the AFE circuit 150, the imaging sensitivity reduction by the gain control is performed with priority. Even when the gain control alone cannot achieve a reduction in imaging sensitivity corresponding to the D range expansion amount, the gain is reduced to the minimum, and the insufficient sensitivity reduction is compensated by exposure correction. In this case, even if noise is amplified during gamma correction, the noise itself is not large because the gain of the AFE circuit 150 has already been reduced to a level corresponding to the lowest sensitivity. For this reason, deterioration in image quality can be minimized.

(Other embodiments)
In the above-described embodiment, the case where the D range expansion according to the present invention is applied to still image capturing has been described. However, the present invention can be similarly applied to EVF image capturing or moving image capturing. In this case, the timing for setting these parameters is adjusted so that the gain control of the AFE circuit 150 (and exposure correction as necessary) and the gamma correction in the signal processing circuit 140 are applied to the same image.

The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.).
Therefore, in order to realize the above-described embodiment by a computer, a program itself supplied to the computer also realizes the present invention. That is, the program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A program for realizing the above-described embodiment is supplied to a computer by a computer-readable recording medium or wired / wireless communication. Examples of computer-readable recording media for supplying programs include magnetic recording media such as flexible disks, hard disks, and magnetic tapes, optical / magneto-optical recording media such as MO, CD, and DVD, and non-volatile semiconductor memories. There is.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is recorded in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a recording medium in which a computer program for realizing the above-described embodiment is encrypted and distributed is distributed, key information for decrypting is supplied to a user who satisfies a predetermined condition, and Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.
Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

Claims (7)

A whiteout amount calculating means for calculating a whiteout amount of the image from an image captured using the imaging element;
D range expansion amount determining means for determining a dynamic range expansion amount for reducing the whiteout amount based on the size of the whiteout amount;
A correction amount determination means for determining a reduction amount of the imaging sensitivity according to the dynamic range expansion amount and a correction amount for correcting the brightness of an image captured with the reduced imaging sensitivity;
Setting means for setting an imaging condition for realizing a reduction amount of the imaging sensitivity, and when the reduction amount can be realized by setting the imaging sensitivity, the imaging sensitivity for realizing the reduction amount is set as the imaging condition. If the reduction amount cannot be realized by setting the imaging sensitivity, the imaging sensitivity reduced as much as possible and the exposure condition that realizes the reduction amount by a combination of the imaging sensitivity are used as the imaging condition. Setting means for setting;
An imaging control unit that performs imaging according to the imaging condition set by the setting unit using the imaging element;
An image pickup apparatus comprising: correction means for correcting the brightness of an image picked up by the image pickup control means according to the correction amount.   The setting means sets an imaging sensitivity and an exposure condition corresponding to the maximum realizable reduction amount as the imaging condition when it is determined that the reduction amount cannot be realized even by a combination of imaging sensitivity setting and exposure condition. The imaging apparatus according to claim 1, wherein the correction amount is corrected to a value corresponding to the maximum realizable reduction amount. It further has face detection means for detecting a face area from the captured image,
The whiteout amount calculating means obtains the whiteout amount for each of the face area and the entire image,
The determining means determines the dynamic range expansion amount based on the dynamic range expansion amount for each of the face region and the entire image.
The imaging apparatus according to claim 1 or 2, wherein   In the imaging mode for capturing a person, the determining unit determines the dynamic range expansion amount by giving priority to the dynamic range expansion amount for the face area over the dynamic range expansion amount for the entire image. The imaging device according to claim 3. The calculation of the whiteout amount by the whiteout amount calculation unit, the determination of the dynamic range expansion amount by the determination unit, and the determination of the imaging sensitivity and correction amount by the correction amount determination unit are performed at the time of standby for still image shooting. To the EVF image
5. The imaging control unit according to claim 1, wherein the imaging control unit performs the imaging using the imaging sensitivity and the correction amount determined immediately before in response to an imaging start instruction. The imaging device described. A whiteout amount calculating step for calculating a whiteout amount of the image from an image captured using the imaging element;
A D range expansion amount determination step for determining a dynamic range expansion amount for reducing the whiteout amount based on the size of the whiteout amount;
A correction amount determining step for determining a reduction amount of the imaging sensitivity according to the dynamic range expansion amount and a correction amount for correcting the brightness of an image captured with the reduced imaging sensitivity;
A setting step for setting an imaging condition for realizing the reduction amount of the imaging sensitivity, and when the reduction amount can be realized by setting the imaging sensitivity, the imaging sensitivity for realizing the reduction amount is set as the imaging condition; If the reduction amount cannot be realized by setting the imaging sensitivity, the imaging sensitivity reduced as much as possible and the exposure condition that realizes the reduction amount by a combination of the imaging sensitivity are used as the imaging condition. A setting step to set;
An imaging control step of performing imaging according to the imaging conditions set in the setting step using the imaging element;
A control method for an imaging apparatus, comprising: a correction step of correcting the brightness of the image captured in the imaging control step according to the correction amount.   The program for making the computer of the imaging device which has an imaging device perform each step of the control method of the imaging device of Claim 6.