JP2014110598A - Image pickup device, and control method and control program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suitably maintain an evaluation value such as an exposure condition when generating a dynamic range-expanded image by synthesizing a plurality of images differing in exposure condition, even in cases where the image has high noise.SOLUTION: When obtaining a dynamic range-expanded image HDR having had its dynamic range expanded by synthesizing a plurality of photographed images Fr(n) and Fr(n+1) differing in exposure condition from each other, an addition-processed image {Fr(n-2)+Fr(n)} is obtained as an evaluation value calculation image by processing a plurality of photographed images photographed under a high exposure condition which is specific one of exposure conditions by adding up and averaging. Then, a control-use evaluation value including the exposure condition and used when controlling an image pickup device is obtained according to the evaluation value calculation image.

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a control program, and more particularly, to an imaging apparatus capable of expanding a dynamic range when capturing a moving image.

  In recent years, many imaging devices such as digital still cameras or digital video cameras using solid-state imaging devices such as CCD or CMOS image sensors have been developed.

  In these imaging apparatuses, the dynamic range of a captured image usually depends on the dynamic range of the solid-state imaging device. For this reason, under shooting conditions such as backlight where the main subject and the background have a large difference in brightness, so-called overexposure or underexposure occurs in the background even if the main subject is adjusted to an appropriate exposure.

  In order to cope with such a problem, a method for expanding a dynamic range by combining a plurality of images is known (see Patent Document 1). In this method, image processing is performed by using the portion where the contrast of the subject remains from a plurality of captured images with different exposure conditions for each screen area, and the dynamic range in the captured image is substantially expanded. Yes.

  Conventionally, dynamic range expansion as described above has been mainly performed on still images. However, dynamic range expansion processing is performed on moving images as the readout speed of the image sensor increases. ing.

  FIG. 7 is a timing chart for explaining the driving of the image sensor when acquiring a plurality of images with different exposure conditions when the dynamic range is expanded in a moving image.

  In FIG. 7, the output synchronization signal is a synchronization signal indicating the timing of outputting a composite image after combining a plurality of images, and the cycle of the output synchronization signal corresponds to the frame rate when the composite image is output. The shooting synchronization signal is a synchronization signal that indicates the timing of shooting one frame of the shot image, and this period corresponds to the frame rate at the time of shooting the image. In the illustrated example, the cycle of the imaging synchronization signal is twice that of the output synchronization signal.

  In the following description, it is assumed that two captured images are combined into one composite image, and in the two captured images, the first frame is referred to as a 1st frame and the second frame is referred to as a 2nd frame. The high exposure image is High, the low exposure image is Low, and the composite image (post-composition output image) is HDR. Each photographed image is indicated by Fr (n) with the number n of frames from the start of photographing as an argument. Also, various exposure evaluation value calculation images for calculating exposure conditions are indicated by Fr (n).

  Now, at the timing of time T1, an exposure condition calculation circuit (not shown) sets the appropriate exposure condition of the subject calculated according to the various exposure evaluation value calculation images Fr (n−2) that are high-exposure photographed images. Based on the appropriate exposure condition, a high-exposure condition of plus one step (plus one-step high exposure condition) is calculated. An imaging device (not shown) takes a high-exposure image (High) under the plus one-step high exposure condition. Here, when calculating the appropriate exposure condition, for example, a luminance value for each region of the image is used.

  The captured image Fr (n) is input to the exposure condition calculation circuit as various exposure evaluation value calculation images Fr (n) and is used for calculating the exposure condition at the timing of time T3.

  At the timing of time T2, the exposure condition calculation circuit calculates a minus one step low exposure condition (minus one step low exposure condition) than the above-described proper exposure condition. In the imaging apparatus, a low-exposure image (Low) is captured under the minus one-stage low-exposure condition to obtain a captured image Fr (n + 1).

  Subsequently, at the timing of time T3, the imaging apparatus displays an expanded dynamic range image generated from the high-exposure image Fr (n) captured two frames ago and the low-exposure image Fr (n + 1) captured one frame ago. Output as a moving image. At this time, similarly to the time T1, the exposure condition calculation circuit is one step higher than the appropriate exposure condition based on the appropriate exposure condition of the subject calculated from the various exposure evaluation value calculation images Fr (n). Calculate high exposure conditions. And in an imaging device, a high exposure image is image | photographed on this high exposure condition, and the captured image Fr (n + 2) is acquired.

  Similarly, the captured images Fr (n + 3) to Fr (n + 5) are obtained. In this way, an operation for switching between high-exposure and low-exposure image capturing is repeated at a speed twice the frame rate of the moving image, thereby generating a dynamic range expanded image.

Japanese Patent Laid-Open No. 63-306777

  However, if a high exposure image and a low exposure image for calculating the proper exposure condition are used as described above, the obtained image becomes high noise depending on the subject and the exposure condition. The accuracy of the evaluation value may be reduced.

  Therefore, an object of the present invention is to maintain good accuracy of evaluation values such as exposure conditions even when noise of an image is high when a dynamic range expanded image is generated by combining a plurality of images with different exposures. An object of the present invention is to provide an imaging device capable of performing the above.

  In order to achieve the above object, an image pickup apparatus according to the present invention is an image pickup apparatus including an image pickup unit that continuously takes an image as a shot image with different exposure conditions, and is obtained by exposing each other obtained by the image pickup unit. Dynamic range expansion means for combining a plurality of captured images with different conditions to obtain a dynamic range expanded image in which the dynamic range is expanded, and a plurality of captured images captured under one specific exposure condition among the exposure conditions An image addition unit that obtains an addition-processed image as an evaluation value calculation image by performing an averaging process, and a control evaluation value that is used when controlling the imaging device including the exposure condition, according to the evaluation value calculation image Evaluation value generation means to be obtained.

  A control method according to the present invention is a control method for an imaging apparatus including an imaging unit that continuously captures images as captured images with different exposure conditions, and a plurality of exposure conditions obtained by the imaging unit are different from each other. A dynamic range expansion step for synthesizing captured images to obtain a dynamic range expanded image in which the dynamic range is expanded, and a plurality of captured images captured under one specific exposure condition among the exposure conditions are added and averaged An image addition step for obtaining an addition-processed image as an evaluation value calculation image, and an evaluation value generation step for obtaining a control evaluation value used in the control of the imaging apparatus including the exposure condition according to the evaluation value calculation image It is characterized by having.

  A control program according to the present invention is a control program used in an imaging apparatus including an imaging unit that continuously captures images as captured images with different exposure conditions, and the computer includes the imaging unit. A dynamic range expansion step for obtaining a dynamic range expanded image obtained by combining a plurality of obtained captured images having different exposure conditions to expand a dynamic range, and was captured under a specific one of the exposure conditions An image addition step of adding and averaging a plurality of photographed images to obtain an addition processed image as an evaluation value calculation image, and calculating an evaluation value for control used in controlling the imaging apparatus including the exposure condition And an evaluation value generation step obtained according to the image for use.

  According to the present invention, when a dynamic range enlarged image is generated by combining a plurality of images with different exposures, the accuracy of evaluation values such as exposure conditions can be maintained well even if the image noise is high.

1 is a block diagram illustrating a configuration of an example of an imaging apparatus according to a first embodiment of the present invention. 3 is a timing chart for explaining driving of an image sensor when acquiring a plurality of images with different exposure conditions in the image pickup apparatus shown in FIG. 1. 4 is a flowchart for explaining a photographing process in the imaging apparatus shown in FIG. 1. It is a flowchart for demonstrating the imaging | photography process in the imaging device by the 2nd Embodiment of this invention. 10 is a timing chart for explaining driving of an image sensor when acquiring a plurality of images with different exposure conditions in an imaging apparatus according to a third embodiment of the present invention. It is a flowchart for demonstrating the imaging | photography process in the imaging device by the 3rd Embodiment of this invention. 6 is a timing chart for explaining driving of an image sensor when acquiring a plurality of images having different exposure conditions when expanding a dynamic range in a moving image.

  Hereinafter, an example of an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing the configuration of an example of an imaging apparatus according to the first embodiment of the present invention.

  The illustrated imaging apparatus is, for example, a digital video camera (hereinafter simply referred to as a camera), and includes a photographing lens unit (hereinafter simply referred to as a photographing lens) 110. The photographing lens 110 includes a plurality of lenses such as a focus lens and a zoom lens, and includes a diaphragm mechanism. The lens control unit 111 controls focus, zoom, and aperture in the photographing lens 110 under the control of the system control circuit 180.

  The optical image (subject image) that has passed through the photographing lens 110 is formed on the image sensor 120. The illustrated image sensor 120 is a CMOS image sensor, for example, and generates an electrical signal (analog signal) corresponding to an optical image by photoelectric conversion. The image sensor 120 not only reads pixel signals from all pixels, but also adds pixel signals of specific pixels. In addition, pixel signals can be read by thinning out specific rows or columns. .

  The image sensor drive circuit 121 drives the image sensor 120 under the control of the system control circuit 180. The image sensor drive circuit 121 changes the exposure time and signal amplification amount of the image sensor 120 during photographing. The analog signal is converted into a digital signal (image data) by A / D conversion, and then input to an image processing circuit 130 including a dynamic range expansion circuit 200.

  As described later, the dynamic range expansion circuit 200 uses image data (hereinafter referred to as a captured image) obtained as a result of shooting using high exposure image data and low exposure image data stored in the memory unit 140 in advance. A dynamic range enlarged image is generated. Here, the dynamic range expanded image indicates an image in which the dynamic range is expanded as compared with each exposure image before composition. Also, in this embodiment, the dynamic range is expanded when the composition is performed so that the gradation of the subject with a predetermined brightness (for example, a bright part and a dark part) is obtained as compared with each exposure image before the composition. Means. Therefore, as a result of the synthesis, for example, a case where a 16-bit synthesized image is generated by synthesizing 8-bit exposed images is included, and a synthesized image is also synthesized so that it becomes 8 bits. .

  As illustrated, the image processing circuit 130 includes an exposure condition calculation circuit 210, a white balance coefficient calculation circuit 220, a face detection circuit 230, a scene determination circuit 240, and an image addition circuit 300.

  The exposure condition calculation circuit 210 sets the appropriate exposure as the appropriate exposure based on the photographed image and sends the appropriate exposure to the system control circuit 180. The white balance coefficient calculation circuit 220 calculates a white balance (WB) coefficient used when performing image processing according to the photographed image, and sends the WB coefficient to the system control circuit 180.

  The face detection circuit 230 discriminates the face area of the subject in the captured image and sends the face area discrimination result to the system control circuit 180. This face area discrimination result is used when the face area in the live image displayed on the LCD 150 is displayed as a frame. Furthermore, the face area discrimination result is used when performing exposure control, focus control, and WB processing with priority on the face area. That is, the face area discrimination result is used when assisting the user's shooting.

  In the illustrated example, the image processing circuit 130 includes only the face detection circuit 230. However, for example, a detection circuit such as person detection or pet detection may be included.

  The scene discrimination circuit 240 discriminates the scene closest to the shot image from the scenes stored in advance in the memory unit 140 according to the composition, brightness, and color of the shot image. Then, the scene discrimination circuit 240 sends the scene discrimination result to the system control circuit 180. The scene determination result is used when assisting the user's shooting when performing exposure conditions, focus control, and WB processing according to the scene.

  The image addition circuit 300 adds or averages pixels in a plurality of captured images to generate an added image or an averaged image in which noise is reduced compared to a reference captured image.

  The image processing circuit 130 performs various signal processing such as gamma processing and WB processing on the captured image. At this time, the image processing circuit 130 writes / reads the captured image to / from the memory 140. Further, the image processing circuit 130 displays the captured image on the LCD 150.

  The captured image that is the output of the image processing circuit 130 is compressed by the image conversion circuit 160 and written to the memory card 170 as a compressed captured image. Further, the image conversion circuit 160 decompresses the compressed captured image read from the memory card 170 and outputs the decompressed image to the image processing circuit 130.

  The system control circuit 180 receives the output of the image processing circuit 130 and performs TTL (through-the-lens) type autofocus (AF) processing, automatic exposure (AE) processing, flash pre-flash (EF) processing, and the like. Do. The system control circuit 180 controls the lens control unit 111 and the image sensor driving circuit 121 during AF processing and AE processing.

  The operation unit 190 connected to the system control circuit 180 has, for example, a release button and a mode switching dial, and the user operates the operation unit 190 at the time of shooting to cause the system control circuit 180 to perform various shootings. Give instructions.

  FIG. 2 is a timing chart for explaining the driving of the image sensor when acquiring a plurality of images with different exposure conditions in the imaging apparatus shown in FIG.

  In FIG. 2, the output synchronization signal, shooting synchronization signal, HDR, and Fr (n) are the same as in FIG. 7, and various evaluation value calculation images Eva are used for exposure processing, WB processing, face detection, scene discrimination, and the like. Used.

  FIG. 3 is a flowchart for explaining a photographing process in the imaging apparatus shown in FIG.

  Referring to FIGS. 2 and 3, now, at time T1, the system control circuit 180 determines that the proper exposure condition (that is, the proper exposure value) obtained from the exposure condition calculation circuit 210 is greater than the proper exposure condition. A one-step higher (ie, plus one-step) high exposure condition (high exposure value) is obtained, and the high exposure condition is set in the lens control unit 111 and the image sensor driving circuit 120 (step S201). Thus, the first frame Fr (n) is shot under the high exposure condition, and the shot image is stored in the memory 140 (step S202).

  Subsequently, at time T2, the system control circuit 180 obtains a low exposure condition (low exposure value) that is one step lower (that is, minus one step) than the appropriate exposure condition, and sets the low exposure condition to the lens control unit 111 and the low exposure condition. The image sensor driving circuit 120 is set (step S203). As a result, the second frame Fr (n + 1) is shot under the low exposure condition, and the shot image is stored in the memory 140 (step S204).

  Subsequently, under the control of the system control circuit 180, the image addition circuit 300 captures the high-exposure photographed image Fr (n) from the memory 140 in the same manner two frames before the high-exposure photographed image Fr (n). The high-exposure photographed image Fr (n−2) thus read is read, and an averaging process is performed to generate an adding average image (also referred to as an adding process image) (step S205). Then, the image addition circuit 300 stores the addition average image in the memory unit 140 as various evaluation value calculation images Eva.

  Next, under the control of the system control circuit 180, the image processing circuit 130 reads various evaluation value calculation images Eva from the memory unit 140 and calculates various evaluation values (also referred to as control evaluation values) (step S206). For example, the exposure condition calculation circuit 210 calculates appropriate exposure conditions according to various evaluation value calculation images Eva. The white balance condition calculation circuit 220 calculates an optimal white balance coefficient for white balance processing according to various evaluation value calculation images Eva.

  The face detection circuit 230 detects a face area from various evaluation value calculation images Eva, and obtains face area information (area detection information) as a face detection result. The scene discrimination circuit 240 discriminates the scene closest to the various evaluation value calculation images Eva from the scenes stored in advance in the memory unit 140 to obtain scene discrimination information.

  Subsequently, at time T <b> 3, under the control of the system control circuit 180, the dynamic range expansion circuit 200 reads the high exposure image Fr (n) and the low exposure image Fr (n + 1) from the memory unit 140. Then, the dynamic range expansion circuit 200 combines the high exposure image Fr (n) and the low exposure image Fr (n + 1) to generate a one frame dynamic range expansion image HDR (step S207). Note that a composition processing method for expanding the dynamic range using a high-exposure image and a low-exposure image is described in Patent Document 1 and the like, and thus description thereof is omitted here.

  Subsequently, under the control of the system control circuit 180, the image processing circuit 130 performs each image processing on the dynamic enlarged image HDR of one frame, and then saves it as an output image in the memory card 170 as the image conversion circuit 160 and also stores it in the LCD 150. It is displayed (step S208).

  Then, the system control circuit 180 determines whether or not there is an instruction to end shooting (step S209). If there is no instruction to end shooting (NO in step S209), the system control circuit 180 returns to the process of step S201 and similarly performs the shooting process for the next frame. That is, the averaging process is performed on the high-exposure photographed image Fr (n + 2) and the high-exposure photographed image Fr (n) photographed in the same manner two frames before the high-exposure photographed image Fr (n + 2). Various evaluation value calculation images Eva are generated. Further, the high-exposure image Fr (n + 2) and the low-exposure image Fr (n + 3) are combined to generate a one-frame dynamic range expanded image HDR.

  On the other hand, when there is an instruction to end shooting (YES in step S209), system control circuit 180 ends the shooting process.

  In this way, the imaging apparatus performs the processing described in FIG. 3 for each frame to capture and record a moving image.

  As described above, in the first embodiment of the present invention, when various evaluation values such as calculation of appropriate exposure are calculated, an averaged image obtained by averaging two high-exposure images (for example, Fr (n−2)). ) + Fr (n)) is used as the various evaluation value calculation images Eva, so that this averaged image can improve the accuracy of various evaluation values as a result of suppressing noise.

  In the first embodiment described above, for the sake of convenience of explanation, the high exposure and low exposure conditions are changed from the appropriate exposure condition to plus one step and minus one step, respectively, but the high exposure and low exposure conditions are appropriately changed. The same effect can be obtained.

  Furthermore, in the first embodiment, for convenience of explanation, only the high-exposure image is used for calculation of various evaluation values. However, depending on the brightness of the subject, the low-exposure image is used for calculation of various evaluation values. May be.

  In the first embodiment of the present invention, the addition average image is generated using the two high-exposure images for convenience of explanation, but the addition average image is obtained using the plurality of high-exposure images. In this way, noise can be similarly suppressed.

  As described above, in the first embodiment of the present invention, when generating a dynamic range expanded image by combining a plurality of captured images with different exposures, the influence of noise on the captured images is reduced, and exposure and WB are reduced. The accuracy of evaluation values used for processing and the like can be maintained satisfactorily.

[Second Embodiment]
Next, an example of an imaging apparatus according to the second embodiment of the present invention will be described. Note that the configuration of the imaging apparatus according to the second embodiment is the same as that of the imaging apparatus shown in FIG.

  In the second embodiment, when the subject is darker than the preset brightness, addition is performed by averaging a plurality of high-exposure images as various evaluation value calculation images as in the first embodiment. An average image is used (see FIG. 2). On the other hand, when the subject is brighter than a preset brightness, one high-exposure image is used as various evaluation value calculation images (see FIG. 7).

  FIG. 4 is a flowchart for explaining shooting processing in the imaging apparatus according to the second embodiment of the present invention. In FIG. 4, the same steps as those shown in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

  2, 4, and 7, after the process of step S <b> 204, system control circuit 180 obtains gain increase amount G according to the high exposure condition and the appropriate exposure condition calculated in step S <b> 501. . Then, the system control circuit 180 compares the gain increase amount (gain increase amount) G with a predetermined threshold value Th, and determines whether or not the gain increase amount G is larger than the threshold value Th (step S505). When the gain increase amount G is equal to or less than the threshold Th (less than the threshold) (NO in step S505), the system control circuit 180 controls the image processing circuit 130, and the high-exposure photographed image Fr (n) obtained in step S202. ) As various evaluation value calculation images Eva in the memory 140 (step S506).

  That is, in step S505, since the high-exposure photographed image Fr (n) is determined to be low noise whose noise is lower than a predetermined value, it is not necessary to suppress noise by addition averaging by the image addition circuit 300.

  On the other hand, when gain increase amount G is larger than threshold value Th (YES in step S505), system control circuit 180 controls image processing circuit 130 and performs the process of step S205 described in FIG. That is, in step S505, the high-exposure photographed image Fr (n) is determined to be high noise whose noise is equal to or higher than a predetermined value. Therefore, addition averaging processing by the image addition circuit 300 is performed, and various evaluation values are obtained. The calculation image Eva is obtained.

  As described above, in the second embodiment of the present invention, as a result of selecting the various evaluation value calculation images Eva according to the gain increase amount, not only can the accuracy of the various evaluation values be improved. When the gain increase is small and it is not necessary to suppress the noise, the averaging is not performed, so that it is possible to avoid the possibility that the accuracy of various evaluation values is reduced due to the subject synthesis error that occurs during the addition process. it can. In addition, an increase in processing load and processing time and an increase in power consumption due to the processing of the image addition circuit can be suppressed.

  As described above, in the second embodiment of the present invention, when generating a dynamic range expanded image by combining a plurality of captured images having different exposures, the influence of noise of the captured image is reduced, and exposure and WB processing are performed. It is possible to maintain the accuracy of the evaluation value used for the above. Furthermore, the processing load, processing time, and power consumption can be reduced.

[Third Embodiment]
Subsequently, an example of an imaging apparatus according to the third embodiment of the present invention will be described. Note that the configuration of the imaging apparatus according to the third embodiment is the same as that of the imaging apparatus shown in FIG.

  In the third embodiment, as will be described later, a high-exposure image and a low-exposure image are taken continuously for a plurality of frames.

  FIG. 5 is a timing chart for explaining the driving of the image sensor when acquiring a plurality of images with different exposure conditions in the imaging apparatus according to the third embodiment of the present invention.

  In FIG. 5, the output synchronization signal, shooting synchronization signal, HDR, and Fr (n) are the same as in FIG. 7, and various evaluation value calculation images Eva are used for exposure processing, WB processing, face detection, scene discrimination, and the like. Used.

  FIG. 6 is a flowchart for explaining shooting processing in the imaging apparatus according to the third embodiment of the present invention.

  Referring to FIGS. 5 and 6, now, at time T1, system control circuit 180 has a low exposure condition that is minus one step lower than the proper exposure condition, depending on the proper exposure condition obtained from exposure condition calculation circuit 210. And the low exposure condition is set in the lens control unit 111 and the image sensor driving circuit 120 (step S601). Thus, the first frame Fr (n) is shot under the low exposure condition, and the shot image is stored in the memory 140 (step S602).

  Subsequently, at time T2, the system control circuit 180 obtains a high exposure condition that is one step higher than the proper exposure condition, and sets the high exposure condition in the lens control unit 111 and the image sensor driving circuit 120 (step S603). . Thus, the 2nd frame Fr (n + 1) is shot under the high exposure condition, and the shot image is stored in the memory 140 (step S604).

  Next, at time T <b> 3, under the control of the system control circuit 180, the dynamic range expansion circuit 200 reads the low exposure image Fr (n) and the high exposure image Fr (n + 1) from the memory unit 140. Then, the dynamic range expansion circuit 200 combines the low-exposure image Fr (n) and the high-exposure image Fr (n + 1) to generate a one-frame dynamic range expansion image HDR (step S605).

  Subsequently, under the control of the system control circuit 180, the image processing circuit 130 performs each image processing on the dynamic enlarged image HDR of one frame, and then saves it as an output image in the memory card 170 as the image conversion circuit 160 and also stores it in the LCD 150. It is displayed (step S606).

  Further, at time T3, the system control circuit 180 captures the first frame Fr (n + 2) under the above-described high exposure condition, and stores the captured image in the memory 140 (step S607). At time T4, under the control of the system control circuit 180, the image addition circuit 300 reads the high-exposure photographed image Fr (n + 1) and the high-exposure photographed image Fr (n + 2) one frame before from the memory 140, An addition average process is performed to generate an addition average image (step S608). The image addition circuit 300 stores the addition average image in the memory unit 140 as various evaluation value calculation images Eva.

  Next, under the control of the system control circuit 180, the image processing circuit 130 reads various evaluation value calculation images Eva from the memory unit 140 and calculates various evaluation values (step S609). Examples of the various evaluation values include an appropriate exposure condition, a white balance coefficient, face area information, and a scene discrimination result.

  Subsequently, at time T5, the system control circuit 180 obtains a minus one-stage low exposure condition from the proper exposure condition, and sets the low exposure condition in the lens control unit 111 and the image sensor driving circuit 120 (step S610). As a result, the first frame Fr (n + 3) is shot under low exposure conditions, and the shot image is stored in the memory 140 (step S611).

  Under the control of the system control circuit 180, the dynamic range expansion circuit 200 reads the high exposure image Fr (n + 2) and the low exposure image Fr (n + 3) from the memory unit 140. Then, the dynamic range expansion circuit 200 synthesizes the high-exposure image Fr (n + 2) and the low-exposure image Fr (n + 3) to generate a one-frame dynamic range expansion image HDR (step S612).

  Subsequently, under the control of the system control circuit 180, the image processing circuit 130 performs each image processing on the dynamic enlarged image HDR of one frame, and then saves it as an output image in the memory card 170 as the image conversion circuit 160 and also stores it in the LCD 150. It is displayed (step S613).

  Then, the system control circuit 180 determines whether or not there is an instruction to end shooting (step S614). If there is no instruction to end the shooting (NO in step S614), the system control circuit 180 returns to the process in step S601 and similarly performs the shooting process for the next frame. On the other hand, when there is an instruction to end shooting (YES in step S614), system control circuit 180 ends the shooting process.

  In this way, the imaging apparatus performs the processing described with reference to FIG. 6 for each frame to capture and record a moving image.

  As described above, in the third embodiment of the present invention, each of a high-exposure image and a low-exposure image is continuously photographed over a plurality of frames, and a plurality of images with different exposures are combined to be dynamic. Generate a range-enlarged image. In addition, when obtaining various evaluation value calculation images, the high-exposure images taken continuously are subjected to addition averaging processing, so the difference in time at which the high-exposure images were taken, that is, the time difference is shortened. As a result, it is possible to suppress an error due to the composition of a moving subject when performing an averaging process on a high-exposure image.

  As described above, in the third embodiment of the present invention, when generating a dynamic range expanded image by combining a plurality of captured images with different exposures, the influence of noise on the captured images is reduced, and exposure and WB are reduced. The accuracy of evaluation values used for processing and the like can be maintained satisfactorily.

  As is clear from the above description, in the example shown in FIG. 1, the photographing lens 110, the image sensor 120, the image processing circuit 130, the lens control unit 111, and the image sensor driving circuit 121 function as an imaging unit. In addition, the system control circuit 180 and the dynamic range expansion circuit 200 function as dynamic range expansion means, and the image creation circuit 300 and the system control circuit 180 function as image addition means. The exposure condition calculation circuit 210, the white balance condition calculation circuit 220, the face detection circuit 230, the scene determination circuit 240, and the system control circuit 180 function as evaluation value generation means. The system control circuit 180 functions as a determination unit.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the imaging apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the imaging apparatus. The control program is recorded on a computer-readable recording medium, for example.

  Each of the above control method and control program has at least a dynamic range expansion step, an image addition step, and an evaluation value generation step.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

DESCRIPTION OF SYMBOLS 120 Image sensor 130 Image processing circuit 160 Image conversion circuit 180 System control circuit 200 Dynamic range expansion circuit 210 Exposure condition calculation circuit 220 White balance condition calculation circuit 230 Face detection circuit 240 Scene determination circuit 300 Image addition circuit

Claims (9)

An imaging apparatus including an imaging unit that continuously captures images as captured images with different exposure conditions,
Dynamic range expansion means for obtaining a dynamic range expanded image obtained by combining a plurality of captured images obtained by the imaging means and having different exposure conditions to expand a dynamic range;
Image addition means for performing an averaging process on a plurality of photographed images photographed under a specific one of the exposure conditions to obtain an addition processed image as an evaluation value calculation image;
An evaluation value generating means for obtaining an evaluation value for control used in the control of the imaging apparatus including the exposure condition according to the image for calculating the evaluation value;
An imaging device comprising: Determination means for determining whether or not the brightness of the subject in the captured image is brighter than a predetermined brightness;
When the determination unit determines that the brightness of the subject is brighter than a predetermined brightness, the evaluation value generation unit captures the image captured under the specific one exposure condition instead of the addition processing image. The imaging apparatus according to claim 1, wherein an image is used as the evaluation value calculation image.   The determination unit compares a captured image captured under the one specific exposure condition and a captured image acquired when the predetermined exposure condition is captured, and an increase in gain is equal to or less than a predetermined threshold. The imaging apparatus according to claim 2, wherein the imaging device determines that the subject is brighter than a predetermined brightness.   The imaging apparatus according to claim 1, wherein the imaging unit continuously captures the specific one captured image with a plurality of frames for capturing the captured image. As the exposure condition, there are a high exposure condition in which an exposure value is higher than a predetermined exposure value, and a low exposure condition in which an exposure value is lower than the predetermined exposure value,
The imaging apparatus according to claim 1, wherein the high exposure condition is used as the specific one exposure condition.   The imaging apparatus according to claim 5, wherein the imaging unit performs imaging under the high exposure condition and the low exposure condition in one frame in which the dynamic range expanded image is output.   In addition to the exposure condition, the control evaluation value includes at least a white balance coefficient for white balance processing, area detection information that is a result of detecting a predetermined area of the subject in the evaluation value calculation image, and the evaluation value calculation image. The imaging apparatus according to claim 1, further comprising scene discrimination information indicating a scene where the image is captured. A control method for an image pickup apparatus including an image pickup means for continuously shooting images as shot images with different exposure conditions,
A dynamic range expansion step for obtaining a dynamic range expanded image in which a dynamic range is expanded by combining a plurality of captured images having different exposure conditions obtained by the imaging means;
An image addition step of obtaining an addition processed image as an evaluation value calculation image by performing an averaging process on a plurality of photographed images photographed under a specific one of the exposure conditions,
An evaluation value generating step for obtaining a control evaluation value used in controlling the imaging apparatus including the exposure condition according to the evaluation value calculation image;
A control method characterized by comprising: A control program used in an imaging device including an imaging unit that continuously captures images as captured images with different exposure conditions,
In the computer provided in the imaging device,
A dynamic range expansion step for obtaining a dynamic range expanded image in which a dynamic range is expanded by combining a plurality of captured images having different exposure conditions obtained by the imaging means;
An image addition step of obtaining an addition processed image as an evaluation value calculation image by performing an averaging process on a plurality of photographed images photographed under a specific one of the exposure conditions,
An evaluation value generating step for obtaining a control evaluation value used in controlling the imaging apparatus including the exposure condition according to the evaluation value calculation image;
A control program characterized by causing

Images