JP2014027460A - Imaging apparatus, camera system, control method of imaging apparatus, program and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce unnaturalness of an image quality change when a normal imaging mode and a gradation expression range expansion imaging mode are switched.SOLUTION: An imaging apparatus includes: imaging means (303) which photoelectrically converts subject light via an imaging optical system and generates an image signal; exposure control means (109) which controls an exposure value of the imaging means; image synthesizing means (104, 105 and 106) which synthesize a plurality of image signals obtained by imaging by the imaging means with a plurality of different exposure values determined by the exposure control means, and generate a synthesized image signal whose gradation expression range is expanded; and imaging mode switching means (103) which changes over a normal imaging mode for performing normal moving image imaging and a gradation expression range expansion imaging mode for generating the synthesized image signal. When the imaging mode switching means changes over the imaging mode, the image synthesizing means stepwise changes gradation expression width of the synthesized image signal in accordance with the exposure value determined by the exposure control means for generating the synthesized image signal.

Description

  The present invention switches between a mode for outputting an image from standard exposure shooting (normal shooting mode) and a mode for outputting a composite image by combining processing from short exposure shooting and long exposure shooting (tone expression range extended shooting mode). The present invention relates to tone control of a composite image at the time.

  Japanese Patent Application Laid-Open No. 2004-133867 discloses a normal shooting mode for acquiring an image captured with standard exposure and a gradation expression range expansion for generating a gradation expression range expanded image with a wide dynamic range by combining images captured with different exposures. A technique for switching between shooting modes is disclosed.

JP 2002-218326 A

  However, in Patent Document 1, since special processing is not performed when switching from the normal shooting mode to the gradation expression range expansion shooting mode during moving image shooting, the gradation expression range suddenly changes between consecutive frames. There is a case where it is expanded and there is a sense of incongruity in the time-series image quality change.

  This case will be described with reference to FIG. FIG. 2A shows a state in which the person is blacked out because the image is shot in the normal shooting mode and gradation expression cannot be performed up to a dark part. Fig. 2 (b) is shot in the gradation expression range expansion shooting mode, the gradation expression range is expanded to the dark part, and the gradation expression of the human part that was crushed black in Fig. 2 (a) is reproduced. It shows the state of being completed.

  If the shooting mode is switched from the normal shooting mode in FIG. 2A to the gradation expression range extended shooting mode in FIG. 2B during moving image shooting, the gradation expression of the person changes suddenly, It will have a sense of incongruity in a series. Conversely, the same problem occurs when the shooting mode is switched from the gradation expression range extended shooting mode of FIG. 2B to the normal shooting mode of FIG.

  An object of the present invention is to reduce the unnaturalness of image quality change when switching between a normal shooting mode and a gradation expression range extended shooting mode.

  An imaging apparatus according to one aspect of the present invention includes an imaging unit that photoelectrically converts subject light via an imaging optical system to generate an image signal, an exposure control unit that controls an exposure value of the imaging unit, and the exposure control Image synthesizing means for synthesizing a plurality of image signals picked up by the image pickup means with different exposure values determined by the means to generate a combined image signal having an expanded gradation expression range; and normal moving image shooting A shooting mode switching unit that switches between a normal shooting mode for performing the above and a gradation expression range extended shooting mode for generating the composite image signal, and the image synthesis unit switches the shooting mode by the shooting mode switching unit. The gradation expression width of the composite image signal is stepwise according to an exposure value determined by the exposure control means to generate the composite image signal. And changes.

  When switching between the normal shooting mode and the gradation expression range extended shooting mode, it is possible to reduce the unnaturalness of the image quality change.

It is a block diagram which shows the structural example of the imaging device of Example 1 in this invention. It is a figure for demonstrating the image quality change which arises when switching from normal imaging | photography mode to gradation expression range expansion imaging | photography mode. 2 is a block diagram illustrating a configuration example of an imaging unit 101. FIG. 3 is a block diagram illustrating a configuration example of a signal synthesis unit 104. FIG. FIG. 5 is a diagram illustrating a description of a synthesis range control unit 105. FIG. 5 is a diagram illustrating the composition ratio control unit 106. It is a figure which shows description of an exposure value. It is a figure which shows the flowchart of the dynamic range expansion control of Example 1 in this invention. It is a block diagram which shows the structural example of the imaging device of Example 2 in this invention. It is a figure which shows the flowchart of the dynamic range expansion control of Example 2 in this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration example of the imaging apparatus according to the first embodiment of the present invention. In this embodiment, an imaging device (video camera or digital camera) capable of still image shooting and moving image shooting is used. Furthermore, the image pickup apparatus of the present embodiment can realize gradation expression range extended photographing with a wide dynamic range by combining images picked up with different exposure amounts. Hereinafter, a configuration example of the imaging apparatus according to the first embodiment will be described with reference to FIG. In this embodiment, a lens-integrated video camera will be described. However, the present invention can also be applied to a lens interchangeable single-lens reflex camera (camera system).

  In FIG. 1, reference numeral 101 denotes an imaging unit (imaging unit) that captures subject light based on an exposure control value (hereinafter also referred to as an exposure value) delivered from an exposure control unit 109 described later and outputs a captured image signal. As will be described later, the imaging unit 101 includes a photographing optical system (an imaging optical unit 301 in FIG. 3) such as a lens including an aperture and a mechanical shutter, and an imaging element that photoelectrically converts subject light (an imaging element unit 303 in FIG. 3). Etc. are included.

  Reference numeral 102 denotes a video signal that is subjected to predetermined image processing such as white balance processing, interpolation processing, shading correction processing, color space conversion processing, and gamma conversion processing on the captured image signal output from the imaging unit 101 and is converted into an output video signal. It is a processing unit. In this embodiment, the video signal processing unit 102 is positioned after the switch 111 and the signal synthesis unit 104 and performs the above-described processes. However, part of these processes is performed after the imaging unit 101. It may be performed before 111.

  Reference numeral 103 denotes a shooting mode switching unit (shooting mode switching unit) that performs a switching process between a normal shooting mode for shooting a normal moving image shot by the imaging unit 101 and a gradation expression range extended shooting mode. The normal shooting mode is a mode for acquiring an image captured by the standard exposure of the imaging device. The gradation expression range expansion shooting mode obtains a long-time exposure (EH) image having high reproducibility of the gradation expression of the dark part of the subject and low time of high reproducibility of the gradation expression of the bright part of the subject. In this mode, an exposure (EL) image is acquired. Further, in the gradation expression range extended photographing mode, as described later, an EH image and an EL image are combined to generate a gradation expression range extended image, and an image with high reproducibility of gradation expression can be acquired. In accordance with the mode selected by the shooting mode switching unit 103, the imaging unit 101 outputs a captured image signal corresponding to each mode. In the gradation expression range extended shooting mode, at least two exposure values are passed from the exposure control unit 109 to the imaging unit 101, and in the normal shooting mode, one exposure value is passed from the exposure control unit 109.

  Reference numeral 104 denotes a signal synthesis unit (image synthesis unit) that synthesizes a plurality of captured image signals input by a switching process of the switch 111 described later and generates a synthesized image signal.

  Reference numeral 105 denotes a synthesis range control unit that calculates a gradation range expansion width and a gain value from the exposure value output from the exposure control unit 109 described later and the variation value of the exposure value detected by the exposure variation detection unit 110 described later. is there. The synthesis range control unit 105 outputs the calculated tone range expansion width and gain value as control amounts to the signal synthesis unit 104, and controls the tone expression width of the synthesized image signal.

  106 calculates a composition ratio corresponding to a variation value of the exposure value detected by an exposure fluctuation detection unit 110 described later, and passes a control amount of the composition ratio to the signal composition unit 104, thereby changing the composition ratio of the composite image signal. It is a composition ratio control unit to be controlled.

  An output video signal recording unit 107 records the output video signal output from the video signal processing unit 102.

  An output image display unit 108 displays the output video signal output from the video signal processing unit 102 on the imaging apparatus.

  An exposure control unit (exposure control unit) 109 calculates a shutter speed control value and an aperture control value based on the captured image signal so that the brightness of the captured image signal is appropriate. The exposure control unit 109 sends the calculated shutter speed control value to the timing control unit 306 (to be described later) of the imaging unit 101, and sends the calculated aperture control value to the optical system drive unit 305 (to be described later) of the imaging unit 101. As described above, the exposure control unit 109 can control the exposure value of the imaging unit 101.

  Reference numeral 110 denotes an exposure fluctuation detection unit (exposure fluctuation detection means) that detects the amount of fluctuation of the exposure value controlled by the exposure control unit 109 when switching between the normal shooting mode and the gradation expression range extended shooting mode. is there.

  Reference numeral 111 denotes a switch for switching whether to send a captured image signal output from the imaging unit 101 to the signal synthesis unit 104 or to the video signal processing unit 102 in accordance with the shooting mode selected by the shooting mode switching unit 103. .

  Hereinafter, a method of detecting the exposure value fluctuation amount of the exposure fluctuation detection unit 110 will be described.

  The exposure fluctuation detection unit 110 detects the amount of fluctuation of the exposure value when switching between the normal photographing mode and the gradation expression range extended photographing mode. For example, an EV value is used as a numerical value representing the exposure value. As shown in FIG. 7, the EV value is expressed by the sum of the Av value obtained by converting the aperture value into a light amount ratio and the Tv value obtained by converting the shutter speed control value into a light amount. The exposure fluctuation detection unit 110 receives the exposure value (EVn) of the captured image signal in the normal shooting mode from the exposure control unit 109 when switching between the normal shooting mode and the gradation expression range extended shooting mode. Further, the exposure value (EV1) of the short-time exposure captured image signal and the exposure value (EV2) of the long-time exposure captured image signal in the gradation expression range extended shooting mode are passed from the exposure control unit 109. In the exposure fluctuation detection unit 110, the exposure value (EVn) of the captured image signal in the normal imaging mode, the exposure value (EV1) of the short-time exposure captured image signal in the gradation expression range expansion mode, and the long-time exposure captured image The difference from the exposure value (EV2) of the signal is detected as the magnitude of the exposure value variation. If the difference between EVn and EV1 or EVn and EV2 is larger than a predetermined value, it can be understood that the change in the gradation expression width in the two modes is large.

  Hereinafter, the imaging unit 101, the signal synthesis unit 104, the exposure fluctuation detection unit 110, the synthesis range control unit 105, and the synthesis ratio control unit 106 will be described in detail.

  A configuration example of the imaging unit 101 will be described in detail with reference to FIG. The imaging unit 101 includes an imaging optical unit 301, an aperture 302, an imaging element unit 303, an A / D conversion unit 304, an optical system driving unit 305, and a timing control unit 306. The imaging optical unit 301 guides subject light from the subject to a subsequent imaging element unit 303. The optical system drive unit 305 controls the aperture area of the stop 302 based on the stop control value sent from the exposure control unit 109. The imaging element unit 303 photoelectrically converts subject light that has passed through the imaging optical unit 301 and converts it into an electrical signal. The imaging element unit 303 is configured by a CCD, a CMOS, or the like. The timing control unit 306 controls the exposure time of the subject light (accumulation time of the image sensor unit 303) based on the shutter speed control value sent from the exposure control unit 109. The A / D conversion unit 304 converts an analog signal into a digital signal.

  A configuration example of the signal synthesis unit 104 will be described in detail with reference to FIG. The signal synthesis unit 104 is a block that operates during the gradation expression range extended shooting mode processing. In the gradation expression range extended shooting mode, a short-exposure captured image signal and a long-exposure captured image signal having different exposure amounts are continuously input from the imaging unit 101 (see FIG. 1). The signal synthesis unit 104 includes switches 401 and 404, a synthesis range correction unit 402, a synthesis processing unit 403, and an image storage unit 405.

  The switch 401 is a switch for switching whether a captured image signal input from the switch 111 (see FIG. 1) is output to a synthesis range correction unit 402 (to be described later) or a synthesis processing unit 403 (to be described later).

  The composite range correction unit 402 is a correction unit that corrects the captured image signal according to the control amount output from the composite range control unit 105 for the captured image signal input from the switch 401.

  The composition processing unit 403 directly combines the captured image signal corrected by the composition range correction unit 402 from the switch 401 and the captured image signal from the image storage unit 405 described later, with the composition ratio calculated by the composition ratio control unit 106. It is a processing part which synthesizes according to The combination processing unit 403 combines a plurality of captured image signals and outputs one combined image signal. However, as an exception, when there is no data in the image storage unit 405 described later, since the synthesis processing unit 403 has only one captured image signal to be input, the process of generating the composite image signal is not performed and the input is performed. The captured image signal is output as it is.

  The switch 404 sends the output destination of the synthesized image signal output from the synthesis processing unit 403 or the captured image signal input to the synthesis processing unit 403 to the video signal processing unit 102 or an image storage unit 405 described later. Switch to switch.

  The image storage unit 405 is a storage unit that stores a captured image signal or a composite image signal output from the composition processing unit 403 and passed from the switch 404.

  Next, a process for synthesizing two captured image signals with different exposure times that are imaged using the signal combining unit 104, that is, a short-time exposure captured image signal and a long-time exposure captured image signal will be described.

  In the gradation expression range extended shooting mode, the image pickup unit 101 in FIG. 1 continuously picks up images with two different exposure values and outputs two different picked-up image signals. More specifically, for example, the imaging unit 101 in FIG. 1 captures a short-time exposure captured image signal. The short-time exposure captured image signal is input to the synthesis range correction unit 402 when the switch 401 is switched. The short-time exposure captured image signal corrected by the synthesis range correction unit 402 is input to the synthesis processing unit 403. At this time, since the image processing unit 403 does not record captured image signal data in the image storage unit 405, the compositing process is not performed, and the short-time exposure captured image signal is output from the compositing processing unit 403. In response to this output, the switch 404 switches the connection to the image storage unit 405, and the short-time exposure captured image signal is stored in the image storage unit 405. Subsequently, the imaging unit 101 in FIG. 1 captures a long-time exposure captured image signal. The long-exposure captured image signal is input to the composition processing unit 403 when the switch 401 is switched. At this time, the short-time exposure captured image signal stored in the image storage unit 405 is also input to the synthesis processing unit 403. The combination processing unit 403 combines the two input captured image signals based on the combination ratio calculated by the combination ratio control unit 106. Finally, the synthesized image signal from the synthesis processing unit 403 is output to the video signal processing unit 102 by switching the switch 404.

  The processing for combining two captured image signals with different exposure times has been described above. However, the signal synthesis unit 104 can also synthesize three or more captured image signals with different exposures. Hereinafter, a case where three or more captured image signals having different exposures are combined will be described. First, it is assumed that a combined image signal obtained by combining two captured image signals is output from the combining processing unit 403. The method of image synthesis processing for synthesizing two captured image signals is as described above. The output destination of the composite image signal output from the composite processing unit 403 is switched by the switch 404 and input to the image storage unit 405. Finally, the third captured image signal picked up by the image pickup unit 101 in FIG. 1 and the combined image signal from the image storage unit 405 are combined by the combining processing unit 403, and a new combined image signal is output. In this way, even when three or more captured image signals are combined, it can be realized by sequentially repeating the process of combining the two captured image signals.

  The synthesis range correction unit 402 will be described in detail. In the synthesis range correction unit 402, the synthesis processing unit 403 performs preprocessing for synthesizing a plurality of captured image signals having different exposure times. Specifically, signal level adjustment of a plurality of captured image signals having different exposure times and control of gradation expression width are performed.

  With reference to FIG. 5A, a signal level adjustment method for a plurality of captured image signals having different exposure times performed by the synthesis range correction unit 402 will be described. In FIG. 5A, the vertical axis represents the signal intensity of the captured image signal, and the horizontal axis represents the brightness of the subject to be imaged. A solid line represents a long-time exposure captured image signal, and a dotted line represents a short-time exposure captured image signal. In the case of the long-exposure captured image signal in FIG. 5A, the signal intensity varies depending on the brightness of the subject in the range from 0 to Lx1 of the subject (that is, the first region below a predetermined value). Therefore, gradation expression is possible. However, the signal intensity does not change and remains constant between the brightness Lx1 and Lx4 of the subject (that is, the second region larger than the predetermined value), and the signal intensity that can be expressed by the captured image signal is saturated and the levels after Lx1. The key expression is not made. On the other hand, in the case of a short-exposure captured image signal, gradation expression is obtained because the signal intensity changes in accordance with the brightness of the subject even during Lx1 to Lx4 after Lx1 (that is, the second region larger than a predetermined value). Is done. That is, by applying the short-time exposure captured image signal to the area Lx1 and later where the long-exposure captured image signal is saturated, it is possible to obtain a captured image signal with an expanded gradation expression width after Lx1. However, it is necessary to calculate the gain according to the difference between the exposure values of the long-exposure captured image signal and the short-exposure captured image signal, and to multiply the short-exposure captured image signal by the gain. This is called signal level adjustment. . Conversely, a gain may be applied to the captured image signal of high exposure (long exposure) to match the level of the captured image signal of low exposure (short exposure). By this processing, two captured image signals having different exposure times can be synthesized.

  In FIG. 5B, the gradation expression width control performed by the synthesis range correction unit 402 will be described in detail. The horizontal bar represents the gradation expression width corresponding to the brightness of the subject in FIG. Further, the shading of the horizontal bar represents the gradation expression width that can be expressed by the long-time exposure captured image signal and the short-time exposure captured image signal.

  The gradation range control performed by the synthesis range correction unit 402 limits the maximum value of the signal intensity expressed by replacing the signal intensity after Lx1 saturated with the long-exposure captured image signal with the short-exposure captured image signal. It is realized by doing. That is, when it is not desired to expand the gradation expression width so much, it can be realized by limiting the maximum value of the signal intensity expressed by the short-time exposure captured image signal as shown in (b1) of FIG.

  On the contrary, when it is desired to greatly expand the gradation expression width, the maximum value of the short-exposure captured image signal can be set to the maximum value of the short-exposure captured image signal as shown in (b3) of FIG. This can be achieved by limiting the signal strength and not limiting it.

  The method for limiting the gradation expression width is not limited to this. For example, the gradation expression width may be controlled by reducing the maximum bit width that represents the signal intensity of the short-time exposure captured image signal.

  In FIG. 6, the control of the composite ratio in the composite ratio control unit 106 of FIG. 1 will be described in detail.

  The vertical axis in FIG. 6 represents the composition ratio of the long-time exposure captured image signal, the horizontal axis represents the brightness of the subject, and the subject brightness 0 to Lx1 on the horizontal axis in FIG. (I.e., the first region below a predetermined value).

  The composition ratio control unit 106 in FIG. 1 calculates the composition ratio of the long exposure captured image signal according to the brightness of the subject in the image of the captured image signal. In this embodiment, the brightness of the subject in the image of the captured image signal is calculated as the brightness value of each pixel of the captured image signal output from the image capturing unit 101, and is the brightness of the subject in each region. On the other hand, the combination ratio of the short-time exposure captured image signal is determined so that the sum of the combination ratio with the combination ratio of the long-exposure captured image signal is 1. In the synthesis processing unit 403 in FIG. 4, signals are synthesized based on two synthesis ratios. FIG. 6 shows (c1), (c2), (c3) and the composition ratio corresponding to the brightness of the three subjects, and the long-time exposure captured image signal in the section from 0 to Lx1 is synthesized. Indicates that the ratio is different. The long exposure captured image signal has a better SN than the short exposure captured image signal. It can be said that the SN in the section from 0 to Lx1 is better as the ratio of the long-time exposure captured image signal is larger. Therefore, (c1) has the highest SN in the synthesis ratios (c1), (c2), and (c3), and (c3 ) Is the lowest SN. Therefore, by changing the synthesis ratios (c1), (c2), and (c3), the synthesis processing unit 403 in FIG. 4 can generate a synthesized image signal in which the SN in the section from 0 to Lx1 is changed. The composition ratio calculated by the composition ratio control unit 106 in FIG. 1 is not limited to calculating the composition ratio of the long-exposure captured image signal, but the composition ratio of the short-exposure captured image signal is calculated. Alternatively, the synthesis ratio of the two captured image signals may be obtained independently.

  The control of the gradation expression width performed by the synthesis range correction unit 402 and the control of the synthesis ratio by the synthesis ratio control unit 106 in FIG. 1 have been described above. In the dark portion of the captured image signal (in the 0 to Lx1 interval in FIG. 5 or FIG. 6), the synthesis ratio is controlled, and in the bright portion of the captured image signal (in the Lx1 to Lx4 interval in FIG. 5), the gradation expression width is controlled. It is desirable.

  In the dark part of the captured image signal with low signal intensity, the short exposure image signal has a lower signal strength and noise level than the long exposure image signal, so the image quality is low. Therefore, the SN of the dark part of the captured image signal can be adjusted by changing the composition ratio and changing the composition ratio of the long exposure captured image signal.

  On the other hand, in the long-exposure captured image signal having a high signal intensity, the signal intensity of the captured image signal is likely to be saturated, and the gradation expression width of the bright portion is likely to be crushed. Therefore, the bright portion of the captured image signal is replaced with a short-exposure image signal whose gradation expression width is adjusted (a captured image signal with a minimum exposure when combining three or more captured image signals with different exposures). Thus, the gradation expression width of the bright part can be expressed.

  The control of the gradation expression width and the composition ratio and the composition processing of the short-time exposure captured image signal and the long-time exposure captured image signal have been described above.

  Next, the overall processing of the method of controlling the gradation expression width at the time of switching between the normal photographing mode and the gradation expression range extended photographing mode, which is a characteristic operation of the present invention, will be described with reference to the flowchart of FIG. . In the present embodiment, the control performed when switching from the normal shooting mode to the gradation expression range extended shooting mode during moving image shooting will be described as an example.

  As a premise, it is assumed that the imaging unit 101 of FIG. 1 has acquired an appropriate time exposure captured image signal imaged with an appropriate exposure time and has been captured in the normal shooting mode.

  In step 801 (hereinafter, step is expressed as S), the shooting mode switching unit 103 in FIG. 1 detects switching between the normal shooting mode and the gradation expression range extended shooting mode. If it is detected that the shooting mode has been switched, the process proceeds to S802.

  In S802, the exposure fluctuation detection unit 110 in FIG. 1 detects the fluctuation amount of the exposure value after switching from the normal photography mode to the gradation expression range extended photography mode. At this time, it is determined whether or not the exposure value difference is 1 EV or more. If it is 1 EV or more (predetermined value or more), the process proceeds to S803. If it is not 1 EV or more (smaller than a predetermined value), the control is not performed and the flowchart is terminated.

  In S803, the composite range control unit 105 in FIG. 1 calculates a gain for adjusting the signal level of the short-exposure captured image signal and the long-exposure captured image signal shown in FIG. 5A, and the process proceeds to S804.

  In S804, the gradation expression width shown in FIG. 5B is calculated by the synthesis range control unit 105 in FIG. The composite range control unit 105 in FIG. 1 determines the gradation expression width according to the gradation expression width determined in the previous frame of the gradation expression range extended shooting mode, and the process proceeds to S805. For example, when the gradation expression width of (b1) in FIG. 5B is not selected in the previous frame, the gradation expression width of (b1) in FIG. 5B is selected. That is, immediately after switching from the normal shooting mode to the gradation expression range extended shooting mode, if the previous frame of the gradation expression range extended shooting mode has not been generated, the gradation of (b1) in FIG. An expression width (minimum gradation expression width) is selected. When the gradation expression width of (b1) in FIG. 5B is selected for the previous frame in the gradation expression range expansion shooting mode, the gradation expression width of (b2) in FIG. 5B is selected. Is done. When the gradation expression width of (b2) in FIG. 5B is selected as the previous frame in the gradation expression range expansion shooting mode, the gradation expression width (maximum) of (b3) in FIG. ) Is selected. By repeating this process, the composition ratio selected from (b1) to (b3) in FIG. 5B is changed according to the gradation expression width selected in the previous frame of the gradation expression range extended shooting mode. . In this embodiment, when the normal shooting mode is switched to the gradation expression range expansion shooting mode, the gradation expression width of FIG. 5B is set to (b1), (b2), (b3) with the passage of time. In this order, it is gradually changed (in steps). Here, the example in which the gradation expression width is changed according to the gradation expression width determined in the previous frame (that is, for each frame) has been described. However, when the gradation expression width seems to change rapidly in time series, the gradation expression width may be changed after waiting for several frames. For example, the number of frames may be changed to the next gradation expression width when, for example, 10 frames are counted by a counter (not shown). In this way, when switching between the normal shooting mode and the gradation expression range expansion shooting mode, the gradation expression width is gradually changed to further reduce the unnaturalness of the image quality change. You can eliminate the sense of incongruity. The magnitudes of the gradation expression widths (b1), (b2), and (b3) are changed according to the amount of change in the exposure value. For example, in the case of two cases where the variation amount of the exposure value is different, the control amount performed by the synthesis range control unit 105 is larger when the variation amount of the exposure value is large than when the variation amount of the exposure value is small. In other words, the composite range control unit 105 increases the control amount of the gradation expression width (for example, the width of (b3)-(b1)) when the exposure value fluctuation amount is large than when the exposure value fluctuation amount is small. To control. Further, when the variation amount of the exposure value is small, the control amount of the gradation expression width (for example, the width of (b3)-(b1)) is controlled to be smaller than when the variation amount of the exposure value is large.

  In S805, the composition ratio shown in FIG. 6 is determined by the composition ratio control unit 106 in FIG. In the composition ratio control unit 106 in FIG. 1, the composition ratio is determined according to the composition ratio determined in the previous frame in the gradation expression range extended shooting mode, and the process proceeds to S806. For example, when the combination ratio of (c3) in FIG. 6 is not selected in the previous frame, the combination ratio of (c3) in FIG. 6 is selected. That is, immediately after switching from the normal shooting mode to the gradation expression range extended shooting mode, if the previous frame of the gradation expression range extended shooting mode has not been generated, the combination ratio (minimum) of (c3) in FIG. Composition ratio) is selected. When the composite ratio of (c3) in FIG. 6 is selected for the previous frame in the gradation expression range extended shooting mode, the composite ratio of (c2) in FIG. 6 is selected. When the composition ratio of (c2) in FIG. 6 has been selected for the previous frame in the gradation expression range expanded shooting mode, the composition ratio (maximum composition ratio) of (c1) in FIG. 6 is selected. By repeating this processing, the composition ratio selected from (c3) to (c1) is changed according to the composition ratio selected in the previous frame in the gradation expression range expansion mode. In this embodiment, when the normal shooting mode is switched to the gradation expression range expansion shooting mode, the composition ratios in FIG. 6 are gradually increased in the order of (c3), (c2), and (c1) with the passage of time. It is changing greatly (in steps). Here, an example in which the composition ratio is changed according to the composition ratio determined in the previous frame (that is, for each frame) has been described. However, if the gradation expression width appears to change rapidly in time series, the composition ratio may be changed after waiting for several frames. For example, the number of frames may be changed to the next composition ratio when, for example, 10 frames are counted by a counter (not shown). In this way, when switching between the normal shooting mode and the gradation expression range expansion shooting mode, the unnaturalness of the image quality change can be further reduced by gradually changing the composition ratio, and the photographer feels uncomfortable. You can eliminate it. Further, the ratios of the synthesis ratios (c1), (c2), and (c3) are changed according to the amount of change in the exposure value. For example, in the case of two cases where the variation amount of the exposure value is different, the control amount performed by the composition ratio control unit 106 is larger when the variation amount of the exposure value is large than when the variation amount of the exposure value is small. In other words, the composition ratio control unit 106 performs control so that the control amount of the composition ratio is increased (for example, the value of (c3) is decreased) when the variation amount of the exposure value is large than when the variation amount of the exposure value is small. . Further, when the exposure value fluctuation amount is small, the control of the composition ratio is controlled to be smaller (for example, the value of (c3) is larger) than when the exposure value fluctuation amount is large.

  In S806, the short-time exposure captured image signal and the long-time exposure captured image signal are combined by the signal combining unit 104 in FIG. 1 according to the gradation expression width and the combination ratio calculated in S803 to S805, and the combined image signal is obtained. Then, the process proceeds to S807.

  In S807, in order to generate the next composite image signal, the imaging unit 101 in FIG. 1 captures the low-time exposure captured image signal and the long-time exposure captured image signal, and the process proceeds to S802.

  Steps S802 to S806 are repeated, and the flowchart ends when the exposure difference becomes 1 EV or less in step S802.

  Here, the processing of the flowchart is ended depending on whether or not the exposure value difference is a predetermined value, that is, 1 EV or more in S802. However, the predetermined value of the exposure value difference may be changed as the processing end condition in S802. .

  Further, the flow processing described with reference to FIG. 8 has been described as processing executed when the normal shooting mode is shifted to the dynamic range expansion mode. However, it is also possible to control so that the gradation expression width does not change abruptly even when the dynamic range expansion mode is shifted to the normal shooting mode. At this time, the process changed in FIG. 8 will be described.

  In S804, in order to select the gradation expression width of FIG. 5B to be shorter than the gradation expression width of the synthesis range control unit 105 of FIG. The gradation expression width is determined so that the gradation expression width is determined in the order of (b3), (b2), and (b1).

  In S805, since the selection is performed so that the SN is low with respect to the determination of the composite ratio of the composite ratio control unit 106 in FIG. 1, (c1), (c2), (c3) with respect to the composite ratio in FIG. ) To determine the composition ratio.

  In this embodiment, the description has been given with the switching in three stages for the sake of simplification. However, the gradation expression can be switched more smoothly by switching in three or more stages.

  In the first embodiment, an imaging apparatus has been described that can record an image quality change comfortably by expanding or reducing the gradation expression width when switching between the normal imaging mode and the gradation expression range extended imaging mode.

  However, when switching between the normal shooting mode and the gradation expression range expansion mode, it may be desirable that a change in image quality due to the expansion or reduction of the gradation expression width appears.

  For example, there is a case where it is desired to compare the difference in gradation expression width that can be captured in the normal imaging mode and the gradation expression range extended imaging mode, that is, when the image display unit 108 in FIG. That is, check the live view display).

  In this case, when the normal shooting mode and the gradation expression range expansion shooting mode are switched, it is preferable that the time-series change in the gradation expression width is large.

  Also, when shooting a movie in the normal shooting mode and shooting a still image in the gradation range extended shooting mode (still image recording shooting when recording a movie), the gradation range is expanded immediately after the shooting mode is switched. It is preferable that recording can be performed with the maximum gradation expression width that can be taken in the shooting mode.

  Therefore, in the second embodiment, when changing between the normal shooting mode and the gradation expression range expansion shooting mode, the time-series expansion change in the gradation expression width is made larger than that in the first embodiment in accordance with the shooting method and the exposure fluctuation amount. Change to In other words, in this embodiment, the time constant of the gradation control of the image combining means for generating a combined image signal by combining a plurality of captured image signals photographed with different exposure values is smaller than in the first embodiment. .

  FIG. 9 is a block diagram illustrating a configuration example of the imaging apparatus according to the second embodiment of the present invention. Hereinafter, a configuration example of the second embodiment will be described with reference to FIG. Note that the same reference numerals as those in FIG. 1 are used for the same operation and processing as in the first embodiment of the present invention, and the description thereof is omitted.

  Reference numeral 901 denotes an imaging method switching unit that performs processing for switching an imaging method using the imaging apparatus.

  As one of the photographing methods here, for example, there is a live view display photographing method in which the output image signal processed by the image signal processing unit 102 is displayed in real time by the output image display unit 108. There is also a moving image recording still image recording and shooting method in which a still image is recorded in the output video signal recording unit 107 during the moving image recording and shooting method. As described above, there are various imaging methods using the imaging apparatus.

  Reference numeral 902 denotes a signal level adjustment based on the exposure value output from the exposure control unit 109, the exposure fluctuation value output from the exposure fluctuation detection unit 110, and the current shooting method information output from the shooting method switching unit 901. It is a synthesis range control unit that determines the gradation expansion width.

  Hereinafter, a method for determining the gradation expansion width in switching from the normal photographing mode to the gradation expression range extended photographing mode will be described.

  The control of the gradation expression width determined by the synthesis range control unit 902 is (b1), (b2), and (b3) in FIG. 5B shown in FIG. 5B. However, the maximum values of the gradation expression widths of (b1) and (b2) in FIG. That is, in this embodiment (for example, live view display shooting method), the maximum value of the gradation expression widths in (b1) and (b2) of FIG. 5B is the same as that in Embodiment 1 (for example, moving image recording shooting method). It is set larger than the maximum value of the gradation expression width of (b1) and (b2).

  However, the maximum value of the gradation expression width of each of (b1), (b2), and (b3) in FIG. 5B is limited by the magnitude of the exposure fluctuation value output from the exposure fluctuation detection unit 110.

  For example, the maximum value of the gradation expression width of (b1) of the present embodiment is limited so as not to exceed (b2) of Embodiment 1, and the maximum value of the gradation expression width of (b2) of the present embodiment. The value is limited so as not to exceed (b3) of Example 1.

  Also, when switching from normal shooting mode to gradation range extended shooting mode, the gradation expression width increases, so the limit value of the maximum gradation expression width is limited to be larger as the exposure fluctuation value is larger. Also good. Specifically, this is realized by changing the selection method of (b1), (b2), and (b3) in FIG.

  As an example 1, in the case of the live view display confirmation method, the gradation expression width is changed in two steps (b2) and (b3) without selecting (b1) in FIG. To do. By changing in this way, control is performed so that the change in the gradation expansion width in time series at the time of switching between the normal imaging mode and the gradation expression range expansion imaging mode becomes large.

  However, it is not limited to two steps, and control may be performed so as to change the gradation expression width in one step.

  As an example 2, in the case of the still image recording shooting method during moving image recording, control is performed so as not to change the time-series gradation expression width when switching between the normal shooting mode and the gradation expression range extended shooting mode. That is, the gradation expression width of (b1) and (b2) in FIG. 5B in the first embodiment is not controlled, and the gradation expression width is controlled to select (b3). That is, control is performed so that recording can be performed with the maximum gradation expression width that can be expressed in the gradation expression range expansion shooting mode.

  Next, reference numeral 903 denotes a composite ratio for calculating a composite ratio of captured image signals based on the exposure fluctuation value output from the exposure fluctuation detection unit 110 and the current shooting method information output from the shooting method switching unit 901. It is a control unit.

  Hereinafter, a method for determining the gradation expansion width in switching from the normal photographing mode to the gradation expression range extended photographing mode will be described.

  The composite ratios of the captured image signals determined by the composite ratio control unit 903 are (c1), (c2), and (c3) in FIG. However, the maximum values of the synthesis ratios of (c3) and (c2) in FIG. That is, in this embodiment (for example, a live view display shooting method), the maximum value of the combination ratio of (c3) and (c2) in FIG. 6 is (c3) and ( It is set to be larger than the maximum value of the composition ratio of c2).

  However, the maximum value of the combination ratio is limited in each of (c1), (c2), and (c3) in FIG. 6 depending on the magnitude of the exposure fluctuation value output from the exposure fluctuation detection unit 110.

  For example, the maximum value of the synthesis ratio of (c3) in this example is limited so as not to exceed (c2) of Example 1, and the maximum value of the synthesis ratio of (c2) in this example is Limiting not to exceed (c1) of Example 1.

  Also, when switching from the normal shooting mode to the gradation expression range expansion shooting mode, the gradation expression width of the captured image signal changes in the widening direction, so the exposure fluctuation value increases the limit value of the maximum value of the composite ratio. You may do it. Specifically, this is realized by changing the selection method of (c1), (c2), and (c3) in FIG.

  As an example 1, in the case of the live view display confirmation method, the combination ratio of two steps (c2) and (c1) is changed without selecting (c3) in FIG. . By changing in this way, control is performed so that the change in the gradation expansion width in time series at the time of switching between the normal imaging mode and the gradation expression range expansion imaging mode becomes large.

  However, it is not limited to two stages, and control may be performed so as to change the synthesis ratio in one stage.

  As an example 2, in the case of the still image recording and shooting method during moving image recording, control is performed so as not to change the time-series composition ratio when switching between the normal shooting mode and the gradation expression range extended shooting mode. Specifically, the composition ratio is controlled so as to select (c1) without performing the composition ratio control according to (c2) and (c3) in FIG. In other words, control is performed so that recording can be performed at the maximum composition ratio that can be expressed in the gradation expression range expansion shooting mode.

  As described above, the control of the gradation expression width and the composition ratio according to the shooting method information and the exposure variation value have been described.

  In the following, with respect to the image pickup apparatus according to the second embodiment, when the normal shooting mode and the gradation expression range extended shooting mode are switched, the time-series gradation expression width expansion change is changed according to the shooting method and the exposure fluctuation value. Will be described.

  The entire process will be described with reference to the flowchart of FIG. As preprocessing, the imaging unit 101 in FIG. 9 acquires an appropriate time exposure captured image signal, and is captured in the normal shooting mode. In addition, about the thing of the same code | symbol as the structural example of this invention Example 1, operation and a process similar to Example 1 are performed, and description is abbreviate | omitted.

  First, in S801 and S802, the same processing as S801 and S802 in FIG. 8 is performed, and the processing proceeds to S1001.

  Next, in S1001, the shooting method switching unit 901 in FIG. 9 detects the current shooting method when the normal shooting mode is switched to the gradation expression range extended shooting mode, and the process proceeds to S803.

  In S803, the same process as S803 in FIG. 8 is performed, and the process proceeds to S1002.

  In S1002, the synthesis range control unit 902 in FIG. 9 determines the expansion of the gradation expression width according to the above-described exposure variation value and the current shooting method, and the process proceeds to S1003 in the signal synthesis unit 104 in FIG.

  In S1003, the composition ratio control unit 903 in FIG. 9 calculates a composition ratio in accordance with the exposure variation value and the photographing method described above, passes them to the signal composition unit 104 in FIG. 9, and proceeds to processing in S806.

Hereinafter, the processing after S806 is the same as the processing after S806 in FIG.
(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

  The present invention can be suitably used for an imaging apparatus such as a digital camera or a video camera that can generate an HDR image during moving image shooting. In addition, the present invention is not limited to devices such as digital cameras, but includes built-in or external connection of imaging devices such as mobile phones, personal computers (laptop type, desktop type, tablet type, etc.), game machines, etc. It can be applied to any device. Therefore, the “imaging device” in this specification includes any electronic device having an imaging function.

DESCRIPTION OF SYMBOLS 101 Image pick-up part 103 Shooting mode switching part 104 Signal composition part 105 Composition range control part 106 Composition ratio control part 109 Exposure control part

Claims (13)

Imaging means for photoelectrically converting subject light via a photographing optical system to generate an image signal;
Exposure control means for controlling the exposure value of the imaging means;
Image synthesizing means for synthesizing a plurality of image signals picked up by the image pickup means with different exposure values determined by the exposure control means to generate a combined image signal having an extended gradation expression range;
A shooting mode switching means for switching between a normal shooting mode for performing normal moving image shooting and a gradation expression range extended shooting mode for generating the composite image signal;
The image synthesizing unit has a gradation of the composite image signal according to an exposure value determined by the exposure control unit to generate the composite image signal when the photographing mode is switched by the photographing mode switching unit. An imaging apparatus characterized by changing an expression width stepwise.   The image synthesizing unit changes the gradation representation width of the composite image signal so as to increase stepwise when the normal photographing mode is switched to the gradation expression range expanded photographing mode. The imaging apparatus according to claim 1, wherein when the range expansion shooting mode is switched to the normal shooting mode, the gradation expression width of the composite image signal is changed stepwise.   The image synthesizing unit changes a synthesis ratio of the plurality of image signals in a first region where the brightness of the subject in the image of the image signal used for generating the synthesized image signal is smaller than a predetermined value, 3. The imaging according to claim 1, wherein in a second region where the brightness of the image is larger than a predetermined value, a gradation expression width of an image signal having a minimum exposure amount among the plurality of image signals is changed. apparatus. Exposure fluctuation detecting means for detecting fluctuations in the exposure value of the exposure control means when the shooting mode is switched;
The image synthesizing unit changes the gradation expression width of the synthesized image signal in a stepwise manner in accordance with the amount of variation of the exposure value detected by the exposure variation detecting unit. The imaging device according to any one of the above.   The imaging apparatus according to claim 4, wherein the image synthesizing unit changes a gradation expression width of the synthesized image signal in a stepwise manner when a variation amount of the exposure value is a predetermined value or more.   The imaging apparatus according to claim 4, wherein the image synthesizing unit does not change a gradation expression width of the synthesized image signal when a variation amount of the exposure value is smaller than a predetermined value.   The image synthesizing unit changes a time constant of gradation control of the image synthesizing unit according to a current photographing method when the photographing mode is switched. Imaging device.   When the current photographing method is a live view display photographing method, the time constant of gradation control of the image composition means is made smaller than when the current photographing method is a moving image recording photographing method. The imaging device according to claim 7.   8. The image pickup apparatus according to claim 7, wherein when the current photographing method is a still image recording photographing method at the time of the moving image recording photographing method, a time constant of gradation control of the image composition means is minimized. . Photographic optics,
A camera system comprising: the imaging device according to claim 1. A method for controlling an imaging apparatus having imaging means for photoelectrically converting subject light via an imaging optical system to generate an image signal,
An exposure control step for controlling an exposure value of the imaging means;
An image synthesis step for synthesizing a plurality of image signals picked up by the image pickup means with a plurality of different exposure values determined in the exposure control step to generate a combined image signal having an extended gradation expression range;
A shooting mode switching step for switching between a normal shooting mode for performing normal moving image shooting and a gradation expression range extended shooting mode for generating the composite image signal;
In the image synthesis step, when the shooting mode is switched by the shooting mode switching step, the gradation of the composite image signal according to the exposure value determined by the exposure control step to generate the composite image signal A method for controlling an imaging apparatus, wherein the expression width is changed stepwise.   A computer-executable program in which a procedure of a control method for an imaging apparatus according to claim 11 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of the imaging apparatus according to claim 11.