JP2010178164A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which performs continuous exposure and does not cause missing exposure information even when an exposure time of second exposure is shorter than a first exposure by almost one frame transfer time. <P>SOLUTION: The imaging apparatus includes an imaging optical system, a CMOS sensor having a rolling shutter function, and a control unit for controlling the CMOS sensor. In the imaging apparatus, the control unit calculates an appropriate exposure time Ts of a subject from subject image data from the CMOS sensor, calculates a short exposure time T1 and a long exposure time T2 on the basis of the appropriate exposure time Ts, causes the exposure to be performed by a rolling shutter system in the short exposure time T1 and the long exposure time T2, composites short/long exposure images obtained by this exposure, and generates an image of a wide dynamic range. The control unit calculates a transfer time of one frame transfer time of the CMOS sensor minus one pixel row, compares the long exposure time T2 with this time, and changes the long exposure time T2 to this time when the former is shorter. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that generates an image with a wide dynamic range by combining short-exposure and long-exposure images.

2. Description of the Related Art Conventionally, there has been known an imaging apparatus including an imaging system that generates a wide dynamic range image by combining a plurality of images captured under different exposure conditions (for example, Patent Document 1). This imaging system has a lens system, a CCD, and control means for processing an image captured by the CCD.
The control means includes a photometric evaluation unit for obtaining an appropriate exposure time by obtaining the brightness level from an image photographed through a lens system and a CCD, and an exposure time at which the exposure is, for example, 1/8 compared to the appropriate exposure time. And an image composition unit that synthesizes images captured by continuous exposure for both the appropriate exposure time and a control unit that controls each part of the control means including the photometry evaluation unit and the image composition unit. is doing.
Further, as a similar imaging device, there is known an imaging device that calculates an appropriate exposure time, calculates long and short exposure times including the appropriate exposure time, and captures and combines two images (for example, Patent Document 2).
This imaging apparatus includes a lens system, an exposure control mechanism (exposure control system) that controls the aperture of the lens system, a CCD, a digital process circuit that performs various digital processes, a digital process circuit, and a system controller that controls the CCD. It has.
This exposure control system uses a normal exposure time, that is, an appropriate exposure time (appropriate exposure time) based on a user setting or a result calculated by a known automatic exposure adjusting means (AE) (a result of known photometry using an output signal from a CCD). (Representative exposure time) is calculated. Then, short exposure and long exposure are performed with an appropriate exposure time and another exposure time calculated based on the appropriate exposure time, and both images are synthesized.

  However, in the imaging devices of Patent Documents 1 and 2, the imaging element is a CCD, and as shown in FIG. 6A, for example, when performing continuous exposure twice, the first exposure is performed. Since this image is transferred to the control means, and then the second exposure is performed and this image is transferred to the control means, a gap between the first exposure time and the second exposure time during the imaging sequence is not exposed for the transfer time. There was a problem that G1 occurred.

  The gap G1 also causes the exposure timings of the two to differ, and the amount of positional deviation between the first exposure and the second exposure increases due to subject blurring or camera shake that occurs when the gap G1 occurs. As a result, the process of synthesizing both becomes heavy. In addition, there is a problem that a portion that is not combined increases due to an increase in the amount of positional deviation. Further, there is a problem that exposure cannot be performed at the time when the gap G1 is generated.

  Therefore, the first exposure and the second exposure are performed substantially continuously using a CMOS sensor with a rolling shutter, thereby exposing each pixel row as shown in FIG. 6B. Thus, the first exposure and the second exposure are performed as one exposure, and the exposure is divided according to the exposure times of the first and second times to be the first exposure and the second exposure. Exposure, and the gap problem is almost solved.

  Specifically, the imaging with the rolling shutter system will be described. The control unit calculates the exposure time and the exposure start point of each of the first exposure and the second exposure for each pixel row based on the appropriate exposure time. Based on the result, a control command for continuously capturing images is issued. This exposure start time differs for each pixel row.

  Specifically, when the subject is surrounded by a single virtual frame when viewed from the camera, assuming that the upper left point of the virtual frame is the origin, exposure of one pixel row extending in the X direction from the origin is first performed. Then, the exposure for one pixel row is sequentially performed from the top to the bottom along the Y axis of the virtual frame.

  In the exposure for each pixel row, when the exposure is completed, the charge information of this pixel is transferred to the control means, and this pixel row starts the next exposure (charge accumulation).

  As described above, when the charge information of one pixel row is transferred in the rolling shutter method of the CMOS sensor, compared with the CCD imaging method in which the next exposure is started after the transfer of all the charge information of one image, the next exposure is performed. Since it is started, the time gap until the next exposure becomes only a transfer time of one pixel row, and becomes small (see the comparison in FIGS. 6A and 6B).

In the case where the first and second exposures are successively performed by the rolling shutter method, for example, the time when the exposure of the first pixel row in the second exposure is transferring the image obtained by the first exposure. Since the signals being transferred are combined when they overlap with the band, the exposure end point is set to be after the frame transfer end point of the first exposure.
For this reason, when the pixel row exposure time of the second exposure is shorter than a predetermined time (1 frame transfer time-1 pixel row transfer time), as shown in FIG. The second exposure of the first pixel row is performed so as to coincide with the frame transfer end time of the second exposure. Since the second exposure is performed in this way, when each pixel row exposure time is shorter than a predetermined time (1 frame transfer time-1 pixel row transfer time), the exposure is performed between the first exposure and the second exposure. An unclear time zone (gap) occurs.
Due to this gap, no exposure time occurs, and the image obtained by combining the images obtained by the first and second exposures lacks exposure information in this time zone, and the subject appears as a double image. It becomes.

  The present invention has been made paying attention to the above problem, and even when the long exposure time calculated from the appropriate exposure time is shorter than a predetermined time (1 frame transfer time-1 pixel row transfer time), the exposure information of the long exposure is obtained. An object is to provide an imaging device that is not lost.

In order to achieve the above object, the invention of claim 1 includes an imaging optical system, a CMOS sensor having a rolling shutter function, and control means for controlling the CMOS sensor. The appropriate exposure time for the subject is calculated from the image data of the output subject, and the short exposure time and the long exposure time are calculated based on the appropriate exposure time, and the short exposure time and the long exposure are calculated by a rolling shutter method. In the imaging device that generates an image with a wide dynamic range by combining the short-exposure image and the long-exposure image obtained by the exposure at each exposure time, the control means transfers one frame of the CMOS sensor The time obtained by subtracting the transfer time of one pixel row from the time is calculated, and the calculated time is compared with the long exposure time. If shorter than the time exposure time is the calculated, and changes the long exposure time to the calculated time.
According to a second aspect of the present invention, in the imaging apparatus according to the first aspect, the control means is at least one of a diaphragm amount of the imaging optical system, a light reduction amount of the neutral density filter, and an analog gain of the CMOS sensor. To adjust the exposure amount of the long exposure increased by the change of the long exposure time to be the same as the exposure amount before the change, and the exposure amount of the short exposure reduced by the adjustment is the same as the exposure amount before the decrease. Thus, the short exposure time is extended.
According to a third aspect of the present invention, in the imaging apparatus according to the second aspect, a process for increasing the gain is performed on the first image obtained by the exposure with the short exposure time, and the luminance of the first image is set to the long exposure time. The same as the luminance of the second image obtained by exposure in step 2, and a portion where the luminance is saturated in the second image is specified, and this portion is replaced with the first image portion corresponding to this portion It is characterized by doing.
According to a fourth aspect of the present invention, in the imaging apparatus according to the third aspect, when the increased exposure amount of the long exposure cannot be made the same as the exposure amount before the change by the adjustment, the luminance is saturated in the second image. For the pixels included in the non-image part, each pixel output value data is bit-shifted to reduce the pixel output value of this image part, and this image part is the same as the one exposed for the long exposure time before the change. It is characterized by.

  According to the imaging apparatus of the present invention, even when the long exposure time calculated from the appropriate exposure time is shorter than the predetermined time (1 frame transfer time-1 pixel row transfer time), the exposure information for the long exposure is not lost.

(A) is a front view which shows the digital camera as an example of the imaging device which concerns on embodiment of this invention. (B) is a top view of (a). (C) is a rear view of (a). It is a block diagram which shows the outline | summary of the system configuration | structure in the digital camera shown in FIG. (A) (b) It is a timing chart which shows the exposure process in short exposure-long exposure using the rolling shutter function of a CMOS sensor for every pixel row. It is a flowchart explaining the process which calculates long and short exposure time in embodiment of this invention. It is a flowchart explaining the process which synthesize | combines the image of a short exposure and the image of a long exposure in embodiment of this invention. (A) It is a timing chart which shows the case where it image | photographs continuously with a CCD sensor. (B) It is a timing chart which shows the exposure process in 1st imaging-2nd imaging using the rolling shutter function of a CMOS sensor for every pixel row. It is a timing chart which shows the exposure process in the 1st imaging-2nd imaging using the rolling shutter function of a CMOS sensor for every pixel row.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

(Appearance structure of digital camera)
As shown in FIGS. 1A, 1B, and 1C, a release button (shutter button) 2, a power button 3, and a shooting / playback switching dial 4 are provided on the top surface of the digital camera 100 according to the present embodiment. A lens barrel unit 6 having a photographing lens system (imaging optical system) 5, a strobe light emitting unit (flash) 7, and an optical finder 8 are provided on the front (front) side of the digital camera 1.

  On the back side of the digital camera 100, there are a liquid crystal monitor (LCD monitor) 9, an eyepiece 8 a of the optical viewfinder 8, a wide-angle zoom (W) switch 10, a telephoto zoom (T) switch 11, and a menu (MENU) button 12. , A confirmation button (OK button) 13 and the like are provided.

In addition, a memory card storage unit 15 for storing a memory card 14 (see FIG. 2) for storing captured image data is provided inside the side surface of the digital camera 100.
(Digital camera system configuration)
As shown in FIG. 2, the digital camera 100 includes a photographing lens system 5 including a focus lens (focusing optical system), a mechanical shutter 17 provided between the photographing lens system 5 and the CMOS sensor 16, and the photographing lens system. 5 and a CMOS sensor 16 that picks up a subject image incident through a mechanical shutter 17, a motor driver 18 that displaces at least a focus lens of the photographing lens system 5 within a movable range along its optical axis, and a photographing start The operation unit 19 including the release button 2 and the like (see FIGS. 1A to 1C) to which the operation of the front end is input, and the F / E of the front end (F / E) that mainly performs signal reading processing from the CMOS sensor 16 E signal processing unit 20, gyro sensor 21 for detecting shake of the digital camera 100, and shake of the digital camera 100 Is a camera shake correction means 22 for correcting camera shake and control for reading a signal from the CMOS sensor 16, processing of the read signal, drive control of the motor driver 18, input processing of an operation signal from the operation unit 19, CMOS Based on the image signal obtained by reading from the sensor 16, a digital signal processing IC (control unit) 23 for performing various arithmetic processes such as an autofocus evaluation value that is an index value representing the degree of focus of the image represented by the image signal. And.
<Mechanical shutter 17>
The mechanical shutter 17 is inserted in an optical path between the photographing lens system 5 and the CMOS sensor 16 to open and close the optical path and limit the exposure of the CMOS sensor 16.
<CMOS sensor 16>
A CMOS stage 24 is provided inside the digital camera 100 so as to be movable in the XY direction, and the CMOS sensor 16 is mounted on the CMOS stage 24. The CMOS stage 24 is moved in the XY direction by a drive mechanism (not shown). The drive mechanism is a control device (control means: control means: later-described) equipped with a CPU or the like based on a shake amount detected by the gyro sensor 21. Controlled by a positional deviation detection means: image processing means) 25.

  The CMOS sensor 16 converts an optical image incident on the light receiving surface in an exposed state into an electrical signal (photoelectric conversion), and transfers and outputs the image as an image signal. The exposure is performed for each pixel row of the CMOS sensor 16, and the pixel row from which the signal is output starts to accumulate charges from that point, so that the charge accumulation start timing is shifted for each pixel row, so-called rolling shutter. As described later, using this rolling shutter function, images are sequentially captured at a plurality of different exposure times.

  An RGB primary color filter (not shown) as a color separation filter is disposed on each pixel constituting the CMOS sensor 16, and an electrical signal (analog RGB image signal) corresponding to the three primary colors of RGB is generated in each pixel.

The pixel output value of the CMOS sensor 16 takes a value of 0 to 4095 in 12 bits (3 bits are assigned to R, G, and B, and there are two G).
<F / E signal processor 20>
The F / E signal processing unit 20 includes a CDS (correlated double sampling) circuit 26, an AGC (automatic gain control) circuit 27, an A / D (analog / digital) converter 28, and a timing generator (TG) 29. ing.

  The CDS circuit 26 performs correlated double sampling on the analog RGB image signal to remove signal noise.

  An AGC (automatic gain control) circuit 27 performs automatic gain control on the signal from which noise has been removed by the CDS circuit 26 to adjust it to a desired signal level.

  The A / D (analog / digital) converter 28 converts the analog RGB image signal from the AGC circuit 27 into digital RGB image data (hereinafter referred to as “RAW-RGB data”).

The timing generator (TG) 29 performs digital signal processing in response to a horizontal synchronization drive signal (HD) and a vertical synchronization drive signal (VD) from a camera interface (camera I / F) 31 of a digital signal processing IC 23 described later. In cooperation with the control device 25 of the IC 23, timing signals are sent to the CMOS sensor 16, the CDS circuit 26, the AGC circuit 27 and the A / D converter 28, respectively, and these are properly synchronized.
<Digital signal processing IC 23>
On the other hand, the digital signal processing IC 23 includes a camera interface 31, a memory controller 32, a display output control unit 33, a compression processing unit 34, a YUV conversion unit 35, a resize processing unit 36, a media interface (media I / F) 37, and a control device 25. Etc.
The control device 25 controls the ROM 38, the frame memory (SDRAM: storage means) 41, the liquid crystal monitor (LC display) 9, the audio output device 39, and the memory card 14 that are connected, the distance measuring sensor 40, and the operation unit 19. And a signal from the ROM 38 is input.

  The camera interface 31 takes in RAW-RGB data output from the A / D converter 28 of the F / E signal processing unit 20.

  Under the control of the control device 25, the memory controller 32 is RAW-RGB data from the camera interface 31, YUV data YUV converted by the YUV converter 35, and image data compressed by the compression processor 34, for example, in JPEG format. , And writing OSD (on-screen display) image data to the frame memory 41 and reading image data from the frame memory 41.

  The display output control unit 33 displays the image data read from the frame memory 41 on the liquid crystal monitor (LCD monitor) 9 and performs TV output for display on an external television (TV) or the like.

  The frame memory 41 is a semiconductor memory such as an SDRAM, and is used for storing data such as various image data.

  Under the control of the control device 25, the compression processing unit 34 encodes the given image data by a predetermined method to generate compressed image data.

  The YUV conversion unit 35 is a luminance level determination unit that determines the luminance level of the RAW-RGB data captured by the camera interface 31, a bit compression conversion unit that bit-shifts the RAW-RGB data and performs bit compression, and an auto white balance control unit. An RGB-YUV converter that converts RAW-RGB data into YUV format data in accordance with an auto white balance control value given from the control device 25, a brightness histogram generator that generates a brightness histogram from the converted YUV format data, and the like have.

  The resizing processing unit 36 resizes RAW-RGB data captured by the camera interface 31, image data read from the frame memory 41, and the like.

  The media interface 37 writes the RAW-RGB data captured by the camera interface 31 and the image data read from the frame memory 41 to the memory card 14 under the control of the memory controller 32 and the control device 25.

  The ROM 38 stores an operation program, data, and the like of the control device 25 for performing various processes related to the photographing operation and an autofocus control process described later.

  The liquid crystal monitor 9 displays an image (a monitoring image described later, an image read from a frame memory, etc.) and necessary information from the display output control unit 33 and provides various information related to imaging (setting of a desired exposure time, etc.). Is done.

  The motor driver 18 drives the lens driving motor of the photographing lens system 5 for focusing, zooming, and the like based on the control of the control device 25, and opens and closes the mechanical shutter 17. This opening / closing operation is performed in cooperation with the timing generator 29.

The distance measuring sensor 40 constitutes a distance measuring means for periodically measuring a subject distance by a so-called triangulation method.
<Monitoring operation>
The photographer operates the menu button 12 (see FIG. 1C) of the digital camera 100 to display a shooting setting screen (not shown) on the liquid crystal monitor 9. Then, by selecting the “still image shooting mode” displayed on the shooting setting screen, the digital camera 100 is set to the still image shooting mode under the control of the control device 25.

At this time, the digital camera 100 performs a monitoring operation under the control of the control device 25. A moving image received by the digital camera 100 is displayed on the liquid crystal monitor 9. During this monitoring operation, an AE (automatic exposure) process and a known AWB process are repeatedly performed.
<AE processing>
Further, the control device 25 calculates an AE (automatic exposure) evaluation value from the integration value of each of the RGB values in the RAW-RGB data. For example, the screen corresponding to the light receiving surface of all the pixels of the CMOS sensor 16 is equally divided into 1024 areas (horizontal 32 divisions and vertical 32 divisions), and the RGB integration of each area is calculated.
Then, the control device 25 calculates the luminance of each area of the screen from the calculated RGB integrated values, and determines an appropriate exposure amount (appropriate exposure time Ts) from the luminance distribution.
<Imaging Operation in this Embodiment>
The imaging operation in the present embodiment will be described with reference to the flowchart shown in FIG. Here, it is assumed that one frame transfer time L1 is 20 fps (one image is transferred in 1/20 second). As shown in FIG. 6B, since the transfer of the first pixel row of short exposure is started (time ta1) and the transfer of the Nth pixel row of the last row is completed (time ta2), one frame transfer time L1 Is the sum of the transfer times of the first to Nth pixel rows at time point ta2−time point ta1. This one-frame transfer time L1 is unique to the camera and does not change in principle.

  In the present embodiment, when calculating the exposure time, first, in step S1, the luminance level of the subject is determined from RGB-RAW input to the luminance level determination unit of the YUV conversion unit 35 during the monitoring operation.

  In step S2, the control device 25 calculates an appropriate exposure time Ts from the brightness level. Here, for example, it is assumed that the appropriate exposure time is calculated as 1/40 seconds.

Next, in step S3, the control device 25 calculates a short exposure time T1 and a long exposure time T2 from the appropriate exposure time Ts. When imaging in the dynamic range ± m (EV) is scheduled, the short exposure time T1 and the long exposure time T2 are calculated by the following expressions: T1 = Ts * 1/2 ^m and T2 = Ts * 2 ^m .

  When the expansion of the dynamic range of ± 2 EV is expected, since the appropriate exposure time is 1/40 seconds, the short exposure time T1 is 1/160 seconds and the long exposure time T2 is 1/10 seconds according to the above formula.

  In step S4, it is determined whether or not the long exposure time T2 calculated in step S3 is shorter than a predetermined time (1 frame transfer time-1 pixel row transfer time). If NO, the process proceeds to step S9, and if YES is determined. If so, the process proceeds to step S5.

Here, a specific example will be described. When the subject is brighter than the above, for example, when the appropriate exposure time is calculated as 1/160 seconds, the short exposure time T1 is 1/640 seconds, and the long exposure time T2 is 1/160 seconds. It is calculated as 40 seconds.
The long exposure time T2 is 1/40 second, which is shorter than about 1/20 second of the predetermined time (1 frame transfer time−1 pixel row transfer time). Therefore, if the exposure start time of the long exposure is set so that the frame transfer of the long exposure does not overlap the frame transfer of the short exposure and is continuous, a gap G2 in a time zone in which no exposure is performed between the short exposure and the long exposure occurs. (See FIG. 3 (a)).

  In step S5, in order to fill this gap G2, the long exposure time T2 calculated in step S4 (exposure time for each long pixel row) is set to a predetermined time T2 ′ (one frame transfer time—one pixel row transfer time, FIG. 3 (b)). As a result, the short exposure and the long exposure are continuous, and the loss of exposure information (double image of the subject) due to the gap G2 can be prevented.

  In step S6, it is determined whether or not the ISO sensitivity of the digital camera 100 is the minimum value. If NO, the process proceeds to step S7, and if YES, the process proceeds to step S9.

  In step S7, the exposure amount of the long exposure time T2 ′ adjusted and extended in the direction of decreasing the ISO sensitivity is made equal to the exposure amount of the long exposure time T2 before the change (extension).

Here, the reason for adjusting the ISO sensitivity will be described. When the photographer originally takes an image of ± 2 EV and expects to expand the dynamic range of 4 EV as a composite result, the long exposure time is set in step S5 as described above. By changing (extending) T2, the long exposure time T2 is reduced from 1/40 second to about 1/20 second, and the dynamic range is expanded more than planned.
Therefore, if it is determined in step S6 that the ISO sensitivity can be adjusted, the gain (ISO sensitivity) is adjusted to be the same so that the exposure amount of the extended long exposure and the exposure amount of the long exposure before the extension are the same. The extended long exposure image is equivalent to the long exposure image captured in the intended dynamic range.
For example, if the ISO sensitivity is set from 200 to 100, the signal gain is halved. Therefore, if the exposure time is doubled, the same long exposure image before extension can be obtained. That is, when the exposure time is doubled, the ISO sensitivity may be set to 200 to 100, for example.

  In this embodiment, the ISO sensitivity is adjusted. However, the present invention is not limited to this, and the diaphragm amount of the mechanical shutter 17 and the amount of light reduction by the ND (dimming) filter may be adjusted, or a combination thereof may be performed. .

  Among them, for example, a diaphragm priority mode that prioritizes the diaphragm may be used to fix the diaphragm according to the photographer's intention. Therefore, it is more preferable to adjust the exposure amount to the same by adjusting the ISO sensitivity.

Next, in step S8, a process for extending the short exposure time T1 in accordance with the ISO sensitivity adjustment is performed.
By adjusting to lower the ISO sensitivity, the exposure amount of the long exposure after the extension can be made the same as the exposure amount of the long exposure before the extension, but the exposure amount of the short exposure is reduced by this ISO sensitivity adjustment. The exposure time T1 is extended to T1 ′ (> T1) so as to be the same as the exposure amount of the short exposure before the ISO sensitivity adjustment. Since the exposure amount for short exposure also changes due to the adjustment of the aperture amount and the adjustment using the neutral density filter, the short exposure time T1 is extended in the same manner as described above according to the adjustment.
In step S9, short exposure and long exposure are continuously performed by a rolling shutter system.
When the short exposure time T1 and the long exposure time T2 are changed (NO in step S6), the respective exposures are performed with the short exposure time T1 ′ and the long exposure time T2 ′. When only the long exposure time T2 is changed without changing the short exposure time T1 (in the case of YES at step S6), the respective exposures are performed with the short exposure time T1 and the long exposure time T2 ′. When neither the short exposure time T1 nor the long exposure time T2 is changed (NO in step S4), the respective exposures are performed with the short exposure time T1 and the long exposure time T2.
In step S10, the short exposure image (first image) exposed at the short exposure time T1 (T1 ′) and the long exposure image (second image) exposed at the long exposure time T2 (T2 ′) are combined. Processing operations are performed.
The processing operation will be described below based on the flowchart shown in FIG.

In step S11, as shown in FIG. 5, the brightness of each of the short exposure image exposed at the short exposure time T1 (T1 ′) and the long exposure image exposed at the long exposure time T2 (T2 ′) is examined.
In step S12, the short exposure image is subjected to gain processing to match the brightness of the long exposure image.
In step S13, it is determined whether or not there is a luminance saturation portion by specifying a portion where saturation (saturation) has occurred in the long exposure image. If NO, the process proceeds to step S15, and if YES, the process proceeds to step S14.
In step S14, a synthesis process is performed in which the luminance saturation portion of the long exposure image identified in step S13 is replaced with a short exposure image portion (image portion corresponding to the luminance saturation portion) subjected to gain processing.

  In step S15, it is determined whether or not the ISO sensitivity is the minimum value with reference to the determination result in step S6. If NO, the process proceeds to step S16, and if YES, the process ends.

In step S16, the long exposure image portion that has not been replaced in step S14 is bit-shifted.
Here, the reason why the bit shift process of step S16 is performed will be described.
If YES is determined in step S6 and the ISO sensitivity is not adjusted, the short exposure image used for composition in step S14 is an image exposed for a short exposure time T1, and thus the short exposure is performed in the composition process. The long exposure image portion replaced with the image reproduces the dynamic range intended by the photographer.

  However, the long-exposure image portion that has not been replaced uses the long-exposure image exposed at the long exposure time T2 ′ extended to fill the gap G2, and this portion is used as the dynamic image intended by the photographer. The range is not reproduced.

That is, in the long exposure, the extra exposure is performed for the time of the long exposure time T2′−the long exposure time T2, and therefore, the process that the photographer intends by cutting the extra exposure information, for example, bit Shift processing is required.
<Bit shift processing>
For example, when the original long exposure time T2 is 1/80 second, the exposure is performed with a long exposure time T2 ′ of about 1/20 second (one frame transfer time−one pixel row transfer time) in accordance with one frame transfer time. As a result, the exposure time is about four times longer, so that information is amplified about four times and appears in the long exposure image portion that has not been replaced with the short exposure image.
Therefore, for the pixels included in this image portion, the pixel output value data of each pixel is bit-shifted by 2 bits to make it a quarter.

With this process, even if YES is determined in step S6 and the ISO sensitivity is not adjusted, the image portion can be made substantially the same as the one exposed with the long exposure time T2, and the dynamic range intended by the photographer can be set. A reflected image can be generated.
Note that when the ISO sensitivity is adjusted to be lowered in step S7, the exposure amount of the long exposure after the extension is the same as the exposure amount of the long exposure before the extension, so this bit shift processing is not performed (in step S15). If yes).
As shown in FIG. 6B, when the long exposure time T2 is calculated to be longer than the predetermined time T2 ′ (one frame transfer time−one pixel row transfer time) in step S4, a specific example is a long time. When the exposure time T2 is calculated as 1/10 second, the long exposure time T2 is longer than about 1/20 second of the predetermined time T2 ′ (one frame transfer time−one pixel row transfer time), so the gap G2 Does not occur and the exposure time is not extended. That is, in this case, since the long exposure is performed with the long exposure time T2, this bit shift processing is not performed (NO in step S4).
Next, functions and effects of the present invention will be described.
According to the first aspect of the present invention, when the long exposure time T2 calculated based on the appropriate exposure time Ts calculated by the control device 25 is shorter than the predetermined time T2 ′ (one frame transfer time−one pixel row transfer time). However, since the long exposure is performed by changing the long exposure time T2 to the predetermined time T2 ′ (one frame transfer time−one pixel row transfer time), a temporal gap G2 is generated between the short exposure and the long exposure. The short exposure and the long exposure can be performed substantially continuously.
According to the invention of claim 2, the long exposure time is adjusted by adjusting at least one of the aperture amount of the mechanical shutter 17 of the imaging optical system 5, the light reduction amount of the neutral density filter, and the analog gain of the CMOS sensor 16. The exposure amount of the long exposure after the change of T2 can be made the same as the exposure amount before the change.
Moreover, the exposure amount of the short exposure reduced by the adjustment can be made the same as the exposure amount of the short exposure before the adjustment by extending the short exposure time T1.
For this reason, each of the short exposure and the long exposure becomes the same as the short and long exposure amounts calculated by the appropriate exposure time Ts, and a composite image having a dynamic range intended by the photographer can be obtained.
According to the invention of claim 3, the short exposure image obtained by exposure with the short exposure time T1 (T1 ′) is subjected to the processing for increasing the gain, and the short exposure image is exposed with the long exposure time T2 (T2 ′). And the same as the luminance of the long-exposure image obtained in this way, and a portion in which the luminance is saturated in the long-exposure image is specified, and this portion is replaced with the portion of the short-exposure image corresponding to this portion. Therefore, even if there is a region where luminance saturation occurs in the long exposure image, the subject image corresponding to this region can be reproduced, and the dynamic range intended by the photographer can be reproduced in this region.
According to the invention of claim 4, when the above-described various adjustments (ISO sensitivity adjustment, ND filter adjustment, mechanical shutter 17 diaphragm adjustment) cannot be performed, an image portion whose luminance is not saturated in the long exposure image is obtained. For the included pixels, bit shift processing is performed on the bits of each pixel output value data, so that even if the above-mentioned various adjustments cannot be made, the excess image information of the long exposure image is cut to extend the exposure amount of the extended long exposure. The exposure amount of the previous long exposure can be made the same, and the dynamic range intended by the photographer can be reproduced in this portion.
As described above, the imaging apparatus according to the present invention has been described based on the above embodiments. However, the specific configuration is not limited to these embodiments, and the gist of the invention according to each claim of the claims. As long as they do not deviate, design changes and additions are permitted.

5 Shooting lens system (imaging optical system)
16 CMOS sensor
25 Control device Ts Proper exposure time T1 Short exposure time (exposure time)
T2 Long exposure time (exposure time)
100 Digital camera (imaging device)
L1 1 frame transfer time
PxL1L 1 pixel row transfer time

JP 2002-305684 A JP 2002-156138 A

Claims (4)

An imaging optical system, a CMOS sensor having a rolling shutter function, and control means for controlling the CMOS sensor;
The control unit calculates an appropriate exposure time for the subject from the image data of the subject output from the CMOS sensor, calculates a short exposure time and a long exposure time based on the appropriate exposure time, and a rolling shutter. In an imaging apparatus that generates an image with a wide dynamic range by combining the short exposure image and the long exposure image obtained by the exposure with the exposure time of each of the short exposure time and the long exposure time in a method,
The control means calculates a time obtained by subtracting a transfer time of one pixel row from one frame transfer time of the CMOS sensor, compares the calculated time with the long exposure time, and calculates the long exposure time. An imaging apparatus characterized by changing the long exposure time to the calculated time when the time is shorter than the time.   The control means adjusts at least one of a diaphragm amount of the imaging optical system, a light reduction amount of a neutral density filter, and an analog gain of the CMOS sensor, and increases the long exposure time that is increased by changing the long exposure time. The exposure amount is the same as the exposure amount before the change, and the short exposure time is extended so that the exposure amount of the short exposure reduced by this adjustment becomes the same as the exposure amount before the reduction. The imaging apparatus according to 1.   The first image obtained by the exposure with the short exposure time is processed to increase the gain so that the luminance of the first image is the same as the luminance of the second image obtained by the exposure with the long exposure time. The imaging apparatus according to claim 2, wherein a combining process is performed in which a portion where luminance is saturated in the second image is specified and the portion is replaced with the first image portion corresponding to the portion.   When the increased exposure amount of the long exposure cannot be made the same as the exposure amount before the change by the adjustment, the pixel output value data is bit-shifted for the pixels included in the image portion where the luminance is not saturated in the second image. 4. The image pickup apparatus according to claim 3, wherein the pixel output value of the image portion is decreased to make the image portion the same as that exposed for the long exposure time before the change.

Images