JP2014053777A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress image distortion by divisional reading of charges from a solid-state imaging element, to suppress a spread damage and to obtain a natural progress image.SOLUTION: A digital camera 100 includes: an imaging element 14; an image display section 28 which displays an image obtained in the imaging element 14; and a system control section 50 which entirely controls a rolling shutter for clearing and reading charges accumulated in the imaging element 14 in a line unit, and a global shutter for clearing and reading all the charges accumulated in the imaging element 14 in the digital camera 100. The system control section 50 captures the image by using the global shutter when exposure of the imaging element 14 is started and terminated, and by using the rolling shutter during reading in the middle of exposure from the imaging element 14.

Description

  The present invention relates to an image pickup apparatus including a solid-state image pickup device, and more particularly to divided read control of charges accumulated in a solid-state image pickup device.

  Imaging devices such as home-use digital video cameras and digital still cameras are spreading. As a solid-state image pickup device used in such an image pickup device, a CCD image pickup device or a CMOS image pickup device is used. In recent years, an image pickup device using a CMOS image pickup device has become mainstream.

  The reason is that the CMOS image sensor can provide an advantageous function in continuous shooting and moving image shooting since it can be read at a higher speed than the CCD image sensor. In addition, unlike the CCD image sensor, the CMOS image sensor does not generate smear because of its structure. Therefore, it is possible to check the progress of accumulation of long-second shooting by reading out the charge accumulated during shooting for long seconds and performing digital addition using this characteristic. It is a convenient shooting function.

  However, the CMOS image pickup device has a problem that spread damage is likely to occur due to the element structure, that is, there is a problem that charge accumulation due to dark current is saturated during long-second accumulation and peripheral pixels spread. In order to avoid this problem, it is desirable to obtain a long-second shot image in which spread flaws are suppressed by dividing the exposure time and performing halfway reading and performing digital addition during long-second shooting. In view of this, a technique has been proposed for making it possible to easily view a halfway progress image at the time of long-time shooting (see, for example, Patent Document 1).

JP 2010-81313 A

  However, if divided readout is performed when long-time shooting is performed with an imaging apparatus equipped with a CMOS imaging device, the exposure timing is shifted within the screen. Therefore, there is a problem that the subject image and the background are distorted when a moving subject is present in the screen or when panning is performed.

  It is an object of the present invention to provide an imaging device that can suppress image distortion due to divided readout of charges from a solid-state imaging device, minimize a spread scar, and obtain a natural halfway image.

  An image pickup apparatus according to the present invention includes an image pickup unit having an image pickup element that converts an optical image into an electrical signal, a display unit that displays an image obtained from the image pickup unit, and clearing of a charge signal accumulated in the image pickup element. Rolling shutter means that sequentially performs readout in line units, global shutter means that collectively clears and reads out charge signals of all lines accumulated in the image sensor, and exposure start operation and exposure of the image sensor Control means for controlling the end operation using the global shutter means and controlling the readout operation of the charge signal during the exposure of the image sensor using the rolling shutter means. And

  According to the present invention, it is possible to suppress the image distortion when the divided reading is performed at the time of long-second shooting using the CMOS image sensor and the progress is previewed, and the spread damage can be minimized. Thereby, a natural midway progress image can be obtained.

1 is a block diagram illustrating a schematic configuration of a digital camera according to an embodiment of the present invention. It is a flowchart which shows the imaging | photography process sequence with the digital camera which concerns on this embodiment. It is a figure which shows typically the aspect of the division | segmentation exposure in the digital camera which concerns on this embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present embodiment, description will be made by taking a digital camera using a CMOS image sensor as a solid-state image sensor as an example of the image pickup apparatus.

<Schematic configuration of digital camera>
FIG. 1 is a block diagram showing a schematic configuration of a digital camera 100 according to an embodiment of the present invention. The digital camera 100 includes a photographing lens 10, a mechanical shutter 12 having a diaphragm function, a CMOS image sensor 14 that converts an optical image into an electric signal, and an analog signal output from the CMOS image sensor 14 that is converted into a digital signal. D converter 16.

  In addition, the digital camera 100 includes a timing generation circuit 18 that supplies a clock signal and a control signal to the CMOS image sensor 14 and the A / D converter 16, and the timing generation circuit 18 includes a memory control circuit 22 and system control described later. Controlled by circuit 50. In the digital camera 100, in addition to exposure control of the CMOS image sensor 14 by the mechanical shutter 12, the charge accumulation time can be controlled by an electronic shutter function that controls the reset timing of the CMOS image sensor 14 by the timing generation circuit 18. This electronic shutter function is used when performing moving image shooting or the like.

  Further, the digital camera 100 includes an image processing circuit 20, a memory control circuit 22, an image display unit 28, a memory 30, a nonvolatile memory 31, and a compression / decompression circuit 32.

  The image processing circuit 20 performs pixel interpolation processing and color conversion processing on the data acquired from the A / D converter 16 and the data acquired from the memory control circuit 22. Further, the image zooming process is performed by the image processing circuit 20 to realize an electronic zoom function. Further, the image processing circuit 20 performs predetermined arithmetic processing on the captured image data. Based on the calculation result thus obtained, the system control circuit 50 controls an exposure control unit 40 and a distance measurement control unit 42 which will be described later, TTL AF (autofocus) processing, AE (automatic exposure) processing, EF (Electronic flash) processing is performed. The image processing circuit 20 performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

  The memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the memory 30, and the compression / decompression circuit 32. Data output from the A / D converter 16 is written into the memory 30 via the image processing circuit 20 and the memory control circuit 22 or directly via the memory control circuit 22.

  The image display unit 28 is, for example, a liquid crystal display (LCD) such as a TFT, and the display image data written in the memory 30 is displayed on the image display unit 28 via the memory control circuit 22. The captured image data is sequentially displayed on the image display unit 28, thereby realizing an electronic viewfinder function. The image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50. By turning off the display, the power consumption of the digital camera 100 can be greatly reduced. It becomes possible to reduce.

  The memory 30 is a storage medium for storing captured still images and moving images, and has a sufficient storage capacity for storing a predetermined number of still images and moving images for a predetermined time. Thereby, even in the case of continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, a large amount of image writing can be performed on the memory 30 at high speed. The memory 30 is also used as a work area for the system control circuit 50.

  The nonvolatile memory 31 is, for example, a flash ROM. The program code executed by the system control circuit 50 is written in the nonvolatile memory 31, and the system control circuit 50 executes the program code while reading the program code from the nonvolatile memory 31 sequentially. The nonvolatile memory 31 is provided with an area for storing system information and user setting information of the digital camera 100, and the system control circuit 50 reads and restores the information at the next startup. As a result, the user is relieved from the work of adjusting to the desired setting condition at each activation. The compression / decompression circuit 32 reads image data stored in the memory 30 by adaptive discrete cosine transform (ADCT) or the like, performs compression processing or decompression processing, and writes the processed data in the memory 30.

  The digital camera 100 further includes an exposure control unit 40, a distance measurement control unit 42, a zoom control unit 44, a strobe 48, and a system control circuit 50.

  The exposure control unit 40 controls the mechanical shutter 12 having an aperture function, and realizes a flash light control function in conjunction with the flash 48. The distance measurement control unit 42 controls the focusing of the photographing lens 10. The zoom control unit 44 controls zooming of the taking lens 10. The strobe 48 has an AF auxiliary light projecting function and a strobe dimming function. The exposure control unit 40 and the distance measurement control unit 42 are controlled by using the TTL method, and the system control circuit 50 performs the exposure control unit 40 and the distance measurement control unit 40 based on the calculation result obtained by calculating the captured image data by the image processing circuit 20. Control is performed for the distance measurement control unit 42. The system control circuit 50 performs overall control of the entire digital camera 100.

  The digital camera 100 includes operation means configured by a single or a plurality of combinations such as a switch, a dial, a touch panel, pointing by gaze detection, a voice recognition device, etc. for a user to give various operation instructions to the system control circuit 50. Prepare. In the present embodiment, as an example, a mode dial 60, a shutter switch 62, a display changeover switch 64, an operation unit 70, and a zoom switch 72 will be described.

  The mode dial 60 is used to set each function mode such as an automatic shooting mode, a shooting mode, a panoramic shooting mode, a moving image shooting mode, a playback mode, and a PC connection mode. The shutter switch 62 includes switches SW1 and SW2. The switch SW1 is turned on during the operation of the shutter switch 62 (half-pressed state), and instructs to start operations such as AF processing, AE processing, and AWB processing. When the shutter switch 62 is completely pressed, the switch SW2 is turned on and the mechanical shutter 12 operates. At this time, when performing strobe shooting, after performing the strobe pre-flash processing, the image sensor 14 is exposed for the exposure time determined by the AE processing, and the strobe 48 is caused to emit light during the exposure period. Simultaneously with the end of the exposure time, the exposure controller 40 blocks light.

  When exposure to the image sensor 14 is thus completed, an analog signal is output from the image sensor 14 to the A / D converter 16, and the A / D converter 16 converts the analog signal into a digital signal. The digital signal output from the A / D converter 16 is written into the memory 30 via the memory control circuit 22, or after being subjected to predetermined image processing by the image processing circuit 20, via the memory control circuit 22. It is written in the memory 30. Further, the compression / decompression circuit 32 reads out image data from the memory 30 and performs compression processing, and the compressed image data is written in a recording medium 202 described later.

  The display changeover switch 64 performs display switching of the image display unit 28. Power can be saved by cutting off the power supply to the image display unit 28 when photographing using the optical viewfinder 104. The operation unit 70 includes various buttons, a touch panel, a rotary dial, and the like. Specifically, the operation unit 70 includes a menu button, a set button, a macro button, a multi-screen playback page break button, a strobe setting button, and a single shooting / continuous shooting / self-timer switching button. Further, the operation unit 70 includes a menu movement + (plus) button, a menu movement-(minus) button, a reproduction image movement + (plus) button, a reproduction image movement-(minus) button, and a photographing image quality selection button. , An exposure compensation button, a date / time setting button, and the like.

  The zoom switch 72 is a zoom operation means for a user to give an instruction to change the magnification of a captured image. The zoom switch 72 includes a tele switch that changes the captured field angle to the telephoto side and a wide switch that changes the wide angle side. The operation of the zoom switch 72 instructs the zoom control unit 44 to change the imaging field angle of the photographing lens 10 and serves as a trigger for performing an optical zoom operation. The operation of the zoom switch 72 also serves as a trigger for electronic zooming change of the imaging angle of view by image cutting by the image processing circuit 20 or pixel interpolation processing.

  The digital camera 100 includes a power supply unit 86, an interface (I / F) 90, a connector 92, and a recording unit 200. The recording unit 200 includes a recording medium 202, an interface (I / F) 204, and a connector 206.

  The power supply unit 86 is a power supply unit including, for example, a primary battery such as an alkaline battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like. By connecting the connector 92 and the connector 206, communication between the recording medium 202 and the system control circuit 50 becomes possible via the I / F 90 and the I / F 204. The recording medium 202 is, for example, a memory card (semiconductor memory), a magnetic disk, a hard disk, or the like, and may be built in the main body of the digital camera 100 or detachable from the main body.

  The digital camera 100 includes an optical viewfinder 104, a communication unit 110, and a connector 112. Without using the electronic viewfinder function of the image display unit 28, the user can check the subject through the optical viewfinder 104 and take a picture. The communication unit 110 has various communication functions such as USB, IEEE 1394, LAN, and wireless communication. The connector 112 connects the digital camera 100 to other devices via the communication unit 110. When wireless communication is performed by the communication unit 110, the connector 112 serves as an antenna.

<Shooting process flow>
FIG. 2 is a flowchart showing a photographing processing procedure in the digital camera 100. When the shutter switch 62 is pressed, the system control circuit 50 starts a photographing operation. Note that the shutter speed follows a time determined in advance using the exposure control unit 40 or a time arbitrarily designated by the user.

  The system control circuit 50 calculates the number of exposure divisions according to the following [Equation 1] based on the shutter speed (step S101).

Number of divisions = shutter speed / division exposure unit time [Equation 1]
Here, the divided exposure unit time is an exposure time for one divided exposure, and is set individually according to the characteristics of the CMOS image sensor 14. For example, when the shutter speed is 4 seconds and the divided exposure unit time is 1 second, the number of divisions = 4 seconds / 1 second = 4. When a remainder is generated as a result of the calculation according to [Expression 1] (that is, when a fractional part is generated in the quotient), exposure is performed in a time shorter than the divided exposure unit time in the last division.

  Next, the system control circuit 50 performs a global reset operation by the electronic shutter function on the CMOS image sensor 14 in order to start exposure, and simultaneously collects the accumulated charges in the pixels of all the rows of the CMOS image sensor 14. Clear (step S102). Subsequently, the system control circuit 50 starts a charge accumulation operation by exposure of the CMOS image sensor 14 (step S103), and then determines whether or not it is the last exposure of the divided exposure (step S104).

  If it is not the final exposure of the divided exposure (NO in S104), the system control circuit 50 sequentially reads out the accumulated charges in each row of the CMOS image sensor 14 by the rolling shutter drive in units of lines (step S105). In other words, the image is read by reading the accumulated charge signal of the CMOS image sensor 14 during the exposure in units of lines.

  Next, the system control circuit 50 performs image integration processing by sequentially digitally adding the images based on the accumulated charge signals read by the rolling shutter drive (step S106). The accumulated image thus obtained is an unnatural image with different brightness at the top and bottom of the screen because the exposure time (charge accumulation time) differs at the top and bottom of the screen at this point. Therefore, the system control circuit 50 corrects the luminance of the integrated image (step S107), and displays the display image with uniform luminance obtained thereby on the image display unit 28 (step S108). Details of step S107 will be described later with reference to FIG. After step S108, the system control circuit 50 returns the process to step S103. That is, the operations in steps S103 to S108 are repeated until the last exposure of the divided exposure.

  If the system control circuit 50 determines in step S104 that the exposure is the last of the divided exposures (YES in S104), it closes the mechanical shutter 12 and puts the CMOS image sensor 14 in a light shielding state (step S109). The light shielding operation of the CMOS image sensor 14 by the mechanical shutter 12 in step S109 substantially serves as a global shutter. That is, the charge accumulation in the pixels in all the rows of the CMOS image sensor 14 is finished simultaneously. Subsequently, the system control circuit 50 sequentially reads the accumulated charges in each row of the CMOS image sensor 14 line by line in the same manner as in step S105 (step S110), and performs image integration processing in the same manner as in step S106. (Step S111), and the photographing operation is terminated.

<Details of luminance correction of integrated image (step S107)>
Details of the luminance correction of the integrated image executed in step S107 will be described with reference to FIG. FIG. 3 is a diagram schematically showing an aspect of divided exposure in the digital camera 100. FIG. 3A shows the relationship between the division unit exposure time and the division readout. When exposure is started, a global reset operation is performed by the electronic shutter function of the CMOS image sensor 14, and after exposure for the division unit exposure time, a rolling shutter is performed. First division reading is performed by driving. At this time, the exposure time (charge accumulation time) is different between the upper part of the image and the lower part of the screen, and the brightness is low because the exposure time is short in the upper part of the image, and the brightness is high because the exposure time is longer toward the lower part of the screen. Yes. Therefore, if an image acquired in this state is preview-displayed as an intermediate progress image on the image display unit 28, an unnatural image with non-uniform luminance in the screen is generated. Therefore, correction to make the luminance uniform is performed.

  FIG. 3B is a diagram schematically showing the correction contents for the exposure time of the first divided readout. After the division unit exposure time (Tunit) has elapsed, line-by-line readout is performed by rolling shutter drive. The reading time required for this reading is “Tread”. In this case, the exposure time of the top line of the screen is “Tunit”, but the exposure time of the bottom line of the screen is “Tunit + Tread”. In order to correct this, if the correction gain magnification of the luminance of the bottom line of the screen (the bottom line of the image sensor 14) is “x1.0 (that is, equal magnification)”, The correction gain magnification (Gain) of the luminance of the upper line (the uppermost line of the image sensor 14) is obtained by the following [Equation 2].

Gain = (1.0 + Tread / Tunit) [Formula 2]
Note that the correction gain magnification of the brightness of the intermediate line between the top and bottom of the screen may be determined by linearly interpolating the correction gain magnifications at the top and bottom of the screen. The correction gain magnification of the brightness of the intermediate line may be determined for each line or may be determined for each of a plurality of lines.

  As described above, in the present embodiment, luminance correction is performed for each division readout and a display image is obtained. Therefore, the difference in exposure time in the screen is eliminated by reading out and integrating the last exposure division. . As a result, luminance correction after the last reading (step S110) and image integration (step S111) becomes unnecessary, image distortion due to rolling shutter driving can be suppressed, and spread flaws can be minimized. Thereby, a natural midway progress image can be obtained.

<Other embodiments>
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. For example, in the above-described embodiment, the luminance correction gain magnification of the lowermost line on the screen is set to “x1.0” and is increased toward the top of the screen. However, the present invention is not limited to this, and the correction gain magnification of the brightness of the line in the center of the screen is set to “x1.0”, and the correction gain magnification is increased from the center of the screen toward the top of the screen, and from the center of the screen toward the bottom of the screen. The correction gain magnification may be reduced. Alternatively, the correction gain magnification of the luminance of the line on the top of the screen may be set to “x1.0”, and the correction gain magnification may be lowered toward the bottom of the screen. That is, “α1 to α3”, “β1 to β3”, and “γ1 to γ3” shown in FIG. 3A indicate the luminance correction gain magnifications. Various settings can be made without departing from the gist of the invention.

  In the above embodiment, the electronic shutter is used for the exposure start operation, and the mechanical shutter 12 is used for the exposure end operation. However, the present invention is not limited to this, and the mechanical shutter 12 may be used for the exposure start operation and the exposure end operation, or a mechanism similar to the global shutter operation such as a focal plane shutter may be used.

  Furthermore, regarding the number of exposure divisions, in the above-described embodiment, the exposure time is divided by the division unit exposure time. However, the present invention is not limited to this, and an appropriate division unit exposure time may be calculated each time according to the characteristics of the image sensor 14 or the shooting scene, or the user may specify the number of divisions. Further, the reading may be performed with non-uniform division times.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

DESCRIPTION OF SYMBOLS 10 Shooting lens 12 Mechanical shutter 14 Image sensor 18 Timing generation circuit 28 Image display part 40 Exposure control part 50 System control circuit 100 Digital camera (imaging apparatus)

Claims (7)

An imaging means having an imaging device for converting an optical image into an electrical signal;
Display means for displaying an image obtained from the imaging means;
Rolling shutter means for sequentially clearing and reading out the charge signals accumulated in the image sensor in line units;
Global shutter means for collectively clearing and reading out charge signals of all lines accumulated in the image sensor;
The exposure start operation and the exposure end operation of the image sensor are controlled to be performed using the global shutter unit, and the charge signal reading operation during the exposure of the image sensor is controlled to be performed using the rolling shutter unit. And an imaging device.   The imaging apparatus according to claim 1, wherein the global shutter used for the exposure start operation of the imaging element is an electronic shutter.   2. The image pickup apparatus according to claim 1, wherein the global shutter used for the exposure end operation of the image pickup element is a mechanical shutter.   The control means integrates images based on the charge signal read out during the exposure, and displays the integrated image accumulated every time of reading on the screen of the display means, and determines a predetermined exposure time and the imaging element. The imaging apparatus according to claim 1, wherein brightness correction of the integrated image is performed based on a readout time.   The control means sets the brightness correction gain magnification of the bottom line of the screen in the display means to the same magnification, and increases the brightness correction gain magnification toward the top of the screen. 4. The imaging device according to 4.   The control means sets the luminance correction gain magnification of the top line of the screen in the display means to the same magnification, and lowers the luminance correction gain magnification toward the lower portion of the screen. 4. The imaging device according to 4.   The control means sets the brightness correction gain magnification of the center line of the screen in the display means to the same magnification, increases the brightness correction gain magnification from the center line toward the top of the screen, and starts from the center line. The imaging apparatus according to claim 4, wherein a luminance correction gain magnification is lowered toward a lower portion of the screen.

Images