JP2009246757A - Imaging device, imaging method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve imaging in an excellent exposure state over the entire image while reducing the load of readout processing for an image signal from an imaging element. <P>SOLUTION: An imaging device includes a CMOS image sensor 13 which accumulates electric charges in pixel units according to a subject image made incident on an imaging plane, and outputs an image signal corresponding to the amount of accumulated electric charges, a control unit 17 which obatains exposure amounts by pixels of the image sensor 13 through an image processing unit 14, adjusts exposure times by the pixels of the image sensor 13 in one exposure period according to the obtained exposure amounts by the pixels, and drives through a CMOS driver 23 to obtain an image signal from the image sensor 13, a main memory 18 and a program memory 19, and a buffer memory 15 which stores an image signal of one screen obtained from the image sensor 13. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program suitable for a digital camera using a CMOS image sensor, for example.

Conventionally, a technique has been conceived in which an electronic still camera performs a non-destructive readout from an image sensor a plurality of times during one exposure and performs image composition to obtain a photographed image with more favorable exposure conditions. (For example, Patent Document 1)
Japanese Patent Laid-Open No. 10-173988

  In the technique described in the above-mentioned Patent Document 1, an image with different exposure conditions is partially adopted for each of a plurality of areas in the screen, and the entire screen is obtained during one exposure in order to obtain good exposure conditions over the entire image. Repeatedly perform nondestructive read operation over multiple times. For this reason, there is a problem that a processing load related to signal readout is heavy.

  The present invention has been made in view of the above circumstances, and its purpose is to capture images in a well-exposed state over the entire image while reducing the burden associated with image signal readout processing from the image sensor. It is an object to provide an imaging device, an imaging method, and a program that can be realized.

  According to a first aspect of the present invention, there is provided an imaging device that accumulates charges in units of pixels according to a subject image incident on the imaging surface and outputs an image signal corresponding to the accumulated charge amount, and an imaging surface of the imaging device. An exposure amount acquisition means for acquiring an exposure amount for each of a plurality of areas to be divided, and an exposure amount for each of the plurality of areas obtained by the exposure amount acquisition means, for each of a plurality of areas of the image sensor in one exposure period. And an exposure control means for adjusting the exposure time and driving to obtain an image signal from the image sensor, and a storage means for storing an image signal for one screen obtained from the image sensor.

  According to a second aspect of the present invention, in the first aspect of the invention, the exposure amount acquisition unit is preset after the exposure control unit starts a single exposure of a plurality of areas of the image sensor simultaneously. The exposure amount is obtained by nondestructive readout that does not inhibit charge accumulation based on exposure in the imaging device at timing.

  According to a third aspect of the present invention, in the first aspect of the present invention, the exposure amount acquisition means starts the exposure of a plurality of areas of the image sensor simultaneously by the exposure control means, and then the image sensor at a constant time period. The exposure amount for each of the plurality of areas is obtained by non-destructive reading that does not inhibit charge accumulation based on exposure at the exposure amount, and the exposure control unit is configured to obtain an exposure amount based on the exposure amount for each of the plurality of areas obtained by the exposure amount acquisition unit. An image signal is output from the area at a timing when the value reaches a predetermined value.

  According to a fourth aspect of the present invention, in the first aspect of the present invention, the exposure amount acquisition unit divides the image sensor at the time of pre-exposure prior to the exposure operation of the main photographing for adjusting the exposure time by the exposure control unit. A plurality of areas are exposed at the same time for a preset time to obtain an image signal from the image sensor, and an exposure amount for each area is obtained. The exposure time is adjusted by varying the timing to start exposure for each of the plurality of areas during the exposure for the main shooting according to the exposure amount, and the timing for obtaining the image signal corresponding to the amount of charge accumulated from the plurality of areas is substantially the same. It is characterized by making it.

  According to a fifth aspect of the present invention, in the first aspect of the invention, a shutter mechanism that controls the opening and closing of an object image incident on the imaging element, and the imaging that is performed immediately before photographing exposure by the exposure control means by the shutter mechanism. Measuring means for measuring the dark current for each of the plurality of areas of the image pickup element in a state where the incident on the element is blocked, and correcting the image signal stored in the storage means for each of the plurality of areas based on the measurement result of the measuring means. And correction means for further comprising.

  According to a sixth aspect of the invention, there is provided the shutter according to the fourth aspect of the invention, further comprising a shutter mechanism that controls opening and closing of the subject image incident on the image sensor, wherein the exposure control means includes a plurality of areas of the image sensor. In accordance with the output timing of the image signal from the image sensor, the shutter mechanism blocks the incidence on the image sensor.

  The invention described in claim 7 further comprises exposure time determining means for determining an exposure time in at least one area of the image sensor according to the invention described in claim 2, wherein the exposure amount acquiring means includes the exposure time. The exposure amount is obtained by the non-destructive reading at a timing earlier than the exposure time end timing determined by the determining means.

  The invention described in claim 8 is characterized in that, in the invention described in claim 1, the plurality of areas are in units of pixels constituting an imaging surface of the imaging element.

  According to a ninth aspect of the present invention, in the first aspect of the invention, the exposure control means limits the maximum exposure amount in one exposure period in accordance with the light receiving capacity of the image sensor. And

  According to a tenth aspect of the present invention, in the first aspect of the invention, the exposure control means causes the exposure time of at least one area on the side with a larger exposure amount to correspond to the light receiving capacity of the imaging element. To do.

  The invention described in claim 11 is the invention described in claim 1, further comprising brightness adjusting means for adjusting the brightness of the image signal stored in the storage means for each of a plurality of areas.

  The invention described in claim 12 is the invention described in claim 11, wherein the brightness adjusting means adjusts the brightness of an image signal for each of a plurality of areas based on an exposure time and an exposure amount.

  According to a thirteenth aspect of the present invention, in the first aspect of the invention, the exposure control means corresponds to the exposure amount distribution for each of the plurality of areas obtained by the exposure amount acquisition means. The exposure time is adjusted.

  According to a fourteenth aspect of the present invention, in the first aspect of the invention, the exposure control means sets the longest exposure time corresponding to the distribution of the exposure amount for each of the plurality of areas obtained by the exposure amount acquisition means. It is characterized by that.

  A fifteenth aspect of the invention is characterized in that, in the first aspect of the invention, the exposure control means resets charge accumulation before and after exposure for each of a plurality of areas of the image sensor.

  According to a sixteenth aspect of the present invention, in the first aspect of the invention, the storage unit stores the image signal and the exposure time for each of the plurality of areas in association with each other.

According to a seventeenth aspect of the present invention, there is provided an imaging method in an imaging apparatus including an imaging element that accumulates charges in units of pixels according to a subject image incident on an imaging surface and outputs an image signal corresponding to the accumulated amount of charges. An exposure amount acquisition step for acquiring an exposure amount for each of a plurality of areas that divide the imaging surface of the image sensor, and an exposure amount for each of the plurality of areas obtained in the exposure amount acquisition step. An exposure control step of adjusting and driving an exposure time for each of a plurality of areas of the image sensor during a period to obtain an image signal from the image sensor, and a storage step of storing an image signal for one screen obtained from the image sensor It is characterized by having.

  According to an eighteenth aspect of the present invention, there is provided a computer built in an image pickup apparatus having an image pickup element that accumulates charges in units of pixels in accordance with a subject image incident on the image pickup surface and outputs an image signal corresponding to the accumulated charge amount. 1 is a program for obtaining an exposure amount for each of a plurality of areas dividing the imaging surface of the image sensor, and an exposure amount for each of the plurality of areas obtained in the exposure amount acquisition step. The exposure control step for adjusting and driving the exposure time for each of the plurality of areas of the image sensor during the exposure period and obtaining an image signal from the image sensor, and storing the image signal for one screen obtained from the image sensor The storage step is performed by a computer.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to implement | achieve the imaging of a favorable exposure state over the whole image, reducing the burden concerning the read-out process of the image signal from an image sensor.

An embodiment in which the present invention is applied to a digital camera will be described below with reference to the drawings. In the present embodiment, a CMOS (Complementary Metal Oxide Semiconductor) image sensor is used as the image sensor.
FIG. 1 shows a circuit configuration of a digital camera 10 according to the present embodiment. In the figure, an optical image of a subject is formed on the imaging surface of a CMOS image sensor 13 by a lens optical system 11 via a mechanical shutter 12.

  An image signal obtained by imaging with the CMOS image sensor 13 in the monitor state is sent to the image processing unit 14, and after performing correlation square sampling, automatic gain adjustment, and A / D conversion processing, pixel interpolation processing and γ correction processing are performed. Are processed and temporarily stored in the buffer memory 15.

  The image data held in the buffer memory 15 is read out to the image processing unit 14, sent to the display unit 16, and displayed as a monitor image.

  The control unit 17 performs overall control of the above operations. The control unit 17 is composed of a CPU, and is connected to a main memory 18 and a program memory 19. The main memory 18 is composed of SDRAM (synchronous DRAM). The program memory 19 is composed of a non-volatile memory that stores an operation program, data, and the like including control in a shooting mode to be described later.

  The control unit 17 reads out necessary programs, data, and the like from the program memory 19 and manages the entire digital camera 10 while temporarily expanding and storing them in the main memory 18 as appropriate.

  The control unit 17 executes various control operations in response to a key operation signal directly input from the key operation unit 20, and in addition to the image processing unit 14 and the display unit 16 through the system bus SB, a lens. The optical system drive unit 21, flash drive unit 22, CMOS driver 23, memory card 24, and audio processing unit 25 are also connected.

  The key operation unit 20 includes, for example, a power key, a shooting mode key, a playback mode key, a menu key, a cursor key, a set key, a scene program key, and the like.

  The lens optical system drive unit 21 receives the control signal from the control unit 17 and controls the rotation of the lens motor (M) 26, and a part of the lenses constituting the lens optical system 11, specifically, focus. The positions of the lens and the zoom lens are moved along the optical axis direction.

  In addition, the lens optical system driving unit 21 rotationally drives a shutter motor 27 constituted by a stepping motor based on a control signal from the control unit 17 to perform opening / closing control of the mechanical shutter 12.

  The mechanical shutter 12 is placed in a fully closed state in synchronization with the closing timing of the electronic shutter of the CMOS image sensor 13 during still image shooting, thereby avoiding further exposure operations being suspended.

  The flash drive unit 22 receives a control signal from the control unit 17 at the time of still image shooting and drives the flash unit 28 composed of a plurality of white high-intensity LEDs to be lit in synchronization with the shooting timing.

  The memory card 24 is a memory for recording image data or the like that is detachably attached to the digital camera 10 via the card connector C.

  The sound processing unit 25 includes a sound source circuit such as a PCM sound source, and at the time of recording sound, the sound processing unit 25 digitizes the sound signal input from the microphone unit 29 disposed on the front surface of the digital camera 10, and has a predetermined data file format. For example, the audio data file is created by compressing the data in the AAC (moving picture experts group-4 Advanced Audio Coding) format, and sent to the memory card 24.

  On the other hand, the audio processing unit 25 uncompresses and converts the audio data file read from the memory card 24 at the time of audio reproduction into an analog signal, and drives the speaker unit 30 provided on the back of the digital camera 10 to release the sound. Let it sound.

Next, the operation of the above embodiment will be described.
The shutter key that constitutes a part of the key operation unit 20 has a two-step operation stroke. That is, a moderate click feeling can be obtained in a so-called “half-pressed” state in which the shutter key is pressed for about half of the entire stroke, and the first stage operation state is detected by the switch, The AF (automatic focus) function and the AE (automatic exposure) function are activated, and the focus distance, shutter speed, and aperture value are locked as photographing conditions.

  When the so-called “full-press” state is reached, in which the pressing operation is continued for the entire stroke from this half-pressed state, shooting is performed under the locked shooting conditions, and image data obtained by shooting is recorded in the memory card 24.

  FIG. 2 shows the basic processing contents executed in the still image shooting mode. The control operation is that the control unit 17 develops the operation program stored in the program memory 19 in the main memory 18. Is executed.

  In the figure, first, a subject image is photographed at a predetermined frame rate, for example, 30 [frames / second] in a monitor state, and a through image is displayed on the display unit 16 (step S101). By repeatedly executing the process of determining whether or not a push operation has been performed (step S102), the process waits for the shutter key to be half-pressed while displaying a through image.

  At this time, the through image is captured at a sufficiently high shutter speed corresponding to the frame rate, for example, fixed at 1/125 [seconds], and is controlled in the aperture value variable control by the AE function and in the image by the AF function. In-focus control at a predetermined position is executed.

  When the shutter key of the key operation unit 20 is half-pressed, it is determined in step S102, and AF processing and AE processing are executed again to obtain an appropriate in-focus position and exposure condition (steps S103 and S104). At this time, the shutter speed is newly determined based on the shooting mode, the scene program, etc. set at that time, regardless of the shooting time for the through image.

  Then, it is determined whether or not the determined shutter speed (SS) is faster than a shutter speed (SSth) as a preset threshold, for example, 1/60 [second] (step S105).

  If it is determined that the shutter speed determined here is faster than the shutter speed as a threshold value, it is further determined whether or not the shutter key is still half-pressed to perform a normal photographing operation (step S106). ). If it is confirmed that the shutter key is still half-pressed, it is next determined whether or not the shutter key is fully pressed (step S107).

  If it is determined that the full-press operation has not been performed yet, the process returns to step S106. By repeatedly executing the processes of steps S106 and S107 in this way, the process waits for the half-press operation of the shutter key to be released or to be fully pressed.

When the half-press operation of the shutter key is released, it is determined in step S106, and the process returns to the processing from the through image display in step S101. When the shutter key is fully pressed, it is determined in step S107. And the shooting operation is performed under the shooting conditions determined in the immediately preceding steps S103 and S104 (step S108).

  Furthermore, when photographing in step S108, the mechanical shutter 12 is controlled to be fully closed in synchronization with the timing at which the exposure by the CMOS image sensor 13 is finished, so that further exposure is temporarily stopped.

  The image data obtained from the image processing unit 14 by the photographing operation is subjected to the above-described image processing by the image processing unit 14 and temporarily stored in the buffer memory 15.

  However, the charge stored in each pixel position of the CMOS image sensor 13 is transferred by the readout operation and the reset operation of the charge amount of each pixel in the CMOS image sensor 13 based on the above photographing, and no new charge is stored. It becomes a state.

  Such a charge amount reading operation involving a reset operation is referred to as “destructive reading” in the following text and drawings.

  After maintaining this state for a certain time, for example, 0.3 [seconds], the charge amount is read again by destructive reading for each pixel of the CMOS image sensor 13, thereby causing darkness occurring in the CMOS image sensor 13. Reads noise components based on current. This noise is referred to as “dark noise” in the following text and drawings.

  Using the dark noise thus obtained, the image data stored in the buffer memory 15 is corrected (step S110).

By the processes in steps S109 and S110, dark noise components generated in the CMOS image sensor 13 can be removed from the image data, and accurate image data can be obtained.
Note that the shooting in step S108 is an operation when the shutter speed is faster than the threshold, and it can be considered that the influence of dark noise is negligibly low. For this reason, the processing in steps S109 and S110 may be omitted.

  After that, the image data stored in the buffer memory 15 is subjected to data compression processing such as ADCT (adaptive discrete cosine transform) defined in a predetermined data file format, for example, JPEG (Joint Photographic Experts Group) format, and the amount of data is greatly increased. The data file is converted into a data file in this format and recorded in the memory card 24 as a recording medium of the digital camera 10 (step S111).

  Thus, a series of shooting operations are completed, and the process returns to step S101 again to prepare for the next shooting.

  In addition, when the shutter speed (SS) determined by the process in step S104 immediately before in step S105 is the same as or slower than the shutter speed (SSth) as a preset threshold, for example, 1/60 [second]. If it is determined, it is next determined whether or not the shutter key is still half-pressed (step S112). If it is confirmed that the shutter key is still half-pressed, it is next determined whether or not the shutter key is fully pressed (step S113).

  If it is determined that the full-press operation has not been performed yet, the process returns to step S112. By repeatedly executing the processes of steps S112 and S113 in this way, the process waits for the half-press operation of the shutter key to be released or to be fully pressed.

When the half-press operation of the shutter key is released, it is determined in step S112, and the process returns to the processing from the through image display in step S101. When the shutter key is fully pressed, it is determined in step S113. Next, all the pixels of the CMOS image sensor 13 are reset once and the mechanical shutter 12 is controlled to be fully closed, so that further exposure to the CMOS image sensor 13 is temporarily stopped.

  Due to the fully-closed control of the mechanical shutter 12 and the reset operation of the CMOS image sensor 13, the charge is wiped out at each pixel position of the CMOS image sensor 13, and no new charge is accumulated.

  After maintaining this state for a certain time, for example, 0.3 [second], the charge amount is read again by destructive reading for each pixel of the CMOS image sensor 13, thereby reading out the dark noise of the CMOS image sensor 13. (Step S112).

Thereafter, a photographing process is executed (step S114).
FIG. 3 is a subroutine showing the detailed contents of this photographing process. First, at the beginning of the process, a variable t indicating the exposure time is set to an initial value “0” (step S201). Next, the mechanical shutter 12 is fully opened and exposure by the CMOS image sensor 13 is started (step S202).

  Further, the value of the time variable t is updated and set to “+1” (step S203). It is determined whether or not the value of the updated time variable t has reached a predetermined time value t1 (step S204). If the time value t1 has not yet been reached, the process returns to step S203 again.

  Here, the predetermined time value t1 is set to a value that is 1/4 of the shutter speed SS determined in the immediately preceding step S104. For example, if the shutter speed SS is 1/30 [second] (precisely 1/32 [second]), the time value t1 is 1/125 [second] of 1/4 (accurately 1/128 [second]. ]).

  However, the processes of steps S203 and S204 are repeatedly executed, and when the value of the variable t reaches the time value t1, it is determined in step S204, and the reset operation is not performed from all the pixels of the CMOS image sensor 13, The exposure amount at that time is acquired by non-destructive readout in which charge accumulation by exposure for each pixel is continued thereafter (step S205).

  The exposure time t2 and the longest exposure time tlim for each pixel are determined based on the exposure amounts of all the pixels of the CMOS image sensor 13 thus obtained (step S206).

  Here, the exposure time t2 for each pixel is expected to reach 90 [%] of the saturation capacity of the photodiode constituting the pixel, based on the exposure amount acquired at the time value t1. Set the exposure time.

  In addition, the longest exposure time tlim is expected to reach 90 [%] of the saturation capacity of the photodiodes that constitute the average value, for example, by obtaining an average value of the exposure amounts of all the pixels acquired at the time value t1. Set the exposure time.

  Based on the setting result, a table indicating the relationship between each pixel position and the exposure time t2 is created and stored in the main memory 18 (step S207).

FIG. 4 is a diagram illustrating the contents of the table created in this way.
In the figure, the vertical axis represents the exposure time t2 by the difference from the time value t1, and the horizontal axis represents each pixel position. The code “1” in the table associates the time t2 with the pixel position. In addition, the flag F (t) is set to “1” for the time t2 when the code is “1” at at least one pixel position.

  After the table is created, the time variable t is further updated by “+1”, and the contents of the flag F (t) corresponding to the time t2 in the table are referred to from the updated variable t value (step S208). It is determined whether (t) is set to “1” (step S209).

  If it is determined that “1” is not set, there is no pixel position set with the time variable t at that time as the exposure time t2, and the process returns to step S208 again. .

  By repeating the processes in steps S208 and S209 in this way, the process waits for the appearance of the pixel position set as the exposure time t2 while sequentially updating the time variable t.

  If it is determined in step S209 that the flag F (t) is set to “1”, there is a pixel in which the content of the time variable t at that time is set as the exposure time t2.

  For this reason, the exposure amount of the pixel is obtained by nondestructive readout in which the accumulation of charges by exposure is continued without any reset operation for all pixel positions where the code is “1” in the above table. Then, it is stored in the buffer memory 15 as part of the image data (step S210).

  Thereafter, it is determined whether or not reading of exposure amounts from all pixel positions of the CMOS image sensor 13 is completed (step S211). If it is determined that the time has not ended yet, it is further determined whether or not the time variable t at that time exceeds the longest exposure time tlim (step S212).

  If it is determined that the time variable t does not exceed the longest exposure time tlim, the process returns to step S208.

  As described above, while the time variable t is sequentially updated and set, the exposure amount at the corresponding pixel position is obtained by nondestructive reading each time the time variable t becomes the content set as the exposure time t2, and the image data It is held in the buffer memory 15 as a part.

  Thereafter, if the time variable t exceeds the longest exposure time tlim before the exposure amounts from all the pixel positions of the CMOS image sensor 13 are obtained by non-destructive reading, it is determined in step S212 and read out. The exposure amount is acquired by non-destructive reading from all remaining pixel positions that are not performed, and is stored in the buffer memory 15 as a part of the image data (step S213).

  Then, when exposure values are acquired from all pixel positions remaining beyond the longest exposure time tlim in step S213, or when reading of exposure amounts from all pixel positions is completed in step S211, next immediately before shooting. Using the dark noise for each pixel acquired in step S14, the image data held in the buffer memory 15 is corrected (step S214).

  Then, considering the exposure amount and exposure time for each pixel, if the exposure amount is the same, the exposure amount adjustment for each pixel is performed so that the shorter the exposure time, the larger the exposure amount. (Step S215).

  Further, the image data after adjusting the exposure amount is subjected to various process processes to generate image data as one image (step S216). Thus, the shooting process subroutine of FIG. 3 is terminated, and the process returns to the main routine of FIG.

  In FIG. 2, after the shooting process of step S115, the image data stored in the buffer memory 15 is subjected to a data compression process defined in a predetermined data file format, for example, the JPEG format, to greatly reduce the amount of data. The data file is formed and recorded on the memory card 24 which is a recording medium of the digital camera 10 (step S116).

  Thus, a series of shooting operations are completed, and the process returns to step S101 again to prepare for the next shooting.

A supplementary description will be given of the imaging process described with reference to FIG.
FIG. 5A shows the relationship between the luminance obtained for each pixel and the exposure time. As shown in FIG. 5A, FIG. 5B illustrates the luminance and the exposure amount for each pixel when the exposure time is uniformly set regardless of the brightness of the luminance as in general photography.

  In such general photographing, charge accumulation due to exposure cannot be obtained in a low-luminance portion, and gradation is not obtained, resulting in uniform blackness, a region Abk called “blackout”. Then, exposure is saturated in a portion with high luminance, and a region Awt called “white jump” is generated in which gradation is not obtained and white is uniformly generated.

  Therefore, in FIG. 5B, a region sandwiched between the two regions Abk and Awt is a region capable of expressing gradation with respect to luminance, and this range is a CCD or dynamic range. In general, a solid-state imaging device such as a CCD or a CMOS image sensor has an extremely inferior dynamic range, which is also called latitude, as compared with a silver salt film.

  On the other hand, FIG. 6A shows the relationship between the luminance obtained for each pixel and the exposure time by the exposure method described in FIG. Here, the longest exposure time tlim is set in accordance with the pixels having the average luminance, so that the pixels having the luminance below the average are exposed for the longest exposure time tlim. In addition, an exposure time t2 is set so that the pixels with higher luminance than the average are shorter as the luminance is higher.

  By such setting of the exposure time, the relationship between the brightness contrast and the exposure amount actually obtained in each pixel is as shown in FIG.

  An exposure time t2 is set so that a value slightly lower than a level at which the exposure of the pixel is saturated, for example, 90 [%] of the saturation capacity of the photodiode of the pixel at any pixel position with a luminance higher than the average luminance. By doing so, the exposure amount is equal to a high level.

  Therefore, although it is possible to reliably prevent the occurrence of the area Awt that becomes white-out, if it is recorded as image data as it is, if it is limited to pixel positions having a luminance higher than the average luminance, the whole area is not flat. It will be a natural description.

  On the other hand, by setting a long exposure time for a pixel position having a low luminance, the area Abk that is blackened can be surely narrowed, and the dynamic range can be expanded on the low luminance side.

  Therefore, by adjusting the exposure amount between each pixel based on the exposure time in step S215, it is possible to obtain an image having a greatly expanded dynamic range as a whole, excluding the narrowed blackout area Abk.

  FIG. 7 shows the adjustment result. Except for the narrow blackened area Abk, the overexposed area Awt is not generated, and the dynamic range is greatly expanded, and a very natural and expressive depiction is possible. I can understand.

  FIG. 8 shows the exposure timing of particularly bright pixels, bright pixels, and pixels having a brightness below average according to the photographing process of FIG. The shutter key full-press operation timing is set to S, and a period LC for detecting dark noise is provided for a certain period of time, for example, 0.3 [seconds]. Thereafter, exposure of all pixels is started.

  If the exposure start timing is t0, then the exposure amount is acquired by nondestructive readout for all the pixels set at timing t1, which is set corresponding to the shutter speed set in the AE process. The timing t2 at which the exposure amount is acquired as the image data of the pixel is determined based on the acquired exposure amount, and the exposure time is set to be shorter for the pixel having the larger exposure amount as described above. In addition, even in a pixel with a small exposure amount, an exposure time that is significantly longer than the original shutter speed is set in a range where the charge is not saturated, so that the area where blackout occurs can be greatly reduced.

  For each pixel, image data can be acquired by reading the charge amount twice in total, and the first one of them is obtained only in the charge amount and does not need to be held thereafter. Unlike a technique that requires a plurality of storage units to obtain one piece of image data, the circuit scale can be kept very small.

  Note that, at the pixel position where the exposure amount is large, the charge accumulation is continued even after the charge amount is acquired by nondestructive reading at the timing of the exposure time t2, so the charge accumulated in the photodiode overflows and the pixel concerned If there is a pixel that is adjacent to the pixel and has not yet finished reading out charges, there is a possibility that overexposure occurs at that pixel. The overexposure in this case seems to spread in a circle around a pixel with a large amount of exposure.

  However, since it is generally considered that there is almost no image with an extremely small exposure amount adjacent to a pixel that actually has a large exposure amount, it is unlikely to be affected by the above-described problems. Think.

  In the above embodiment, the longest exposure time tlim is set in accordance with pixels having an average exposure amount. However, particularly when a pixel having a large exposure amount and a pixel having a small exposure amount are close to each other. By setting the longest exposure time tlim short, it is possible to further avoid the influence of the above-mentioned problems.

  Further, as described above, since an image having a very wide dynamic range can be acquired, it is conceivable that an existing JPEG data file cannot express the number of gradations as it is.

  Therefore, by compressing an arbitrary gradation area by gamma correction, a more preferable photographed image suitable for storing image data can be acquired.

  As described above, according to the present embodiment, it is possible to realize imaging with a good exposure state over the entire image while reducing both the processing related to the reading process of the image from the imaging device and the burden on the circuit. It becomes.

  In particular, in this embodiment, exposure is started at the same time for all the pixels, the exposure amount is obtained by nondestructive reading at a time t1 when a predetermined time has elapsed from the start of exposure, and exposure is continued as it is to obtain image data for each pixel. Since the exposure amount acquisition timing t2 is different, immediate exposure can be started after the shutter key is operated, and the so-called release time lag can be minimized.

  In this case, the time t1 from the start of exposure until the exposure amount is acquired by nondestructive reading is set to a sufficiently short time based on the shutter speed determined in the previous AE process. Therefore, it is possible to deal with a pixel having a high possibility that the exposure is saturated at a fast timing with a time margin, and it is possible to avoid the occurrence of overexposure.

  In addition, in the above embodiment, the dark noise is measured by using the mechanical shutter 12 at the time of shooting, and the image data after shooting is corrected, so that the influence of the dark current is eliminated and higher image quality is obtained. Obtainable.

  In the embodiment described above, the exposure time is different for each pixel. However, the present invention is not limited to this, and the imaging surface of the CMOS image sensor 13 is divided into at least a plurality of areas, and the exposure time for each area. By adjusting the, the dynamic range can be expanded by reducing the occurrence range of blackout and overexposure compared with the case where the entire imaging surface is controlled with a uniform exposure time.

  However, by increasing the number of area divisions and using pixel units as in the present embodiment, it is possible to achieve the most precise and widest dynamic range.

Moreover, in the said embodiment, the exposure amount in a pixel with much exposure amount shall be restrict | limited according to the light reception capacity. Thereby, it is possible to suppress the occurrence of overexposure and contribute to the expansion of the dynamic range.In the above embodiment, the exposure time at the pixel position where the amount of exposure is large is increased as much as possible corresponding to the light receiving capacity. While avoiding the occurrence of overexposure, it is possible to create a descriptive expression.

  In the above embodiment, the brightness of the acquired image data is adjusted for each division unit after acquiring the image data of the entire imaging surface. A certain image can be obtained.

  In this respect, specifically, since the brightness of the image data is adjusted based on the exposure time and the exposure amount for each division unit, the adjustment standard is clear, and an image having expressive power can be obtained by simple calculation. Obtainable.

  In the above embodiment, the exposure time is determined for the exposure amount obtained by exposure for a fixed time for each pixel as described above, but in addition to this, referring to the exposure amount distribution pattern, The exposure time may be determined.

  In this way, it is possible to avoid a part of the projection and an exposure amount different from that of the surrounding area, and to avoid a shooting failure.

  Further, if the longest exposure time is set based on the distribution of the exposure amount, it is possible to avoid the occurrence of blurring or the like in the image due to the extended exposure time.

  In the above embodiment, as shown in FIG. 8 above, exposure is started for all the pixels all at once, the exposure amount of each pixel is acquired by nondestructive reading when a predetermined time has elapsed, and the subsequent exposure is performed according to the acquired exposure amount. Although the time is controlled, the present invention is not limited to this.

  FIG. 9 shows another example of such an operation. From the timing S when the shutter key is fully pressed, exposure is started all at once through the dark noise detection period LC, and thereafter a predetermined time interval. Then, the exposure amount of all pixels is continuously sampled by nondestructive readout. Then, acquisition of the exposure amount by nondestructive reading is stopped in order from the pixel where the exposure amount seems to have reached a predetermined amount, and the number of times of nondestructive reading increases, but as shown in FIG. Exposure control can be realized by a very simple control method without the need to create a table.

  Further, as described with reference to FIGS. 8 and 9, it is conceivable to match the end timings instead of matching the exposure start timings of all the pixels.

  FIG. 10 shows an operation example of such exposure control. In the figure, the exposure amount is acquired by destructive readout with a reset operation simultaneously for all the pixels from the timing S when the shutter key is fully pressed until a predetermined time tp, and the exposure time for each pixel is calculated. Thereafter, the exposure is sequentially started from the pixels for which the exposure time is set to be long, so that the timing of the end of the exposure for all the pixels is finally matched.

  By adopting such a control method, non-destructive reading is not performed at all, and the exposure time can be varied for each pixel only by destructive reading with a reset operation.

  In this way, the operation can be realized by using a solid-state imaging device whose reset operation can be controlled in units of pixels or areas. At the present stage of practical use, solid-state imaging devices that perform a reset operation for each divided region are not generalized, but it is technically possible to manufacture a CMOS image sensor capable of such an operation at present. I have confirmed.

  Further, according to the exposure control method described with reference to FIG. 10, since the end of the exposure time can be made coincident with all the pixels, if the mechanical shutter is synchronized with the timing and the fully closed control is performed, the focal plane is accumulated. And the adverse effect of line exposure, also called rolling shutter, can be avoided.

  Furthermore, as described above, by using a solid-state imaging device capable of resetting the charge accumulation for each divided area of the imaging surface, the reset operation is performed at the time when the exposure amount reading of the area is completed, and the exposure is saturated and adjacent due to overflow. It is possible to reliably avoid overexposure to the surrounding area.

  In the above embodiment, the image data is recorded on the recording medium in a data file format that accompanies data compression. However, the image data is recorded without performing any data compression processing. The image data may be processed by using retouching software or the like.

  For example, if a data file format that can be recorded by adding exposure time information in pixel units to RAW data that has not been standardized at present, a higher image quality is required. Can also be used actively.

  Further, in the above embodiment, it is determined whether or not the photographing operation shown in FIG. 3 is executed in accordance with the shutter speed value determined in the AE process. It may be adopted as one of the scene program functions adopted in the above.

FIG. 11 exemplifies a screen for selecting the main photographing process named “dark to bright / subject mixture” together with the sample image SI and the explanatory text as one of such scene program functions. In the figure,
“When dark to bright subjects are included,
Wide dynamic range shooting with all exposures is possible.
Can also be used for backlight compensation.
Note) The release diagram time lag is slightly extended.
The scene program numbered “32” is illustrated together with the sample image SI.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. In addition, the functions executed in the above-described embodiments may be implemented in appropriate combination as much as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination according to a plurality of disclosed structural requirements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

1 is a block diagram showing a functional configuration of an electronic circuit of a digital camera according to an embodiment of the present invention. 6 is a flowchart showing processing contents in a still image shooting mode according to the embodiment. 3 is a flowchart showing the contents of a subroutine of the photographing process in FIG. 2 according to the embodiment. The figure which illustrates the content of the exposure timing table produced by the subroutine of the said FIG. 3 concerning the embodiment. The figure which shows the relationship between the brightness | luminance when exposure time is set uniformly, and exposure amount. The figure which shows the relationship between the brightness | luminance which concerns on the embodiment, exposure time, and exposure amount. The figure which shows the relationship between the brightness | luminance after adjustment which concerns on the embodiment, exposure time, and exposure amount. The figure which shows the difference in the exposure time according to the exposure amount which concerns on the same embodiment. FIG. 6 is a diagram illustrating another shooting control operation according to the embodiment. FIG. 6 is a diagram illustrating another shooting control operation according to the embodiment. The figure which shows the example of a screen display in the scene program function concerning the embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 11 ... Lens optical system, 12 ... Mechanical shutter, 13 ... CMOS image sensor, 14 ... Image processing part, 15 ... Buffer memory, 16 ... Display part, 17 ... Control part, 18 ... Main memory, 19 ... Program memory 20 ... Key operation unit 21 ... Lens optical system drive unit 22 ... Flash drive unit 23 ... CMOS driver 24 ... Memory card 25 ... Audio processing unit 26 ... Lens motor (M) 27 ... Shutter Motor (M), 28 ... flash unit, 29 ... microphone unit, 30 ... speaker unit, C ... card connector, SB ... system bus.

Claims (18)

An image sensor that accumulates charges in pixel units according to a subject image incident on the imaging surface, and outputs an image signal corresponding to the accumulated charge amount;
Exposure amount acquisition means for acquiring an exposure amount for each of a plurality of areas dividing the imaging surface of the imaging element;
Exposure that adjusts the exposure time for each of the plurality of areas of the image pickup device and drives the image pickup device during one exposure period to obtain an image signal from the exposure amount obtained by the exposure amount acquisition means. Control means;
An image pickup apparatus comprising: storage means for storing an image signal for one screen obtained from the image pickup element.   The exposure amount acquisition unit does not impede charge accumulation based on the exposure on the image sensor at a preset timing after the exposure control unit starts a single exposure of a plurality of areas of the image sensor simultaneously. The imaging apparatus according to claim 1, wherein the exposure amount is acquired by nondestructive reading. The exposure amount acquisition means includes a plurality of non-destructive readouts that do not impede charge accumulation based on exposure in the image sensor at a fixed time period after the exposure control means simultaneously starts exposure of a plurality of areas of the image sensor. Get the exposure amount for each area,
2. The imaging according to claim 1, wherein the exposure control unit outputs an image signal from the area at a timing when the exposure amount reaches a predetermined value by the exposure amount for each of the plurality of areas obtained by the exposure amount acquisition unit. apparatus. The exposure amount acquisition means exposes a plurality of areas for dividing the image sensor at the time of pre-exposure prior to the exposure operation of the main photographing that adjusts the exposure time by the exposure control means, and exposes the image from the image sensor at the same time. Obtain the exposure amount for each area by obtaining the signal,
The exposure control means adjusts an exposure time by varying a timing at which exposure is started for each of the plurality of areas at the time of exposure of the main photographing by an exposure amount for each of the plurality of areas obtained by the exposure amount acquisition means, The imaging apparatus according to claim 1, wherein timings for obtaining image signals corresponding to charge amounts accumulated from a plurality of areas are substantially matched. A shutter mechanism for controlling opening and closing of an object image incident on the image sensor;
Measuring means for measuring a dark current for each of a plurality of areas of the image pickup device in a state in which incidence to the image pickup device is cut off immediately before photographing exposure by the exposure control means by the shutter mechanism;
2. The imaging apparatus according to claim 1, further comprising a correction unit that corrects an image signal stored in the storage unit for each of a plurality of areas based on a measurement result of the measurement unit. A shutter mechanism for controlling opening and closing of the subject image incident on the image sensor;
5. The image pickup apparatus according to claim 4, wherein the exposure control means blocks the incidence on the image pickup element by the shutter mechanism in accordance with output timings of image signals from a plurality of areas of the image pickup element. Exposure time determining means for determining an exposure time in at least one area of the image sensor;
3. The imaging apparatus according to claim 2, wherein the exposure amount acquisition unit acquires the exposure amount by the non-destructive reading at a timing earlier than an exposure time end timing determined by the exposure time determination unit.   The image pickup apparatus according to claim 1, wherein the plurality of areas are in units of pixels constituting an image pickup surface of the image pickup element.   The imaging apparatus according to claim 1, wherein the exposure control unit limits a maximum exposure amount in one exposure period in accordance with a light receiving capacity of the imaging element.   2. The imaging apparatus according to claim 1, wherein the exposure control means associates the exposure time of at least one area on the side with a large exposure amount with the light receiving capacity of the imaging element.   2. The imaging apparatus according to claim 1, further comprising brightness adjusting means for adjusting the brightness of the image signal stored in the storage means for each of a plurality of areas.   12. The imaging apparatus according to claim 11, wherein the brightness adjusting unit adjusts the brightness of the image signal for each of a plurality of areas based on an exposure time and an exposure amount.   2. The imaging apparatus according to claim 1, wherein the exposure control unit adjusts an exposure time for each of the plurality of areas in accordance with a distribution of the exposure amount for each of the plurality of areas obtained by the exposure amount acquisition unit. .   The imaging apparatus according to claim 1, wherein the exposure control unit sets a longest exposure time corresponding to a distribution of exposure amounts for each of a plurality of areas obtained by the exposure amount acquisition unit.   The imaging apparatus according to claim 1, wherein the exposure control unit resets charge accumulation before and after exposure for each of a plurality of areas of the imaging element.   The imaging apparatus according to claim 1, wherein the storage unit stores an image signal and an exposure time for each of the plurality of areas in association with each other. An imaging method in an imaging apparatus including an imaging element that accumulates charges in units of pixels according to a subject image incident on an imaging surface and outputs an image signal corresponding to the accumulated charge amount,
An exposure amount acquisition step of acquiring an exposure amount for each of a plurality of areas dividing the imaging surface of the imaging element;
Exposure that adjusts the exposure time for each of a plurality of areas of the image pickup device and drives the image pickup device in one exposure period to obtain an image signal from the exposure amount obtained in the exposure amount acquisition step. Control process;
And a storage step of storing an image signal for one screen obtained from the image sensor. A program for a computer built in an imaging apparatus including an imaging element that accumulates charges in units of pixels according to a subject image incident on an imaging surface and outputs an image signal corresponding to the accumulated charge amount,
An exposure amount acquisition step of acquiring an exposure amount for each of a plurality of areas dividing the imaging surface of the image sensor;
Exposure that adjusts the exposure time for each of the plurality of areas of the image pickup device and drives the image pickup device in one exposure period to obtain an image signal from the exposure amount obtained in the exposure amount acquisition step. Control steps;
A program for causing a computer to execute a storage step of storing an image signal for one screen obtained from the imaging device.

Images