JP7277251B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an imaging device and its control method.

2. Description of the Related Art In an imaging apparatus such as a digital camera, there is known a technique of reducing the transmittance of an imaging light amount for imaging in a very bright environment. Even in a bright environment, it is possible to open the aperture to achieve a shallow depth of field, or to capture a subject (such as a waterfall) without saturation even when performing a long exposure. This is for representing the trajectory of movement, and has conventionally been performed using an optical filter such as an ND filter.

On the other hand, in Patent Document 1, the charges converted by the photoelectric conversion portion are transferred to the storage portion a plurality of times, and the charges transferred a plurality of times are collectively accumulated. Also disclosed is a solid-state imaging device that can be freely changed. By applying this matter to control the operation of the image sensor, the short accumulation period is evenly distributed over the frame period, and multiple accumulations are performed collectively to adjust the light intensity without using an optical filter. , and a natural moving image can be obtained.

Further, Patent Document 2 discloses a solid-state imaging device in which electric charges from a photoelectric conversion portion in a photodiode are transferred to a floating diffusion portion (FD portion) so that the potential of the floating diffusion portion changes according to light. there is In addition, in Patent Document 2, for the fixed pattern noise caused by the reset generated in the FD section, the dark output before transfer of the signal charge is sampled and accumulated in a capacitor, and the fixed pattern noise is obtained by taking the difference from the bright output. Techniques for removing noise are also disclosed. Further, in Japanese Patent Application Laid-Open No. 2002-200000, when multiple exposure image data each having been subjected to different shading corrections are added to create composite image data, dark image data for noise reduction is also subjected to shading correction. An imaging device is disclosed that performs correction in consideration of influence.

JP 2010-157893 A JP-A-9-46596 Japanese Patent No. 05889324

However, in Japanese Patent Laid-Open No. 2002-100001, a configuration for suppressing the influence of fixed pattern noise such as noise generated by resetting in the FD section is not used, and image quality deteriorates due to noise. Further, in the imaging apparatus of Patent Document 2, image quality is improved by subtracting fixed pattern noise due to resetting in the FD section as a dark output. However, when accumulation is performed a plurality of times within one frame as in Patent Document 1, fixed pattern noise of the type that is generated when charges are transferred from the photoelectric conversion unit to the memory unit that holds the charges. exist. In Japanese Patent Application Laid-Open No. 2002-200000, no countermeasures are taken against fixed pattern noise that occurs when transferring to the memory unit, and the data remains without being subtracted, resulting in deterioration of image quality. In addition, in the imaging apparatus of Patent Document 3, the effect of error that occurs in noise reduction calculation due to shading correction being applied to an exposed image is calculated numerically for a dark image used for noise reduction calculation as well. It is canceled by performing calculation. However, when multiple exposures are performed as in Patent Document 1, there are two types of fixed pattern noise to be corrected: one occurring in the floating diffusion region, and the other occurring when passing through the path between the storage section and the photodiode. . Furthermore, fixed pattern noise occurs different times during one imaging cycle. Therefore, it is difficult to reproduce the effects of multiple types of noise, each occurring at different times, from a single dark image. There is a possibility that it will be sufficient and the image quality will deteriorate.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an imaging apparatus capable of suppressing the influence of fixed pattern noise in an imaging element while adjusting the amount of light without using an optical filter.

In order to solve the above-described problems, an image pickup apparatus of the present invention includes an image pickup device in which pixels having at least one charge holding portion for one photoelectric conversion portion are two-dimensionally arranged, and a shutter that controls the image pickup device. acquisition means for acquiring a first image taken with the shutter open and a second image taken with the shutter closed; and correction means for correcting the first image using the second image. And prepare. The acquisition means acquires the first image by accumulating charges in the charge holding unit in a time division manner during one imaging cycle, and acquires the second image in a time division manner corresponding to the first image. and the exposure accumulation time per exposure corresponding to the second image is equal to the exposure accumulation per exposure corresponding to the first image. shorter than time.

According to the present invention, it is possible to provide an imaging apparatus capable of suppressing the influence of fixed pattern noise in an imaging device while performing light amount adjustment without using an optical filter.

It is a figure which shows the external appearance of an imaging device. It is a figure which shows the structure of an imaging device. It is a figure which shows the circuit of an image pick-up element. FIG. 4 is a diagram showing a readout circuit; 4 is a timing chart showing the driving sequence of the imaging element in the first embodiment; 7 is a timing chart showing a driving sequence of an image sensor in a conventional example; 9 is a timing chart showing the driving sequence of the imaging element in the second embodiment; FIG. 11 is a timing chart showing a driving sequence of an imaging element in the third embodiment; FIG. FIG. 11 is a diagram showing the configuration of an imaging device in a third embodiment; FIG. 10 is a flow chart showing imaging operation in the third embodiment.

(First embodiment)
FIG. 1 is an external view of an imaging device 100. FIG. 1A is a front view of the imaging device 100, and FIG. 1B is a rear view of the imaging device 100. FIG. The imaging device 10 is, for example, a digital motion camera capable of capturing still images and moving images. In this embodiment, an example of an imaging device in which an imaging device main body and a lens are integrated will be described, but the imaging device is not limited to this, and may be, for example, a lens-interchangeable digital single-lens camera. Also, the image processing unit that processes the image signal output from the image sensor 184 does not necessarily have to be configured as a part of the imaging device, and is configured by hardware separate from the image sensor 184 and the imaging optical system. may Also, all or part of the functions of the image processing section may be mounted on the imaging device 184 .

The image pickup apparatus 100 includes an image pickup apparatus main body 151 , an image pickup optical system 152 on the front surface of the image pickup apparatus main body 151 , and a switch ST<b>154 on the top surface of the image pickup apparatus main body 151 . The imaging device 100 also includes a display unit 153 , a switch MV 155 , a selection lever 156 , a menu button 157 , an up switch 158 , a down switch 159 , a dial 160 and a playback button 161 on the back of the imaging device main body 151 .

The imaging device main body 151 is a main body portion of the imaging device 100 in which the imaging element 184 and the shutter device are accommodated. The imaging optical system 152 is an optical system for forming an optical image of a subject on the imaging device 184, and has a lens and a diaphragm inside. The display unit 153 is a display unit for displaying shooting information and images. The display unit 153 has a display luminance range that can display an image with a wide dynamic range without suppressing the luminance range. Moreover, the display unit 153 may be provided with a movable mechanism for changing the orientation of the screen as necessary.

The switch ST154 is a shutter button mainly used for taking still images. The switch MV155 is a button for starting and stopping moving image shooting. A selection lever 156 is a changeover switch for selecting a shooting mode. A menu button 157 is a menu button for shifting to a function setting mode for setting the functions of the imaging apparatus 100 . Up switch 158 and down switch 159 are up/down switches for changing various set values. A dial 160 is a dial for changing various set values. The playback button 161 is a button for switching to a playback mode in which the video recorded on the recording medium 193 in the imaging device 100 is played back on the display unit 153 .

FIG. 2 is a block diagram showing a configuration example of the imaging device 100. As shown in FIG. The imaging apparatus 100 includes an imaging optical system 152 , an aperture 181 , an aperture control section 182 , an optical filter 183 , an imaging device 184 , a blur correction lens 185 and a blur correction drive section 186 . The imaging apparatus 100 also includes a digital signal processing section 187 , a timing generation section 189 , a system control CPU 178 , a switch input means 179 and a memory section 190 . The imaging device 100 also includes a display I/F 191 , a display unit 153 , a recording I/F 192 , a recording medium 193 , a print I/F 194 , an external I/F 196 and a wireless I/F 198 .

An optical axis 180 is the optical axis of the imaging optical system 152 . The imaging optical system 152 forms an optical image of a subject on the imaging device 184 . The imaging optical system 152 includes multiple lenses such as a focus lens and a shift lens. A diaphragm 181 adjusts the amount of light passing through the imaging optical system 152 . The aperture control section 182 controls the aperture 181 . The optical filter 183 limits the wavelength of light incident on the imaging device 184 and the spatial frequencies transmitted to the imaging device 184 .

The imaging device 184 converts the optical image of the subject imaged via the imaging optical system 152 into an electrical image signal. The imaging device 184 has sufficient number of pixels, signal readout speed, color gamut, and dynamic range to meet the Ultra High Definition Television standard.

The digital signal processing unit 187 performs analog processing and analog-to-digital conversion on the image signal output from the imaging element 184, performs various corrections on the digital video data, and then compresses the video data. The timing generator 189 outputs various timing signals to the imaging element 184 and the digital signal processor 187 to control various timings. The system control CPU 178 is a CPU (Central Processing Unit) that performs various calculations and controls the entire imaging apparatus 100 . That is, the timing generator 189 and the system control CPU 178 have the function of control means for controlling the transfer, accumulation, and readout timings of signal charges in the imaging device 184 .

The memory unit 190 temporarily stores video data. A display I/F 191 is an interface for displaying a captured image on the display unit 153 . The display unit 153 is a display unit such as a liquid crystal display. A recording I/F 192 is an interface for recording on or reading from the recording medium 193 . A recording medium 193 is a recording medium such as a memory for recording video data, additional data, and the like. The recording medium 193 may be attached to the imaging device 100 or may be detachable.

A print I/F 194 is an interface for outputting a photographed image to an external printer 195 for printing. Printer 195 is a printer such as a small inkjet printer. The external I/F 196 is an interface for communicating with the external device 197 and the like. The external device 197 is a device capable of displaying images, such as a computer or television. A wireless I/F 198 is an interface for communicating with an external network 199 . Network 199 is a computer network such as the Internet. The switch input unit 179 includes a switch ST154, a switch MV155, and a plurality of switches for switching between various modes, and receives operations from the user.

FIG. 3 is a circuit diagram showing the pixel configuration of the imaging device 184. As shown in FIG. The imaging device 184 has a large number of pixel elements (pixel units) arranged two-dimensionally. In FIG. 3, among the many pixel elements of the image sensor 184, a pixel portion 300 at the first row and the first column (1, 1) and a pixel portion 301 at the m row and the first column (m, 1) which is the last row are showing. Since the configurations of the pixel portion 300 and the pixel portion 301 are the same, the same reference numerals are given to the components of each pixel. Note that the basic structure of the imaging device 184 having a signal holding unit is disclosed in Japanese Patent Application Laid-Open No. 2002-200010, and therefore the description thereof is omitted.

<Description of Image Pickup Device Configuration, Image Signal Generation Process, and Light Reduction Means>
One pixel portion 300 has a photodiode 500, a first transfer transistor 501A, a signal holding portion 507A, and a second transfer transistor 502A. Furthermore, the pixel section 300 has a third transfer transistor 503 , a floating diffusion region (hereinafter referred to as an FD region) 508 , a reset transistor 504 , an amplification transistor 505 and a selection transistor 506 . Further, the pixel section 300 includes power lines 520 , 521 and signal output lines 523 . In this embodiment, an example in which each pixel portion has one charge holding portion will be described, but the present invention is not limited to this, and a plurality of charge holding portions may be provided.

The anode of photodiode 500 is connected to the ground line. The cathode of the photodiode 500 is connected to the source of the first transfer transistor 501A and the source of the third transfer transistor 503, respectively. The drain of the first transfer transistor 501A is connected to the source of the second transfer transistor 502A. A connection node between the drain of the first transfer transistor 501A and the source of the second transfer transistor 502A constitutes a charge holding portion 507A.

The drain of the second transfer transistor 502 A is connected to the source of the reset transistor 504 and the gate of the amplification transistor 505 . A connection node of the drain of the second transfer transistor 502A, the source of the reset transistor 504, and the gate of the amplification transistor 505 constitutes an FD region 508. FIG. A source of the amplification transistor 505 is connected to a drain of the selection transistor 506 . A drain of the reset transistor 504 and a drain of the amplification transistor 505 are connected to the power supply line 520 . A drain of the third transfer transistor 503 is connected to the power supply line 521 . The source of select transistor 506 is connected to signal output line 523 . The other end of the signal output line 523 is connected to the readout circuit 600 .

A plurality of pixels of the imaging element 184 are connected in units of rows to control lines arranged in the row direction from the vertical scanning circuit 700 . Each row of control lines includes a plurality of control lines connected to the gates of transfer transistors 501A, 502A, 501B, 502B, 503, reset transistor 504, and select transistor 506, respectively. Each control pulse is sent from the vertical scanning circuit 307 to each transistor based on the control signal from the system control CPU 178 . Each transistor is turned on when the control pulse is at high level and turned off when the control pulse is at low level.

Specifically, the first transfer transistor 501A is controlled by a transfer pulse φTX1A. The second transfer transistor 502A is controlled by a transfer pulse φTX2A. A reset transistor 504 is controlled by a reset pulse φRES. The selection transistor 506 is controlled by a selection pulse φSEL. The third transfer transistor 503 is controlled by a transfer pulse φTX3.

<Description of noise reset circuit configuration>
FIG. 4 is a circuit diagram showing a configuration example of the readout circuit 600 of the image sensor 184. As shown in FIG. The readout circuit 600 can correct reset noise. The readout circuit 600 is connected to each pixel portion through a signal output line 523, and receives charge signals output from each pixel portion. The reading circuit 600 includes a first switch 414, a second switch 415, a third switch 418, a fourth switch 419, a first capacitor 410, a second capacitor 411, a first horizontal output line 424, a 2 horizontal output lines 425 and an output amplifier 421 .

A first switch 414 is a switch that controls writing of a pixel signal to the capacitor 410 . The first switch 414 is a switch controlled by the transfer pulse Ts, and is turned on when the transfer pulse Ts is at high level to connect the output terminal of the signal output line 523 and the capacitor 410 . A second switch 415 is a switch that controls writing of a pixel signal to the capacitor 411 . The second switch 415 is a switch controlled by the transfer pulse Tn, is turned on when the transfer pulse Tn is at high level, and connects the output terminal of the signal output line 523 and the capacitor 411 .

A third switch 418 is a switch that controls the output of the pixel signal held in the capacitor 410 to the output amplifier 421 . A fourth switch 419 is a switch that controls the output of the pixel signal held in the capacitor 411 to the output amplifier 421 . The third switch 418 and the fourth switch 419 are turned on according to the control signal from the horizontal shift register 431 . As a result, the signal written to the capacitor 410 is output to the output amplifier 421 via the third switch 418 and horizontal output line 424 . Also, the signal written in the capacitor 411 is output to the output amplifier 421 via the fourth switch 419 and the horizontal output line 425 .

The output amplifier 421 outputs a differential signal of the signals from the horizontal output lines 424 and 425 . The output of the output amplifier 421 is sent to the digital signal processing section 187 and converted into a digital signal by an A/D converter. The transfer pulse Ts, the transfer pulse Tn, and the signal from the horizontal shift register 431 are signals supplied from the timing generator 189 under the control of the system control CPU 178 .

Details of the operation of the imaging element 184 will be described with reference to FIG. FIG. 5 is a timing chart showing the driving sequence of the imaging device 184 in the first embodiment. In the following description, it is assumed that moving images are captured under the condition of 30 fps, and image signals are obtained by adding 1/480 second accumulation four times during one shooting cycle of 1/30 second. As an example, the timing of control of the image sensor will be described. Timing information such as photoelectric conversion, accumulation, and discharge shown in the timing chart is stored in the memory unit 190. When the system control CPU 178 performs photographing, it is read out from the memory unit 190 and transmitted to the image pickup device 184 for photographing. It controls the operation of element 184 .

The image sensor 184 has many rows of pixel columns in the vertical direction. In the following description, the first row including the pixel portion 300 (1, 1) in the circuit diagram shown in FIG. 3 is taken as an example to control the image sensor. explain the timing of The accumulation operation of all the pixels of the imaging device 184 is performed by scanning these controls in the vertical direction by the horizontal synchronizing signal. Note that the timing chart of FIG. 5 schematically shows the time interval, which differs from the actual control time interval.

In FIG. 5, the period from time t1 to time t16 is defined as one imaging period (the Nth imaging period). In the following, as shooting conditions, a moving image is obtained by adding 1/480 second accumulation four times as a bright output signal, which is the main component of the moving image signal, during a period of 1/30 second. A case of obtaining an exposure amount equivalent to is shown as an example.

First, at time t1, the transfer pulse φTX3(1) for the first row supplied from the vertical scanning circuit 700 changes from high level to low level. As a result, the third transfer transistor 503 is turned off, the reset of the photodiodes 500 in the first row is released, and the photodiodes 500 start accumulating signal charges. At the same time, the transfer pulse φTX1(1) of the first row changes from low level to high level. As a result, the first transfer transistor 501A is turned on, and the signal charge accumulated in the photodiode 500 starts to be transferred to the signal holding portion 507A holding the charge of the moving image in the first row. This time point is the start of the imaging cycle. Note that the shutter is closed at time t1, and a so-called dark image is captured.

At time t2, the transfer pulse φTX1(1) of the first row becomes low level. As a result, the first transfer transistor 501A is turned off, and transfer of the signal charge accumulated in the photodiode 500 to the signal holding portion 507A is completed. At the same time, the transfer pulse φTX3(1) of the first row changes from low level to high level. This turns off the third transfer transistor 503 and resets the photodiodes 500 in the first row again.

At time t3, as at time t1, the transfer pulse φTX3(1) of the first row again changes from high level to low level. As a result, the third transfer transistor 503 is turned off, the reset of the photodiodes 500 in the first row is released, and the photodiodes 500 start accumulating signal charges. At the same time, the transfer pulse φTX1(1) of the first row changes from low level to high level. As a result, the first transfer transistor 501A is turned on, and the signal charge accumulated in the photodiode 500 starts to be transferred to the signal holding portion 507A holding the charge of the moving image in the first row.

At time t4, as at time t2, the transfer pulse φTX1(1) of the first row becomes low level. As a result, the third transfer transistor 501A is turned off, and transfer of the signal charge accumulated in the photodiode 500 to the signal holding portion 507A is completed. At the same time, the transfer pulse φTX3(1) of the first row changes from low level to high level. This turns off the third transfer transistor 503 and resets the photodiodes 500 in the first row again.

The operation from time t3 to time t4 is a repeating operation of the operation from time t1 to t2. 1, 601-2, 601-3, 601-4 are performed a total of four times. Note that the control operations in the accumulation times 601-3 and 601-4 are all the same as those in the accumulation times 601-1 and 601-2, so description thereof will be omitted. The charge signals generated by the above four operations are accumulated and held in the signal holding section 507A. During accumulation times 601-1 to 601-4, exposure accumulation is performed with the shutter closed, and a dark output signal (second image signal) is obtained.

In the present embodiment, an example in which photoelectrically converted charges are transferred to the signal holding unit 507A four times has been described. The number of accumulations in the signal holding unit 507A does not matter as long as the accumulation is performed a plurality of times. It is assumed that the four dark output signal accumulation times, such as times t1 to t2, are each shorter than the light output signal accumulation time, which will be described later. Desirably, it is required to be the minimum time unit controllable by the imaging apparatus 100 .

After the above four repetitions are completed, at time t5, the reset pulse φRES(1) of the first row becomes low level. As a result, the reset transistor 504 in the first row is turned off, and the reset state of the FD region 508 is released. At the same time, the selection pulse φSEL(1) for the first row becomes high level. As a result, the select transistor 506 in the first row is turned on, and readout of the charge signal in the first row becomes possible.

At time t6, the transfer pulse φTX2(1) of the first row becomes high level. As a result, the second transfer transistor 502A in the first row is turned on, and the charge signal held in the signal holding portion 507A is transferred to the FD region 508. FIG. An output corresponding to the change in potential of the FD region 508 is read out to the signal output line 523 through the amplification transistor 505 and the selection transistor 506 and supplied to the read circuit 600 . At this time, the transfer pulse Tn in the readout circuit 600 becomes high level at the same time. As a result, the second switch 415 in the circuit is turned on, and the charge signal supplied to the reading circuit 600 through the signal output line 523 is transferred to the signal holding portion 411 and held.

At time t7, the reset pulse φRES(1) of the first row becomes high level. As a result, the reset transistor 504 is turned on, and the FD region 508 of the first row and the first signal holding section 507A are reset. At time t7, the selection pulse φSEL(1) for the first row, the transfer pulse φTX2(1) for the first row, and the transfer pulse Tn for the readout circuit 600 are at low level. The signal transferred to and held by the signal holding unit 411 is a dark output signal that mainly includes a fixed pattern noise signal.

At time t8, the transfer pulse φTX3(1) for the first row supplied from the vertical scanning circuit 700 again changes from high level to low level. As a result, the third transfer transistor 503 is turned off, the reset of the photodiodes 500 in the first row is released, and the photodiodes 500 start accumulating signal charges for moving images.

At time t9, the transfer pulse φTX1(1) of the first row changes from low level to high level. As a result, the first transfer transistor 501A is turned on, and the signal charge that has been accumulated in the photodiode 500 since time t8 begins to be transferred to the signal holding portion 507A that holds the moving image charge in the first row.

At time t10, the transfer pulse φTX1(1) of the first row changes from high level to low level. As a result, when the transfer transistor 501A in the first row is turned off, the charge transfer from the photodiode to the signal holding portion 507A is completed. At the same time, the transfer pulse φTX3(1) of the first row returns from low level to high level, and the photodiodes 500 of the first row are reset again.

The period from time t8 to time t10 corresponds to one accumulation time of 1/480 second of the moving image in the shooting cycle, and is illustrated as accumulation time 602-1 in the region of the upward slanted line. Such an accumulation operation is discretely performed four times, and is illustrated as accumulation times 602-1, 602-2, 602-3, and 602-4 in areas with diagonal lines rising to the right. By adding these four accumulations, an accumulation time equivalent to one accumulation (1/480 sec.times.4 times=1/120 sec.) is obtained. Note that the control operations in the accumulation times 602-2, 602-3, and 602-4 are all the same as in the accumulation time 602-1, so description thereof will be omitted. During accumulation times 602-1 to 602-4, exposure accumulation is performed with the shutter open, and a bright output signal (first image signal) is obtained.

At time t11, the reset pulse φRES(1) of the first row becomes low level. As a result, the reset transistor 504 in the first row is turned off, and the reset state of the FD region 508 is released. At the same time, the selection pulse φSEL(1) for the first row becomes high level. As a result, the select transistor 506 in the first row is turned on, and readout of the charge signal in the first row becomes possible.

At time t12, the transfer pulse φTX2(1) of the first row becomes high level. As a result, the second transfer transistor 502A in the first row is turned on, and the charge signal held in the signal holding portion 507A is transferred to the FD region 508. FIG. An output corresponding to the change in potential of the FD region 508 is read out to the signal output line 523 through the amplification transistor 505 and the selection transistor 506 and supplied to the read circuit 600 . At this time, the transfer pulse Ts in the readout circuit 600 becomes high level at the same time. As a result, the first switch 414 in the circuit is turned on, and the charge signal supplied to the reading circuit 600 through the signal output line 523 is transferred to the signal holding portion 410 and held.

At time t13, the reset pulse φRES(1) of the first row becomes high level. As a result, the reset transistor 504 is turned on, and the FD region 508 of the first row and the first signal holding section 507A are reset. At time t13, the selection pulse φSEL(1) for the first row, the transfer pulse φTX2(1) for the first row, and the transfer pulse Ts of the readout circuit 600 are at low level.

Here, the signal transferred to and held in the signal holding unit 410 is the bright output signal that is the main component of the final image output signal. In this embodiment, an example in which the light output signal and the dark output signal are equally exposed and accumulated four times will be described. The number of repetitions should be equal to or less than the number of repetitions of exposure/accumulation of the bright output signal.

At time t14, the signal Td from the horizontal shift register 431 becomes high level. As a result, the third switch 418 and the fourth switch 419 are turned on at the same time. Then, the bright output signal, which is the main component of the final image output signal written in the signal holding unit 410 , is output to the output amplifier 421 via the horizontal output line 424 . Also, the dark output signal mainly containing the fixed pattern noise signal written in the signal holding unit 411 is output to the output amplifier 421 via the horizontal output line 425 . The output amplifier 421 outputs a differential signal of the signals from the horizontal output lines 424 and 425 . As a result, it is possible to extract the image signal from which the residual fixed pattern noise is removed from the bright output signal, which is the main component of the final image output signal.

At time t15, the pulse signal Td changes from high level to low level. As a result, the third switch 418 and the fourth switch 419 are turned off, the output to the output amplifier 421 ends, and the photographing cycle ends.

At time t16, the transfer pulse φTX3(1) of the first row changes from high level to low level as at time t1. As a result, the third transfer transistor 503A is turned off, the reset state of the photoelectric conversion unit 500 is released, and the next imaging cycle illustrated from time t16 to t17 is started. Since this is a repetition of the same operation as the operation in the previous imaging cycle explained from time t1 to t15, detailed explanation is omitted.

Note that the timing chart in the second line is executed in synchronization with the horizontal synchronous vibration φH immediately after time t1. That is, the timing chart for all rows starts between time t1 and time t16. For example, the timing chart started by the horizontal synchronizing signal φH at time t0 is the m-th row. The switch signals at that time can be expressed as φSEL(m), φRES(m), φTX3(m), φTX1A(m), φTX1B(m), φTX2A(m), and φTX2B(m).

<Effect of Light Amount Adjustment by Dimming Means>
According to the timing chart as described above, the bright output signal, which is the main component of the moving image signal, is time-divided into 1/480 seconds and added four times during one shooting period of 1/30 seconds. An exposure amount equivalent to one second exposure can be obtained. This amount of exposure is 1/4 of the exposure time for full exposure of 1/30 second, which is one shooting period. Therefore, a light reduction effect equivalent to 2 steps of ND, which reduces the total exposure amount to 1/4, is exhibited.

In the present embodiment, an example of exhibiting the effect of two stages of ND has been described. You can adjust the effect arbitrarily. For example, when the exposure/accumulation/transfer operation is repeated four times during one shooting period of 1/30 second, the exposure time that can be set for each operation is at most 1/30 second, which is one shooting period. It becomes 1/120 seconds corresponding to /4. At this time, since the 1/120 second exposure is repeated four times, the total exposure time is multiplied by four to 1/30 second, which is equal to 1/30 second, which is the time of one shooting cycle. This corresponds to the state without the ND effect.

Also, if the exposure/accumulation/transfer operation is repeated four times during one shooting cycle of 1/30 second, and the exposure time per time is 1/240 second, which is half, the total exposure time is 1. Multiplying /240 seconds by 4 gives 1/60 seconds. Since this is 1/2 of 1/30 second, which is the time of one photographing cycle, the amount of light received is 1/2 compared to the case where the entire 1/30 second is exposed. In other words, the amount of light is halved, which is an effect equivalent to one ND step. Similarly, if the exposure time per exposure is 1/480 second, the total exposure time is 1/120 second by multiplying 1/480 second by four. Since this is 1/4 of 1/30 second, which is the time of one photographing cycle, the amount of light received is 1/4 compared to the case where the entire 1/30 second is exposed. In other words, the light amount is reduced to 1/4, which is an effect equivalent to 2 stages of ND. Similarly, when the exposure time per time is 1/960 seconds, the effect of 3 ND stages where the amount of light is 1/8, and further when the exposure time per time is 1/1920 seconds, the amount of light is reduced. The effect of 4 stages of ND, which is 1/16, is exhibited. In this way, by adjusting the exposure/accumulation time per time, it is possible to adjust the amount of light attenuation.

A moving image obtained by the photographing operation of this embodiment is configured to obtain one image signal by adding short accumulation times set at approximately equal intervals during a photographing period of 1/30 second. Therefore, it is possible to obtain a high-quality moving image without frame-by-frame flickering. In the above-described embodiment, the number of times of accumulation and addition of moving images is assumed to be 4, but the number of times of accumulation and addition can be 8, 16, 32, and 64, for example.

<Removal of fixed pattern noise>
However, the bright output signal, which is the main component of the moving image signal, contains residual fixed pattern noise in the pixels, which may cause deterioration in image quality. In the present embodiment, in order to suppress image quality deterioration due to residual fixed pattern noise, a fixed pattern noise signal is added to subtract a signal of fixed pattern noise generated in pixels from a bright output signal, which is the main component of a moving image signal. We set a period for making an estimate. The period from time t1 to time t7 in FIG. 5 corresponds to this.

Conventionally, as a countermeasure against fixed pattern noise, Patent Document 1 introduces a technique of assuming reset noise generated in the FD area 508 and removing its influence. FIG. 6 shows an imaging timing chart when the conventional technique is applied. FIG. 6 is a timing chart showing the driving sequence of the imaging device 184 in the conventional example. As in FIG. 5, FIG. 6 shows an image signal obtained by adding four accumulations of 1/480 seconds in one shooting cycle of 1/30 seconds on the assumption that moving images are shot at 30 fps. It corresponds to the case of obtaining One shooting cycle is from time T1 to time T12.

The difference between the conventional example (FIG. 6) and the present embodiment (FIG. 5) lies in the operation during the period for estimating fixed pattern noise. In the conventional example, when the reset pulse φRES(1) of the first row becomes low level at time T1 in the imaging cycle, the reset transistor 504 of the first row is turned off, and the reset state of the FD region 508 is released. starting from the point in time. At the same time, when the selection pulse φSEL(1) for the first row becomes high level, the selection transistor 506 for the first row is turned on, and the charge signal for the first row can be read. This is the same operation as the operation at time t5 in this embodiment. In this embodiment, the transfer/accumulation from the photodiodes 601-1, 601-2, 601-3, and 601-4 to the signal holding unit 507A via the transfer transistor 501A is repeated four times from t1 to t5. There was a period of However, this does not exist in the conventional timing chart.

At time T2 in the conventional example, when the transfer pulse φTX2(1) of the first row becomes high level, the second transfer transistor 502A of the first row is turned on, and the charge held in the signal holding portion 507A is turned on. Signals are transferred to the FD area 508 . An output corresponding to the change in potential of the FD region 508 is read out to the signal output line 523 through the amplification transistor 505 and the selection transistor 506 and supplied to the read circuit 600 . At this time, the transfer pulse Tn in the readout circuit 600 becomes high level at the same time, the second switch 415 in the circuit is turned on, and the charge signal supplied to the readout circuit 600 through the signal output line 523 is transferred to the signal holding unit 411 . transferred to and retained. This is the same operation as the operation at time t6 in this embodiment, but as described above, the transfer/accumulation operations during the accumulation times 601-1 to 601-4 that exist in this embodiment do not exist in the conventional example. Therefore, the charge transferred and held in the signal holding unit 411 at time T2 in the conventional example does not include the charge derived from the photodiode 500 and the transfer transistor 501A, and only the reset noise signal generated in the FD region 508 is the dark output signal. is retained as

At time T3, the reset pulse φRES(1) of the first row becomes high level. As a result, the reset transistor 504 is turned on, and the FD region 508 of the first row and the first signal holding section 507A are reset. At time T3, the selection pulse φSEL(1) of the first row, the transfer pulse φTX2(1) of the first row, and the transfer pulse Tn of the readout circuit 600 are at low level. This is equivalent to the operation at time t7 in FIG. 5 as described above.

The operation from time T4 to time T9 in FIG. 6 is an operation to obtain a bright output signal through the four exposure/accumulation periods of accumulation times 602-1, 602-2, 602-3, and 602-4. This operation is equivalent to the operation of obtaining a bright output signal through the exposure/accumulation period repeated four times during the accumulation times 602-1 to 602-4 from times t8 to t13 in FIG. can get. Therefore, the description is omitted. The obtained bright output signal is held in the signal holding section 410 .

At time T10 in FIG. 6, as at time t14 in FIG. 5, the signal Td from the horizontal shift register 431 of the readout circuit goes high. As a result, the third switch 418 and the fourth switch 419 are turned on at the same time. Then, the bright output signal, which is the main component of the image output signal written in the signal holding section 410 , is output to the output amplifier 421 via the horizontal output line 424 . Also, the dark output signal mainly containing the fixed pattern noise signal written in the signal holding unit 411 is output to the output amplifier 421 via the horizontal output line 425 . The output amplifier 421 outputs a differential signal of the signals from the horizontal output lines 424 and 425 . As a result, it is possible to extract the image signal from which the residual fixed pattern noise is removed from the bright output signal, which is the main component of the final image output signal.

At this time, the dark output signal held in the signal holding unit 411 in FIG. 6 is only the reset noise signal generated in the FD area 508, which is subtracted from the bright output signal as a fixed pattern noise signal. In Patent Document 1, the image quality is improved by subtracting only the reset noise signal (hereinafter referred to as first noise signal SR) of the FD region 508 from the value of the bright output signal (hereinafter referred to as Ss). I am planning.

On the other hand, in the case of an image pickup device having the charge holding portion 507A in each pixel in the path between the photoelectric conversion portion 500 and the FD region 508 as in this embodiment, the floating diffusion 508 is not the only source of fixed pattern noise. . A charge signal generated in the photoelectric conversion unit 500 is photoelectrically converted by switching off/on the first transfer transistor 501A that controls transfer between the photoelectric conversion unit and the charge holding unit 507A as in the operation from time t9 to time t10. From portion 500 is moved to charge holding portion 507A. At this time, in the charge holding unit 507A, in addition to the charge signal generated in the photoelectric conversion unit 500, there is fixed pattern noise S _T caused by transfer via the first transfer transistor 501A, which is not found in the above-described conventional example. are added and held.

As an example, in FIG. 5, let Sa1 be the total charge held in the charge holding unit 507A after one exposure/transfer/accumulation operation indicated by the accumulation time 602-1 between time t8 and time t10. If Sp is the charge generated by photoelectric conversion in the photoelectric conversion unit 500 in one exposure time (1/480 second), then Sa1=Sp+ _ST .

Further, in the present embodiment, where exposure and accumulation are repeated multiple times, transfer from the photoelectric conversion unit 500 to the charge holding unit 507A is repeatedly performed multiple times by switching off/on the first transfer transistor 501A. there is Therefore, substantially the same amount of fixed pattern noise _ST resulting from transfer via the first transfer transistor 501A is generated for each transfer. In the charge holding unit 507A, in addition to the charge finally generated by the photoelectric conversion in the photoelectric conversion unit 500, the amount of fixed pattern noise that is approximately linearly proportional to the number of times the first transfer transistor 501A is switched off/on is held. It will be done. In this case, the total charge SaM held in the charge holding unit 507A after being transferred M times is SaM=M*Sp+M* _ST . In the above example, since four transfers are performed, the total charge is Sa4=4*Sp+4* _ST .

After that, after releasing the reset of the FD region 508 at time t11, the signal charge is transferred from the signal holding unit 507A to the FD region 508 and the signal output line 523 via the second transfer transistor 502A. At this time, when passing through the FD area 508, the reset noise signal SR in the FD area 508 is added. Therefore, the charge signal output from the signal output line 523 to the readout circuit 600 as the bright output signal Ss is Ss=M*Sp+M*S _T + _SR , where M is the number of transfers. Since four transfers are performed in this embodiment, the bright output signal Ss is Ss=4*Sp+4*S _T +S _R. Thus, in addition to the charge Sp generated by the photoelectric conversion, the bright output signal Ss includes M times of _ST , which is noise caused by transfer via the transfer transistor 501A, and one time of FD, as fixed pattern noise. of reset noise S _R is included.

When only the reset noise _SR in the FD region 508 is subtracted from the bright output signal Ss as in the conventional example, the final output signal SF after subtraction is given by the following, assuming that the number of transfers is M times. is represented by the formula
SF = Ss - S _R
= (M x Sp + M x S _T + S _R ) - S _R
= M x Sp + M x S _T
As described above, in the conventional example, the final output signal SF contains all the noise components of the noise M× _ST caused by the transfer through the first transfer transistor 501A for M times, which causes deterioration of the image quality. turn into. When four transfers are performed, the final output signal Ss is Ss=4*Sp+4* _ST , and the noise component _ST for four times remains.

On the other hand, in the present embodiment, in addition to acquiring the reset noise S _R , the noise S _T caused by the transfer caused by the ON/OFF switching of the first transfer transistor 501A a plurality of times is acquired during the period from time t1 to time t7 in FIG. Acquisition of the noise correction signal Sn including the noise is performed. Therefore, noise caused by transfer via the first transfer transistor 501A can be removed.

Specifically, in a very short period from time t1 to time t2 in FIG. 5, the transfer pulse φTX3 changes from high level to low level, and reset of the photodiodes 500 in the first row is released. At the same time, the transfer pulse φTX1 changes from low level to high level, and the first transfer transistor 501A is turned on. At this time, assuming that a small amount of electric charge generated by photoelectric conversion in the photodiode 500 in a very short time is Sp_min, this electric charge Sp_min is transferred to the signal holding portion 507A. Further, at this time, transfer from the photoelectric conversion unit 500 to the signal holding unit 507A is performed via the first transfer transistor 501A. resulting noise _ST . The noise _ST caused by this transfer is added and accumulated in the signal holding unit 507A together with a small amount of charge Sp_min and held. As a result, the total charge Sa1 held in the charge holding unit 507A after one exposure/transfer/accumulation operation shown in the accumulation time 601-1 from time t1 to time t2 is Sa1=Sp_min+ _ST .

After time t3, the same operation as in accumulation time 601-1 is repeated four times in total for accumulation times 601-2, 601-3, and 601-4. Since the noise S _T caused by passing through the first transfer transistor 501A is integrated in substantially linear proportion to the number of transfers, the final charge SaM held in the charge holding unit 507A after M transfers is SaM=M*Sp_min+M* _ST . In this embodiment, since M=4 times, the final charge Sa4 is Sa4=4*Sp_min+4* _ST .

After the reset of the FD region 508 is released from time t5 to time t7, the signal charge is transferred from the signal holding unit 507A to the FD region 508 and, by extension, the signal output line 523 via the second transfer transistor 502A. At this time, when passing through the FD area 508, the reset noise signal _SR in the FD area 508 is added. The dark output signal Sn output from the signal output line 523 to the readout circuit 600 is Sn=M*Sp_min+M*S _T + _SR when transferred M times. Since M=4 times in this embodiment, Sn=4*Sp_min+4*S _T + _SR . This dark output signal Sn is held in the signal holding unit 411 .

In addition, after time t8, the charge of the light output signal Ss obtained by exposure/transfer/accumulation four times for each 1/480 second indicated by the accumulation times 602-1, 602-2, 602-3, and 602-4 The quantity is Ss=4*Sp+4*S _T +S _R. The charge of this bright output signal Ss is held in the signal holding section 410 . In the case of M transfers, Ss=M*Sp+M*S _T + _SR .

From time t14 to time t15, the final output signal SF obtained by subtracting the dark output signal Sn stored in the signal holding unit 411 from the bright output signal Ss stored in the signal holding unit 410 is obtained by transferring M times. is represented by the following formula.
SF = Ss - Sn
= (M x Sp + M x S _T + S _R ) - (M x Sp_min + M x S _T + S _R )
= M × (Sp-Sp_min)
At this time, the exposure time for generating the electric charge Sp_min is the minimum time unit controllable by the imaging device 100 and is extremely short, so that Sp>>Sp_min. In this case, SF=M×(Sp-Sp_min) can be approximated to SF=M×Sp, the influence of noise can be eliminated, and only the term of the charge signal Sp generated by photoelectric conversion can be extracted. In this embodiment, M=4 times and SF=4×Sp. In the present embodiment, the exposure time Sp_min is the minimum time unit that can be controlled by the imaging device, but the exposure time is not limited to this, and may be shorter than the exposure time per acquisition of the bright output signal. .

However, there are cases where the charge Sp_min generated from time t1 to time t2 is not so small as to be negligible with respect to the charge Sp generated from time t8 to time t10. For example, it may occur when the rate of light attenuation by the light attenuation means increases. In this embodiment, assuming a 2-stage ND, the exposure accumulation time per acquisition of the bright output signal was 1/480 second. However, the exposure accumulation time for obtaining a bright output signal is shortened to 1/960 seconds for ND 3 stages and 1/1920 seconds for ND 4 stages. In this case, the value of the charge Sp obtained by exposure accumulation per one bright output signal becomes small. On the other hand, the value of the charge Sp_min remains unchanged since the time taken to acquire the dark output signal does not change. As a result, the difference between the charge Sp, which had a sufficiently larger value than the charge Sp_min, decreases as the number of ND stages increases, and the influence of the charge Sp_min cannot be ignored.

Therefore, in the set number of ND stages, if the charge Sp_min is not so small as to be negligible with respect to the charge Sp, the final output signal SF=M×(Sp−Sp_min) is designed so that the noise can be removed from the signal. . For this purpose, the exposure time for each acquisition of the bright output signal should be increased by the amount corresponding to the amount of decrease due to the charge Sp_min. As an example, if there is a 10% reduction in Sp by subtracting Sp_min, then the exposure time of 1/480 second per bright output signal acquisition is increased by 10% to counteract the effect of the reduction. and set 11/4800 sec as the exposure time per time. That is, when the amount of light reduction by the light reduction means exceeds a predetermined amount of light reduction, the exposure time per acquisition of the bright output signal is reduced to the exposure accumulation time per acquisition of the dark output signal. You can increase it. In this way, when the amount of light reduction by the light reduction means exceeds a predetermined ratio, the influence of noise can be suppressed by increasing the exposure time per time when acquiring the bright output signal.

As described above, according to the present embodiment, even when an imaging signal is obtained by evenly distributing a short accumulation period into one frame period and accumulating a plurality of times collectively, complex fixed pattern noise components are removed from the imaging signal, Image quality can be improved.

(Second embodiment)
In the first embodiment, four exposure/accumulation operations (602-1 to 602-4) are repeated four times during the period for obtaining the bright output signal, and the noise correction signal is generated. exposure/accumulation operations (601-1 to 601-4) for obtaining a dark output of . In this embodiment, an example in which noise correction information is obtained based on the number of dark output exposure/accumulation times will be described. In the present embodiment, the number of exposure/accumulation operations during the period of obtaining the bright output signal is one, which is less than the number of exposure/accumulation operations. However, the present invention is not limited to this. , the number of exposure accumulation operations for obtaining a bright output or less.

FIG. 7 is a timing chart showing the driving sequence of the imaging element 184 in the second embodiment. In the following description, as in the first embodiment (FIG. 5), it is assumed that moving images are shot under the condition of 30 fps, and 1/480 second is accumulated four times during one shooting cycle of 1/30 second. The timing of control of the imaging element will be described by taking as an example a case where an image signal is obtained by adding times. Timing information such as photoelectric conversion, accumulation, and discharge shown in the timing chart is stored in the memory unit 190. When the system control CPU 178 performs photographing, it is read out from the memory unit 190 and transmitted to the image pickup device 184 for photographing. It controls the operation of element 184 . In FIG. 7, from time t1 to time t17 is defined as one imaging cycle (Nth imaging cycle), and after time t17, it is repeated.

At time t1, the transfer pulse φTX3(1) for the first row supplied from the vertical scanning circuit 700 changes from high level to low level. As a result, the third transfer transistor 503 is turned off, the reset of the photodiodes 500 in the first row is released, and the photodiodes 500 start accumulating signal charges. At the same time, the transfer pulse φTX1(1) of the first row changes from low level to high level. As a result, the first transfer transistor 501A is turned on, and the signal charge accumulated in the photodiode 500 begins to be transferred to the signal holding portion 507A holding the charge of the moving image in the first row. This time t1 is assumed to be the start of the imaging cycle.

At time t2, the transfer pulse φTX1(1) of the first row becomes low level. As a result, the first transfer transistor 501A is turned off, and transfer of the signal charge accumulated in the photodiode 500 to the signal holding portion 507A is completed. At the same time, the transfer pulse φTX3(1) of the first row changes from low level to high level. This turns off the third transfer transistor 503 and resets the photodiodes 500 in the first row again.

In the second embodiment, since one accumulation/transfer operation is performed from time t1 to time t2, the dark output signal stored in the signal holding unit 507A is equal to the charge Sp_min generated during this short accumulation time. The noise _ST due to transfer is added only once. Therefore, the total charge Sa1'' held in the charge holding unit 507A after one exposure/transfer/accumulation operation shown in the accumulation time 701-1 from time t1 to time t2 is Sa1''=Sp_min+ _ST . Become. Time t1 to time t2 are the same as in the first embodiment, but in the first embodiment four accumulation/transfer operations are performed to obtain a dark output signal, whereas in the second embodiment dark output signals are obtained. The accumulation/transfer operation to the signal holding unit 507A for signal acquisition is completed only once.

At time t3, the reset pulse φRES(1) of the first row becomes low level. As a result, the reset transistor 504 in the first row is turned off, and the reset state of the FD region 508 is released. At the same time, the selection pulse φSEL(1) for the first row becomes high level. As a result, the select transistor 506 in the first row is turned on, and readout of the charge signal in the first row becomes possible.

At time t4, the transfer pulse φTX2(1) of the first row becomes high level. As a result, the second transfer transistor 502A in the first row is turned on, and the charge signal held in the signal holding portion 507A is transferred to the FD region 508. FIG. An output corresponding to the change in potential of the FD region 508 is read out to the signal output line 523 via the amplification transistor 505 and the selection transistor 506 . At this time, the signal to which the reset noise _SR is added when passing through the FD region 508 is read out to the signal output line 523. Assuming that this read dark output signal is Sn1'', Sn1'' =Sp_min+S _T +S _R.

Also, in the second embodiment, unlike the first embodiment, the signal output line 523 is directly connected to the digital signal processing section 187 . The information of the dark output signal output to the digital signal processing section 187 through the signal output line 523 is stored in the memory section 190 by the CPU 178 .

At time t5, the reset pulse φRES(1) of the first row becomes high level. As a result, the reset transistor 504 is turned on, and the FD region 508 in the first row and the first signal holding section 507A are reset. At time t5, the selection pulse φSEL(1) for the first row and the transfer pulse φTX2(1) for the first row are at low level.

At time t6, the reset pulse φRES(1) of the first row becomes low level again. As a result, the reset transistor 504 in the first row is turned off, and the reset state of the FD region 508 is released. At the same time, the selection pulse φSEL(1) for the first row becomes high level. As a result, the select transistor 506 in the first row is turned on, and readout of the charge signal in the first row becomes possible.

At time t7, the transfer pulse φTX2(1) of the first row becomes high level. As a result, the second transfer transistor 502A in the first row is turned on, and the charge signal held in the signal holding portion 507A is transferred to the FD region 508. FIG. An output corresponding to the change in potential of the FD region 508 is read out to the signal output line 523 via the amplification transistor 505 and the selection transistor 506 . At this time, since no exposure/accumulation operation is performed between time t5 and time t7, the signal read out to the signal output line 523 includes the reset signal in the FD area 508 as in the case of the conventional example. Only noise _SR is included. This is referred to as a second dark output signal Sn2″ (=SR). The second dark output signal Sn2″ is output from the signal output line 523 to the digital signal processing section 187, and information is stored in the memory section 190 by the CPU 178. be done. Both of the two types of signals obtained from time t1 to time t8 are dark output signals mainly composed of fixed pattern noise. Information on the dark output signal is stored in the memory section 190 through the digital signal processing section 187 .

The operation from time t9 to time t14 is an operation for obtaining a bright output signal through exposure/accumulation periods that are repeated four times. This is the same as the operation of obtaining the bright output signal through the exposure/accumulation period (accumulation times 602-1 to 602-4) repeated four times from time t8 to time t13 in the first embodiment (FIG. 5). omitted. The same bright output signal is obtained in the first and second embodiments. However, the signal read out to the signal output line 523 is directly output to the digital signal processing section 187 and is stored in the memory section 190 by the CPU 178 as bright output information.

When exposure and accumulation are performed four times, the bright output signal Ss includes fixed pattern noise signals S _T and S _R and is expressed by Ss=4×Sp+4×S _T + _SR . Since the fixed pattern noise signal is included, there is a concern that the image quality is degraded as it is. The bright output signal Ss differs in the number of times that the noise _ST caused by the transfer is included four times and the reset noise _SR is included once. Therefore, the noise signal to be removed from the bright output signal Ss cannot be simply estimated by the integer multiple of the first dark output signal Sn1''=Sp_min+S _T + _{SR including S T and S R once each} .

Therefore, in the present embodiment, the first dark output signal Sn1″ and the second dark output signal Sn2″ stored in the memory unit 190 are combined and subtracted from the bright output signal Ss to remove the effect of noise. . Assuming that the final image output signal after the noise elimination operation is SF and the number of exposure/accumulation repetitions when acquiring the bright output signal is M, the noise effect can be eliminated by the following equation.
SF=Ss−M×(Sn1″−Sn2″)− _SR
At this time, since Sn1''=Sp_min+ _ST + _SR , Sn2''= _SR , and Ss=M*Sp+M* _ST + _SR , the above equation can be expanded as follows.
SF = Ss - M x {(Sp_min + S _T + S _R ) - S _R } - S _R
=Ss−M×(Sp_min+S _T )−S _R
=Ss-(M*Sp_min+M*S _T -S _R )
= M x Sp + M x S _T + S _R - (M x Sp_min + M x S _T - S _R )
= M x Sp - M x Sp_min
= M × (Sp-Sp_min)

As in the first embodiment, the exposure time of Sp_min is the minimum time unit that can be controlled by the imaging apparatus, and assuming that it is an extremely short time, and Sp>>Sp_min, SF=M×Sp can be approximated. Therefore, the effect of noise can be eliminated from the acquired bright output signal, and only the term of the charge signal Sp generated by photoelectric conversion can be extracted. In this embodiment, the number of transfers M is 4, so SF=4×Sp. Also, as in the first embodiment, the exposure time Sp_min is not limited to the minimum time unit that can be controlled by the imaging device, and may be shorter than the exposure time per acquisition of the bright output signal. .

As described above, according to the present embodiment, complex fixed pattern noise components can be removed even when using a dark output signal obtained by exposure/accumulation repetition times smaller than the number of exposure/accumulation repetitions when obtaining a bright output signal. , it is possible to acquire an image with improved image quality.

(Third Embodiment)
In the first embodiment and the second embodiment, a dark output signal is obtained every imaging cycle. However, in an environment where the value of the dark output signal does not change, it is not necessary to obtain the dark output signal each time. good. In the third embodiment, an example in which noise is removed from a captured image using previously recorded dark output signal information as a correction signal will be described.

FIG. 8 is a timing chart showing the driving sequence of the imaging element 184 in the second embodiment. In the following description, as in the second embodiment (FIG. 7), it is assumed that moving images are shot under the condition of 30 fps, and 1/480 seconds are accumulated four times during one shooting cycle of 1/30 seconds. The timing of control of the imaging element will be described by taking as an example a case where an image signal is obtained by adding times. Timing information such as photoelectric conversion, accumulation, and discharge shown in the timing chart is stored in the memory unit 190. When the system control CPU 178 performs photographing, it is read out from the memory unit 190 and transmitted to the image pickup device 184 for photographing. It controls the operation of element 184 . In FIG. 8, the period from time t1 to time t17 is defined as one imaging period (the Nth imaging period), and the period from time t17 to time t23 is defined as the N+1th imaging period. Below, points different from the second embodiment will be described.

In the third embodiment, information on the dark output signal recorded in the past is used as a correction signal to perform correction for removing noise from the captured image. Therefore, the dark output signal is obtained in the Nth imaging cycle, and the period Pn for obtaining the dark output signal is not provided in the (N+1)th imaging cycle. An example of doing so will be described. That is, in FIG. 8, the acquisition periods Pn and Ps for both the dark output signal and the bright output signal are provided in the N-th imaging period, but the period Pn for obtaining the dark output signal is not provided in the N+1-th imaging period. Only the period Ps' for acquiring the bright output signal is provided.

The operation of the N-th imaging cycle from time t1 to t17 in this embodiment (FIG. 8) is the same as the operation from time t1 to t17 in the second embodiment (FIG. 7). Both the dark output signal (the first dark output signal Sa1'' and the second dark output signal Sa2'') and the bright output signal Ss are obtained from this N-th imaging cycle, and the CPU 178 obtains the information thereof. Save in memory unit 190 . Noise is removed by subtracting a value obtained by numerically calculating the dark output signal from the bright output signal Ss. In this embodiment, the information of the dark output signal is stored even after the noise removal processing of the Nth imaging cycle is completed, and is also used in the noise removal processing of the next N+1 imaging cycle.

The operation of the (N+1)-th imaging cycle from time t17 to time t23 in this embodiment includes only the bright output signal acquisition operation and does not include the dark output signal acquisition operation. The bright output signal acquisition operation from time t17 to time t23 is the same as the bright output signal acquisition operation from time t9 to time t16 in the second embodiment (FIG. 7). The acquired bright output signal is stored in the memory unit 190 by the CPU 178 .

Here, since the dark output signal is not acquired in the (N+1)th imaging cycle, the dark output signal to be subtracted from the bright output signal for noise removal cannot be obtained from the same imaging cycle. Therefore, the dark output signal information acquired in the Nth imaging cycle is held in the memory unit 190, and the dark output signal acquired in the Nth imaging cycle is calculated from the bright output signal acquired in the (N+1)th imaging cycle. and subtracting it, noise can be removed.

According to this embodiment, it is possible to reduce the amount of calculation of the CPU 178 because it is not necessary to process the dark output signal for each imaging cycle while reducing noise. In addition, in this embodiment, since the time required for transferring the dark output signal is eliminated, there is no limitation on the time that can be used for exposing and accumulating the bright output signal in one shooting cycle, and the degree of freedom in setting the shutter speed is increased. .

If the imaging device 184 heats up by shooting a moving image for a long period of time, the dark output signal changes, and the stored dark output signal information may not be able to achieve ideal noise removal. Therefore, as a modified example of the third embodiment, a configuration will be described in which a temperature detecting means is provided to detect the temperature of the image pickup element and a dark output signal is re-obtained according to the detection result. FIG. 9 is a diagram illustrating the configuration of an imaging device 100 in a modified example of the third embodiment. A temperature detection unit 188 is added to the image pickup apparatus 100 shown in FIG. 9 in contrast to the image pickup apparatus 100 shown in FIG. 2, and other configurations are the same.

FIG. 10 is a flow chart showing moving image shooting processing assuming a change in temperature. The flow of FIG. 10 is started by the start of moving image shooting.
In step S<b>001 , the same imaging operation as the imaging operation in the N-th imaging cycle in FIG. Also, the noise-removed signal obtained by subtracting the dark output signal from the bright output signal is stored in the memory 190 as the final imaging signal in this imaging cycle. A sequence that acquires both the dark output signal and the bright output signal is referred to as a first imaging sequence. When the acquisition and storage of the final imaging signal are completed, the process proceeds to step S002.

In step S002, information is updated with the dark output signal acquired in step S001 as the latest dark output signal and stored in the memory 190, and the process proceeds to step S003.
In step S003, it is determined whether or not the user has performed an operation to end the moving image capturing. If the operation to terminate has been performed, the flow is terminated, and if not, the process proceeds to step S004.

In step S004, it is determined whether or not the temperature of the image sensor 184 detected by the temperature detection unit 188 has changed by a predetermined amount or more. If the temperature has changed by more than the predetermined temperature, the process returns to step S001 to obtain the dark output signal as well as the bright output signal. On the other hand, if the change in temperature is less than the predetermined temperature, the process proceeds to step S005.

In step S<b>005 , the same imaging operation as the imaging operation in the N+1th imaging cycle in FIG. Further, noise is removed by subtracting the dark output signal stored in step S002 from the bright output signal obtained in this step. A signal obtained by subtracting the dark output signal from the bright output signal is stored in the memory 190 as the final imaging signal in this imaging cycle. This sequence in which only the bright output signal is acquired and the previously stored dark output signal is used for correction is referred to as a second imaging sequence. When the acquisition and storage of the final imaging signal are completed, the process returns to step S003.

According to this modified example, it is possible to reduce the amount of calculation by eliminating the need to obtain the dark output signal for each imaging cycle, but when an error occurs in the dark output signal due to temperature change, a new dark output signal is obtained again. Thus, the influence of fixed pattern noise can be removed with high accuracy. Although temperature change has been described as a condition for obtaining a new dark output signal, the present invention is not limited to this. For example, even when the number of divisions for obtaining the bright output signal is changed, or when the exposure time for obtaining the bright output signal is changed, a dark output signal suitable for noise correction of the bright output signal is obtained. Therefore, a new dark output signal should be obtained.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist thereof.

100 imaging device 184 imaging device 187 digital signal processing unit 178 system control CPU
500 photoelectric conversion unit 507A first signal holding unit

Claims (10)

an imaging device in which pixels each having at least one charge holding unit are arrayed two-dimensionally with respect to one photoelectric conversion unit;
acquisition means for controlling the imaging device to acquire a first image captured with the shutter open and a second image captured with the shutter closed;
a correction means for correcting the first image using the second image;
The acquisition means is
obtaining the first image by accumulating charges in the charge holding unit in a time division manner during one imaging cycle;
acquiring the second image by accumulating charges in the charge holding unit at a number of times equal to or less than the number of time divisions corresponding to the first image;
An imaging apparatus, wherein a single exposure accumulation time corresponding to the second image is shorter than a single exposure accumulation time corresponding to the first image. 2. The imaging apparatus according to claim 1, wherein the number of times of exposure accumulation corresponding to said second image is equal to the number of times of exposure accumulation corresponding to said first image. 3. The image pickup apparatus according to claim 1, wherein the acquisition unit acquires the second image when the temperature of the image sensor changes by a predetermined temperature or more. 4. The imaging according to any one of claims 1 to 3, wherein said acquisition means acquires said second image when an exposure accumulation time corresponding to said first image is changed. Device. 5. The method according to any one of claims 1 to 4, wherein said acquisition means acquires said second image when the number of time divisions corresponding to said first image is changed. Imaging device. 3. The correction means generates correction information for noise correction from the second image, and corrects the first image by subtracting the correction information from the first image. 6. The imaging device according to any one of 1 to 5. 7. The imaging apparatus according to claim 6, further comprising a memory for storing said correction information. 8. The imaging apparatus according to any one of claims 1 to 7, wherein the exposure accumulation time per exposure corresponding to the second image is a minimum time unit controllable by the imaging apparatus. When the amount of light attenuation of the first image by time division exceeds a predetermined amount of light attenuation, the exposure accumulation time per exposure corresponding to the first image is changed to the exposure accumulation time per exposure corresponding to the second image. 9. The imaging apparatus according to any one of claims 1 to 8, wherein the time is increased by the exposure accumulation time. A control method for an imaging device having an imaging device in which pixels each having at least one charge holding unit for one photoelectric conversion unit are two-dimensionally arranged, comprising:
a step of acquiring a first image when the shutter is opened by accumulating charges in the charge holding unit in a time division manner during one imaging cycle;
obtaining a second image when the shutter is closed by accumulating charges in the charge holding unit a number of times equal to or less than the number of time divisions corresponding to the first image;
and correcting the first image using the second image;
A control method, wherein a single exposure accumulation time corresponding to the second image is shorter than a single exposure accumulation time corresponding to the first image.

Images