JP5428792B2 - Solid-state imaging device, driving method thereof, camera system, and program - Google Patents

Description

  The present invention relates to a solid-state imaging device typified by a CCD or CMOS image sensor, a driving method thereof, a camera system, and a program.

  A solid-state imaging device typified by a CCD or CMOS (Complementary Metal Oxide Semiconductor) used in a video camera, a digital still camera, or the like performs photoelectric conversion by a photoelectric conversion device that accumulates electric charge according to the amount of incident light.

In recent years, CMOS image sensors have attracted attention as solid-state imaging devices (image sensors) that replace CCDs.
This is because the CMOS image sensor overcomes the following problems.
That is, a dedicated process is required for manufacturing a CCD pixel, a plurality of power supply voltages are required for its operation, and a plurality of peripheral ICs need to be operated in combination.
This is because, in the case of such a CCD, the CMOS image sensor overcomes various problems such as a very complicated system.

The CMOS image sensor can be manufactured by using a manufacturing process similar to that of a general CMOS integrated circuit, can be driven by a single power source, and further, an analog circuit or logic using the CMOS process. Circuits can be mixed in the same chip.
For this reason, the CMOS image sensor has a plurality of great merits such that the number of peripheral ICs can be reduced.

The output circuit of a CCD is mainly a 1-channel (ch) output using an FD amplifier having a floating diffusion layer (FD).
In contrast, a CMOS image sensor has an FD amplifier for each pixel, and the output is mainly a column parallel output type in which a row in a pixel array is selected and read out in the column direction at the same time. It is.
This is because it is difficult to obtain a sufficient driving capability with an FD amplifier arranged in a pixel, and therefore it is necessary to lower the data rate, and parallel processing is advantageous.

  Such a CMOS image sensor is widely used as an image pickup element in an image pickup apparatus such as a digital camera, a camcorder, a surveillance camera, or an in-vehicle camera.

  FIG. 1 is a diagram illustrating an example of a pixel circuit of a CMOS image sensor including four transistors.

The pixel circuit 10 includes a photoelectric conversion element 11 made of, for example, a photodiode (PD).
The pixel circuit 10 has four transistors, that is, a transfer transistor 12, a reset transistor 13, an amplification transistor 14, and a selection transistor 15, as active elements, for the one photoelectric conversion element 11.

The photoelectric conversion element 11 photoelectrically converts incident light into an amount of electric charges (here, electrons) corresponding to the amount of light.
The transfer transistor 12 is connected between the photoelectric conversion element 11 and the floating diffusion FD, and a transfer signal (drive signal) TRG is given to the gate (transfer gate) through the transfer control line LTRG.
Thereby, the electrons photoelectrically converted by the photoelectric conversion element 11 are transferred to the floating diffusion FD.

The reset transistor 13 is connected between the power supply line LVREF and the floating diffusion FD, and a reset signal RST is given to the gate through the reset control line LRST.
As a result, the potential of the floating diffusion FD is reset to the potential of the power supply line LVREF.

The gate of the amplification transistor 14 is connected to the floating diffusion FD. The amplification transistor 14 is connected to the vertical signal line 16 via the selection transistor 15, and constitutes a constant current source and a source follower outside the pixel portion.
Then, an address signal (selection signal) SEL is given to the gate of the selection transistor 15 through the selection control line LSEL, and the selection transistor 15 is turned on.
When the selection transistor 15 is turned on, the amplification transistor 14 amplifies the potential of the floating diffusion FD and outputs a voltage corresponding to the potential to the vertical signal line 16. The voltage output from each pixel through the vertical signal line 16 is output to the readout circuit.

  In a CMOS image sensor, pixels having such a configuration are arranged in a two-dimensional array.

  In general, the photoelectric conversion element has an upper limit on the amount of accumulated charge, and when the amount of light exceeds a certain level, the amount of accumulated charge reaches a saturation level, and subject areas with a certain level of brightness are set to a saturated luminance level. So-called overexposure occurs.

In order to prevent such a phenomenon, in the solid-state imaging device, the exposure time is adjusted by controlling the charge accumulation time in the photoelectric conversion device according to the change in external light, etc., and the sensitivity is controlled to the optimum value, etc. Processing is performed.
For example, for a bright subject, the exposure time is shortened by cutting the shutter speed at a high speed, the charge accumulation time in the photoelectric conversion element is shortened, and an electric signal is output before the accumulated charge amount reaches the saturation level.
By such processing, it is possible to output an image in which the gradation corresponding to the subject is accurately reproduced.

However, when shooting a subject in which a bright place and a dark place are mixed, if the shutter is opened at a high speed, a sufficient exposure time cannot be taken in a dark portion, so that the S / N deteriorates and the image quality deteriorates.
As described above, in order to accurately reproduce the brightness levels of the bright and dark portions in the captured image of the subject in which the bright and dark portions are mixed, the following processing is necessary.
That is, it is necessary to realize a high S / N as a long exposure time for pixels with little incident light on the image sensor, and to avoid saturation for pixels with much incident light.

  In response to such a situation, in the CMOS image sensor, techniques for expanding the dynamic range are proposed in Patent Documents 1, 2, 3, and the like.

  In the solid-state imaging device disclosed in Patent Document 1, a plurality of control voltages are sequentially supplied to the gate electrode of the transfer transistor 12, and signal charges transferred by the transfer transistor 12 at that time are read out twice or more.

In the solid-state imaging device disclosed in Patent Document 2, an intermediate voltage having the same voltage value is supplied to the gate electrode of the transfer transistor 12 a plurality of times, and the signal charge transferred by the transfer transistor 12 at that time is read twice or more.
Here, the intermediate voltage (intermediate potential) is not a level at which the transfer transistor 12 is completely turned off as usual, but if electrons are accumulated more than a certain level, the surplus can overflow into the FD portion. Is a level.

  According to the solid-state imaging devices disclosed in Patent Documents 1 and 2, it is possible to acquire a signal with a high S / N linearity without narrowing a normal saturation level at low illuminance, and linear with respect to incident light exceeding the normal saturation level The dynamic range can be expanded while realizing a good S / N in the region.

In the imaging device disclosed in Patent Document 3, in a process of inputting a long exposure image and a short exposure image and generating a wide dynamic range image in which each effective image value is selectively combined, a luminance change pixel Is detected and pixel values are replaced.
In this process, an output pixel value is determined by applying a pixel value corresponding to the pixel position of the luminance change pixel in the blurred image based on the wide dynamic range image and the wide dynamic range image.
According to this imaging apparatus, it is possible to prevent the occurrence of defective gradation and false color of luminance change pixels caused by movement of a subject, and to obtain a natural-looking high-quality image.

Hereinafter, based on these techniques, the operation of a solid-state imaging device that performs reading at an intermediate potential will be described.
In the following description, for the sake of simplification, description of a method of applying an intermediate potential multiple times for suppressing variations in transistors is omitted. In an actual operation, a plurality of times of application at an intermediate potential is performed. In this case as well, the operation of the present invention can be used without change.

FIG. 2 is a diagram illustrating a temporal change in charge in a pixel in a solid-state imaging device that performs intermediate potential readout.
The horizontal axis S101 is a time axis, and the period S102 is the accumulation time of a long accumulation (hereinafter, long accumulation) frame. The vertical axis S103 represents the amount of charge stored in the pixel, S104 represents the saturation level of charge accumulation in the pixel, and S105 represents the intermediate potential for reading.

In the method of realizing a larger dynamic range by combining two frames, one short-time accumulation (hereinafter, short-lived) frame is used in addition to the long accumulation frame.
The accumulation time S108 of the short accumulation frame is from the start time S106 to the end time S107 of the short accumulation frame. Reading of the short accumulation frame is an intermediate time of the accumulation time of the long accumulation frame.
FIG. 2 shows the state of charge increasing with time for three color components in a certain adjacent pixel.
For example, component S109 is green (G), component S110 is red (R), component S111 is blue (B), and the like.
The rate of increase in the charge of each color component varies depending on the color of the subject and the color sensitivity of the image sensor.
FIG. 2 shows a case where the subject is dark and the rate of increase in the charge amount in all colors is slow.
At this time, in all color components, the final value accumulated in the pixel is obtained by reading the value of the long accumulation frame, and the image is reconstructed (demosaiced) from these color data. At this time, there is no output from the short accumulation frame.

FIG. 3 is a diagram illustrating a temporal change in charge in a pixel in a solid-state imaging device that reads out an intermediate potential when the subject is bright. S101 to S111 are the same as those in FIG.
In this case, the increase in the charge amount of each of the color components S109, S110, and S111 is faster than in the case of FIG. 2, and the intermediate potential MV is exceeded or saturated by the start time of accumulation of the short accumulation frame.
At this time, the values S202, S203, and S204 are read as the values of the short accumulation frame of each color as the respective color components by the readout after the accumulation time of the short accumulation frame after the reset S201 at the intermediate potential S105. An image is reconstructed (demosaiced) from the color data of these three short accumulation frames.

  In the example of FIGS. 2 and 3, adjacent color components are read out from the same short accumulation frame or the same long accumulation frame, and correct color restoration is performed from the set of these values.

FIG. 4 is a diagram illustrating temporal changes in charge in pixels in a solid-state imaging device that performs intermediate potential reading when there is a large difference in charge increase speed for each color.
The component S109 indicates a case where the intermediate potential S105 is exceeded by the accumulation start time of the short accumulation frame and accumulation in the short accumulation frame is performed.
The component S109 is reset at the start time S301 of the short accumulation frame, and then the charge S302 stored in the accumulation period of the short accumulation frame is read out.
On the other hand, in the other color components S110 and S111, accumulation in the short accumulation frame is not performed, and the charges S303 and S304 are read out as the output of the long accumulation frame.
In the adjacent pixels, the short accumulation value and the long accumulation value coexist. In this case, it is possible to restore (demosaic) the correct color.

In these three cases (FIGS. 2 to 4), it is assumed that there is no change in the brightness or color of the subject within the accumulation time of the long accumulation frame.
Therefore, the correct value can be obtained for each color from the value of the short accumulation frame and the long accumulation frame, the demosaiced from these color components, the restored color becomes the correct color, and the phenomenon such as generation of false color does not occur .

JP 2007-151069 A Japanese Patent Laid-Open No. 2008-99158 Japanese Patent Laid-Open No. 2008-227697

However, in a solid-state imaging device that performs reading using such an intermediate potential, if the subject changes in brightness or color during the accumulation time of the long-lived frame, such as when the subject moves, the right side of the short accumulation frame side is correct. There is a disadvantage that the value cannot be read.
For this reason, generation of a false color may occur when demosaicing using this color component.

FIG. 5 is a diagram illustrating a first example of a problem.
FIG. 5 shows a case where the image is dark before the start of the short accumulation frame (S401) and then becomes brighter (S402).

The accumulation time of the short accumulation frame is already bright, and as shown in FIG. 5, charges are accumulated at a faster rate than the accumulation in the previous dark condition (S109) (S403).
When the value S403 in the short frame is read and multiplied by the gain of the ratio of the accumulation time of the short accumulation frame and the long accumulation frame, the expected value S409 is obtained from the component values S406, S407, and S408 in FIG.
However, the actual value is the correct value S413 from the values S109, S410, S411, and S412 shown in FIG.
Therefore, the expected value S409 obtained by multiplying the read value from the simple short accumulation frame by the gain is larger than the actual value S413 by the component value S414.
When this value is used, the other colors read in the long accumulation frame, S404 and S405 are correct values. When these values are demosaiced in this part and the color is obtained, it is different from the actual color. In some cases, the image quality of the moving subject portion is degraded.

FIG. 6 is a diagram illustrating a second example of a problem.
FIG. 6 shows a case where the image is bright until the start of the short accumulation frame (S501) and then becomes dark (S502).

The accumulation start time of the short accumulation frame is already dark, and as shown in FIG. 6, charges are accumulated at a slower rate than the accumulation in the previous bright situation (S109) (S503).
When the value S503 in the short accumulation frame is read and multiplied by the gain of the ratio of the accumulation times of the short accumulation frame and the long accumulation frame, an expected value S507 is obtained from S506 in FIG.
However, the actual value is the correct value in S510 from the values S109, S508, and S509 shown in FIG.
For this reason, the predicted value S507 from the value read from the simple short accumulation frame is smaller by S511 than the actual value S510.
When this value is used, the other color components S504 and S505 read out in the long accumulation frame are correct values, and when demosaicing is performed together with these values and the color of this portion is obtained, it is different from the actual color. In some cases, the image quality of the moving subject portion is degraded.

In order to solve such a problem, in Patent Document 3, a new image is created and replaced as a linear combination of a wide dynamic range image and a blurred image generated from the generated false color in the false color occurrence portion, thereby making the false color inconspicuous. Such a method is used.
However, this technique cannot suppress the occurrence of false colors. In addition, there is a problem that blur is increased by using a blurred image.

  The present invention can be expected to improve the accuracy of the estimated value from a plurality of short-time accumulation values even when the subject moves and changes in brightness or color, and can suppress the occurrence of false colors, etc. The object is to provide a solid-state imaging device capable of obtaining a more natural image, a driving method thereof, a camera system, and a program.

A solid-state imaging device according to a first aspect of the present invention includes a pixel including a photoelectric conversion element that converts an optical signal into a signal charge, and a transfer gate that transfers the signal charge photoelectrically converted by the photoelectric conversion element to an output node. A pixel array unit arranged in a matrix, a shutter operation of the pixel array unit, a pixel driving unit that controls the operation of the pixel so as to perform reading from the pixel of the pixel array unit, and a reading by the pixel driving unit A signal processing system for processing the output signal, wherein the pixel driving unit is intermediate during a first accumulation period of a first exposure time for acquiring an image between an electronic shutter operation and a normal readout operation. A read scan is performed by voltage transfer, and includes a second exposure time for image acquisition according to a read operation by the intermediate voltage transfer, for transfer by an intermediate voltage of the same or different voltage value. A plurality of second accumulation periods of a second exposure time shorter than the first exposure time are set during the first accumulation period of the first exposure time, signal processing system uses the value read by the plurality of second accumulation period, have rows process of estimating the output value of the second accumulation period, when the subject has changed from dark to bright, the output value In the process of estimating the maximum possible value as an actual value is increased from the end of the second accumulation period set in the dark period, and then the second accumulation period set in the bright period is increased. As the output value when increasing at the accumulation speed, the minimum possible value as the actual value is increased from the start time of the second accumulation period set in the bright period, and then the second accumulation When it increases at the accumulation rate of the period When the subject changes from light to dark in the process of estimating the output value, the maximum possible value as the actual value is set in the dark period at the speed of the component that exceeds the intermediate potential in the light period. The output value is increased at the speed of the component until the start of the second accumulation period, and then increased at the accumulation speed of the second accumulation period set in the dark period, The minimum possible value as an actual value increases at the speed of the component until the end of the second accumulation period set in the bright period, and then the second accumulation period set in the dark period The output value in the second accumulation period is estimated from the output value as the maximum value and the output value as the minimum value .

A camera system according to a second aspect of the present invention includes a solid-state imaging device and an optical system that forms a subject image on the solid-state imaging device, and the solid-state imaging device converts an optical signal into a signal charge. A pixel array unit in which pixels including a photoelectric conversion element and a transfer gate that transfers a signal charge photoelectrically converted by the photoelectric conversion element to an output node are arranged in a matrix; a shutter operation of the pixel array unit; and A pixel driving unit that controls the operation of the pixel so as to perform reading from the pixels of the pixel array unit, and a signal processing system that processes a signal read by the pixel driving unit, and the pixel driving unit includes: During the first accumulation period of the first exposure time for acquiring an image between the electronic shutter operation and the normal readout operation, readout scanning is executed by intermediate voltage transfer, and the readout operation by the intermediate voltage transfer is performed. Including a second exposure time for image acquisition, a function of executing one or more selected scans for transfer using an intermediate voltage of the same or different voltage value, and a first scan time shorter than the first exposure time. A plurality of second accumulation periods of two exposure times are set during the first accumulation period of the first exposure time, and the signal processing system uses the values read in the plurality of second accumulation periods to There line processing for estimating the output value of the accumulation period, when the subject has changed from dark to bright, the process of estimating the output value, a maximum possible value as the actual value, the set in the dark period As the output value when increasing from the end of the second accumulation period and then increasing at the accumulation speed of the second accumulation period set in the light period, the minimum possible value as the actual value is Set in the above light period A process for estimating the output value when the subject changes from light to dark when it is increased from the start of the second accumulation period and then increased at the accumulation speed of the second accumulation period. The maximum possible value as an actual value is increased at the speed of the component at the speed of the component exceeding the intermediate potential in the light period until the start of the second accumulation period set in the dark period, and thereafter The output value when increasing at the accumulation speed of the second accumulation period set in the dark period, and the minimum value possible as the actual value is the second value set in the bright period. The output value increases when the accumulation period ends until the end of the accumulation period, and then increases at the accumulation speed of the second accumulation period set in the dark period, and the output value as the maximum value And most The output value of the second accumulation period is estimated from the output value as a small value .

A solid-state imaging device driving method according to a third aspect of the present invention includes a photoelectric conversion element that converts an optical signal into a signal charge, and a transfer gate that transfers the signal charge photoelectrically converted by the photoelectric conversion element to an output node. When driving the pixel array unit in which the pixels including the pixel array are arranged, the first exposure time and the second accumulation period of the second exposure time shorter than the first exposure time are defined as the first accumulation period of the first exposure time. During the first accumulation period of the first exposure time in which multiple images are set and an image is acquired between the electronic shutter operation and the normal readout operation, readout scanning is executed by intermediate voltage transfer, and readout operation by the intermediate voltage transfer And a second scanning time for image acquisition according to the above, and a selective scan for transfer with an intermediate voltage of the same or different voltage value is executed once or a plurality of times and read out in a plurality of second accumulation periods. Value There are, have rows process of estimating the output value of the second accumulation period, when the subject has changed from dark to bright, the process of estimating the output value, a maximum possible value as the actual value, the dark period The output value is increased when the second accumulation period set at the end of the second accumulation period is reached and then increased at the accumulation speed of the second accumulation period set during the light period, and can be an actual value. The minimum value is set as an output value when increasing from the start time of the second accumulation period set in the bright period and then increasing at the accumulation speed of the second accumulation period. When the output value is changed from dark to dark, in the process of estimating the output value, the maximum possible value as an actual value is set to the second accumulation set in the dark period at the speed of the component exceeding the intermediate potential in the bright period. Period The output value when increasing at the speed of the component up to the start time and then increasing at the accumulation speed of the second accumulation period set in the dark period, and the minimum possible value as the actual value , Increased at the rate of the component until the end of the second accumulation period set in the light period, and then increased at the accumulation rate of the second accumulation period set in the dark period The output value of the second accumulation period is estimated from the output value as the maximum value and the output value as the minimum value .

According to a fourth aspect of the present invention, pixels including a photoelectric conversion element that converts an optical signal into a signal charge and a transfer gate that transfers the signal charge photoelectrically converted by the photoelectric conversion element to an output node are arranged in a matrix. A process of setting a plurality of first exposure times and a second accumulation period of a second exposure time shorter than the first exposure time during the first accumulation period of the first exposure time when driving the pixel array unit In the first accumulation period of the first exposure time for acquiring an image between the electronic shutter operation and the normal reading operation, the reading scan is executed by the intermediate voltage transfer, and the image acquisition corresponding to the reading operation by the intermediate voltage transfer is performed. A process of executing a selective scan for transfer with an intermediate voltage of the same or different voltage value, including a second exposure time for the first and second values, and a value read in a plurality of second accumulation periods make use of, A process of performing processing for estimating an output value of the second accumulation period, wherein the case where the subject is changed from dark to bright, the process of estimating the output value, a maximum possible value as the actual value, The output value is an actual value that increases from the end of the second accumulation period set in the dark period and then increases at the accumulation speed of the second accumulation period set in the light period. As the output value when increasing from the start of the second accumulation period set in the light period and then increasing at the accumulation speed of the second accumulation period, Is changed from light to dark, in the process of estimating the output value, the maximum possible value as the actual value is set in the dark period at the speed of the component exceeding the intermediate potential in the light period. 2 accumulation It increases at the speed of the component until the start point in between, and then the output value when increasing at the accumulation speed of the second accumulation period set in the dark period, the minimum possible value as the actual value The value increases at the speed of the component until the end of the second accumulation period set in the light period, and then increases at the accumulation speed of the second accumulation period set in the dark period. This is a program for causing a computer to execute the driving process of the solid-state imaging device that estimates the output value of the second accumulation period from the output value as the maximum value and the output value as the minimum value .

According to the present invention, even when there is a change in brightness or color due to movement of a subject, an improvement in the accuracy of an estimated value can be expected from a plurality of short-time accumulation values, and generation of false colors can be suppressed. .
Thereby, a more natural image can be obtained.

It is a figure which shows the pixel example of the CMOS image sensor comprised by four transistors. It is a figure which shows the time change of the electric charge in the pixel in the solid-state image sensor which reads intermediate potential. It is a figure which shows the time change of the electric charge in the pixel in the solid-state image sensor which reads intermediate potential when a to-be-photographed object is bright. It is a figure which shows the time change of the electric charge in the pixel in the solid-state image sensor which performs intermediate potential read when there exists a big difference in the increase rate of an electric charge for every color. It is a figure which shows the 1st example used as a problem. It is a figure which shows the 2nd example used as a problem. It is a figure which shows the structural example of the CMOS image sensor (solid-state image sensor) which concerns on embodiment of this invention. It is a figure which shows an example of the pixel of the CMOS image sensor comprised by four transistors which concern on this embodiment. It is a figure for demonstrating the drive method which concerns on this 1st Embodiment when a to-be-photographed object changes from dark to bright. It is a figure shown about the estimation method of a correct value from the value of the short accumulation in the condition of FIG. It is a figure for demonstrating the drive method which concerns on this 1st Embodiment in the case of changing from light to dark. It is a figure shown about the estimation method of a correct value from the value of the short accumulation in the condition of FIG. It is a figure which shows the example which changed the brightness different from FIG. It is a figure which shows the method of estimation with the drive method in FIG. It is a figure for demonstrating the drive method which concerns on the 2nd embodiment. It is a block diagram including the structural example of the solid-state image sensor containing the estimation circuit which estimates the correct value of the short accumulation frame which concerns on this embodiment. It is a figure which shows an example of a structure of the camera system with which the solid-state image sensor which concerns on embodiment of this invention is applied.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The description will be given in the following order.
1. 1. Overall configuration example of solid-state imaging device First Embodiment 3 Second Embodiment 4. Third Embodiment 5 Fourth embodiment 6 and other supplementary explanations

<1. Example of overall configuration of solid-state image sensor>
FIG. 7 is a diagram illustrating a configuration example of a CMOS image sensor (solid-state imaging device) according to the embodiment of the present invention.

  The CMOS image sensor 100 includes a pixel array unit 110, a row selection circuit (Vdec) 120 as a pixel driving unit, and a readout circuit (AFE) 130.

  In the pixel array section 110, a plurality of pixel circuits 110A are arranged in a two-dimensional shape (matrix shape) of M rows × N columns.

  FIG. 8 is a diagram illustrating an example of a pixel of a CMOS image sensor including four transistors according to the present embodiment.

The pixel circuit 110A includes a photoelectric conversion element 111 made of, for example, a photodiode (PD).
The pixel circuit 110 </ b> A has four transistors, which are a transfer transistor 112, a reset transistor 113, an amplification transistor 114, and a selection transistor 115, as active elements for the one photoelectric conversion element 111.

The photoelectric conversion element 111 photoelectrically converts incident light into an amount of electric charges (here, electrons) corresponding to the amount of light.
The transfer transistor 112 is connected between the photoelectric conversion element 111 and the floating diffusion FD as an output node, and a transfer signal TRG as a control signal is given to a gate (transfer gate) through the transfer control line LTRG.
Thereby, the transfer transistor 112 transfers the electrons photoelectrically converted by the photoelectric conversion element 111 to the floating diffusion FD.

The reset transistor 113 is connected between the power supply line LVREF and the floating diffusion FD, and a reset signal RST, which is a control signal, is given to the gate of the reset transistor 113 through the reset control line LRST.
As a result, the reset transistor 113 resets the potential of the floating diffusion FD to the potential of the power supply line LVREF.

The gate of the amplification transistor 114 is connected to the floating diffusion FD. The amplification transistor 114 is connected to the vertical signal line LSGN via the selection transistor 115, and constitutes a constant current source and a source follower outside the pixel portion.
A selection signal SEL, which is a control signal corresponding to the address signal, is supplied to the gate of the selection transistor 115 through the selection control line LSEL, and the selection transistor 115 is turned on.
When the selection transistor 115 is turned on, the amplification transistor 114 amplifies the potential of the floating diffusion FD and outputs a voltage corresponding to the potential to the vertical signal line LSGN.
The voltage output from each pixel through the signal line LSGN is output to the readout circuit 130.
These operations are performed simultaneously for each pixel for one row because, for example, the gates of the transfer transistor 112, the reset transistor 113, and the selection transistor 115 are connected in units of rows.
For example, the transfer signal TRG for turning on and off the transfer transistor 112 is set and supplied to an intermediate voltage value (referred to as an intermediate voltage or an intermediate potential) by a driving process of the row selection circuit 120, as will be described later.

A reset control line LRST, a transfer control line LTRG, and a selection control line LSEL wired to the pixel array unit 110 are wired as a set for each row of the pixel array.
These reset control line LRST, transfer control line LTRG, and selection control line LSEL are driven by the row selection circuit 120.

The row selection circuit 120 controls the operation of pixels arranged in an arbitrary row in the pixel array unit 110. The row selection circuit 120 controls pixels through the control lines LSEL, LRST, and LTRG.
The row selection circuit 120, switch to a global shutter system that performs exposure in a rolling shutter mode or all pixels simultaneously perform exposure the exposure system for every row in response to a shutter mode switching signals generated by the control system (not shown), performs image drive control .

Row selection circuit 120 includes, for example, a shift register or an address decoder.
The row selection circuit 120 appropriately generates pixel drive signals such as a transfer signal TRG, a reset signal RST, and a selection signal SEL under the control of a control system (not shown).
Thereby, the row selection circuit 120 selects each pixel circuit 100A of the pixel array unit 110 while scanning the electronic shutter row and the readout row in the vertical direction in units of rows.
Then, the row selection circuit 120 discards (resets) the signal of the pixel circuit 110A in the row for the electronic shutter row, and performs readout for reading out the signal of the pixel circuit 110A in the row for readout. Drive.

Although not shown, the row selection circuit 120 has a readout scanning system for performing an operation of reading out the signal VSL of each pixel circuit 110A in the readout row while performing selective scanning sequentially in units of rows of the pixel circuit 110A.
Although not shown, the row selection circuit 120 has an electronic shutter scanning system that performs an electronic shutter operation on the same row (electronic shutter row) ahead of the readout scanning by the readout scanning system by a time corresponding to the shutter speed. .

The period from the timing when the unnecessary charge of the photoelectric conversion element 111 is reset by the electronic shutter operation by the electronic shutter scanning system to the timing when the signal of the pixel circuit 110A is read by the reading operation by the readout scanning system is the first exposure time. It becomes.
This first exposure time corresponds to a first accumulation period of signal charges in the pixel circuit 110A.
That is, the electronic shutter operation is an operation of resetting the signal charge accumulated in the photoelectric conversion element 111 and starting accumulation of a new signal charge after the reset.
In the present embodiment, the second accumulation of the second exposure time determined by the time interval for driving the transfer transistor 112 of the pixel circuit 110A that outputs the video signal read during the first exposure time during the first exposure time. A period is set.

The row selection circuit 120 has a preceding selection function of selectively scanning a plurality of rows, for example, two rows, at equal intervals prior to the readout row to be selectively scanned.
This advance selection function can be configured including, for example, a shift register or an address decoder.
The preceding selection function selectively scans two rows at equal intervals in advance of the readout row to be selectively scanned by scanning the selected scan under the control of a control system (not shown) and appropriately generating the transfer signal TRG. To do.
In this selective scanning, an operation of transferring the signal charge accumulated in the photoelectric conversion element 111 to the FD based on the transfer signal TRG is performed.

  The row selection circuit 120 outputs a transfer signal TRG, a reset signal RST, and a selection signal SEL for turning on and off the transfer transistor 112, the reset transistor 113, and the selection transistor 115 of the pixel circuit 110A in synchronization with the selection scan. To supply. The row selection circuit 120 is a transfer signal TRG of an intermediate voltage (intermediate voltage or intermediate potential) between the voltage for turning on and off each transistor of the pixel circuit 110A and the reference voltage in synchronization with the selection scanning by the preceding selection function. Is supplied to the transfer transistor 112 of the pixel circuit 110A.

The row selection circuit 120 executes readout scanning by intermediate voltage transfer during the first exposure time period in which an image in a low illuminance area is acquired between the electronic shutter operation and the normal readout operation.
The row selection circuit 120 is preceded by a second exposure time for acquiring an image in a high illuminance region before the read operation by the intermediate voltage transfer, and performs row selection scanning for dummy transfer by the intermediate voltage of the same or different voltage value. Is executed once or a plurality of times.
In the present embodiment, a plurality of second exposure time periods shorter than the first exposure time are set during the first exposure time period.
In the present embodiment, a plurality of second accumulation periods having a short second exposure time are set during the first exposure time period. For example, a frame having a short second accumulation period set to the same accumulation time Ts is used. Have multiple.
Alternatively, in the present embodiment, there are a plurality of sets of frames having short second accumulation periods with different accumulation times, and one set of frames has a plurality of frames of short second accumulation periods set to the same accumulation time. .

The readout circuit 130 performs a predetermined process on the signal VSL output through the vertical signal line LSGN from each pixel circuit 110A of the readout row selected or preselected by the driving of the row selection circuit 120, for example, signal processing The subsequent pixel signal is temporarily held.
A circuit configuration including a sample hold circuit that samples and holds a signal output through the vertical signal line LSGN can be applied to the readout circuit 130.
Alternatively, the readout circuit 130 includes a sample-and-hold circuit, and a circuit configuration including a function for removing pixel-specific fixed pattern noise such as reset noise and threshold variation of the amplification transistor 114 by CDS (correlated double sampling) processing is applicable. It is.
In addition, the reading circuit 130 can have a configuration in which an analog-digital (AD) conversion function is provided and a signal level is a digital signal.

  The signal processing system or the subsequent signal processing system included in the readout circuit 130 of the present embodiment has an estimation processing function for performing the following estimation processing from the readout signals at the first exposure time and the second exposure time described above.

The signal processing system according to the present embodiment has a plurality of second exposure times even when there is a change in brightness in the accumulation period, such as when a moving subject is photographed in a solid-state imaging device that performs readout at an intermediate potential. Estimate the correct value of the corresponding short-time accumulation (short accumulation) frame. The short accumulation frame corresponds to the frame of the second accumulation period.
Thereby, the signal processing system can reproduce a correct color.

In the present embodiment, a plurality of short accumulation frames set in the same accumulation period are provided in a long accumulation (long accumulation) frame period corresponding to the first exposure time. The correct value is estimated using the output value from the frame. The long accumulation frame corresponds to the frame of the first accumulation period.
When the dynamic range is expanded using frames of two different accumulation periods, a frame rate twice as high as that of the final video is usually required.
In this embodiment, a case where two short accumulation frames are added will be described. In this case, an operation at a frame rate four times that of the final video is required.

As described above, the solid-state imaging device 100 of the present embodiment is capable of reading at an intermediate potential, and has a short second accumulation period set to the same accumulation time Ts in the long accumulation time period TI. It is characterized by having multiple frames.
Then, the signal processing system estimates the output value from the frame of the short second accumulation period using the values from the frames of the plurality of short second accumulation periods (values read in the second accumulation period). It has a function.

Alternatively, as described above, the solid-state imaging device 100 of the present embodiment is capable of reading at an intermediate potential, and has a short second accumulation period with a different accumulation time Ts in a long accumulation time period TI. Have multiple sets. One set of frames has a plurality of frames having a short accumulation time Ts set in the same accumulation period.
The signal processing system has a function of estimating the output value of each set of frames having a short accumulation period using the values from the plurality of frames having a short accumulation time Ts. That is, the signal processing system has a function of estimating from output values of frames having the same accumulation period.

When the signal processing system of the solid-state imaging device 100 of the present embodiment uses a plurality of short second accumulation period frames having a plurality of different accumulation times, it is combined with output values of other short accumulations having different accumulation times. The function of estimating the correct output value of the short accumulation or detecting the occurrence of false color is provided.
That is, the signal processing system can be configured to estimate the output value together with the output values of other frames having different accumulation times.

The signal processing system assumes that the accumulation time of a desired short accumulation frame is Ts, the accumulation time of each of the N short frames is Ts / N, and the N short frames are predicted as the output value of the short frames. The total value of output from livestock frames.
That is, the signal processing system can be configured to have a function of estimating as a total value.

The signal processing system is set to a first set of short accumulation frames set to a short accumulation time where N accumulation times are TS1 / N and a short accumulation time where M accumulation times are Ts2 / M. With a second set of short storage frames.
The signal processing system then calculates the total output from the N short frames as the expected value of the first set of short accumulation time outputs, and the predicted output value of the second set of short accumulation periods. It is also possible to configure so that the total output from the M short frames is used.

  Hereinafter, a plurality of processes in the signal processing system will be described in detail with reference to FIGS.

<2. First Embodiment>
FIG. 9 is a diagram for explaining the driving method according to the first embodiment when the subject changes from dark to bright as in FIG. 5.

Here, in order to facilitate understanding, the same components as those in FIGS. 2 to 6 are denoted by the same reference numerals S101 to S111.
FIG. 2 is a diagram illustrating a temporal change in charge in a pixel in a solid-state imaging device that performs intermediate potential readout.
The horizontal axis S101 is a time axis, and the period S102 is the accumulation time of a long accumulation (hereinafter, long accumulation) frame. The vertical axis S103 represents the amount of charge stored in the pixel, S104 represents the saturation level of charge accumulation in the pixel, and S105 represents the intermediate potential for reading.

In a method of realizing a larger dynamic range by combining two frames, a plurality of short-time accumulation (hereinafter, short-lived) frames are used in addition to a long accumulation frame.
FIG. 9 also shows the state of charge that increases with time for three color components in a certain adjacent pixel.
For example, component S109 is green (G), component S110 is red (R), component S111 is blue (B), and the like.
The rate of increase in the charge of each color component varies depending on the color of the subject and the color sensitivity of the image sensor.

  In FIG. 9, S601 indicates a dark (low illuminance) period, and S602 indicates a bright (high illuminance) period.

In the first embodiment, three short accumulations Ts1 (S603), Ts2 (S604), and Ts3 (S605) set to the same accumulation time are set.
These three short values are read out and used to estimate the correct value. The case where it becomes dark until the start time of the middle short accumulation Ts2 (S601) and then becomes brighter (S602) will be described.

The component S110 is read from the long accumulation as S608 without increasing to the intermediate potential although the rate of increase of the charge amount increases after the start of Ts2. Similarly, the component S111 is read from the long accumulation as S609.
On the other hand, the component S109 does not reach the intermediate potential until the first short accumulation Ts1, the output of the short accumulation Ts1 becomes zero, but exceeds the intermediate potential at the beginning of the second short accumulation Ts2, The charge is accumulated in a bright subject, and the value 606 is read out by reading out the short accumulation Ts2.
S607 is similarly read out in the third short accumulation Ts3.
For convenience of the following description, it is assumed that the brightness of a bright subject does not change, and S606 and S607 have the same value.
From the output values from these three short accumulations, the actual accumulated charge amount can be estimated more accurately.

  FIG. 10 is a diagram illustrating a method for estimating a correct value from the short accumulation value in the situation of FIG.

From the short accumulation output value of FIG. 9, the maximum possible value as an actual value is an intermediate potential at the end of the short accumulation Ts1, and is an output value when increasing at the accumulation rate of S701 and then the short accumulation Ts2. S702.
Further, the minimum possible value as the actual value is the intermediate potential S703 at the start of the short accumulation Ts2, and then the output value S704 when increasing at the increasing speed of the short accumulation Ts2.
Therefore, the range estimated as the correct value is the section of S705 in FIG. As the most probable estimated value, an intermediate value between S702 and S704 is adopted.

  FIG. 11 is a diagram for explaining the driving method according to the first embodiment when the light changes to dark as in FIG. 6.

Similarly to FIG. 9, three short accumulations Ts1 (S803), Ts2 (S804), and Ts3 (S805) set to the same accumulation time are set.
These three short values are read out and used to estimate the correct value.
A case will be described in which the light is bright until the start time of the middle short accumulation Ts2 (S801) and then dark (S802).

The component S110 is read from the long accumulation as S809 after the rate of increase of the charge after Ts2 decreases. The component S111 is also read from the long accumulation as S810.
On the other hand, in the component S109, the accumulated charge amount reaches the intermediate potential by the first short accumulation Ts1, and the output of the short accumulation Ts1 is read as S806.
At the beginning of the second short accumulation Ts2, the intermediate potential is exceeded, charge accumulation is performed on a dark subject, and the value S807 is read by reading the short accumulation Ts2.
Similarly, the value S808 is read out in the third short accumulation Ts3.
For convenience of the following description, it is assumed that there is no change in the brightness of a dark subject, and S807 and S808 have the same value. The actual accumulated charge amount can be estimated from the output values from these three short accumulations Ts1, Ts2, and Ts3.

  FIG. 12 is a diagram illustrating a method for estimating a correct value from the short accumulation value in the situation of FIG.

The maximum possible value as the actual value is the output value S902 when increasing at the speed of the component S109 until the start of the short accumulation Ts2, and then increasing at the accumulation speed of the short accumulation Ts2.
Further, the minimum possible value as an actual value is the output value S904 when increasing at the speed of the component S109 until the end of the short accumulation Ts1, and then increasing at the increase speed of the short accumulation Ts2.
Therefore, the estimated range of the correct value is S905 in FIG.
A method such as adopting an intermediate value between S902 and S904 as the most probable estimated value is employed.
By using the estimation method of this example, it is possible to detect a change in the brightness or color of a subject that could not be detected from a single short accumulation output value. It is possible to obtain a correct value.
In the case of other changes in brightness, the correct value range can be estimated in the same manner.

FIG. 13 is a diagram showing an example in which the brightness changes different from FIG.
FIG. 14 is a diagram showing an estimation method in the driving method in FIG.

The output of the short accumulation frame is S1106, S1107, and S1108 when the short accumulation Ts1 is brightened until the time of reading and then darkened.
Based on outputs from these three short accumulation frames, an estimation method similar to that shown in FIG. 11 is used as shown in FIG. 14, and an intermediate value between the estimated maximum value S1202 and the minimum value S1204 is appropriate as the most probable value.
In the case of the change in brightness shown in FIG. 14, the actual value is S1204.
In this way, it is possible to estimate a correct value in a similar manner for various cases of changes in brightness.

<3. Second Embodiment>
FIG. 15 is a diagram for explaining the driving method according to the second embodiment.

  In the second embodiment, an example in which the present invention is applied to a case where a frame having three different accumulation times is combined to extend a high dynamic range will be described. In this case, the necessary frame rate is 8 times.

In this example, four short accumulations S1001 (Tss1), S1002 (Tss2), S1003 (Tss3), and S1004 (Tss4) having a short accumulation time are used.
In this example, three short accumulations S1005 (Ts1), S1006 (Ts2), and S1007 (Ts3) set to accumulation times longer than the short accumulations Tss1 to Tss4 are used, and alternately read as shown in the figure. Do.
Use four short short accumulations to capture light and dark changes and estimate the correct value for the accumulation time Tss.
Similarly, a short accumulation having three long accumulation times is used to catch a change in light and dark, and a correct value at the accumulation time Ts is estimated. The correct value estimation method for each frame can be performed in the same manner as in the first embodiment.

<4. Third Embodiment>
In the third embodiment, in the conventional case where a wide dynamic range is realized by synthesizing four frames, the output data of other short accumulation frames having different accumulation times are used together, and each short accumulation is correct. A method for estimating a value is provided.
This method can also be used effectively when detecting the occurrence of a false color or correcting the false color using the detection.

In FIG. 9, even when different accumulation times are used for the three short accumulation frames S603, S604, and S606, a correct value can be estimated by the same method.
By changing the accumulation time of the short accumulation frame in this way, it is possible to realize a wide dynamic range by combining four frames, and even in this case, it is possible to estimate a correct value as well.

<5. Fourth Embodiment>
In this example, in FIG. 9, in the case of three short accumulations Ts1 (S603), Ts2 (S604), and Ts3 (S605) set to the same accumulation time that is one third of the desired short livestock accumulation time Ts As the estimation of the correct value of the short accumulation, the total of the output values from the three short accumulations is obtained.
Not limited to a total of three short蓄, it can be realized similarly in the case of the N short slaughter with storage time Ts / N.
Similarly, in the case of being composed of a plurality of sets of frames having different accumulation times of short accumulation, similarly, when estimating the correct value by summing the values from the accumulation frames in each group Can be applied similarly.

<6. Other supplementary explanation>
In the description of the embodiment of the present invention, a case where the dynamic range is expanded using two types of frames having two different accumulation times will be mainly described. Among these, the short accumulation having three identical accumulation times is described. The description will be focused on the case of using the frame.
However, the application of the present invention is not limited to such a case.
Easily expand to extend the dynamic range by combining frames with 3 or more different accumulation times, or to estimate the correct value using values from 4 or more short accumulations with the same accumulation time can do.

The actual change in light and darkness is not limited to the case shown in the example, and there are various cases. Even in these cases, it is possible to estimate a more correct value in the same method than in the conventional method.
When a false color remains due to a problem of estimation accuracy or the like, a technique of making it inconspicuous, for example, using saturation compression together is used.

  FIG. 16 is a block diagram including a configuration example of a solid-state imaging device including an estimation circuit that estimates a correct value of a short accumulation frame according to the present embodiment.

In the solid-state imaging device 200, a signal processing system 210 including an estimation circuit and a memory 220 are arranged after the readout circuit 130 of the solid-state imaging device 100 in FIG.
The signal processing system 210 includes a memory controller 211, an estimation circuit 212, a synthesis circuit 213, and a camera signal processing circuit 214.
Here, a description will be given along the operation shown in FIG.

Three short accumulation Ts1, Ts2, Ts3 signals and long accumulation TI (S102) signals read out from the CMOS image sensor (solid-state imaging device) 100 having an intermediate potential readout function are signaled as signal S2111 by the memory controller 211. It is written in the memory 220.
The readout timings of the three short accumulation periods and the long accumulation are as shown in FIG.
The short accumulation signal read from the memory 220 via the signal S220 by the memory controller 211 is input to the estimation circuit 212 as a signal S2112.
In the estimation circuit 212, an estimated value V212 is calculated by the estimation method described in association with FIG. 9 and the like, and is output to the synthesis circuit 213.
Similarly, the signal from the long accumulation is read out from the memory 220 and is input to the synthesis circuit 213 as the signal S2113 through the memory controller 211.
In the synthesis circuit 213, a normal synthesis method takes into account the accumulation time of short accumulation and long accumulation, and synthesis is performed by a method such as applying a gain to the value so as not to cause a defect in the joint portion. Is supplied to the camera signal processing circuit 214 in the subsequent stage.
The camera signal processing circuit 214 performs other camera signal processing and outputs a final signal S214.

As described above, according to the present embodiment, even when the subject moves and there is a change in brightness or color, an improvement in the accuracy of the estimated value can be expected from a plurality of short accumulation values, and false colors are generated. There is an effect to suppress.
As a result, a more natural image can be obtained.
In this embodiment, since the movement itself is not detected, problems such as a new false color due to erroneous detection of the movement of the subject do not occur.
In addition, even when the brightness of the subject itself changes for reasons other than motion, the correct value can be estimated more appropriately, so it can be said that it is superior to the method of estimating the correct value based on motion detection.

  Although the CMOS image sensor according to each embodiment is not particularly limited, for example, it may be configured as a CMOS image sensor equipped with a column parallel type analog-digital converter (hereinafter abbreviated as ADC (Analog digital converter)). Is possible.

  A solid-state imaging device having such an effect can be applied as an imaging device for a digital camera or a video camera.

  FIG. 17 is a diagram illustrating an example of a configuration of a camera system to which the solid-state imaging device according to the embodiment of the present invention is applied.

As shown in FIG. 17, the camera system 300 includes an imaging device 310 to which the CMOS image sensors (solid-state imaging devices) 100 and 200 according to the present embodiment can be applied.
Further, the camera system 300 includes an optical system that guides incident light (images a subject image) to the pixel region of the imaging device 310, for example, a lens 320 that forms incident light (image light) on an imaging surface.
The camera system 300 includes a drive circuit (DRV) 330 that drives the imaging device 310 and a signal processing circuit (PRC) 340 that processes an output signal of the imaging device 310.

  The drive circuit 330 includes a timing generator (not shown) that generates various timing signals including a start pulse and a clock pulse that drive a circuit in the imaging device 310, and drives the imaging device 310 with a predetermined timing signal. .

Further, the signal processing circuit 340 performs predetermined signal processing on the output signal of the imaging device 310.
The image signal processed by the signal processing circuit 340 is recorded on a recording medium such as a memory. The image information recorded on the recording medium is hard copied by a printer or the like. The image signal processed by the signal processing circuit 340 is displayed as a moving image on a monitor including a liquid crystal display.

As described above, in the imaging apparatus such as a digital still camera, the same effect as described above can be obtained by mounting the above-described imaging elements 100 and 200 as the imaging device 310.
In other words, even when the subject moves and changes in brightness or color, it can be expected to improve the accuracy of the estimated value from a plurality of short accumulation values, it is possible to suppress the occurrence of false colors, etc., more natural Obtaining images with low power consumption and a highly accurate camera can be realized.

Note that the method described above in detail can be formed as a program according to the above-described procedure and executed by a computer such as a CPU.
Further, such a program can be configured to be accessed by a recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a floppy (registered trademark) disk, or the like, and to execute the program by a computer in which the recording medium is set.

  DESCRIPTION OF SYMBOLS 100,200 ... Solid-state image sensor, 110 ... Pixel array part, 110A ... Pixel circuit, 120 ... Row selection circuit (pixel drive part), 130 ... Read-out circuit, 140 ... Shutter Mode switching unit 111 ... photoelectric conversion element 112 112 transfer transistor 113 reset transistor 114 amplification transistor 115 selection transistor 210 signal processing system 211 ..Memory controller, 212... Estimation circuit, 213... Synthesis circuit, 214... Camera signal processing circuit, 300... Camera system, 310. ... Lens, 340 ... Signal processing circuit.

Claims (9)

A pixel array unit in which pixels including a photoelectric conversion element that converts an optical signal into a signal charge and a transfer gate that transfers a signal charge photoelectrically converted by the photoelectric conversion element to an output node are arranged in a matrix;
A pixel driver for controlling the operation of the pixel so as to perform a shutter operation of the pixel array unit and reading from the pixels of the pixel array unit;
A signal processing system for processing a signal read by the pixel driving unit,
The pixel driver is
During the first accumulation period of the first exposure time for acquiring an image between the electronic shutter operation and the normal reading operation, the reading scan is executed by the intermediate voltage transfer, and the image acquisition corresponding to the reading operation by the intermediate voltage transfer is performed. Having a second exposure time, and having a function of executing one or more selected scans for transfer with an intermediate voltage of the same or different voltage value,
A plurality of second accumulation periods of a second exposure time shorter than the first exposure time are set during the first accumulation period of the first exposure time;
The signal processing system is
Using the value read in the plurality of second accumulation period, we have rows process of estimating the output value of the second accumulation period,
When the subject changes from dark to light, in the process of estimating the output value,
The maximum possible value as an actual value increases from the end of the second accumulation period set in the dark period, and then increases at the accumulation rate of the second accumulation period set in the bright period. Output value when
Output value when the minimum possible value is increased from the start time of the second accumulation period set in the light period and then increased at the accumulation speed of the second accumulation period age,
In the process of estimating the output value when the subject changes from light to dark,
The maximum possible value as the actual value is increased at the speed of the component that exceeds the intermediate potential in the light period, until the start of the second accumulation period set in the dark period, and then As an output value when increasing at the accumulation speed of the second accumulation period set in the dark period,
The minimum possible value as an actual value increases at the speed of the component until the end of the second accumulation period set in the bright period, and then the second accumulation period set in the dark period As an output value when increasing at the accumulation speed of
A solid-state imaging device that estimates an output value of the second accumulation period from an output value as a maximum value and an output value as a minimum value . 2. The solid-state imaging according to claim 1, wherein a plurality of second accumulation periods of the second exposure time are set during the first accumulation period of the first exposure time and a plurality of second accumulation periods are set in the same accumulation period. element. Among the first accumulation period of the first exposure time, a different second accumulation period of accumulation time comprises a plurality of sets, a plurality of second accumulation period Oite to one set of which is set to the same accumulation period Have
The signal processing system is
The solid-state imaging device according to claim 1, wherein processing for estimating an output value of each set of second accumulation periods is performed using values read out in a plurality of second accumulation periods. The signal processing system is
When a plurality of second accumulation periods having a plurality of different accumulation times are used, a correct output value of the second accumulation period is estimated together with output values of other second accumulation periods having different accumulation times. 3. The solid-state imaging device according to 3. The signal processing system is
When the frame accumulation time of the desired second accumulation period is Ts, the accumulation time of each of the N second accumulation periods is Ts / N, and the N number of the second accumulation periods are used as the predicted output values of the second accumulation period. The solid-state imaging device according to any one of claims 1 to 4, wherein a total value of outputs during two accumulation periods is used. The first set of frames of the storage period set in the second storage period in which N storage times are Ts1 / N and the storage period of the storage period set in the storage period in which M storage times are Ts2 / M Having a second set, the signal processing system is:
As an expected value of the output of the first set of accumulation periods, the sum of outputs of these N livestock periods, and as an expected value of the output of the second set of accumulation periods, the sum of outputs of the M accumulation periods. The solid-state imaging device according to any one of claims 1 to 4. A solid-state image sensor;
An optical system for forming a subject image on the solid-state imaging device,
The solid-state imaging device is
A pixel array unit in which pixels including a photoelectric conversion element that converts an optical signal into a signal charge and a transfer gate that transfers a signal charge photoelectrically converted by the photoelectric conversion element to an output node are arranged in a matrix;
A pixel driver for controlling the operation of the pixel so as to perform a shutter operation of the pixel array unit and reading from the pixels of the pixel array unit;
A signal processing system for processing a signal read by the pixel driving unit,
The pixel driver is
During the first accumulation period of the first exposure time for acquiring an image between the electronic shutter operation and the normal reading operation, the reading scan is executed by the intermediate voltage transfer, and the image acquisition corresponding to the reading operation by the intermediate voltage transfer is performed. Having a second exposure time, and having a function of executing one or more selected scans for transfer with an intermediate voltage of the same or different voltage value,
A plurality of second accumulation periods of a second exposure time shorter than the first exposure time are set during the first accumulation period of the first exposure time;
The signal processing system is
Using the value read in the plurality of second accumulation period, we have rows process of estimating the output value of the second accumulation period,
When the subject changes from dark to light, in the process of estimating the output value,
The maximum possible value as an actual value increases from the end of the second accumulation period set in the dark period, and then increases at the accumulation rate of the second accumulation period set in the bright period. Output value when
Output value when the minimum possible value is increased from the start time of the second accumulation period set in the light period and then increased at the accumulation speed of the second accumulation period age,
In the process of estimating the output value when the subject changes from light to dark,
The maximum possible value as the actual value is increased at the speed of the component that exceeds the intermediate potential in the light period, until the start of the second accumulation period set in the dark period, and then As an output value when increasing at the accumulation speed of the second accumulation period set in the dark period,
The minimum possible value as an actual value increases at the speed of the component until the end of the second accumulation period set in the bright period, and then the second accumulation period set in the dark period As an output value when increasing at the accumulation speed of
A camera system for estimating an output value of the second accumulation period from an output value as a maximum value and an output value as a minimum value . When driving a pixel array unit in which pixels including a photoelectric conversion element that converts an optical signal into a signal charge and a transfer gate that transfers a signal charge photoelectrically converted by the photoelectric conversion element to an output node are arranged in a matrix. ,
A plurality of second accumulation periods of a first exposure time and a second exposure time shorter than the first exposure time are set during the first accumulation period of the first exposure time,
During the first accumulation period of the first exposure time for acquiring an image between the electronic shutter operation and the normal reading operation, the reading scan is executed by the intermediate voltage transfer, and the image acquisition corresponding to the reading operation by the intermediate voltage transfer is performed. Performing a selective scan for transfer with an intermediate voltage of the same or different voltage value one or more times, including a second exposure time for
Using the value read in the plurality of second accumulation period, we have rows process of estimating the output value of the second accumulation period,
When the subject changes from dark to light, in the process of estimating the output value,
The maximum possible value as an actual value increases from the end of the second accumulation period set in the dark period, and then increases at the accumulation rate of the second accumulation period set in the bright period. Output value when
Output value when the minimum possible value is increased from the start time of the second accumulation period set in the light period and then increased at the accumulation speed of the second accumulation period age,
In the process of estimating the output value when the subject changes from light to dark,
The maximum possible value as the actual value is increased at the speed of the component that exceeds the intermediate potential in the light period, until the start of the second accumulation period set in the dark period, and then As an output value when increasing at the accumulation speed of the second accumulation period set in the dark period,
The minimum possible value as an actual value increases at the speed of the component until the end of the second accumulation period set in the bright period, and then the second accumulation period set in the dark period As an output value when increasing at the accumulation speed of
A solid-state imaging device driving method for estimating an output value in the second accumulation period from an output value as a maximum value and an output value as a minimum value . When driving a pixel array unit in which pixels including a photoelectric conversion element that converts an optical signal into a signal charge and a transfer gate that transfers a signal charge photoelectrically converted by the photoelectric conversion element to an output node are arranged in a matrix. ,
A process of setting a plurality of first exposure times and a second accumulation period of a second exposure time shorter than the first exposure time during the first accumulation period of the first exposure time;
During the first accumulation period of the first exposure time for acquiring an image between the electronic shutter operation and the normal reading operation, the reading scan is executed by the intermediate voltage transfer, and the image acquisition corresponding to the reading operation by the intermediate voltage transfer is performed. A process of performing one or more selective scans for transfer with an intermediate voltage of the same or different voltage value, including a second exposure time for
Processing for estimating an output value of the second accumulation period using values read in a plurality of second accumulation periods,
When the subject changes from dark to light, in the process of estimating the output value,
The maximum possible value as an actual value increases from the end of the second accumulation period set in the dark period, and then increases at the accumulation rate of the second accumulation period set in the bright period. Output value when
Output value when the minimum possible value is increased from the start time of the second accumulation period set in the light period and then increased at the accumulation speed of the second accumulation period age,
In the process of estimating the output value when the subject changes from light to dark,
The maximum possible value as the actual value is increased at the speed of the component that exceeds the intermediate potential in the light period, until the start of the second accumulation period set in the dark period, and then As an output value when increasing at the accumulation speed of the second accumulation period set in the dark period,
The minimum possible value as an actual value increases at the speed of the component until the end of the second accumulation period set in the bright period, and then the second accumulation period set in the dark period As an output value when increasing at the accumulation speed of
A program that causes a computer to execute a driving process of a solid-state imaging device that estimates an output value of the second accumulation period from an output value as a maximum value and an output value as a minimum value .

Images