WO2011001616A1

WO2011001616A1 - Solid-state imaging device, and control method therefor

Info

Publication number: WO2011001616A1
Application number: PCT/JP2010/003919
Authority: WO
Inventors: 山田哲生
Original assignee: パナソニック株式会社
Priority date: 2009-06-30
Filing date: 2010-06-14
Publication date: 2011-01-06
Also published as: JP2011015103A

Abstract

The disclosed solid-state imaging device (100) is provided with a plurality of pixels (180) disposed in rows; a vertical transfer unit (181) and a horizontal transfer unit (182); and a light-control unit (112), which controls whether or not light is irradiated onto all of the plurality of pixels (180). The solid-state imaging device is further provided with a control unit (120), which, during a first time period, controls the light-control unit (112) so that light is irradiated onto all of the plurality of pixels (180); at a first point in time during the first time period, resets the signal charges accumulated in a plurality of first photoelectric conversion elements included in the plurality of pixels (180); at a second point in time during the first time period and after the first point in time, resets the signal charges accumulated in a plurality of second photoelectric conversion elements included in the plurality of pixels (180); and during a second time period that directly follows the first time period, controls the light-control unit (112) so that light is not irradiated onto all of the plurality of pixels (180), and causes the vertical transfer unit (181) and the horizontal transfer unit (182) to read the signal charges accumulated in the plurality of pixels (180).

Description

Solid-state imaging device and control method thereof

The present invention relates to a solid-state imaging device and a control method thereof, and more particularly to a solid-state imaging device that outputs signals with different exposure times.

Currently, solid-state imaging devices using a CCD (Charge Coupled Device) image sensor or a MOS (Metal Oxide Semiconductor) image sensor are widely used in digital still cameras, digital video cameras, mobile phones, and the like. These CCD image sensors or MOS image sensors are known to have a narrow dynamic range as compared with a silver salt camera.

On the other hand, Patent Document 1 discloses a technique for expanding the dynamic range of a CCD image sensor.

The technique described in Patent Document 1 expands the dynamic range by differentiating the exposure times (sensitivities) of two adjacent pixels and synthesizing signals having different exposure times.

Hereinafter, the solid-state imaging device described in Patent Document 1 will be described.

FIG. 21 is a diagram showing a configuration of a solid-state imaging device described in Patent Document 1.

In FIG. 21, color filters of three primary colors of red R, green G, and blue B are arranged on a large number of pixels PIX.

In the column direction, two pixels of the same color are arranged side by side, such as odd-numbered columns B1, B2, G1, and G2, and even-numbered columns G1, G2, R1, and R2. In the horizontal direction, two sets of pixels of the same color are arranged at the same column direction position, such as G1 and G2 on the right side of B1 and B2, and B1 and B2 on the right side thereof. The upper pixel groups R1, G1, and B1 are referred to as a first pixel group, and the lower pixel groups R2, G2, and B2 are referred to as a second pixel group. For example, the first pixel group is a high-sensitivity pixel group, and the second pixel group is a low-sensitivity pixel group.

One vertical charge transfer path VCCD is arranged along each column of pixels (on the right side of each column in the figure). A horizontal charge transfer path HCCD is coupled in the lateral direction to the lower ends of the plurality of vertical charge transfer paths VCCD, and an output amplifier OA is connected to one end thereof.

FIG. 22 is a timing chart of drive signals for the solid-state imaging device described in Patent Document 1.

At timings t1 to t2 shown in FIG. 22, the overflow drain voltage VOD becomes high, the accumulated charge of each pixel is extracted to the substrate, all the pixels PIX are cleared, and new exposure is started. The drive signal φ3 becomes a high potential at timing t3, and then the drive signal φ1 becomes a higher potential at timing t4.

In a short period tS from timing t2 to t3, signal charges corresponding to incident light are accumulated in all the pixels PIX. At timing t3, the drive signal φ3 becomes a read voltage, and the accumulated charge for a short period tS is read from the pixel PIX2 of the second pixel group to which φ3 is applied to the vertical charge transfer path VCCD. No readout occurs in the pixel to which φ1 is applied, and charge accumulation according to incident light continues.

Next, at timings t3 to t4, signal charges are accumulated according to incident light. The vertical charge transfer path VCCD holds the charge read at timing t3.

Next, at timing t4, the drive signal φ1 becomes a read voltage, and the accumulated charge of tL for a long period from timing t2 to t4 is read from the pixel PIX1 to which φ1 is applied to the vertical charge transfer path VCCD.

As shown in FIG. 22, after timing t5, vertical transfer is performed by applying four-layer drive signals φ1, φ2, φ3, and φ4 on the vertical charge transfer path. In the horizontal charge transfer path, two-phase drive signals φH1 and φH2 are applied.

Next, charge transfer is performed in the vertical charge transfer path VCCD, and the charges for two pixel rows in which the accumulated charges for short-term exposure and the accumulated charges for long-term exposure are alternately arranged are supplied to the horizontal charge transfer path HCCD. . The horizontal charge transfer path HCCD transfers charges for two pixel rows supplied from the VCCD toward the output amplifier OA. The signal charge is read out through the output amplifier OA.

As described above, the solid-state imaging device described in Patent Document 1 expands the dynamic range by synthesizing signals with different exposure times.

JP 2007-266556 A

However, the solid-state imaging device described in Patent Document 1 first transfers the short-term exposure accumulated charge to the vertical charge transfer path and holds it in the vertical charge transfer path. Thereafter, the solid-state imaging device described in Patent Document 1 reads out the accumulated charge of long-term exposure to the vertical charge transfer path. As described above, the solid-state imaging device described in Patent Document 1 temporarily holds the accumulated charge of short-term exposure in the vertical charge transfer path from the end of short-term exposure to the end of long-term exposure. .

As a result, the following two types of noise charges are superimposed on the accumulated charge during the period in which the accumulated charge of short-time exposure is held in the vertical charge transfer path. One is a so-called smear component due to leakage of incident light, and the other is a dark current component that is thermally generated in the vertical charge transfer path. As a result, it is difficult to obtain a high-quality video signal with the prior art.

Therefore, an object of the present invention is to provide a solid-state imaging device capable of suppressing noise due to smear and dark current and a control method thereof.

In order to achieve the above object, a solid-state imaging device according to the present invention is a solid-state imaging device, which is arranged in a matrix, converts a light into a signal charge, and accumulates the converted signal charge. An element, a plurality of vertical transfer units that transfer the signal charges stored in a plurality of photoelectric conversion elements arranged in the corresponding column, and signals transferred by the plurality of vertical transfer units A horizontal transfer unit configured to transfer charges; a light control unit configured to control whether light is incident on the plurality of photoelectric conversion elements; and a control unit, wherein the control unit includes the plurality of units during a first period. The light control unit controls the light so that light is incident on the photoelectric conversion elements, and is accumulated in the plurality of first photoelectric conversion elements included in the plurality of photoelectric conversion elements at the first time included in the first period. Resetting the signal charge, the first And resetting the signal charges accumulated in the plurality of second photoelectric conversion elements included in the plurality of photoelectric conversion elements at a second time that is included in between and at a second time after the first time, During the second period immediately after, the light control unit is controlled so that light does not enter the plurality of photoelectric conversion elements, and during the first exposure time from the first time to the start time of the second period. Accumulated by the plurality of second photoelectric conversion elements between the first signal charge accumulated in the plurality of first photoelectric conversion elements and a second exposure time from the second time to the start time of the second period. The second signal charges are read out through the vertical transfer unit and the horizontal transfer unit during the second period, and the first photoelectric conversion elements and the second photoelectric conversion elements have a one-to-one correspondence. Each of the first photoelectric conversion elements is connected to the first photoelectric conversion element. Is located within a predetermined distance from said second photoelectric conversion element corresponding to 換素Ko.

According to this configuration, the solid-state imaging device according to the present invention can output signals with different exposure times. Thus, the dynamic range can be expanded by synthesizing signals having different exposure times. Furthermore, in the solid-state imaging device according to the present invention, the start times of the two exposure times are made different, and the end times of the two exposure times are made the same time by the light control unit. Thereby, the solid-state imaging device according to the present invention can hold the signal charge in the photoelectric conversion element during the period from the end of the exposure time to the reading of the signal charge. Therefore, the solid-state imaging device according to the present invention does not need to hold the signal charge in the vertical charge transfer path or the like for a long time, so that noise due to smear and dark current can be suppressed.

Each of the first photoelectric conversion elements is adjacent to the second photoelectric conversion element corresponding to the first photoelectric conversion element in the row direction, the column direction, or the oblique direction, or adjacent to the second photoelectric conversion element in the row direction or the column direction. May be arranged.

According to this configuration, an image signal having a wide dynamic range can be generated by synthesizing signals corresponding to the two corresponding photoelectric conversion elements output from the solid-state imaging device according to the present invention.

The solid-state imaging device further includes a plurality of filters formed on the plurality of photoelectric conversion elements, respectively, and each of the plurality of filters transmits any light in a plurality of wavelength bands. The same type of filter may be formed on the first photoelectric conversion element and the second photoelectric conversion element which are any of a plurality of types of filters and correspond to each other.

Each of the plurality of filters is any one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light, and each of the first photoelectric conversion elements Is arranged next to the second photoelectric conversion element corresponding to the first photoelectric conversion element in the row direction or the column direction, and the first photoelectric conversion element and the second photoelectric conversion element corresponding to each other. May be arranged in a Bayer array.

According to this configuration, the solid-state imaging device according to the present invention can reduce the distance between the first photoelectric conversion element and the second photoelectric conversion element corresponding to each other.

Each of the plurality of filters is any one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light, and is arranged in a Bayer array. Each first photoelectric conversion element formed under the filter is arranged next to the second photoelectric conversion element corresponding to the first photoelectric conversion element in an oblique direction, and the red filter or the blue filter Each 1st photoelectric conversion element arrange | positioned under may be arrange | positioned 2 adjacent in the row direction or column direction of the said 2nd photoelectric conversion element corresponding to the said 1st photoelectric conversion element.

According to this configuration, the solid-state imaging device according to the present invention can arrange the first photoelectric conversion element, the second photoelectric conversion element, and the added centroid of each color spatially and evenly.

Each of the plurality of filters is any one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light, and the green filters are arranged in a checkered pattern. Of the four filters adjacent to each of the red filter and the blue filter, two are the red filter, two are the blue filters, and each of the first photoelectric conversion elements is the first filter. The second photoelectric conversion element corresponding to the photoelectric conversion element may be disposed next to the second photoelectric conversion element in an oblique direction.

The solid-state imaging device further includes a semiconductor substrate on which the plurality of photoelectric conversion elements are formed, and the control unit stores the signal accumulated in the plurality of first photoelectric conversion elements at the first time. The signal charges accumulated in the plurality of first photoelectric conversion elements are reset by discharging the charges to the semiconductor substrate or transferred to the vertical transfer unit, and at the second time, the plurality of second charges The signal charges accumulated in the plurality of second photoelectric conversion elements may be reset by transferring the signal charges accumulated in the two photoelectric conversion elements to the vertical transfer unit.

According to this configuration, the solid-state imaging device according to the present invention can reset the signal charge accumulated in the second photoelectric conversion element without adding a function to the vertical transfer unit or the like.

The solid-state imaging device further converts the first signal charge into a first image signal, converts the second signal charge into a second image signal, and converts the converted first image signal and second image signal. You may provide the output part which outputs a signal, and the signal processing part which synthesize | combines a said 1st image signal and a said 2nd image signal.

According to this configuration, the solid-state imaging device according to the present invention can generate an image signal with a wide dynamic range by synthesizing signals with different exposure times.

Further, the signal processing unit generates a first image signal after correction by setting the first image signal to the first value when the first image signal is larger than a first value. You may provide a clip process part and the synthetic | combination part which synthesize | combines the said 1st image signal after a correction | amendment, and a said 2nd image signal.

According to this configuration, the solid-state imaging device according to the present invention can reduce image unevenness caused by variations in the saturation output voltage of the photoelectric conversion elements.

The solid-state imaging device has a first operation mode and a second operation mode. In the first operation mode, the control unit reads the first signal charge and the second signal charge, and The output unit converts the first signal charge into a first image signal, converts the second signal charge into a second image signal, outputs the converted first image signal and the second image signal, and The signal processing unit synthesizes the first image signal and the second image signal, and in the second operation mode, the control unit transfers the first signal charge and the second signal charge in the vertical transfer unit or The mixed signal charge may be generated by mixing in the horizontal transfer unit, and the output unit may convert the mixed signal charge into a mixed image signal and output the converted mixed image signal.

According to this configuration, the solid-state imaging device according to the present invention can generate an image signal with a wide dynamic range by operating in the first operation mode. Furthermore, the solid-state imaging device according to the present invention can generate a highly sensitive image signal by operating in the second operation mode.

Further, the signal processing unit further sets the second image after correction by setting the mixed image signal to the second value when the mixed image signal is larger than the second value in the second operation mode. A second white clip processing unit for generating a signal, wherein the second value is a mixed image signal value corresponding to a saturation signal charge of the photoelectric conversion element is VSAT, the first exposure time is t1, When the second exposure time is t2, it may be equal to or less than the value represented by VSAT × (1 + t2 / t1).

The control unit may set the first exposure time to be twice or more the second exposure time and set the second exposure time to 90% or more of the first exposure time in the second operation mode. .

The solid-state imaging device further has a third operation mode, and in the third operation mode, the control unit reads the first signal charge and the second signal charge during the second period. The output unit converts the first signal charge and the second signal charge read by the reading unit into a third image signal, and outputs the converted third image signal. In the third operation mode, the second exposure time may be 90% or more of the first exposure time.

According to this configuration, the solid-state imaging device according to the present invention can further generate a high-resolution image signal by operating in the third operation mode.

In addition, the solid-state imaging device further includes a brightness acquisition unit that acquires the brightness of the object scene, and when the brightness acquired by the brightness acquisition unit is greater than a third value, the first operation mode And when the brightness acquired by the brightness acquisition unit is smaller than the third value, a mode selection unit that selects the second operation mode, and the solid-state imaging device includes the mode selection unit The operation may be performed in the first operation mode or the second operation mode selected by.

According to this configuration, the solid-state imaging device according to the present invention can automatically select the optimum operation mode and operate in the selected operation mode.

Further, the reading unit may read the signal charges accumulated in the plurality of photoelectric conversion elements in a plurality of times during the second period.

According to this configuration, since the solid-state imaging device according to the present invention only needs to provide one vertical transfer electrode for each photoelectric conversion element, the circuit area of the solid-state imaging device can be reduced.

The light control unit may be a mechanical shutter, a liquid crystal, a MEMS mirror, or an optical element that can be electrically controlled.

The present invention can be realized not only as such a solid-state imaging device, but also as a control method of a solid-state imaging device using characteristic means included in the solid-state imaging device as a step, or such characteristic steps. It can also be realized as a program for causing a computer to execute. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM and a transmission medium such as the Internet.

Furthermore, the present invention can be realized as a semiconductor integrated circuit (LSI) that realizes part or all of the functions of such a solid-state imaging device, or can be realized as a camera or a digital still camera equipped with such a solid-state imaging device. it can.

As described above, the present invention can provide a solid-state imaging device capable of suppressing noise due to smear and dark current and a control method thereof.

FIG. 1 is a block diagram showing a configuration of a solid-state imaging device according to an embodiment of the present invention. FIG. 2 is a diagram showing the configuration of the image sensor according to the embodiment of the present invention. FIG. 3 is a timing chart showing the operation of the solid-state imaging device in the wide DR mode according to the embodiment of the present invention. FIG. 4 is a diagram schematically showing the operation of the image sensor in the wide DR mode according to the embodiment of the present invention. FIG. 5 is a cross-sectional view of the vertical transfer unit according to the embodiment of the present invention. FIG. 6 is a diagram showing a potential distribution of the vertical overflow drain portion according to the embodiment of the present invention. FIG. 7 is a diagram showing a relationship between opening / closing of the light control unit and opening / closing of the electronic shutter according to the embodiment of the present invention. FIG. 8A is a diagram illustrating a configuration example of the vertical transfer electrode in the solid-state imaging device according to the embodiment of the present invention. FIG. 8B is a diagram illustrating a configuration example of the vertical transfer electrode in the solid-state imaging device according to the embodiment of the present invention. FIG. 9 is a timing chart showing a modified example of the operation of the solid-state imaging device in the wide DR mode according to the embodiment of the present invention. FIG. 10 is a diagram showing the photoelectric conversion characteristics of the photoelectric conversion element in the short exposure time and the long exposure time according to the embodiment of the present invention. FIG. 11 is a diagram showing the characteristics of the composite image signal according to the embodiment of the present invention. FIG. 12 is a timing chart showing the operation of the solid-state imaging device in the high sensitivity mode according to the embodiment of the present invention. FIG. 13A is a diagram showing a pixel arrangement example and the distribution of the added centroid according to the embodiment of the present invention. FIG. 13B is a diagram showing an example of pixel dimensions according to the embodiment of the present invention. FIG. 14A is a diagram showing a pixel arrangement example and the distribution of the added centroid according to the embodiment of the present invention. FIG. 14B is a diagram showing a two-dimensional Nyquist limit before and after pixel mixing according to the embodiment of the present invention. FIG. 15 is a diagram illustrating a configuration example of the vertical transfer electrode in the solid-state imaging device according to the embodiment of the present invention. FIG. 16 is a diagram showing a pixel arrangement example and the distribution of the added centroid according to the embodiment of the present invention. FIG. 17 is a diagram showing a pixel arrangement example and the distribution of the added centroid according to the embodiment of the present invention. FIG. 18 is a diagram showing a pixel arrangement example and the distribution of the added centroid according to the embodiment of the present invention. FIG. 19 is a flowchart showing a flow of operations of the solid-state imaging device according to the embodiment of the present invention. FIG. 20A is a diagram illustrating a configuration example of an imaging unit when a MEMS mirror is used according to an embodiment of the present invention. FIG. 20B is a diagram illustrating a configuration example of an imaging unit when a MEMS mirror is used according to the embodiment of the present invention. FIG. 21 is a diagram illustrating a configuration of a conventional solid-state imaging device. FIG. 22 is a timing chart of a driving signal of a conventional solid-state imaging device.

Hereinafter, embodiments of a solid-state imaging device according to the present invention will be described in detail with reference to the drawings.

(Embodiment)
The solid-state imaging device according to the embodiment of the present invention can expand the dynamic range by outputting signals with different exposure times and combining the signals with different exposure times. Furthermore, in the solid-state imaging device according to the embodiment of the present invention, the start times of the two exposure times are made different, and the end times of the two exposure times are made the same time by the mechanical shutter. Thereby, the solid-state imaging device according to the embodiment of the present invention does not need to hold the signal charge in the vertical charge transfer path or the like for a long time, so that it is possible to suppress noise due to smear and dark current.

First, the configuration of the solid-state imaging device 100 according to the embodiment of the present invention will be described.

FIG. 1 is a block diagram showing a configuration of a solid-state imaging device 100 according to an embodiment of the present invention.

1 is used for a digital still camera, for example. The solid-state imaging device 100 converts the light 150 into an image signal 151 that is an electrical signal, and outputs an image signal 151.

The solid-state imaging device 100 has a wide dynamic range mode (hereinafter, “wide DR mode”), a high sensitivity mode, and a high resolution mode.

Wide DR mode (first operation mode) is an operation mode that can realize a wider dynamic range than the high sensitivity mode and the high resolution mode. Specifically, in the wide DR mode, the solid-state imaging device 100 generates signals with different exposure times, and generates an image signal 151 with a wide dynamic range by combining signals with different exposure times.

The high sensitivity mode (second operation mode) is an operation mode having higher sensitivity than the wide DR mode and the high resolution mode, and is effective when the amount of light in the object scene is small (in a dark place). Specifically, in the high sensitivity mode, the solid-state imaging device 100 mixes signal charges generated by two pixels and outputs an image signal 151 corresponding to the mixed signal charges.

The high resolution mode (third operation mode) is an operation mode for generating an image signal 151 having a higher resolution than the wide DR mode and the high sensitivity mode. Specifically, in the high resolution mode, the solid-state imaging device 100 outputs an image signal 151 corresponding to the signal charge generated at each pixel. Thereby, the solid-state imaging device 100 generates an image signal 151 having a resolution twice as high as that in the wide DR mode and the high sensitivity mode in the high resolution mode, for example.

Further, for example, a designation mode signal 152 corresponding to a user operation is input to the solid-state imaging device 100. The solid-state imaging device 100 selects any one of the wide DR mode, the high sensitivity mode, and the high resolution mode according to the designation mode signal 152, and operates in the selected mode.

Further, as shown in FIG. 1, the solid-state imaging device 100 includes an imaging unit 110, a control unit 120, a mode determination unit 130, and a signal processing unit 140.

The imaging unit 110 converts the light 150 in the object scene into an image signal 153 that is an electrical signal and outputs the image signal 153. The imaging unit 110 includes

lenses

111 and 113, a light control unit 112, and an imaging element 114.

The lens 111 condenses the light 150 in the object field on the light control unit 112.

The light control unit 112 controls whether or not the light 150 collected by the lens 111 is incident on all of the plurality of photoelectric conversion elements included in the imaging element 114. For example, the light control unit 112 is a mechanical shutter.

The lens 113 collects the light 150 that has passed through the light control unit 112 on the image sensor 114.

The image sensor 114 converts the light 150 collected by the lens 113 into an image signal 153 that is an electrical signal and outputs the image signal 153.

The mode determination unit 130 determines the operation mode of the solid-state imaging device 100 according to the designation mode signal 152. Further, the mode determination unit 130 outputs a selection mode signal 160 indicating the determined operation mode to the control unit 120 and the signal processing unit 140. The mode determination unit 130 includes a mode acquisition unit 131, a brightness acquisition unit 132, and a mode selection unit 133.

The signal processing unit 140 generates an image signal 151 by performing signal processing according to the operation mode determined by the mode determination unit 130 on the image signal 153 output from the image sensor 114, and the generated image signal 151 is Output. The signal processing unit 140 includes a first white clip processing unit 141, a synthesis unit 142, an output unit 143, and a second white clip processing unit 144.

The control unit 120 controls the operation of the imaging unit 110 according to the operation mode determined by the mode determination unit 130. The control unit 120 includes a first drive unit 121 and a second drive unit 122. The first drive unit 121 generates a first drive signal 154 that drives the image sensor 114. The second drive unit 122 generates a second drive signal 155 that controls the light control unit 112.

Next, the configuration of the image sensor 114 will be described.

FIG. 2 is a diagram illustrating a configuration of the image sensor 114.

The image sensor 114 shown in FIG. 2 is a CCD image sensor, and includes a plurality of pixels 180 arranged in a matrix, a plurality of vertical transfer units 181 provided for each column, a horizontal transfer unit 182, and an output unit 183. With.

Each pixel 180 includes a photoelectric conversion element and a filter. The photoelectric conversion element converts light into signal charges and accumulates the converted signal charges.

In FIG. 2, an example in which 8 × 8 pixels 180 are arranged is shown, but the number of pixels 180 may be other than this.

The filter is formed on the photoelectric conversion element and transmits only light in a predetermined wavelength band to the photoelectric conversion element. Each filter is one of a plurality of types of filters that transmit any one of a plurality of wavelength bands. Specifically, the plurality of filters are any one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light. Note that the filter may include a filter that transmits a wavelength band other than the above, such as a filter that transmits colorless (visible light) or a filter that transmits infrared light.

In the following description, a pixel 180 that includes a red filter and photoelectrically converts red light is referred to as an R pixel, a green filter is included, a pixel 180 that photoelectrically converts green light is referred to as a G pixel, a blue filter is included, and blue light A pixel 180 that performs photoelectric conversion is referred to as a B pixel.

The R pixel R1, the G pixel G1, and the B pixel B1 are pixels 180 that perform photoelectric conversion during a short exposure time in the wide DR mode, and are referred to as a first pixel below. The R pixel R2, the G pixel G2, and the B pixel B2 are pixels 180 that perform photoelectric conversion during the long exposure time in the wide DR mode, and are referred to as second pixels below.

Here, the plurality of first pixels and the plurality of second pixels are arranged in a one-to-one correspondence, and one corresponding first pixel and one corresponding second pixel form a set. In the example shown in FIG. 2, pixels 180 of the same color adjacent in the column direction form one set. In the wide DR mode, the signal processing unit 140 generates an image signal 151 having a wide dynamic range by synthesizing the image signals 153 corresponding to the pixels 180 forming this set. In the high sensitivity mode, the image sensor 114 mixes and outputs the signal charges photoelectrically converted by the pixels 180 forming this set.

Further, each first pixel is arranged within a predetermined distance from the second pixel corresponding to the first pixel. For example, in FIG. 2, the two pixels 180 forming this set are arranged next to each other in the column direction. In addition, this set is arranged in a Bayer array.

Here, the Bayer arrangement means that G pixels are arranged in a checkered pattern, and rows (or columns) in which R pixels are arranged and rows (or columns) in which B pixels are arranged are alternately arranged. Is an array. In other words, the Bayer array is an array in which unit pixel cells composed of four 2 × 2 pixels are arranged in a plane. This unit pixel cell includes two G pixels, one R pixel, and one B pixel that are arranged obliquely to each other.

The plurality of vertical transfer units 181 and the horizontal transfer unit 182 correspond to a reading unit of the present invention, and read signal charges accumulated by the plurality of pixels 180.

The vertical transfer unit 181 is a vertical transfer CCD that transfers signal charges accumulated in a plurality of pixels 180 arranged in a corresponding column in the column direction (vertical direction).

The horizontal transfer unit 182 is a horizontal transfer CCD that transfers a plurality of signal charges transferred by the plurality of vertical transfer units 181 in the row direction (horizontal direction).

The output unit 183 converts the signal charge transferred by the horizontal transfer unit 182 into an image signal 153, and outputs the converted image signal 153. Specifically, in the wide DR mode, the output unit 183 converts the signal charge accumulated in the first pixel into the long exposure image signal 153a, and converts the signal charge accumulated in the second pixel into the short exposure image signal. 153b and the converted long exposure image signal 153a and short exposure image signal 153b are output to the signal processing unit 140.

In the high sensitivity mode, the output unit 183 converts the mixed signal charge obtained by mixing the signal charges accumulated in the first pixel and the second pixel to form a mixed image signal 153c, and converts the mixed image signal into a mixed image signal. 153c is output to the signal processing unit 140. In the high resolution mode, the output unit 183 converts the signal charges of all the pixels 180 into the high resolution image signal 153d, and outputs the converted high resolution image signal 153d to the signal processing unit.

Hereinafter, the operation of the solid-state imaging device 100 in the wide DR mode will be described.

In the wide DR mode, the control unit 120 controls the light control unit 112 so that the light 150 is incident on all of the plurality of pixels 180 during the first period. In addition, the control unit 120 resets the signal charge accumulated in the second pixel at a first time included in the first period, and at a second time included in the first period and after the first time. The signal charge accumulated in the first pixel is reset. In addition, during the second period immediately after the first period, the control unit 120 controls the light control unit 112 so that the light 150 does not enter all of the plurality of pixels 180 and accumulates the light in the plurality of pixels 180. The signal charge is read through the plurality of vertical transfer units 181 and horizontal transfer units 182.

Specifically, the control unit 120 discharges the signal charges accumulated in all of the plurality of pixels 180 to the semiconductor substrate at the first time, thereby obtaining the signal charges accumulated in the plurality of second pixels. Reset. In addition, the control unit 120 resets the signal charges accumulated in the plurality of first pixels by transferring the signal charges accumulated in the plurality of first pixels to the vertical transfer unit 181 at the second time. To do.

Thus, the short exposure time of the first pixel and the long exposure time of the second pixel are started at different timings and are simultaneously ended.

FIG. 3 is a timing chart showing the operation of the solid-state imaging device 100 in the wide DR mode. FIG. 4 is a diagram schematically showing the operation of the image sensor 114 in the wide DR mode.

As shown in FIG. 3, before the time t10, the second drive unit 122 opens the light control unit 112 to cause the light 150 to enter the image sensor 114.

Next, the first drive unit 121 resets signal charges accumulated in all the pixels 180 by applying a substrate sweep pulse to the signal Vsub between time t10 and time t11 (S101 in FIG. 4). .

Here, the substrate sweeping operation will be described.

FIG. 5 is a cross-sectional view of the pixel 180 and the vertical transfer unit 181. As shown in FIG. 5, the image sensor 114 has a so-called vertical overflow drain (VOD) structure. Specifically, the imaging element 114 includes an n-type semiconductor substrate 191, a p-well 192 formed in the n-type semiconductor substrate 191, a photodiode 196 (photoelectric conversion element) formed in the p-well, And a p + region 193 formed on the diode 196. The vertical transfer unit 181 includes a vertical transfer channel 194 and a vertical transfer electrode 195 formed on the vertical transfer channel 194.

FIG. 6 is a diagram showing a potential distribution in the x section shown in FIG. As shown in FIG. 6, the voltage VSB is applied to the n-type semiconductor substrate 191 when the photodiode 196 is accumulated (other than when the substrate is swept). As a result, the photodiode 196 accumulates signal charges corresponding to the amount of incident light 150.

On the other hand, at the time of substrate sweeping, the first drive unit 121 applies, for example, 15 V to the pulse φES. Thereby, the voltage VSB + φES is applied to the n-type semiconductor substrate 191. Accordingly, as shown in FIG. 6, the signal charge accumulated in the photodiode 196 is swept out to the n-type semiconductor substrate 191 via the p-well 192.

FIG. 7 is a diagram showing a relationship between opening / closing of the light control unit 112 and opening / closing of the electronic shutter. As shown in FIG. 7, in a state where the light control unit 112 is open, a period from when the substrate sweep is performed until the light control unit 112 is closed is a state where the electronic shutter is open (long exposure time). Become.

Again, description will be made with reference to FIGS.

3 between time t11 and time t12 in FIG. 3, the first pixel and the second pixel convert the incident light 150 shown in FIG. 1 into signal charges, and accumulate the converted signal charges. The circles shown in FIG. 4 indicate signal charges. The size of the circle indicates the amount of signal charge, and the larger the circle, the greater the signal charge.

Next, during the time t12 to the time t13, the first driving unit 121 has accumulated in the first pixel by applying the VCCD sweep pulse (readout pulse) to φV1 and φV5 applied to the first pixel. The signal charge is transferred (swept out) to the vertical transfer unit 181. Thereby, the first drive unit 121 resets the signal charge accumulated in the first pixel (S102 in FIG. 4).

Next, between the time t13 and the time t14, the first pixel and the second pixel convert the incident light 150 into a signal charge and accumulate the converted signal charge (S103 in FIG. 4).

Next, at time t14, the second drive unit 122 closes the light control unit 112. Thereby, the light 150 does not enter the image sensor 114 after the time t14. That is, the exposure time ends.

As described above, the short exposure time of the first pixel is from time t13 to time t14 from when the VCCD sweep pulse is applied until the light control unit 112 is closed, and the long exposure time of the second pixel is the substrate sweep pulse. From time t11 to time t14 until the light control unit 112 is closed.

Note that before and after time t12 shown in FIG. 3, pulses are applied to φV3 and φV7 applied to the second pixel. This is done to prevent the potential of the p-well 192 from fluctuating due to the application of the VCCD sweep pulse to φV1 and φV5. It is not necessary to apply a pulse to φV3 and φV7 at this timing.

Next, at time t14 to time t19, the signal charges accumulated in the first pixel and the second pixel are read out. Here, the first drive unit 121 performs so-called N-field reading that causes the image sensor 114 to read the signal charges accumulated in the plurality of pixels 180 in N times.

Specifically, the signal charges held in the vertical transfer unit 181 are transferred to the horizontal transfer unit 182 during the dummy field period from time t14 to time t15. As a result, the signal charge held in the vertical transfer unit 181 is reset. That is, the signal charge swept out by the vertical transfer unit 181 from time t12 to time t13 is reset.

Also, the dummy field period has an effect of making the state of the vertical transfer unit 181 at the start of the first field period immediately after the dummy field period the same as the state at the start of each of the subsequent second to fourth field periods. As a result, the tendency of variation between the signal charge read in the first field period and the signal charge read in the second to fourth field periods can be made closer.

Note that if only to reset the signal charge held in the vertical transfer unit 181, so-called high-speed transfer is performed in which the transfer rate in the dummy field period is faster than the transfer rate in the first to fourth field periods. May be.

Next, in the first field period from time t15 to time t16, the signal charge accumulated in the second pixel to which φV7 is applied is read (S104 in FIG. 4).

Next, in the second field period from time t16 to time t17, the signal charge accumulated in the second pixel to which φV3 is applied is read (S105 in FIG. 4).

Next, in the third field period from time t17 to time t18, the signal charge accumulated in the first pixel to which φV5 is applied is read (S106 in FIG. 4).

Finally, in the fourth field period from time t18 to time t19, the signal charge accumulated in the first pixel to which φV1 is applied is read (S107 in FIG. 4).

As described above, the signal charges accumulated in all the pixels 180 are read out.

In addition, although the case where reading is performed by dividing into four fields has been described as an example here, the number of fields is not limited to four. Further, it is not necessary to divide into fields and perform reading. That is, the signal charges accumulated in all the pixels 180 may be read at a time.

Here, the signal charge of the first pixel is read after reading the signal charge of the second pixel. However, the signal charge of the second pixel may be read after reading the signal charge of the first pixel. Then, the signal charge of the first pixel and the signal charge of the second pixel may be alternately read out.

Next, the configuration of the vertical transfer electrode 195 of the image sensor 114 will be described.

FIG. 8A is a diagram illustrating a configuration example of the vertical transfer electrode 195. As shown in FIG. 8A, two vertical transfer electrodes 195 are formed for one pixel 180. In addition, when performing the above-described four-field reading, the first drive unit 121 controls the vertical transfer unit 181 with eight-phase drive pulses of φV1 to φV8.

As described above, the solid-state imaging device 100 according to the embodiment of the present invention can output the long exposure image signal 153a and the short exposure image signal 153b. Thereby, the signal processing unit 140 synthesizes the long exposure image signal 153a and the short exposure image signal 153b having different exposure times, so that the dynamic range can be expanded. Furthermore, in the solid-state imaging device 100 according to the embodiment of the present invention, the start times of the short exposure time t2 and the long exposure time t1 are different, and the light control unit 112 sets the short exposure time t2 and the long exposure time t1. Make the end time the same. As a result, the solid-state imaging device 100 according to the embodiment of the present invention does not need to hold the signal charge in the vertical transfer unit 181 for a long time as in the technique described in Patent Document 1, so that noise due to smear and dark current is reduced. Can be suppressed.

Further, in order to realize an increase in the number of pixels of the image sensor 114 and a reduction in size of the image sensor 114, as shown in FIG. 8A, two vertical transfer electrodes 195 are arranged for one pixel, and then N times. N-field reading is used to read signal charges separately.

Here, in the technique described in Patent Document 1, such N field readout cannot be performed because signal charges are temporarily accumulated in the vertical charge transfer path (vertical transfer unit 181).

Also, if the technique described in Patent Document 1 is to be realized, it is necessary to arrange one transfer stage for one pixel. Here, since it is necessary to arrange at least three vertical transfer electrodes 195 in one transfer stage, the structure of the vertical transfer electrodes 195 becomes complicated. Furthermore, since the area that can be occupied by one transfer stage is limited to one pixel, the amount of charge that can be transferred is reduced. On the other hand, the pixel size of a solid-state imaging device mounted on a recent digital still camera is miniaturized to 2 μm or less. It is extremely difficult to apply the conventional technique to such a fine pixel.

On the other hand, in the solid-state imaging device 100 according to the embodiment of the present invention, the signal charge is temporarily accumulated in the pixel 180, so that the N-field readout described above can be performed. As a result, the solid-state imaging device 100 according to the embodiment of the present invention only has to arrange two vertical transfer electrodes 195 in one pixel as shown in FIG. 8A.

Furthermore, it is possible to arrange only one vertical transfer electrode 195 per pixel. FIG. 8B is a diagram illustrating a configuration example of the vertical transfer electrode 195 in this case. FIG. 8B shows a configuration example of the vertical transfer electrode 195 in the case where the Bayer array is used. Details of the case in which such a Bayer array is used will be described later.

As shown in FIG. 8B, since the solid-state imaging device 100 according to the embodiment of the present invention only needs to arrange one vertical transfer electrode 195 per pixel, the imaging element 114 can be downsized, that is, the solid-state imaging device. 100 can be reduced in size and cost. Further, the present invention can be easily applied to a solid-state imaging device in which the pixel size is miniaturized, such as a solid-state imaging device for a digital still camera.

In the above description, at time t10 to time t11 shown in FIG. 3, the first drive unit 121 resets the signal charge accumulated in the second pixel by applying a substrate sweep pulse to the image sensor 114. However, the signal charge accumulated in the second pixel may be reset by applying a VCCD sweep pulse (readout pulse) to φV3 and φV7 applied to the second pixel.

FIG. 9 is a timing chart of the solid-state imaging device 100 when the first driving unit 121 resets the signal charge accumulated in the second pixel using the VCCD sweep pulse.

As shown in FIG. 9, the first driving unit 121 applies a VCCD sweep pulse (readout pulse) to φV3 and φV7 applied to the second pixel between time t20 and time t21 to thereby apply the second pixel to the second pixel. The accumulated signal charge may be reset.

In addition, during the time t22 to t23 shown in FIG. 9, the first driving unit 121 accumulates in the first pixel by applying the VCCD sweep pulse (readout pulse) to φV1 and φV5 applied to the first pixel. Although the signal charge is reset, the VCCD sweep pulse need not be applied. However, the first drive unit 121 applies the VCCD sweep pulse, and once resets the signal charge accumulated in the first pixel, the second VCCD sweep pulse at the time t12 to t13 is surely performed. The signal charge stored in one pixel can be reset. Therefore, as shown in FIG. 9, it is more preferable to apply a VCCD sweep pulse (readout pulse) to the first pixel between times t22 and t23.

Further, the first driving unit 121 may simultaneously apply the VCCD sweep pulse to the first pixel and the second pixel from time t20 to t21. However, when the VCCD sweep pulse is simultaneously applied to the first pixel and the second pixel, φV1, φV3, φV5, and φV7 simultaneously change to a high potential. Thereby, for example, the potential of the p well 192 changes. Therefore, as shown in FIG. 9, it is more preferable to apply the VCCD sweep pulse to the first pixel and the second pixel at different times.

As described above, by resetting the second pixel by applying the VCCD sweep pulse to the second pixel at the start time of the long exposure time t1, the image sensor 114 causes the vertical overflow drain (VOD) described above to be reset. There is no need to have a structure. Thereby, the structure of the image pick-up element 114 can be simplified.

Next, the operation of the signal processing unit 140 in the wide DR mode will be described.

As described above, in the wide DR mode, the long exposure image signal 153a corresponding to the long exposure time t1 and the short exposure image signal 153b corresponding to the short exposure time t2 are input to the signal processing unit 140. In addition, the signal processing unit 140 generates the image signal 151 having a wide dynamic range by combining the long exposure image signal 153a and the short exposure image signal 153b.

Hereinafter, the principle of obtaining the image signal 151 having a wide dynamic range by synthesizing the long exposure image signal 153a and the short exposure image signal 153b will be described.

FIG. 10 is a diagram showing the photoelectric conversion characteristics of the pixel 180 in the short exposure time and the long exposure time. The horizontal axis in FIG. 10 indicates the exposure amount (the integrated value of the amount of light received during the exposure time), and the vertical axis indicates the output level of the image signal 153.

As shown in FIG. 10, for the same exposure amount, the output level of the long exposure image signal 153a is equal to the output level of the short exposure image signal 153b and the output level of the long exposure image signal 153a and the short exposure image signal 153b. It increases according to the ratio of the exposure time (integration time). Further, the slope of the sensitivity curve shown in FIG. 10 represents the sensitivity. That is, the longer the exposure time, the higher the sensitivity.

Also, there is an upper limit on the amount of signal charge that can be accumulated in each pixel 180. As a result, when the output level of the image signal 153 reaches the output level corresponding to the upper limit (saturation output voltage VSAT), the output level of the image signal 153 is a constant value (saturation output) even if the exposure amount further increases. Voltage VSAT). The dynamic range with respect to the light amount (brightness) is determined according to the exposure amount (saturation exposure amount) reaching the saturation output voltage VSAT.

Therefore, the dynamic range of the long exposure image signal 153a with high sensitivity is narrow, and the dynamic range of the short exposure image signal 153b with low sensitivity is wide.

On the other hand, high sensitivity in the solid-state imaging device 100 is the most important requirement. However, when high sensitivity is realized, as shown in FIG. 10, the dynamic range for the contrast of the subject becomes narrow. In other words, the output level of the bright part is saturated and the details are crushed (generally called overexposure). That is, it is preferable that the sensitivity is high for a dark portion and the sensitivity is low for a bright portion.

To realize this, the long exposure image signal 153a and the short exposure image signal 153b may be synthesized. FIG. 11 is a diagram illustrating the characteristics of a combined image signal obtained by combining the long exposure image signal 153a and the short exposure image signal 153b. As shown in FIG. 11, the composite image signal can realize high sensitivity in a dark region (low exposure amount), and low sensitivity in a bright region (high exposure amount). Further, the composite image signal can provide a wide dynamic range determined by low sensitivity characteristics.

Hereinafter, the operation of the signal processing unit 140 shown in FIG. 1 will be described.

The first white clip processing unit 141 generates a corrected long exposure image signal 156 by performing white clip processing on the long exposure image signal 153a. Here, the white clip processing is processing for setting the level of the long exposure image signal 153a to the white clip level when the level of the long exposure image signal 153a is higher than the white clip level.

Here, in general, the saturation output voltage VSAT of the pixel 180 varies between elements. This causes unevenness in the video. That is, the knee points shown in FIG. 11 vary. In order to avoid this, the white clip process is performed. In other words, it is necessary to read out the image signals 153 of all the pixels 180 independently in order to perform this white clip processing. The white clip level is a level lower than the saturation output voltage VSAT.

The synthesizing unit 142 generates a synthesized image signal 157 having high sensitivity and a wide dynamic range by synthesizing the corrected long exposure image signal 156 and the short exposure image signal 153b generated by the first white clip processing unit 141. To do.

The output unit 143 outputs the composite image signal 157 as the image signal 151 to the outside. Note that the output unit 143 may generate the image signal 151 by performing predetermined signal processing such as gain adjustment, noise removal, level correction, and matrix conversion on the composite image signal 157. Further, the signal processing unit 140 may perform part or all of the predetermined signal processing on the long exposure image signal 153a, the short exposure image signal 153b, or the corrected long exposure image signal 156 before synthesis.

As described above, in the wide DR mode, the signal processing unit 140 can generate the image signal 151 having a wide dynamic range by combining the long exposure image signal 153a and the short exposure image signal 153b having different exposure times.

Next, the operation of the solid-state imaging device 100 in the high sensitivity mode will be described.

In the high sensitivity mode, the control unit 120 controls the light control unit 112 so that the light 150 is incident on all of the plurality of pixels 180 during the first period. In addition, the control unit 120 resets the signal charge accumulated in the second pixel at the first time included in the first period, and accumulates in the first pixel at the second time included in the first period. Reset the signal charge. In addition, the control unit 120 controls the light control unit 112 so that light does not enter all of the plurality of pixels 180 during the second period after the first period, and the first and second pixels forming the set The mixed signal charge is generated by mixing the signal charges accumulated in the two pixels in the plurality of vertical transfer units 181 or the horizontal transfer unit 182.

Also, the output unit 183 converts the mixed signal charge into a mixed image signal 153c, and outputs the converted mixed image signal 153c to the signal processing unit 140.

FIG. 12 is a timing chart showing the operation of the solid-state imaging device 100 in the high sensitivity mode. As shown in FIG. 12, in the high sensitivity mode, the control unit 120 makes the long exposure time t1 and the short exposure time t2 substantially equal.

Specifically, before the time t30, the second drive unit 122 opens the light control unit 112 to cause the light 150 to enter the image sensor 114.

Next, the first driving unit 121 applies a VCCD sweep pulse (readout pulse) to φV3 and φV7 applied to the second pixel between time t30 and time t31, so that the signal accumulated in the second pixel is applied. Reset the charge.

Next, the first driving unit 121 applies a VCCD sweep pulse (readout pulse) to φV1 and φV5 applied to the first pixel between time t32 and time t33, thereby storing the signal accumulated in the first pixel. Reset the charge.

Next, between time t33 and time t34, the first pixel and the second pixel convert the incident light 150 into signal charges, and store the converted signal charges.

Next, at time t34, the second drive unit 122 closes the light control unit 112. Thereby, the light 150 does not enter the image sensor 114 after the time t34. That is, the exposure time ends.

As described above, the short exposure time t2 of the first pixel is from time t33 to time t34 from when the VCCD sweep pulse is applied until the light control unit 112 is closed, and the long exposure time t1 of the second pixel is VCCD. It is from time t31 to time t34 from when the sweep pulse is applied until the light control unit 112 is closed.

Note that the first drive unit 121 may simultaneously apply the VCCD sweep pulse to the first pixel and the second pixel from time t30 to t31. However, when the VCCD sweep pulse is simultaneously applied to the first pixel and the second pixel, φV1, φV3, φV5, and φV7 simultaneously change to a high potential. Thereby, for example, the potential of the p well 192 changes. Therefore, as shown in FIG. 12, it is more preferable to apply the VCCD sweep pulse to the first pixel and the second pixel at different times.

Next, at time t34 to time t37, the signal charges accumulated in the first pixel and the second pixel are mixed and read out. Here, the first drive unit 121 performs so-called N-field reading that causes the image sensor 114 to read the signal charges accumulated in the plurality of pixels 180 in N times.

Specifically, the signal charge held in the vertical transfer unit 181 is transferred to the horizontal transfer unit 182 during the dummy field period from time t34 to time t35. As a result, the signal charge held in the vertical transfer unit 181 is reset. That is, the signal charges swept out by the vertical transfer unit 181 from time t30 to t31 and from time t32 to time t33 are reset.

Next, in the first field period from time t35 to time t36, the signal charges accumulated in the first pixel and the second pixel to which φV5 and φV7 are applied are mixed and read out.

Next, in the second field period from time t36 to time t37, the signal charges accumulated in the first pixel and the second pixel to which φV1 and φV3 are applied are mixed and read out.

Specifically, the first drive unit 121 mixes signal charges of pixels of the same color arranged adjacent to each other in the vertical direction in the vertical transfer unit 181.

As described above, the signal charges accumulated in all the pixels 180 are mixed and read out.

In addition, although the case where reading is performed in two fields has been described as an example here, the number of fields is not limited to two. Further, it is not necessary to divide into fields and perform reading. That is, the signal charges accumulated in all the pixels 180 may be read at a time.

As described above, in the high sensitivity mode, the solid-state imaging device 100 according to the embodiment of the present invention can increase the signal charge amount by mixing the signal charges of the pixels 180 forming the group in the imaging element 114. wear. Thereby, the effective sensitivity can be increased.

For example, when shooting a dark subject, sensitivity has priority over the dynamic range. In such a case, the high sensitivity mode is selected.

Next, the operation of the signal processing unit 140 in the high sensitivity mode will be described.

As described above, the mixed image signal 153c is input to the signal processing unit 140 in the high sensitivity mode.

The second white clip processing unit 144 illustrated in FIG. 1 generates a corrected mixed image signal 158 by performing white clip processing on the mixed image signal 153c. Here, the white clip level VM used by the second white clip processing unit 144 has the relationship of the following formula (1) with respect to the long exposure time t1, the short exposure time t2, and the saturation output voltage VSAT of the pixel 180. Set to meet.

VM ≦ VSAT × (1 + t2 / t1) (1)
Thereby, it is possible to avoid the unevenness of the saturation output voltage VSAT from appearing in the image. Further, if t1≈t2, the mixed charge amount is approximately doubled, and the effective sensitivity is approximately doubled.

The output unit 143 outputs the corrected mixed image signal 158 to the outside as the image signal 151. The output unit 143 may generate the image signal 151 by performing predetermined signal processing such as gain adjustment, noise removal, level correction, and matrix conversion on the corrected mixed image signal 158. Further, the signal processing unit 140 may perform part or all of the predetermined signal processing on the mixed image signal 153c.

As described above, in the high sensitivity mode, the solid-state imaging device 100 can mix the signal charges generated by the two pixels and output the highly sensitive image signal 151 corresponding to the mixed signal charges.

Hereinafter, the gravity center of the pixel mixture when the signal charges of the same color pixels adjacent in the upper and lower directions are mixed in the high sensitivity mode will be described. In addition, in the wide DR mode, the addition centroid when the image signals 153 of the same color pixels adjacent in the vertical direction are combined is also the same.

FIG. 13A is a diagram showing the distribution of the added centroids on the imaging surface when signal charges of pixels of the same color adjacent in the vertical direction are mixed. The center of gravity Rg shown in FIG. 13A is the added center of gravity of the R pixels R1 and R2, the center of gravity Gg is the added center of gravity of the G pixels G1 and G2, and the center of gravity Bg is the added center of gravity of the B pixels B1 and B2. As shown in FIG. 13A, since the adjacent pixels are mixed, the added centroids are regularly arranged. Thereby, deterioration of image quality (resolution) can be suppressed as much as possible.

FIG. 13B is a diagram illustrating an example of pixel dimensions. As shown in FIG. 13B, isotropic resolution can be ensured by setting the vertical pixel size to about ½ of the horizontal pixel size. For example, compared with the case where the vertical pixel size is equal to the horizontal pixel size, the vertical pixel size is made smaller than the horizontal pixel size and larger than 1/4, so that the resolution can be reduced. The directionality can be improved. Here, the pixel size is a size of a region including the pixel 180 (photodiode 196) and part of the vertical transfer unit 181 corresponding to the pixel 180 (two vertical transfer electrodes 195 in the above example). It is.

In the above description, the example in which the first pixel and the second pixel forming the set are adjacent to each other in the column direction is shown. However, the first pixel and the second pixel forming the set may be adjacent to each other in the row direction.

Moreover, the arrangement shown below may be used for the arrangement relationship between the first pixel and the second pixel.

FIG. 14A is a diagram illustrating an arrangement example of the first pixel and the second pixel, and a distribution of the added centroids.

As shown in FIG. 14A, the pixels forming a group may be arranged next to each other in an oblique direction. In addition, the dotted line frame shown to FIG. 14A has shown this group.

Specifically, the G pixels are arranged in a checkered pattern, and the R pixels and the B pixels are arranged in an oblique stripe form. That is, the R pixel and the B pixel are alternately arranged via the G pixel in the row direction and the column direction. In addition, columns in which only the first pixels are arranged and columns in which the second pixels are arranged are alternately arranged. In other words, two of the four pixels 180 diagonally adjacent to each R pixel and each B pixel filter are R pixels, and two are B pixels.

When the pixels forming the set are adjacent to each other in the oblique direction, the spatial sampling pitch of the G pixel, the B pixel, and the R pixel can be made equal in both the horizontal and vertical directions. Furthermore, the distribution of the spatial mixing centroid due to the mixing of signal charges between the pixels forming the set can be made uniform. Thereby, good resolution characteristics can be obtained.

FIG. 14B is a diagram showing a two-dimensional Nyquist limit before and after mixing. Before the mixing, the resolution characteristics in the oblique direction of the R pixel and the B pixel are not uniform, but as shown in FIG. 14B, by mixing the two pixels, the R pixel, the B pixel, and the G pixel are evenly distributed. Resolution characteristics can be realized.

Hereinafter, a pixel mixing method in the pixel arrangement shown in FIG. 14A will be described.

FIG. 15 is a diagram schematically illustrating a configuration example of the vertical transfer electrode 195 and a reading operation in the pixel arrangement illustrated in FIG. 14A.

As shown in FIG. 15, the image sensor 114 further includes a position adjusting unit 184 and a line memory 185.

The position adjustment unit 184 includes the signal charges transferred by the plurality of first column vertical transfer units 181 in which only the first pixels are arranged, and the plurality of second column vertical transfer units 181 in which only the second pixels are arranged. The signal charges transferred by the above are transferred at different stages. For example, in the example shown in FIG. 15, the signal charge transferred by the vertical transfer unit 181 in the first column is transferred in three stages, and the signal charge transferred by the vertical transfer unit 181 in the second column is transferred in four stages. .

The line memory 185 temporarily holds the signal charge transferred by the position adjustment unit 184. Also, the line memory 185 is. The signal charge to be held is transferred to the horizontal transfer unit 182.

Hereinafter, the case where the signal charges accumulated in the first pixel 180A and the second pixel 180B forming the set shown in FIG. 15 are mixed will be described as an example. Note that the numerical values in circles in FIG. 15 indicate the number of transfers.

As shown in FIG. 15, the signal charges of the first pixel 180A and the second pixel 180B forming a set are simultaneously read out to the vertical transfer unit 181. Next, each read signal charge is input to the position adjustment unit 184 by being sequentially transferred in the vertical direction (downward in FIG. 15). The position adjustment unit 184 transfers the input signal charges. Thus, the signal charges of the first pixel 180A and the second pixel 180B forming a set in the line memory 185 are simultaneously held. Next, the line memory 185 transfers the signal charges of the first pixel 180A and the second pixel 180B forming this set to the horizontal transfer unit 182. Next, the horizontal transfer unit 182 mixes the signal charges of the first pixel 180A and the second pixel 180B transferred from the line memory 185 in the horizontal transfer unit 182 and transfers the mixed signal charges in the horizontal direction. . Next, the output unit 183 converts the signal charge transferred by the horizontal transfer unit 182 into a mixed image signal 153c, and outputs the converted mixed image signal 153c.

As described above, the signal charges of the first pixels and the signal charges of the second pixels arranged adjacent to each other in the oblique direction can be mixed in the image sensor 114.

In addition, as a pixel arrangement in which the first pixel and the second pixel are arranged adjacent to each other in the oblique direction, the following pixel arrangement may be used.

16 and 17 are diagrams showing a variation of the pixel arrangement and the distribution of the added centroids. Note that the dotted frame shown in FIGS. 16 and 17 indicates a pair of corresponding first pixel and second pixel.

In the pixel arrangement example shown in FIG. 16, G pixels are arranged in a checkered pattern, and R pixels and B pixels are arranged in vertical stripes every two columns. That is, the R pixel and the B pixel are alternately arranged via the G pixel in the row direction. In addition, a column in which only the first pixels of the G pixel and the R pixel are alternately arranged, a column in which only the second pixel of the G pixel and the R pixel are alternately arranged, and only the first pixel of the G pixel and the B pixel. Are alternately arranged, and a column in which only the second pixels of the G pixel and the B pixel are alternately arranged is repeatedly arranged in this order.

In the pixel arrangement example shown in FIG. 17, the G pixels are arranged in a checkered pattern, and the R pixels and the B pixels are arranged in a horizontal stripe every two rows. That is, the R pixel and the B pixel are alternately arranged via the G pixel in the column direction. In addition, only the first pixel of the G pixel and the R pixel, a row where only the second pixel of the G pixel and the R pixel are alternately arranged, and only the first pixel of the G pixel and the B pixel are arranged. Are alternately arranged, and rows in which only the second pixels of the G pixel and the B pixel are alternately arranged are repeatedly arranged in this order.

In the case of the arrangement shown in FIGS. 16 and 17, the same effect as the arrangement shown in FIG. 14A can be obtained.

Also, the following pixel arrangement may be used as the arrangement of the first pixel and the second pixel.

FIG. 18 is a diagram illustrating a variation of the pixel arrangement and the distribution of the added centroids. Note that the dotted frame shown in FIG. 18 indicates a set of the corresponding first pixel and second pixel.

In the pixel array shown in FIG. 18, R pixels, G pixels, and B pixels are Bayer arrayed. Specifically, G pixels are arranged in a checkered pattern. In addition, rows in which only G pixels and R pixels are alternately arranged and rows in which only G pixels and B pixels are alternately arranged are alternately arranged, and only G pixels and R pixels are alternately arranged. And columns in which only G pixels and B pixels are alternately arranged are alternately arranged.

Further, the G pixel forms a pair between pixels adjacent in the oblique direction, and the R pixel and the B pixel form a pair between pixels of the same color that are spatially closest. For example, in the example illustrated in FIG. 18, the R pixel and the B pixel form a group between pixels of the same color that are arranged next to each other in the column direction. Note that the R pixel and the B pixel may form a pair between pixels of the same color that are arranged next to each other in the row direction.

By using this pixel arrangement, the added centroids of the G pixel, R pixel, and B pixel can be arranged spatially and evenly. Therefore, uniform resolution characteristics can be realized in each of the R pixel, the B pixel, and the G pixel.

Also, by using the Bayer array shown in FIG. 18, it is possible to realize uniform resolution characteristics in each of the R pixel, the B pixel, and the G pixel in the high resolution mode described later.

Next, the operation of the solid-state imaging device 100 in the high resolution mode will be described.

In the high resolution mode, the control unit 120 causes the light control unit 112 to control so that light enters all of the plurality of pixels 180 during the first period. In addition, the control unit 120 resets the signal charges accumulated in the plurality of second pixels at the first time included in the first period, and the plurality of first pixels at the second time included in the first period. The signal charge accumulated in the is reset. In addition, during the second period after the first period, the control unit 120 controls the light control unit 112 so that light does not enter all of the plurality of pixels 180 and is stored in the plurality of pixels 180. The signal charges are read through the plurality of vertical transfer units 181 and horizontal transfer units 182 respectively.

Also, the output unit 183 converts the read signal charge into a high resolution image signal 153d, and outputs the converted high resolution image signal 153d to the signal processing unit 140.

Next, the overall operation flow of the solid-state imaging device 100 will be described.

FIG. 19 is a flowchart showing an operation flow of the solid-state imaging device 100.

First, the mode acquisition unit 131 acquires the designation mode signal 152 corresponding to the user's operation, and outputs the obtained designation mode signal 152 to the mode selection unit 133 (S201). Here, the designation mode signal 152 indicates any one of a wide DR mode, a high sensitivity mode, a high resolution mode, and an auto mode, which are operation modes of the solid-state imaging device 100.

Also, the brightness acquisition unit 132 acquires brightness information 159 indicating the brightness of the object scene, and outputs the acquired brightness information 159 to the mode selection unit 133 (S202). For example, the brightness acquisition unit 132 acquires brightness information output from an optical sensor provided in the solid-state imaging device 100 or provided in an imaging device (such as a digital still camera) on which the solid-state imaging device 100 is mounted. Note that the brightness acquisition unit 132 may determine the brightness of the object scene from the

image signal

151 or 153 captured in advance.

Next, the mode selection unit 133 selects an operation mode using the designation mode signal 152 and the brightness information 159, and outputs a selection mode signal 160 indicating the selected operation mode to the control unit 120.

Specifically, when the auto mode is designated by the designated mode signal 152 (Yes in S203), the mode selection unit 133 determines that the brightness of the object scene indicated by the brightness information 159 is equal to or greater than a predetermined threshold value. It is determined whether or not (S204).

When the brightness of the scene shown in the brightness information 159 is equal to or greater than a predetermined threshold (Yes in S204), that is, when the scene is bright, the mode selection unit 133 selects the wide DR mode. The selection mode signal 160 indicating the wide DR mode is output to the control unit 120.

Thereby, the first drive unit 121 generates the first drive signal 154 that causes the image sensor 114 to perform the above-described wide DR mode operation. Specifically, the first drive signal 154 includes a substrate sweep pulse (Vsub), a drive signal for driving the vertical transfer unit 181 (φV1 to φV8, etc.), and a drive signal for driving the horizontal transfer unit 182 (φH1 to φH4). ).

Specifically, the control unit 120 operates the image sensor 114 with non-addition readout, and makes the long exposure time t1 longer than the short exposure time t2. For example, the control unit 120 sets the long exposure time t1 to at least twice the short exposure time t2 (S205).

Thereby, the image sensor 114 generates the long exposure image signal 153a corresponding to the long exposure time t1 and the short exposure image signal 153b corresponding to the short exposure time t2 (S206).

Next, the first white clip processing unit 141 generates a corrected long exposure image signal 156 by performing white clip processing on the long exposure image signal 153a. Next, the synthesizing unit 142 generates the synthesized image signal 157 by synthesizing the corrected long exposure image signal 156 and the short exposure image signal 153b (S207).

Next, the output unit 143 determines whether or not the operation mode selected by the mode selection unit 133 is appropriate for the state of the object scene (S208). Specifically, the output unit 143 determines that the operation mode is not appropriate when overexposure or underexposure occurs. For example, the output unit 143 determines that the operation mode is appropriate when the average value of the signal levels of the composite image signal 157 is within a predetermined range, and determines that the operation mode is not appropriate when outside the range. To do. Note that the output unit 143 determines that the operation mode is appropriate when the number of pixels included in the composite image signal 157 whose signal level is outside the predetermined range is less than a predetermined number, and the number is equal to or greater than the number. In this case, it may be determined that the operation mode is not appropriate.

When the operation mode is appropriate (Yes in S208), the output unit 143 outputs the composite image signal 157 to the outside as the image signal 151 (S216).

On the other hand, when the brightness of the object scene indicated in the brightness information 159 is less than a predetermined threshold value in step S204 (No in S204), that is, when the object scene is dark, the mode selection unit 133 The high sensitivity mode is selected, and a selection mode signal 160 indicating the high sensitivity mode is output to the control unit 120.

Thereby, the first drive unit 121 generates the first drive signal 154 that causes the image sensor 114 to perform the above-described operation in the high sensitivity mode.

Specifically, the control unit 120 operates the image sensor 114 with mixed readout, and makes the long exposure time t1 and the short exposure time t2 substantially equal. For example, the control unit 120 sets the long exposure time t1 within the range of 90 to 110% of the short exposure time t2 (S210).

Thereby, the image sensor 114 generates the mixed image signal 153c (S211).

Next, the second white clip processing unit 144 generates a corrected mixed image signal 158 by performing white clip processing on the mixed image signal 153c.

Next, the output unit 143 determines whether or not the operation mode selected by the mode selection unit 133 is appropriate for the state of the object scene (S208).

If the operation mode is appropriate (Yes in S208), the output unit 143 then outputs the corrected mixed image signal 158 to the outside as the image signal 151 (S216).

On the other hand, when the wide DR mode is designated by the designated mode signal 152 (No in S203 and Yes in S212), the mode selection unit 133 selects the wide DR mode and outputs the selection mode signal 160 indicating the wide DR mode. Output to the control unit 120. The operation in this case is the same as that in the case where the brightness of the object scene indicated in the brightness information 159 is equal to or greater than a predetermined threshold value (Yes in S204).

When the high sensitivity mode is designated by the designated mode signal 152 (No in S203, No in S212, and Yes in S213), the mode selection unit 133 selects the high sensitivity mode and indicates the high sensitivity mode. The selection mode signal 160 is output to the control unit 120. Note that the operation in this case is the same as that in the case where the brightness of the object scene indicated in the brightness information 159 is less than a predetermined threshold in the auto mode (No in S204).

On the other hand, when the high resolution mode is designated by the designated mode signal 152 (No in S203, No in S212, and No in S213), the mode selection unit 133 selects the high resolution mode and indicates the high resolution mode. The selection mode signal 160 is output to the control unit 120.

Thereby, the first drive unit 121 generates the first drive signal 154 that causes the image sensor 114 to perform the operation in the high resolution mode.

Specifically, the control unit 120 operates the image sensor 114 with non-addition readout, and makes the long exposure time t1 and the short exposure time t2 substantially equal. For example, the control unit 120 sets the long exposure time t1 within the range of 90 to 110% of the short exposure time t2 (S214).

Thereby, the image sensor 114 generates the high-resolution image signal 153d (S215). The high resolution image signal 153d has twice the resolution of the long exposure image signal 153a, the short exposure image signal 153b, and the mixed image signal 153c.

If the operation mode is appropriate (Yes in S208), the output unit 143 then outputs the high resolution image signal 153d as an image signal 151 to the outside (S216).

On the other hand, if the operation mode is not appropriate in each operation mode (No in S208), the brightness acquisition unit 132 acquires the brightness information 159 again and outputs the acquired brightness information 159 to the mode selection unit 133. (S209).

Next, the mode selection unit 133 performs the same process as when the auto mode is designated by the designation mode signal 152 (Yes in S203).

If it is not appropriate in the high resolution mode (No in S208), the exposure time t1≈t2 may be changed based on the brightness information 159, and the process of step S214 may be performed again. Specifically, when the object scene is bright, the exposure time t1≈t2 may be shortened, and when the object scene is dark, the exposure time t1≈t2 may be increased.

As described above, the solid-state imaging device 100 according to the embodiment of the present invention generates the long exposure image signal 153a and the short exposure image signal 153b having different exposure times in the wide DR mode, and the short exposure image signal 153a and the short exposure image signal 153a. By combining the exposure image signal 153b, an image signal 151 having a wide dynamic range can be generated.

Furthermore, in the solid-state imaging device 100 according to the embodiment of the present invention, the start times of the two exposure times are made different, and the end times of the two exposure times are made the same time by the light control unit 112. Thereby, the solid-state imaging device 100 according to the embodiment of the present invention can hold the signal charge in the pixel 180 during the period from the end of the exposure time to the reading of the signal charge. Therefore, the solid-state imaging device 100 does not need to hold the signal charge in the vertical transfer unit 181 or the like for a long time, and thus noise due to smear and dark current can be suppressed.

In the high sensitivity mode, the solid-state imaging device 100 according to the embodiment of the present invention can generate a highly sensitive image signal 151 by mixing signal charges of two pixels in the imaging element 114.

In the high-resolution mode, the solid-state imaging device 100 according to the embodiment of the present invention can generate the high-resolution image signal 151 by reading the signal charges of each pixel 180 individually.

Further, the solid-state imaging device 100 according to the embodiment of the present invention selects an operation mode according to the brightness of the object scene. Thereby, the solid-state imaging device 100 can automatically select an optimal operation mode.

The solid-state imaging device 100 according to the embodiment of the present invention has been described above, but the present invention is not limited to this embodiment.

For example, in the above embodiment, the light control unit 112 is a mechanical shutter, but other than the mechanical shutter as long as it can control whether light is incident on all of the plurality of pixels 180. But you can.

Specifically, the light control unit 112 may be an optical element that can electrically control light transmittance, such as a liquid crystal panel. For example, the optical element is disposed on the entire incident surface side of the image sensor 114. In this configuration, the light 150 is controlled not to be incident on the image sensor 114 by lowering the transmittance of the optical element, and the light 150 is incident on the image sensor 114 by increasing the transmittance of the optical element. Can be controlled.

The light control unit 112 may be a device that can control the reflection direction of the light 150 such as a MEMS (Micro Electro Mechanical Systems) mirror.

20A and 20B are diagrams illustrating a configuration example of the imaging unit 110 when the MEMS mirror 112A is used as the light control unit 112. FIG.

As shown in FIG. 20A, the light 150 condensed by the lens 111 can be incident on the image sensor 114 by controlling the reflection direction of the MEMS mirror 112A. In addition, as shown in FIG. 20B, the light 150 collected by the lens 111 can be prevented from entering the image sensor 114 by controlling the reflection direction of the MEMS mirror 112A.

Further, the light control unit 112 may be a light irradiation device that irradiates a subject such as a strobe light. For example, when shooting in a dark place, the light irradiation device irradiates the subject with light, thereby causing the light 150 to enter the imaging device 114, and the light irradiation device does not irradiate the subject with light. It is possible to control so that the light 150 does not enter the beam 114.

In addition, each processing unit included in the solid-state imaging device 100 according to the above embodiment is typically realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

Further, the integration of circuits is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

In addition, a part of or all of the functions of the control unit 120, the mode determination unit 130, and the signal processing unit 140 included in the solid-state imaging device 100 according to the embodiment of the present invention is executed by a processor such as a CPU. It may be realized.

Furthermore, the present invention may be the above program or a recording medium on which the above program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

In the above description, the solid-state imaging device 100 according to the embodiment of the present invention includes the imaging unit 110, the control unit 120, the mode determination unit 130, and the signal processing unit 140. However, the mode determination unit 130 and the signal processing are described. At least one of the units 140 may be formed outside the solid-state imaging device 100. That is, the solid-state imaging device 100 may output the image signal 153 to the outside, and the external device may perform the above-described signal processing on the image signal 153.

Also, the numbers used above are all exemplified for specifically describing the present invention, and the present invention is not limited to the illustrated numbers. Furthermore, the logic levels represented by high / low or the switching states represented by on / off are illustrative for the purpose of illustrating the present invention, and different combinations of the illustrated logic levels or switching states. Therefore, it is possible to obtain an equivalent result. In addition, n-type and p-type such as a transistor and a semiconductor layer are exemplified to specifically describe the present invention, and it is possible to obtain equivalent results by inverting them. In addition, the connection relationship between the components is exemplified for specifically explaining the present invention, and the connection relationship for realizing the function of the present invention is not limited to this.

Furthermore, various modifications in which the present embodiment is modified within the scope conceived by those skilled in the art are also included in the present invention without departing from the gist of the present invention.

Further, at least a part of the functions of the solid-state imaging device 100 and the modification thereof according to the above-described embodiment may be combined.

The present invention can be applied to a solid-state imaging device, and in particular to a digital still camera.

DESCRIPTION OF SYMBOLS 100 Solid-state imaging device 110

Imaging part

111, 113 Lens 112 Light control part 112A MEMS mirror 114 Imaging element 120 Control part 121 1st drive part 122 2nd drive part 130 Mode determination part 131 Mode acquisition part 132 Brightness acquisition part 133 Mode selection Unit 140 signal processing unit 141 first white clip processing unit 142 combining unit 143 output unit 144 second white clip processing unit 150 light 151, 153 image signal 152 designation mode signal 153a long exposure image signal 153b short exposure image signal 153c mixed image signal 153d High-resolution image signal 154 First drive signal 155 Second drive signal 156 Corrected long exposure image signal 157 Composite image signal 158 Corrected mixed image signal 159 Brightness information 160 Selection mode signal 180 Pixel 180A First image Element 180B Second pixel 181 Vertical transfer unit 182 Horizontal transfer unit 183 Output unit 184 Position adjustment unit 185 Line memory 191 N-type semiconductor substrate 192 p well 193 p + region 194 Vertical transfer channel 195 Vertical transfer electrode 196 Photodiode Bg, Gg, Rg Center of gravity B1, B2 B pixel G1, G2 G pixel R1, R2 R pixel

Claims

A solid-state imaging device,
A plurality of photoelectric conversion elements that are arranged in a matrix, convert light into signal charges, and store the converted signal charges;
A plurality of vertical transfer units provided for each column and transferring the signal charges accumulated in a plurality of photoelectric conversion elements arranged in the corresponding column;
A horizontal transfer unit for transferring the signal charges transferred by the plurality of vertical transfer units;
A light control unit that controls whether light is incident on the plurality of photoelectric conversion elements;
A control unit,
The controller is
During the first period, the light control unit is controlled so that light is incident on the plurality of photoelectric conversion elements,
Resetting the signal charges accumulated in the plurality of first photoelectric conversion elements included in the plurality of photoelectric conversion elements at a first time included in the first period;
Resetting the signal charges accumulated in the plurality of second photoelectric conversion elements included in the plurality of photoelectric conversion elements at a second time included in the first period and after the first time;
During the second period immediately after the first period, the light control unit is controlled so that light does not enter the plurality of photoelectric conversion elements,
The first signal charge accumulated in the plurality of first photoelectric conversion elements during the first exposure time from the first time to the start time of the second period, and the start of the second period from the second time Reading out the second signal charges accumulated by the plurality of second photoelectric conversion elements during the second exposure time until the time through the vertical transfer unit and the horizontal transfer unit during the second period,
Each of the first photoelectric conversion elements and each of the second photoelectric conversion elements are arranged in a one-to-one correspondence,
Each of the first photoelectric conversion elements is disposed within a predetermined distance from the second photoelectric conversion element corresponding to the first photoelectric conversion element.
Each of the first photoelectric conversion elements is arranged next to the second photoelectric conversion element corresponding to the first photoelectric conversion element in the row direction, the column direction, or the oblique direction, or adjacent to the second photoelectric conversion element in the row direction or the column direction. The solid-state imaging device according to claim 1.
The solid-state imaging device further includes:
Comprising a plurality of filters respectively formed on the plurality of photoelectric conversion elements;
Each of the plurality of filters is one of a plurality of types of filters that transmit any one of a plurality of wavelength bands,
The solid-state imaging device according to claim 1, wherein the same type of filter is formed on the first photoelectric conversion element and the second photoelectric conversion element corresponding to each other.
Each of the plurality of filters is any one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light.
Each of the first photoelectric conversion elements is arranged next to the second photoelectric conversion element corresponding to the first photoelectric conversion element in the row direction or the column direction,
The solid-state imaging device according to claim 3, wherein the first photoelectric conversion element and the second photoelectric conversion element corresponding to each other are arranged in a Bayer array.
Each of the plurality of filters is one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light, and is arranged in a Bayer array.
Each first photoelectric conversion element formed under the green filter is disposed next to the second photoelectric conversion element corresponding to the first photoelectric conversion element in an oblique direction,
Each 1st photoelectric conversion element arrange | positioned under the said red filter or the said blue filter is arrange | positioned 2 adjacent in the row direction or column direction of the said 2nd photoelectric conversion element corresponding to the said 1st photoelectric conversion element. The solid-state imaging device according to claim 3.
Each of the plurality of filters is any one of a red filter that transmits red light, a green filter that transmits green light, and a blue filter that transmits blue light.
The green filter is arranged in a checkered pattern,
Of the four filters diagonally adjacent to each red filter and each blue filter, two are the red filters and two are blue filters,
The solid-state imaging device according to claim 3, wherein each of the first photoelectric conversion elements is arranged next to the second photoelectric conversion element corresponding to the first photoelectric conversion element in an oblique direction.
The solid-state imaging device further includes:
Comprising a semiconductor substrate on which the plurality of photoelectric conversion elements are formed;
The control unit discharges the signal charges accumulated in the plurality of first photoelectric conversion elements at the first time to the semiconductor substrate or transfers the signal charges to the vertical transfer unit, thereby Resetting the signal charge accumulated in one photoelectric conversion element and transferring the signal charge accumulated in the plurality of second photoelectric conversion elements at the second time to the vertical transfer unit, The solid-state imaging device according to any one of claims 1 to 6, wherein the signal charges accumulated in a plurality of second photoelectric conversion elements are reset.
The solid-state imaging device further includes:
An output unit that converts the first signal charge into a first image signal, converts the second signal charge into a second image signal, and outputs the converted first image signal and the second image signal;
The solid-state imaging device according to any one of claims 1 to 7, further comprising: a signal processing unit that synthesizes the first image signal and the second image signal.
The signal processing unit
A first white clip processing unit that generates a corrected first image signal by setting the first image signal to the first value when the first image signal is greater than a first value;
The solid-state imaging device according to claim 8, further comprising a combining unit that combines the corrected first image signal and the second image signal.
The solid-state imaging device has a first operation mode and a second operation mode,
During the first operation mode,
The controller reads the first signal charge and the second signal charge;
The output unit converts the first signal charge into a first image signal, converts the second signal charge into a second image signal, and outputs the converted first image signal and the second image signal,
The signal processing unit combines the first image signal and the second image signal,
In the second operation mode,
The control unit generates a mixed signal charge by mixing the first signal charge and the second signal charge in the vertical transfer unit or the horizontal transfer unit,
The solid-state imaging device according to claim 8 or 9, wherein the output unit converts the mixed signal charge into a mixed image signal and outputs the converted mixed image signal.
The signal processing unit further includes:
In the second operation mode, when the mixed image signal is larger than a second value, a second white clip processing unit that generates a corrected second image signal by setting the mixed image signal to the second value With
The second value is VSAT × ((2) when the value of the mixed image signal corresponding to the saturation signal charge of the photoelectric conversion element is VSAT, the first exposure time is t1, and the second exposure time is t2. The solid-state imaging device according to claim 10, which is equal to or less than a value represented by 1 + t2 / t1).
The control unit makes the first exposure time more than twice the second exposure time, and sets the second exposure time to 90% or more of the first exposure time in the second operation mode. The solid-state imaging device according to 11.
The solid-state imaging device further includes:
A brightness acquisition unit for acquiring the brightness of the scene;
When the brightness acquired by the brightness acquisition unit is greater than a third value, the first operation mode is selected, and when the brightness acquired by the brightness acquisition unit is less than the third value, A mode selection unit for selecting the second operation mode,
The solid-state imaging device according to any one of claims 10 to 12, wherein the solid-state imaging device operates in the first operation mode or the second operation mode selected by the mode selection unit.
The solid-state imaging device according to any one of claims 1 to 13, wherein the light control unit is a mechanical shutter, a liquid crystal, a MEMS mirror, or an optically controllable optical element.
A plurality of photoelectric conversion elements that are arranged in a matrix, convert light into signal charges, and store the converted signal charges;
A plurality of vertical transfer units that are provided for each column and transfer the signal charges accumulated in the photoelectric conversion elements arranged in the corresponding column;
A horizontal transfer unit for transferring the signal charges transferred by the plurality of vertical transfer units;
A control method of a solid-state imaging device comprising: a light control unit that controls whether light is incident on the plurality of photoelectric conversion elements;
During the first period, causing the light control unit to control the light to enter the plurality of photoelectric conversion elements;
Resetting the signal charges accumulated in the plurality of first photoelectric conversion elements included in the plurality of photoelectric conversion elements at a first time included in the first period;
Resetting the signal charges accumulated in the plurality of second photoelectric conversion elements included in the plurality of photoelectric conversion elements at a second time that is included in the first period and after the first time;
Controlling the light control unit so that light does not enter the plurality of photoelectric conversion elements during a second period immediately after the first period;
The first signal charge accumulated in the plurality of first photoelectric conversion elements during the first exposure time from the first time to the start time of the second period, and the start of the second period from the second time Reading out the second signal charges accumulated by the plurality of second photoelectric conversion elements during the second exposure time until the time through the vertical transfer unit and the horizontal transfer unit during the second period; Including
Each of the first photoelectric conversion elements and each of the second photoelectric conversion elements are arranged in a one-to-one correspondence,
Each said 1st photoelectric conversion element is arrange | positioned within the predetermined distance from the said 2nd photoelectric conversion element corresponding to the said 1st photoelectric conversion element. The control method of a solid-state imaging device.