US20240163585A1 - Imaging device - Google Patents
Imaging device Download PDFInfo
- Publication number
- US20240163585A1 US20240163585A1 US18/550,402 US202218550402A US2024163585A1 US 20240163585 A1 US20240163585 A1 US 20240163585A1 US 202218550402 A US202218550402 A US 202218550402A US 2024163585 A1 US2024163585 A1 US 2024163585A1
- Authority
- US
- United States
- Prior art keywords
- charge accumulation
- accumulation section
- changeover switch
- section
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 142
- 238000009825 accumulation Methods 0.000 claims abstract description 235
- 238000006243 chemical reaction Methods 0.000 claims abstract description 91
- 238000012546 transfer Methods 0.000 claims abstract description 37
- 238000012986 modification Methods 0.000 description 74
- 230000004048 modification Effects 0.000 description 74
- 238000012545 processing Methods 0.000 description 46
- 238000010586 diagram Methods 0.000 description 40
- 238000004891 communication Methods 0.000 description 28
- 238000005516 engineering process Methods 0.000 description 22
- 238000001727 in vivo Methods 0.000 description 15
- 239000002775 capsule Substances 0.000 description 14
- 230000003287 optical effect Effects 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 210000001519 tissue Anatomy 0.000 description 9
- 230000003321 amplification Effects 0.000 description 8
- 238000003199 nucleic acid amplification method Methods 0.000 description 8
- 239000004065 semiconductor Substances 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 238000000034 method Methods 0.000 description 6
- 238000001356 surgical procedure Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 230000005284 excitation Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 229910000673 Indium arsenide Inorganic materials 0.000 description 3
- 208000005646 Pneumoperitoneum Diseases 0.000 description 3
- 239000003990 capacitor Substances 0.000 description 3
- RPQDHPTXJYYUPQ-UHFFFAOYSA-N indium arsenide Chemical compound [In]#[As] RPQDHPTXJYYUPQ-UHFFFAOYSA-N 0.000 description 3
- 230000001678 irradiating effect Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 239000000758 substrate Substances 0.000 description 3
- 229910000530 Gallium indium arsenide Inorganic materials 0.000 description 2
- 229910000661 Mercury cadmium telluride Inorganic materials 0.000 description 2
- 229910052787 antimony Inorganic materials 0.000 description 2
- WATWJIUSRGPENY-UHFFFAOYSA-N antimony atom Chemical compound [Sb] WATWJIUSRGPENY-UHFFFAOYSA-N 0.000 description 2
- 210000004204 blood vessel Anatomy 0.000 description 2
- 239000003153 chemical reaction reagent Substances 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- WPYVAWXEWQSOGY-UHFFFAOYSA-N indium antimonide Chemical compound [Sb]#[In] WPYVAWXEWQSOGY-UHFFFAOYSA-N 0.000 description 2
- MOFVSTNWEDAEEK-UHFFFAOYSA-M indocyanine green Chemical compound [Na+].[O-]S(=O)(=O)CCCCN1C2=CC=C3C=CC=CC3=C2C(C)(C)C1=CC=CC=CC=CC1=[N+](CCCCS([O-])(=O)=O)C2=CC=C(C=CC=C3)C3=C2C1(C)C MOFVSTNWEDAEEK-UHFFFAOYSA-M 0.000 description 2
- 229960004657 indocyanine green Drugs 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000007639 printing Methods 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- JBRZTFJDHDCESZ-UHFFFAOYSA-N AsGa Chemical compound [As]#[Ga] JBRZTFJDHDCESZ-UHFFFAOYSA-N 0.000 description 1
- 229910001218 Gallium arsenide Inorganic materials 0.000 description 1
- 240000004050 Pentaglottis sempervirens Species 0.000 description 1
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 1
- KXNLCSXBJCPWGL-UHFFFAOYSA-N [Ga].[As].[In] Chemical compound [Ga].[As].[In] KXNLCSXBJCPWGL-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000000740 bleeding effect Effects 0.000 description 1
- 210000000746 body region Anatomy 0.000 description 1
- MCMSPRNYOJJPIZ-UHFFFAOYSA-N cadmium;mercury;tellurium Chemical compound [Cd]=[Te]=[Hg] MCMSPRNYOJJPIZ-UHFFFAOYSA-N 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000002674 endoscopic surgery Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 229910052732 germanium Inorganic materials 0.000 description 1
- GNPVGFCGXDBREM-UHFFFAOYSA-N germanium atom Chemical compound [Ge] GNPVGFCGXDBREM-UHFFFAOYSA-N 0.000 description 1
- 230000004313 glare Effects 0.000 description 1
- 210000000936 intestine Anatomy 0.000 description 1
- 230000031700 light absorption Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 210000004400 mucous membrane Anatomy 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002572 peristaltic effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000008929 regeneration Effects 0.000 description 1
- 238000011069 regeneration method Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/62—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels
- H04N25/621—Detection or reduction of noise due to excess charges produced by the exposure, e.g. smear, blooming, ghost image, crosstalk or leakage between pixels for the control of blooming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
- H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/57—Control of the dynamic range
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/57—Control of the dynamic range
- H04N25/59—Control of the dynamic range by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
- H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
- H04N25/771—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising storage means other than floating diffusion
Definitions
- the present disclosure relates to an imaging device.
- An imaging device generally includes a photoelectric conversion element and a readout integrated circuit (ROIC) that reads out charges from the photoelectric conversion element.
- the readout integrated circuit is provided with a charge accumulation section that accumulates the charge read out from the photoelectric conversion element.
- the conversion efficiency indicating the voltage of the output signal per one electron depends on the capacitance of the charge accumulation section.
- Patent Document 1 WO 2019/155841
- the conversion efficiency is also fixed. In this case, for example, the conversion efficiency cannot be changed according to the charge amount from the photoelectric conversion element, and thus it is difficult to expand the dynamic range.
- the present disclosure provides an imaging device capable of expanding a dynamic range.
- An imaging device includes: a photoelectric conversion element provided in each of a plurality of pixels; a first charge accumulation section connected to the photoelectric conversion element; a second charge accumulation section connected in parallel with the first charge accumulation section; a reset transistor that resets a potential of the second charge accumulation section; a transfer transistor disposed between the first charge accumulation section and the second charge accumulation section; a third charge accumulation section connected in parallel with the first charge accumulation section; and a first changeover switch that is disposed between the first charge accumulation section and the third charge accumulation section and switches whether or not to connect the third charge accumulation section to the photoelectric conversion element.
- the first changeover switch may switch whether or not to connect the third charge accumulation section to the photoelectric conversion element according to a light amount of light incident on the photoelectric conversion element.
- the first changeover switch may be in an off state in a case where the light amount is smaller than a reference value, and in an on state in a case where the light amount is larger than the reference value.
- a fourth charge accumulation section disposed between the reset transistor and the second changeover switch or connected in parallel with the second charge accumulation section via the second changeover switch may be further provided.
- the second changeover switch may switch whether or not to connect the fourth charge accumulation section to the second charge accumulation section in synchronization with the first changeover switch.
- an addition value of capacitances of the first charge accumulation section and the third charge accumulation section may be equal to an addition value of capacitances of the second charge accumulation section and the fourth charge accumulation section.
- the reset transistor, the transfer transistor, the first changeover switch, and the second changeover switch may be P-channel MOS transistors.
- a selection transistor that switches whether or not to output a pixel signal corresponding to an amount of charge accumulated in the second charge accumulation section may be further provided, and
- the selection transistor may be turned on again after the first changeover switch is maintained in an off state and the first changeover switch may be subsequently switched from the off state to an on state.
- a capacitance of the third charge accumulation section may be larger than a capacitance of the first charge accumulation section.
- a pixel in which the third charge accumulation section is connected to the photoelectric conversion element by the first changeover switch and a pixel in which the third charge accumulation section is not connected to the photoelectric conversion element by the first changeover switch may be mixed.
- an overflow gate transistor that discharges charge accumulated in the third charge accumulation section may be further provided.
- the reset transistor, the transfer transistor, and the first changeover switch may be P-channel MOS transistors.
- a third changeover switch that switches whether or not to connect the second charge accumulation sections respectively provided in adjacent pixels adjacent to each other among the plurality of pixels may be further provided.
- the third changeover switch may be always in an on state in a case where the adjacent pixels are added, and the third changeover switch may be always in an off state in a case where the adjacent pixels are not added.
- FIG. 1 is a block diagram illustrating an example of a configuration of an imaging device according to a first embodiment.
- FIG. 2 is a circuit diagram illustrating a configuration of a pixel according to the first embodiment.
- FIG. 3 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the first embodiment.
- FIG. 4 is a timing chart illustrating another example of the readout operation of the pixel circuit according to the first embodiment.
- FIG. 5 is a circuit diagram illustrating a configuration of a pixel according to a first modification.
- FIG. 6 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the first modification.
- FIG. 7 is a circuit diagram illustrating a configuration of a pixel according to a second modification.
- FIG. 8 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the second modification.
- FIG. 9 is a circuit diagram illustrating a configuration of a pixel according to a third modification.
- FIG. 10 A is a circuit diagram illustrating a configuration of a pixel according to a fourth modification.
- FIG. 10 B is a circuit diagram illustrating the configuration of the pixel according to the fourth modification.
- FIG. 10 C is a circuit diagram illustrating the configuration of the pixel according to the fourth modification.
- FIG. 11 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the fourth modification.
- FIG. 12 A is a circuit diagram illustrating a configuration of a pixel according to a second embodiment.
- FIG. 12 B is a circuit diagram illustrating the configuration of the pixel according to the second embodiment.
- FIG. 13 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the second embodiment.
- FIG. 14 A is a circuit diagram illustrating a configuration of a pixel according to a fifth modification.
- FIG. 14 B is a circuit diagram illustrating the configuration of the pixel according to the fifth modification.
- FIG. 15 A is a circuit diagram illustrating a configuration of a pixel according to a sixth modification.
- FIG. 15 B is a circuit diagram illustrating the configuration of the pixel according to the sixth modification.
- FIG. 16 A is a circuit diagram illustrating a configuration of a pixel according to a seventh modification.
- FIG. 16 B is a circuit diagram illustrating the configuration of the pixel according to the seventh modification.
- FIG. 17 A is a circuit diagram illustrating a configuration of the pixel according to the sixth modification.
- FIG. 17 B is a circuit diagram illustrating the configuration of the pixel according to the sixth modification.
- FIG. 18 is a functional block diagram illustrating an example of an electronic device.
- FIG. 19 is a block diagram illustrating an example of schematic configuration of an in-vivo information acquisition system.
- FIG. 20 is a diagram illustrating an example of a schematic configuration of an endoscopic surgical system.
- FIG. 21 is a block diagram depicting an example of schematic configuration of a vehicle control system.
- FIG. 22 is a block diagram illustrating an example of a schematic configuration of a vehicle control system.
- FIG. 23 is a diagram of assistance in explaining an example of installation positions of an outside-vehicle information detecting section and an imaging section.
- FIG. 1 is a block diagram illustrating an example of a configuration of an imaging device according to a first embodiment.
- An imaging device 1 illustrated in FIG. 1 is, for example, an infrared image sensor, and has sensitivity also to light having a wavelength of 800 nm or more, for example.
- a rectangular pixel region 10 P and an extra-pixel region 10 B outside the pixel region 10 P are provided in the imaging device 1 .
- a peripheral circuit for driving the pixel region 10 P is provided in the extra-pixel region 10 B.
- the peripheral circuit provided in the extra-pixel region 10 B includes, for example, a row scanning section 201 , a horizontal selection section 203 , a column scanning section 204 , and a system control section 202 .
- the row scanning section 201 includes a shift register, an address decoder, and the like, and is a pixel drive section that drives each pixel P, for example, in units of rows.
- the pixel signal output from each pixel P of the pixel row selectively scanned by the row scanning section 201 is supplied to the horizontal selection section 203 through each of vertical signal lines Lsig.
- the horizontal selection section 203 includes an amplifier, a horizontal selection switch, and the like provided for each vertical signal line Lsig.
- the column scanning section 204 includes a shift register, an address decoder, and the like, and scans and concurrently drives each horizontal selection switch of the horizontal selection section 203 in sequence.
- the pixel signal of each pixel transmitted through each of the vertical signal lines Lsig is sequentially output to the horizontal signal line. Thereafter, the pixel signal is input to a signal processing section (not illustrated) or the like through the horizontal signal line.
- the system control section 202 receives an externally supplied clock, data indicating an operation mode and the like, and further outputs data such as internal information of the imaging device 1 .
- the system control section 202 further includes a timing generator that generates various timing signals, and performs drive control of the row scanning section 201 , the horizontal selection section 203 , the column scanning section 204 , and the like on the basis of the various timing signals generated by the timing generator.
- FIG. 2 is a circuit diagram illustrating a configuration of the pixel P.
- the pixel P includes a photoelectric conversion element 10 and a pixel circuit 20 .
- the photoelectric conversion element 10 is disposed on one semiconductor substrate, and the pixel circuit 20 is disposed on another semiconductor substrate stacked under the one semiconductor substrate.
- the photoelectric conversion element 10 is, for example, a photodiode that photoelectrically converts incident light having a wavelength in an infrared region.
- a photoelectric conversion film (not illustrated) is provided between an anode electrode and a cathode electrode.
- the photoelectric conversion film includes, for example, a compound semiconductor material such as a group III-V semiconductor, and photoelectrically converts light having a wavelength in a visible region to a short infrared region.
- the compound semiconductor material constituting the photoelectric conversion element 10 for example, InGaAs (indium gallium arsenide), InAsSb (indium arsenide antimony), GaAsSb (gallium arsenide antimony), InAs (indium arsenide), InSb (indium antimony), HgCdTe (mercury cadmium telluride), and the like can be used.
- the photoelectric conversion film may include Ge (germanium).
- the photoelectric conversion film may include a semiconductor material having a Type II structure.
- the pixel circuit 20 is an example of a readout integrated circuit that reads out the charge of the photoelectric conversion element 10 .
- the pixel circuit 20 includes a first charge accumulation section (SN 1 ) 21 , a transfer transistor (TRG) 22 , a second charge accumulation section (FD 1 ) 23 , a reset transistor (RST) 24 , an amplification transistor (AMP) 25 , a selection transistor (SEL) 26 , a changeover switch 27 , and a third charge accumulation section 28 .
- the first charge accumulation section 21 includes a capacitor that temporarily holds the charge photoelectrically converted by the photoelectric conversion element 10 .
- One end of the first charge accumulation section 21 is connected to the anode electrode of the photoelectric conversion element 10 and the source of the transfer transistor 22 .
- the other end of the first charge accumulation section 21 is grounded.
- the drain of the transfer transistor 22 is connected to the second charge accumulation section 23 . That is, the transfer transistor 22 is disposed between the first charge accumulation section 21 and the second charge accumulation section 23 .
- the transfer transistor 22 is turned on and off on the basis of a transfer signal input from the row scanning section 201 to the gate. When the transfer transistor 22 is turned on, the charge accumulated in the first charge accumulation section 21 is transferred to the second charge accumulation section 23 via the transfer transistor 22 .
- the second charge accumulation section 23 is a floating diffusion that accumulates the charge transferred from the first charge accumulation section 21 .
- One end of the second charge accumulation section 23 is connected to the source of the reset transistor 24 and the gate of the amplification transistor 25 together with the drain of the transfer transistor. The other end of the second charge accumulation section 23 is grounded.
- the drain of the reset transistor 24 is connected to a power source VDD.
- the reset transistor 24 is turned on and off on the basis of a reset signal input from the row scanning section 201 to the gate.
- a reset potential VRST is applied to the second charge accumulation section 23 .
- the reset potential VRST brings the potential of the second charge accumulation section 23 into an initial state (reset state).
- the gate is connected to the second charge accumulation section 23 , the drain is connected to the power source VDD, and the source is connected to the drain of the selection transistor 26 .
- the amplification transistor 25 configures a source follower circuit with a load metal oxide semiconductor (MOS) as a constant current source connected via a vertical signal line Lsig.
- MOS load metal oxide semiconductor
- the source of the selection transistor 26 is connected to the vertical signal line Lsig.
- the selection transistor 26 is turned on and off on the basis of a selection signal input from the row scanning section 201 to the gate.
- the selection transistor 26 is turned on, the pixel signal generated by the amplification transistor 25 is output to the horizontal selection section 203 via the vertical signal line Lsig.
- the changeover switch 27 corresponds to a first changeover switch and includes an N-channel MOS transistor.
- a drain is connected to one end of the first charge accumulation section 21
- a source is connected to one end of the third charge accumulation section 28 . That is, the changeover switch 27 is disposed between the first charge accumulation section 21 and the third charge accumulation section 28 .
- the changeover switch 27 is turned on and off on the basis of a control signal input from the row scanning section 201 to the gate. When the changeover switch 27 is turned on, the charge photoelectrically converted by the photoelectric conversion element 10 is accumulated in the third charge accumulation section 28 via the changeover switch 27 .
- the third charge accumulation section 28 includes a capacitor that temporarily holds the charge photoelectrically converted by the photoelectric conversion element 10 .
- the other end of the third charge accumulation section 28 is grounded.
- the conversion efficiency changes depending on whether or not the changeover switch 27 connects the third charge accumulation section 28 to the photoelectric conversion element 10 .
- the changeover switch 27 When the changeover switch 27 is turned on, the conversion efficiency decreases. Conversely, when the changeover switch 27 is turned off, the conversion efficiency increases.
- Formula (1) shows conversion efficiency (Low) when the changeover switch 27 is turned on and conversion efficiency (Hi) when the changeover switch 27 is turned off.
- q represents the amount of charge of the photoelectric conversion element 10 .
- C sn1 , C sn2 , and C fd1 are capacitances of the first charge accumulation section 21 , the third charge accumulation section 28 , and the second charge accumulation section 23 , respectively.
- the conversion efficiency (Low) depends on the capacitance of each of the first charge accumulation section 21 , the second charge accumulation section 23 , and the third charge accumulation section 28 .
- the capacitance C sn2 of the third charge accumulation section 28 increases, the saturation signal amount also increases. Therefore, the capacitance C sn2 of the third charge accumulation section 28 is desirably larger than the capacitance C sn1 of the first charge accumulation section 21 .
- readout noise depends on the balance between the addition value of the capacitance C sn1 and the capacitance C sn2 and the capacitance C fd1 .
- the addition value described above be equal to the capacitance C fd1 .
- the readout noise is, for example, input-converted random noise (RN) in double delta sampling (DDS) driving, in other words, kT/C noise.
- RN input-converted random noise
- DDS double delta sampling
- the capacitance C fd1 increases, the amount of charge that can be accumulated in the second charge accumulation section 23 increases.
- the saturation signal amount can be increased by turning on the changeover switch 27 .
- FIG. 3 is a timing chart illustrating an example of the readout operation of the pixel circuit 20 .
- the conversion efficiency (Low) is set. Therefore, the changeover switch 27 (SNG) is always in an on state regardless of a data phase (D phase) and a reset phase (P phase).
- the reset transistor 24 (RST), the transfer transistor 22 (TRG), and the selection transistor 26 (SEL) are in an off state. Therefore, the charge photoelectrically converted by the photoelectric conversion element 10 is accumulated in the first charge accumulation section 21 and the third charge accumulation section 28 .
- the transfer transistor 22 is switched from the off state to the on state.
- the charges accumulated in the first charge accumulation section 21 and the third charge accumulation section 28 are transferred to the second charge accumulation section 23 and amplified as a pixel signal by the amplification transistor 25 .
- the selection transistor 26 is turned on. As a result, the pixel signal is output to the vertical signal line Lsig.
- the selection transistor 26 is turned off within the time from time t 3 to time t 4 . Subsequently, the reset transistor 24 , the transfer transistor 22 , and the selection transistor 26 are simultaneously switched from the off state to the on state. As a result, the charge remaining in the second charge accumulation section 23 is discharged via the reset transistor 24 , so that the second charge accumulation section 23 is reset to the initial state. Thereafter, the above operation is repeated again.
- the changeover switch 27 (SNG) is always in the off state regardless of the data phase (D phase) and the reset phase (P phase). Furthermore, by adjusting the timing of the pulse of the signal input to the gate of each transistor, the pixel region 10 P can be driven by a global shutter method in which all the pixels P accumulate (expose) charges at the same timing, or can be driven by a rolling shutter method in which each pixel P accumulates charges at different timings.
- FIG. 4 is a timing chart illustrating another example of the readout operation of the pixel circuit 20 .
- the charge set to the conversion efficiency (Hi) and the charge set to the conversion efficiency (Low) are read out at different times.
- the charge set to the conversion efficiency (Hi) and the charge set to the conversion efficiency (Low) can be simultaneously read out by one exposure. Also, in this readout operation, the charge having overflowed from the first charge accumulation section 21 via the changeover switch 27 is accumulated in the third charge accumulation section 28 , whereby the dynamic range can be expanded by one exposure.
- the pixel circuit 20 is provided with the changeover switch 27 and the third charge accumulation section 28 . Therefore, by switching on and off the changeover switch 27 according to the illuminance condition, the capacitance of the entire charge accumulation section in the pixel circuit 20 changes. With this change, the conversion efficiency also changes, and thus the dynamic range can be expanded.
- the illuminance condition may be different for each pixel P in some cases. Therefore, in the pixel region 10 P, a pixel P in which the changeover switch 27 is turned on (set to the conversion efficiency (Low)) and a pixel P in which the changeover switch 27 is turned off (set to the conversion efficiency (Hi)) may be mixed. In this case, the conversion rate can be optimized for each pixel P.
- FIG. 5 is a circuit diagram illustrating a configuration of a pixel according to a first modification. Furthermore, FIG. 6 is a timing chart illustrating an example of the readout operation of the pixel circuit according to the first modification.
- a pixel circuit 20 a further includes an overflow gate (OFG) transistor 29 in addition to the components of the pixel circuit 20 described above.
- the overflow gate transistor 29 includes an N-channel MOS transistor.
- the drain is connected to the source of the changeover switch 27 and one end of the third charge accumulation section 28 , and the source is connected to the power source VDD.
- the overflow gate transistor 29 is turned on and off on the basis of a discharge signal input from the row scanning section 201 to the gate.
- the overflow gate transistor 29 is switched from the off state to the on state until the selection transistor 26 is turned off and the selection transistor 26 is turned on within the time from time t 2 to time t 3 .
- the charge remaining in the third charge accumulation section 28 is discharged to the power source VDD via the overflow gate transistor 29 .
- the pixel circuit 20 a since the pixel circuit 20 a includes the overflow gate transistor 29 , the charge remaining in the third charge accumulation section 28 can be discharged. As a result, blooming to other pixels P can be suppressed.
- FIG. 7 is a circuit diagram illustrating a configuration of a pixel according to a second modification. Furthermore, FIG. 8 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the second modification.
- all of the transfer transistor 22 , the reset transistor 24 , and the changeover switch 27 include N-channel MOS transistors.
- all of the transfer transistor 22 , the reset transistor 24 , and the changeover switch 27 include P-channel MOS transistors.
- the conductivity types of the transfer transistor 22 , the reset transistor 24 , and the changeover switch 27 are opposite to those of the first embodiment. Therefore, in the present modification, as illustrated in FIG. 8 , the signal level input to the gate of each transistor is opposite to that of the first embodiment.
- all of the transfer transistor 22 , the reset transistor 24 , and the changeover switch 27 include P-channel MOS transistors. Therefore, readout of signal charges from the photoelectric conversion element 10 is readout of holes that are positive charges. In this case, as compared with a case where these transistors include N-channel MOS transistors, charges are less likely to overflow from the first charge accumulation section 21 , the second charge accumulation section 23 , and the third charge accumulation section 28 . Therefore, also in the present modification, blooming between the pixels P adjacent to each other can be suppressed.
- FIG. 9 is a circuit diagram illustrating a configuration of a pixel according to a third modification.
- the pixel circuit 20 c illustrated in FIG. 9 is further provided with the overflow gate transistor 29 described in the first modification in addition to the components of the pixel circuit 20 b of the second modification described above.
- the overflow gate transistor 29 includes the P-channel MOS transistor.
- the pixel circuit 20 c according to the present modification has a configuration in which the first modification and the second modification are combined. Therefore, the charge remaining in the third charge accumulation section 28 can be discharged by the overflow gate transistor 29 , and furthermore, the charge is less likely to overflow from the first charge accumulation section 21 , the second charge accumulation section 23 , and the third charge accumulation section 28 by configuring the pixel transistor with the P-channel MOS transistor.
- the blooming suppression effect can be enhanced.
- FIGS. 10 A, 10 B, and 10 C are circuit diagrams illustrating a configuration of a pixel according to a fourth modification. Furthermore, FIG. 11 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the fourth modification.
- the changeover switch 30 is disposed between a pixel P 11 and a pixel P 12 .
- the pixel P 11 and the pixel P 12 are commonly connected to the vertical signal line Lsig and are adjacent to each other.
- the changeover switch 30 corresponds to a third changeover switch and includes an N-channel MOS transistor.
- the drain of the changeover switch 30 is connected to the second charge accumulation section 23 of the pixel P 11 , and the source is connected to the second charge accumulation section 23 of the pixel P 12 .
- the changeover switch 30 is turned on and off on the basis of a signal input from the row scanning section 201 to the gate.
- the changeover switch 30 (FDX) is in the always-on state within the time from time t 1 to time t 4 (within the periods of the D phase and the P phase), the second charge accumulation section 23 is shared between the pixel P 11 and the pixel P 12 , and accordingly, the charges accumulated in the second charge accumulation section 23 are added. As a result, the sensitivity and the saturation signal amount are doubled.
- the changeover switch 30 is always in the off state within the above time, since the second charge accumulation section 23 is not shared, the charges accumulated in each second charge accumulation section 23 are not added.
- the changeover switch 30 is disposed between the first charge accumulation section 21 of the pixel P 11 and the first charge accumulation section 21 of the pixel P 12 .
- the changeover switch 30 When the changeover switch 30 is in the always-on state, the first charge accumulation section 21 of the pixel P 11 is connected to the first charge accumulation section 21 of the pixel P 12 via the changeover switch 30 .
- the first charge accumulation section 21 is shared between the pixel P 11 and the pixel P 12 , whereby the charges accumulated in the first charge accumulation section 21 are added. As a result, the signal saturation amount is doubled.
- the changeover switch 30 is always in the off state within the above time, since the first charge accumulation section 21 is not shared, the charges accumulated in each first charge accumulation section 21 are not added.
- the changeover switch 30 is disposed between the third charge accumulation section 28 of the pixel P 11 and the third charge accumulation section 28 of the pixel P 12 .
- the changeover switch 30 is in the always-on state, the third charge accumulation section 28 of the pixel P 11 is connected to the third charge accumulation section 28 of the pixel P 12 via the changeover switch 30 .
- the third charge accumulation section 28 is shared between the pixel P 11 and the pixel P 12 , whereby the charges accumulated in the third charge accumulation section 28 are added.
- the signal saturation amount is doubled.
- the changeover switch 30 is always in the off state within the above time, since the third charge accumulation section 28 is not shared, the charges accumulated in each third charge accumulation section 28 are not added.
- the changeover switch 30 can switch whether or not to connect the first charge accumulation section 21 , the second charge accumulation section 23 , or the third charge accumulation section 28 disposed in the pixels adjacent to each other. As a result, the sensitivity and the saturation signal amount can be adjusted. Note that, in the present modification, the changeover switch 30 is disposed between pixels adjacent in a direction parallel to the vertical signal line Lsig (vertical direction), but may be disposed between pixels adjacent in a direction orthogonal to the vertical signal line Lsig (horizontal direction).
- FIGS. 12 A and 12 B are circuit diagrams illustrating a configuration of a pixel according to a second embodiment. Components similar to those of the first embodiment described above are denoted by the same reference numerals, and detailed description thereof will be omitted.
- a pixel circuit 20 d illustrated in FIG. 12 A and a pixel circuit 20 e illustrated in FIG. 12 B further include a changeover switch 31 and a fourth charge accumulation section 32 in addition to the components of the pixel circuit 20 described in the first embodiment.
- the changeover switch 31 corresponds to a second changeover switch and includes an N-channel MOS transistor.
- the fourth charge accumulation section 32 includes a capacitor.
- the drain is connected to the source of the reset transistor 24 , and the source is connected to the second charge accumulation section 23 .
- the changeover switch 31 is turned on and off on the basis of a signal input from the row scanning section 201 to the gate.
- One end of the fourth charge accumulation section 32 is connected to the source of the reset transistor 24 and the drain of the changeover switch 31 , and the other end is grounded.
- the drain of the changeover switch 31 is connected to the sources of the second charge accumulation section 23 and the reset transistor 24 , and the gate of the amplification transistor 25 .
- the source is connected to one end of the fourth charge accumulation section 32 .
- the other end of the fourth charge accumulation section 32 is grounded. That is, the fourth charge accumulation section 32 is connected to the second charge accumulation section 23 via the changeover switch 31 .
- the conversion efficiency is changed by the changeover switch 27 switching whether or not to connect the third charge accumulation section 28 to the photoelectric conversion element 10 and the changeover switch 31 switching whether or not to connect the fourth charge accumulation section 32 to the second charge accumulation section 23 .
- the changeover switch 27 When the changeover switch 27 is turned on, the conversion efficiency decreases. Conversely, when the changeover switch 27 is turned off, the conversion efficiency increases.
- the following Formula (2) shows conversion efficiency (Low) when the changeover switch 27 and the changeover switch 31 are turned on and conversion efficiency (Hi) when the changeover switch 27 and the changeover switch 31 are turned off.
- C fd2 is the capacitance of the fourth charge accumulation section 32 .
- the conversion efficiency (Low) depends on the capacitance of each of the first charge accumulation section 21 , the second charge accumulation section 23 , the third charge accumulation section 28 , and the fourth charge accumulation section 32 .
- the capacitance C fd2 of the fourth charge accumulation section 32 increases, the saturation signal amount also increases. Therefore, the capacitance C fd2 of the fourth charge accumulation section 32 is desirably larger than the capacitance C fd1 of the second charge accumulation section 23 .
- the readout noise depends on a balance between a first addition value of the capacitance C sn1 and the capacitance C sn2 and a second addition value of the capacitance C fd1 and the capacitance C fd2 .
- the first addition value is desirably equal to the second addition value.
- the saturation signal amount can be increased by turning on the changeover switch 27 and the changeover switch 31 .
- FIG. 13 is a timing chart illustrating an example of the readout operation of the pixel circuits 20 d and 20 e according to the second embodiment.
- the conversion efficiency (Low) is set.
- the changeover switch 27 (SNG) not only the changeover switch 27 (SNG) but also the changeover switch 31 (FDG) is always in an on state regardless of the D phase and the P phase.
- the changeover switch 27 and the changeover switch 31 are always in an off state regardless of the D phase and the P phase. In this manner, the changeover switch 31 operates in synchronization with the changeover switch 27 .
- the reset transistor 24 , the transfer transistor 22 , and the selection transistor 26 may be driven in the timing chart illustrated in FIG. 11 .
- the charge set to the conversion efficiency (Hi) and the charge set to the conversion efficiency (Low) can be simultaneously read out by one exposure.
- the charge having overflowed from the second charge accumulation section 23 via the changeover switch 31 is accumulated in the fourth charge accumulation section 32 , whereby the dynamic range can be expanded by one exposure.
- the conversion efficiency can be switched by controlling on and off of the changeover switch 31 in addition to the changeover switch 27 . Furthermore, by adjusting the balance between an SN capacitance of the first charge accumulation section 21 and the third charge accumulation section 28 and an FD capacitance of the second charge accumulation section 23 and the fourth charge accumulation section 32 , imaging with a high SN ratio can be performed under wider illuminance conditions.
- FIGS. 14 A and 14 B are circuit diagrams illustrating a configuration of a pixel according to a fifth modification.
- a pixel circuit 20 f illustrated in FIG. 14 A and a pixel circuit 20 g illustrated in FIG. 14 B further include the overflow gate transistor 29 similarly to the first modification described above.
- the overflow gate transistor 29 is driven in the timing chart illustrated in FIG. 6 , similarly to the first modification. That is, in the time from time t 2 to time t 3 , the overflow gate transistor 29 is switched from the off state to the on state until the selection transistor 26 is turned off and the selection transistor 26 is turned on. As a result, the charge remaining in the third charge accumulation section 28 is discharged to the power source VDD via the overflow gate transistor 29 .
- the changeover switch 31 is always in an off state together with the changeover switch 27 in a case where the conversion efficiency (Hi) is set, and is always in an on state together with the changeover switch 27 in a case where the conversion efficiency (Low) is set.
- the pixel circuits 20 f and 20 g have the overflow gate transistor 29 , the charge remaining in the third charge accumulation section 28 can be discharged. As a result, blooming to other pixels P can be suppressed.
- FIGS. 15 A and 15 B are circuit diagrams illustrating a configuration of a pixel according to a sixth modification. Comparing a pixel circuit 20 h illustrated in FIG. 15 A with the pixel circuit 20 d illustrated in FIG. 12 A , all of the transfer transistor 22 , the reset transistor 24 , the changeover switch 27 , and the changeover switch 31 include P-channel MOS transistors instead of N-channel MOS transistors.
- all of the transfer transistor 22 , the reset transistor 24 , the changeover switch 27 , and the changeover switch 31 include P-channel MOS transistors instead of N-channel MOS transistors.
- the conductivity types of the transfer transistor 22 , the reset transistor 24 , the changeover switch 27 , and the changeover switch 31 are opposite to those of the second embodiment. Therefore, in the present modification, the signal level input to the gate of each transistor is opposite to that of the second embodiment.
- all of the transfer transistor 22 , the reset transistor 24 , the changeover switch 27 , and the changeover switch 31 include P-channel MOS transistors. Therefore, readout of signal charges from the photoelectric conversion element 10 is readout of holes that are positive charges. Therefore, the charges are less likely to overflow from the first charge accumulation section 21 , the second charge accumulation section 23 , the third charge accumulation section 28 , and the fourth charge accumulation section 32 . Therefore, also in the present modification, blooming between adjacent pixels can be suppressed.
- FIGS. 16 A and 16 B are circuit diagrams illustrating a configuration of a pixel according to a seventh modification. Comparing a pixel circuit 20 j illustrated in FIG. 16 A with the pixel circuit 20 f illustrated in FIG. 14 A , all of the transfer transistor 22 , the reset transistor 24 , the changeover switch 27 , the changeover switch 31 , and the overflow gate transistor 29 include P-channel MOS transistors instead of N-channel MOS transistors.
- all of the transfer transistor 22 , the reset transistor 24 , the changeover switch 27 , the changeover switch 31 , and the overflow gate transistor 29 include P-channel MOS transistors instead of N-channel MOS transistors.
- the pixel circuit 20 k according to the present modification has a configuration in which the fifth modification and the sixth modification are combined. Therefore, the charge remaining in the third charge accumulation section 28 can be discharged by the overflow gate transistor 29 , and furthermore, the charge is less likely to overflow from the first charge accumulation section 21 , the second charge accumulation section 23 , and the third charge accumulation section 28 by configuring the pixel transistor with the P-channel MOS transistor.
- the blooming suppression effect can be enhanced.
- FIGS. 17 A and 17 B are circuit diagrams illustrating a configuration of a pixel according to an eighth modification.
- the changeover switch 30 is disposed between the pixel P 21 and the pixel P 22 .
- the pixel P 21 and the pixel P 22 are commonly connected to the vertical signal line Lsig and are adjacent to each other.
- each of the pixel P 21 and the pixel P 22 includes the pixel circuit 20 d illustrated in FIG. 12 A .
- each of the pixel P 21 and the pixel P 22 includes the pixel circuit 20 e illustrated in FIG. 12 B .
- the drain of the changeover switch 30 is connected between the second charge accumulation section 23 of the pixel P 21 and the changeover switch 31
- the source is connected between the second charge accumulation section 23 of the pixel P 22 and the changeover switch 31 .
- the changeover switch 31 in a case where the charges of the pixel P 21 and the pixel P 22 are added, the changeover switch 31 is always turned on within the periods of the D phase and the P phase. Conversely, in a case where no charge is added, the changeover switch 31 is always in an off state within the D-phase and P-phase periods.
- connection form of the changeover switch 30 is not limited to the arrangement illustrated in FIGS. 17 A and 17 B .
- the changeover switch 30 may switch the connection between the second charge accumulation sections 21 similarly to FIG. 10 B , or may switch the connection between the third charge accumulation sections 28 similarly to in FIG. 10 C .
- the changeover switch 30 can switch whether or not to connect the first charge accumulation section 21 , the second charge accumulation section 23 , or the third charge accumulation section 28 disposed in the pixels adjacent to each other. As a result, the sensitivity and the saturation signal amount can be adjusted. Note that, in the present modification, the changeover switch 30 is disposed between pixels adjacent in a direction parallel to the vertical signal line Lsig, but may be disposed between pixels adjacent in a direction orthogonal to the vertical signal line Lsig.
- FIG. 18 illustrates a schematic configuration of an electronic device 3 (camera) as an example.
- the electronic device 3 is, for example, a camera capable of capturing a still image or a moving image, and includes the imaging device 1 , an optical system 310 , a shutter device 311 , a drive section 313 that drives the imaging device 1 and the shutter device 311 , and a signal processing section 312 .
- the optical system 310 guides image light (incident light) from a subject to the imaging device 1 .
- the optical system 310 may include a plurality of optical lenses.
- the shutter device 311 controls a light irradiation period and a light shielding period for the imaging device 1 .
- the drive section 313 controls a transfer operation of the imaging device 1 and a shutter operation of the shutter device 311 .
- the signal processing section 312 performs various types of signal processing on a signal output from the imaging device 1 .
- a video signal Dout after the signal processing is stored in a storage medium such as a memory or output to a monitor and the like.
- the technology according to the present disclosure can be applied to a wide variety of products.
- the technology according to the present disclosure may be applied to an endoscopic surgical system.
- FIG. 19 is a block diagram illustrating an example of a schematic configuration of a patient in-vivo information acquisition system using a capsule endoscope to which the technology according to the present disclosure (present technology) can be applied.
- An in-vivo information acquisition system 10001 includes a capsule endoscope 10100 and an external control device 10200 .
- the capsule endoscope 10100 is swallowed by a patient at the time of examination.
- the capsule endoscope 10100 has an imaging function and a wireless communication function and, while moving inside an organ such as a stomach and an intestine by peristaltic movement or the like until it is naturally excreted from the patient, sequentially captures images inside the organ (hereinafter also referred to as in-vivo images) at predetermined intervals, and sequentially transmits information regarding the in-vivo images wirelessly to the external control device 10200 outside the body.
- the external control device 10200 centrally controls the operation of the in-vivo information acquisition system 10001 . Furthermore, the external control device 10200 receives information regarding the in-vivo images transmitted from the capsule endoscope 10100 , and generates image data for displaying the in-vivo images on a display device (not illustrated) on the basis of the received information regarding the in-vivo images.
- in-vivo images indicating the patient's internal conditions can be obtained continually from the time the capsule endoscope 10100 is swallowed to the time the capsule endoscope 10100 is excreted.
- the capsule endoscope 10100 includes a capsule-shaped housing 10101 , and includes a light source section 10111 , an imaging section 10112 , an image processing section 10113 , a wireless communication section 10114 , a power supply section 10115 , a power source section 10116 , and a control section 10117 which are housed in the capsule-shaped housing 10101 .
- the light source section 10111 includes a light source such as a light emitting diode (LED) or the like, for example, and irradiates an imaging field of view of the imaging section 10112 with light.
- a light source such as a light emitting diode (LED) or the like, for example, and irradiates an imaging field of view of the imaging section 10112 with light.
- LED light emitting diode
- the imaging section 10112 includes an optical system including an imaging element and a plurality of lenses provided on a preceding stage of the imaging element. Reflected light (hereinafter referred to as observation light) of the light applied to body tissue to be observed is condensed by the optical system and is incident on the imaging element. In the imaging section 10112 , in the imaging element, the observation light incident thereon is photoelectrically converted, and an image signal corresponding to the observation light is generated. The image signal generated by the imaging section 10112 is provided to the image processing section 10113 .
- the image processing section 10113 includes a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or the like, and performs various kinds of signal processing on the image signal generated by the imaging section 10112 .
- the image processing section 10113 provides the image signal that has been subjected to signal processing to the wireless communication section 10114 as RAW data.
- the wireless communication section 10114 performs predetermined processing such as modulation processing or the like on the image signal that has been subjected to the signal processing by the image processing section 10113 , and transmits the image signal to the external control device 10200 via an antenna 10114 A. Furthermore, the wireless communication section 10114 receives a control signal regarding drive control of the capsule endoscope 10100 from the external control device 10200 via the antenna 10114 A. The wireless communication section 10114 provides the control signal received from the external control device 10200 to the control section 10117 .
- the power supply section 10115 includes an antenna coil for power reception, a power regeneration circuit for regenerating power from current generated in the antenna coil, a booster circuit and the like. In the power supply section 10115 , the principle of what is called contactless charging is used to generate power.
- the power source section 10116 includes a secondary battery, and accumulates the power generated by the power supply section 10115 . Although arrows or the like indicating the destination to which power from the power source section 10116 is supplied are not illustrated in FIG. 19 for preventing the illustration from being complex, power accumulated in the power source section 10116 is supplied to the light source section 10111 , the imaging section 10112 , the image processing section 10113 , the wireless communication section 10114 , and the control section 10117 , and may be used to drive these sections.
- the control section 10117 includes a processor such as a CPU, and appropriately controls drives of the light source section 10111 , the imaging section 10112 , the image processing section 10113 , the wireless communication section 10114 , and the power supply section 10115 in accordance with a control signal transmitted from the external control device 10200 .
- a processor such as a CPU
- the external control device 10200 includes a processor such as a CPU, a GPU, or the like, or a microcomputer or a control board or the like on which a processor and a storage element such as a memory are mounted in a mixed manner.
- the external control device 10200 controls the operation of the capsule endoscope 10100 by transmitting the control signal to the control section 10117 of the capsule endoscope 10100 via an antenna 10200 A.
- an irradiation condition of the light to the observation target in the light source section 10111 might be changed by the control signal from the external control device 10200 .
- an imaging condition for example, a frame rate, an exposure value and the like in the imaging section 10112
- an imaging condition might be changed by the control signal from the external control device 10200 .
- the contents of processing in the image processing section 10113 and conditions for transmitting the image signal by the wireless communication section 10114 may be changed by the control signal from the external control device 10200 .
- the external control device 10200 performs various types of image processing on the image signal transmitted from the capsule endoscope 10100 , and generates image data for displaying a captured in-vivo image on a display device.
- image processing for example, various signal processing such as development processing (demosaic processing), image quality enhancement processing (band enhancement processing, super-resolution processing, noise reduction (NR) processing, and/or camera shake correction processing, and the like), and/or enlargement processing (electronic zoom processing) and the like can be performed.
- the external control device 10200 controls driving of the display device to display the in-vivo image captured on the basis of the generated image data.
- the external control device 10200 may also cause a recording device (not illustrated) to record the generated image data, or cause a printing device (not illustrated) to make a printout of the generated image data.
- the technology according to the present disclosure can be applied to, for example, the imaging section 10112 among the above-described configurations. As a result, the dynamic range of the imaging section 10112 is expanded.
- the technology of the present disclosure can be applied to various products.
- the technology according to the present disclosure may be applied to an endoscopic surgical system.
- FIG. 20 is a diagram illustrating an example of a schematic configuration of an endoscopic surgical system to which the technology according to the present disclosure (present technology) can be applied.
- FIG. 20 illustrates a state in which an operator (doctor) 11131 is performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgical system 11000 .
- the endoscopic surgical system 11000 includes an endoscope 11100 , other surgical tools 11110 such as a pneumoperitoneum tube 11111 and an energy device 11112 , a supporting arm apparatus 11120 which supports the endoscope 11100 thereon, and a cart 11200 on which various apparatus for endoscopic surgery are mounted.
- the endoscope 11100 includes a lens barrel 11101 having a region of a predetermined length from a distal end thereof to be inserted into a body cavity of the patient 11132 , and a camera head 11102 connected to a proximal end of the lens barrel 11101 .
- the endoscope 11100 configured as a so-called rigid scope having the rigid lens barrel 11101 is illustrated, but the endoscope 11100 may be configured as a so-called flexible scope having a flexible lens barrel.
- the lens barrel 11101 has, at a distal end thereof, an opening in which an objective lens is fitted.
- a light source device 11203 is connected to the endoscope 11100 such that light generated by the light source device 11203 is introduced to a distal end of the lens barrel 11101 by a light guide extending in the inside of the lens barrel 11101 and is irradiated toward an observation target in a body cavity of the patient 11132 through the objective lens.
- the endoscope 11100 may be a forward-viewing endoscope or may be an oblique-viewing endoscope or a side-viewing endoscope.
- An optical system and an imaging element are provided in the inside of the camera head 11102 such that reflected light (observation light) from the observation target is condensed on the imaging element by the optical system.
- the observation light is photoelectrically converted by the imaging element to generate an electric signal corresponding to the observation light, namely, an image signal corresponding to an observation image.
- the image signal is transmitted as RAW data to a camera control unit (CCU) 11201 .
- CCU camera control unit
- the CCU 11201 includes a central processing unit (CPU), a graphics processing unit (GPU), or the like, and comprehensively controls operation of the endoscope 11100 and a display device 11202 . Moreover, the CCU 11201 receives an image signal from the camera head 11102 , and applies, on the image signal, various types of image processing for displaying an image based on the image signal, for example, development processing (demosaic processing) and the like.
- CPU central processing unit
- GPU graphics processing unit
- the display device 11202 displays thereon an image based on an image signal, for which the image processing has been performed by the CCU 11201 , under the control of the CCU 11201 .
- the light source device 11203 includes a light source such as, for example, a light emitting diode (LED) and supplies irradiation light upon imaging of a surgical site or the like to the endoscope 11100 .
- a light source such as, for example, a light emitting diode (LED) and supplies irradiation light upon imaging of a surgical site or the like to the endoscope 11100 .
- LED light emitting diode
- An input device 11204 is an input interface for the endoscopic surgical system 11000 .
- a user can input various types of information and input instructions to the endoscopic surgical system 11000 via the input device 11204 .
- the user inputs an instruction or the like for changing imaging conditions (a type of irradiation light, a magnification, a focal length, and the like) by the endoscope 11100 .
- a treatment tool control device 11205 controls driving of the energy device 11112 for cautery or incision of a tissue, sealing of a blood vessel or the like.
- a pneumoperitoneum device 11206 feeds gas into a body cavity of the patient 11132 through the pneumoperitoneum tube 11111 to inflate the body cavity in order to secure the field of view of the endoscope 11100 and secure the working space for the operator.
- a recorder 11207 is an apparatus capable of recording various kinds of information relating to surgery.
- a printer 11208 is an apparatus capable of printing various kinds of information relating to surgery in various forms such as a text, an image, a graph, or the like.
- the light source device 11203 that supplies the endoscope 11100 with the irradiation light at the time of imaging the surgical site can include, for example, an LED, a laser light source, or a white light source including a combination thereof.
- a white light source includes a combination of red, green, and blue (RGB) laser light sources, since the output intensity and the output timing can be controlled with a high degree of accuracy for each color (each wavelength), adjustment of the white balance of a captured image can be performed by the light source device 11203 .
- RGB red, green, and blue
- driving of the light source device 11203 may be controlled so as to change the intensity of output light at every predetermined time interval.
- driving of the imaging element of the camera head 11102 in synchronization with the timing of the change of the intensity of light to acquire images time-divisionally and synthesizing the images, an image of a high dynamic range free from underexposed blocked up shadows and overexposed highlights can be created.
- the light source device 11203 may be configured to be able to supply light having a predetermined wavelength band corresponding to special light observation.
- special light observation for example, by utilizing the wavelength dependency of absorption of light in a body tissue to irradiate light of a narrow band in comparison with irradiation light upon ordinary observation (namely, white light), narrow band observation (narrow band imaging) of imaging a predetermined tissue such as a blood vessel of a superficial portion of the mucous membrane or the like in a high contrast is performed.
- fluorescent observation for obtaining an image from fluorescent light generated by irradiation of excitation light may be performed.
- fluorescent observation it is possible, for example, to perform observation of fluorescent light from a body tissue by irradiating excitation light on the body tissue (autofluorescence observation) or to obtain a fluorescent light image by locally injecting a reagent such as indocyanine green (ICG) into a body tissue and irradiating excitation light corresponding to a fluorescent light wavelength of the reagent upon the body tissue.
- a reagent such as indocyanine green (ICG)
- ICG indocyanine green
- the light source device 11203 can be configured to supply such narrow-band light and/or excitation light suitable for special light observation as described above.
- FIG. 21 is a block diagram illustrating an example of a functional configuration of the camera head 11102 and the CCU 11201 illustrated in FIG. 20 .
- the camera head 11102 includes a lens unit 11401 , an imaging section 11402 , a drive section 11403 , a communication section 11404 , and a camera head control section 11405 .
- the CCU 11201 includes a communication section 11411 , an image processing section 11412 , and a control section 11413 .
- the camera head 11102 and the CCU 11201 are communicatively connected to each other by a transmission cable 11400 .
- the lens unit 11401 is an optical system provided at a connection to the lens barrel 11101 .
- the observation light taken in from the distal end of the lens barrel 11101 is guided to the camera head 11102 and is incident on the lens unit 11401 .
- the lens unit 11401 includes a combination of a plurality of lenses including a zoom lens and a focusing lens.
- the number of the imaging elements included in the imaging section 11402 may be one (a so-called single plate type) or plural (a so-called multi-plate type).
- image signals corresponding to R, G, and B may be generated by the respective imaging elements, and a color image may be obtained by combining the generated image signals, for example.
- the imaging section 11402 may also be configured so as to have a pair of imaging elements for acquiring respective image signals for the right eye and the left eye ready for three dimensional (3D) display. By the 3D display, the operator 11131 may grasp a depth of the living tissue in the surgical site more accurately.
- a plurality of systems of lens units 11401 may be provided so as to correspond to the respective imaging elements.
- the imaging section 11402 may not necessarily be provided in the camera head 11102 .
- the imaging section 11402 may be provided immediately behind the objective lens in the inside of the lens barrel 11101 .
- the drive section 11403 includes an actuator and moves the zoom lens and the focusing lens of the lens unit 11401 by a predetermined distance along an optical axis under the control of the camera head control section 11405 .
- the magnification and the focal point of a captured image by the imaging section 11402 can be adjusted suitably.
- the communication section 11404 includes a communication device for transmitting and receiving various kinds of information to and from the CCU 11201 .
- the communication section 11404 transmits an image signal acquired from the imaging section 11402 as RAW data to the CCU 11201 through the transmission cable 11400 .
- the communication section 11404 receives a control signal for controlling driving of the camera head 11102 from the CCU 11201 and supplies the control signal to the camera head control section 11405 .
- the control signal includes, for example, information regarding an imaging condition such as information specifying a frame rate of a captured image, information specifying an exposure value at the time of imaging, and/or information specifying the magnification and focal point of the captured image.
- imaging conditions such as the frame rate, exposure value, magnification, and focus described above may be appropriately specified by the user, or may be automatically set by the control section 11413 of the CCU 11201 on the basis of the acquired image signal.
- an auto exposure (AE) function, an auto focus (AF) function and an auto white balance (AWB) function are incorporated in the endoscope 11100 .
- the camera head control section 11405 controls driving of the camera head 11102 on the basis of a control signal from the CCU 11201 received through the communication section 11404 .
- the communication section 11411 includes a communication device for transmitting and receiving various types of information to and from the camera head 11102 .
- the communication section 11411 receives an image signal transmitted thereto from the camera head 11102 through the transmission cable 11400 .
- the communication section 11411 transmits, to the camera head 11102 , a control signal for controlling driving of the camera head 11102 .
- the image signal and the control signal can be transmitted by electrical communication, optical communication, or the like.
- the image processing section 11412 performs various types of image processing on the image signal being the RAW data transmitted from the camera head 11102 .
- the control section 11413 performs various control related to imaging of the surgical site or the like by the endoscope 11100 and display of a captured image obtained by the imaging of the surgical site or the like. For example, the control section 11413 creates a control signal for controlling driving of the camera head 11102 .
- control section 11413 controls, on the basis of an image signal for which image processes have been performed by the image processing section 11412 , the display device 11202 to display a picked up image in which the surgical site or the like is imaged.
- control section 11413 may recognize various objects in the captured image using various image recognition technologies.
- the control section 11413 can recognize a surgical tool such as forceps, a particular living body region, bleeding, mist when the energy device 11112 is used and the like by detecting the shape, color and the like of edges of objects included in a captured image.
- the control section 11413 may cause, when it controls the display device 11202 to display a picked up image, various kinds of surgery supporting information to be displayed in an overlapping manner with an image of the surgical site using a result of the recognition.
- the surgery support information is superimposed to be displayed, and presented to the operator 11131 , so that it becomes possible to reduce the burden on the operator 11131 and enable the operator 11131 to reliably proceed with surgery.
- the transmission cable 11400 which connects the camera head 11102 and the CCU 11201 to each other is an electric signal cable ready for communication of an electric signal, an optical fiber ready for optical communication or a composite cable ready for both of electrical and optical communications.
- the communication is performed by wire using the transmission cable 11400 , but the communication between the camera head 11102 and the CCU 11201 may be performed wirelessly.
- the technology according to the present disclosure may be applied to the imaging section 11402 among the configurations described above. As a result, the dynamic range of the imaging section 11402 is expanded.
- the endoscopic surgical system has been described as an example, but the technology according to the present disclosure may be applied to, for example, a microscopic surgical system or the like.
- the technology according to the present disclosure may also be achieved as a device mounted on any type of mobile body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a boat, a robot, and the like.
- FIG. 22 is a block diagram depicting an example of schematic configuration of a vehicle control system as an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied.
- the vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001 .
- the vehicle control system 12000 includes a driving system control unit 12010 , a body system control unit 12020 , an outside-vehicle information detecting unit 12030 , an in-vehicle information detecting unit 12040 , and an integrated control unit 12050 .
- a microcomputer 12051 , a sound/image output section 12052 , and a vehicle-mounted network interface (I/F) 12053 are illustrated as a functional configuration of the integrated control unit 12050 .
- the driving system control unit 12010 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs.
- the driving system control unit 12010 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like.
- the body system control unit 12020 controls the operation of various kinds of devices provided to a vehicle body in accordance with various kinds of programs.
- the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like.
- radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the body system control unit 12020 .
- the body system control unit 12020 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle.
- the outside-vehicle information detecting unit 12030 detects information about the outside of the vehicle including the vehicle control system 12000 .
- the outside-vehicle information detecting unit 12030 is connected with an imaging section 12031 .
- the outside-vehicle information detecting unit 12030 makes the imaging section 12031 image an image of the outside of the vehicle, and receives the imaged image.
- the outside-vehicle information detecting unit 12030 may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto.
- the imaging section 12031 is an optical sensor that receives light, and which outputs an electric signal corresponding to a received light amount of the light.
- the imaging section 12031 can output the electric signal as an image, or can output the electric signal as information about a measured distance.
- the light received by the imaging section 12031 may be visible light, or may be invisible light such as infrared rays or the like.
- the in-vehicle information detecting unit 12040 detects information about the inside of the vehicle.
- the in-vehicle information detecting unit 12040 is, for example, connected with a driver state detecting section 12041 that detects the state of a driver.
- the driver state detecting section 12041 for example, includes a camera that images the driver.
- the in-vehicle information detecting unit 12040 may calculate a degree of fatigue of the driver or a degree of concentration of the driver, or may determine whether the driver is dozing.
- the microcomputer 12051 can calculate a control target value for the driving force generating device, the steering mechanism, or the braking device on the basis of the information about the inside or outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040 , and output a control command to the driving system control unit 12010 .
- the microcomputer 12051 can perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like.
- ADAS advanced driver assistance system
- the microcomputer 12051 can perform cooperative control intended for automated driving, which makes the vehicle to travel automatedly without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the information about the outside or inside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 or the in-vehicle information detecting unit 12040 .
- the microcomputer 12051 can output a control command to the body system control unit 12020 on the basis of the information about the outside of the vehicle which information is obtained by the outside-vehicle information detecting unit 12030 .
- the microcomputer 12051 can perform cooperative control intended to prevent a glare by controlling the headlamp so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detecting unit 12030 .
- the sound/image output section 12052 transmits an output signal of at least one of a sound and an image to an output device capable of visually or auditorily notifying information to an occupant of the vehicle or the outside of the vehicle.
- an audio speaker 12061 a display section 12062 , and an instrument panel 12063 are illustrated as the output device.
- the display section 12062 may, for example, include at least one of an on-board display and a head-up display.
- FIG. 23 is a diagram depicting an example of the installation position of the imaging section 12031 .
- the imaging section 12031 includes imaging sections 12101 , 12102 , 12103 , 12104 , and 12105 .
- the imaging sections 12101 , 12102 , 12103 , 12104 , and 12105 are, for example, disposed at positions such as on a front nose, sideview mirrors, a rear bumper, and a back door of the vehicle 12100 as well as a position on an upper portion of a windshield within the interior of the vehicle.
- the imaging section 12101 provided to the front nose and the imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of the vehicle 12100 .
- the imaging sections 12102 and 12103 provided to the sideview mirrors obtain mainly an image of the sides of the vehicle 12100 .
- the imaging section 12104 provided to the rear bumper or the back door obtains mainly an image of the rear of the vehicle 12100 .
- the imaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.
- FIG. 23 depicts an example of photographing ranges of the imaging sections 12101 to 12104 .
- An imaging range 12111 represents the imaging range of the imaging section 12101 provided to the front nose.
- Imaging ranges 1211212113 respectively represent the imaging ranges of the imaging sections 12102 and 12103 provided to the sideview mirrors.
- An imaging range 12114 represents the imaging range of the imaging section 12104 provided to the rear bumper or the back door.
- a bird's-eye image of the vehicle 12100 as viewed from above is obtained by superimposing image data imaged by the imaging sections 12101 to 12104 , for example.
- At least one of the imaging sections 12101 to 12104 may have a function of obtaining distance information.
- at least one of the imaging sections 12101 to 12104 may be a stereo camera constituted of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.
- the microcomputer 12051 can determine a distance to each three-dimensional object within the imaging ranges 12111 to 12114 and a temporal change in the distance (relative speed with respect to the vehicle 12100 ) on the basis of the distance information obtained from the imaging sections 12101 to 12104 , and thereby extract, as a preceding vehicle, a nearest three-dimensional object in particular that is present on a traveling path of the vehicle 12100 and which travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, equal to or more than 0 km/hour). Further, the microcomputer 12051 can set a following distance to be maintained in front of a preceding vehicle in advance, and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), or the like. It is thus possible to perform cooperative control intended for automated driving that makes the vehicle travel automatedly without depending on the operation of the driver or the like.
- automatic brake control including following stop control
- automatic acceleration control including following start control
- the microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a large-sized vehicle, a pedestrian, a utility pole, and other three-dimensional objects on the basis of the distance information obtained from the imaging sections 12101 to 12104 , extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic avoidance of an obstacle.
- the microcomputer 12051 identifies obstacles around the vehicle 12100 as obstacles that the driver of the vehicle 12100 can recognize visually and obstacles that are difficult for the driver of the vehicle 12100 to recognize visually. Then, the microcomputer 12051 determines a collision risk indicating a risk of collision with each obstacle.
- the microcomputer 12051 In a situation in which the collision risk is equal to or higher than a set value and there is thus a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display section 12062 , and performs forced deceleration or avoidance steering via the driving system control unit 12010 .
- the microcomputer 12051 can thereby assist in driving to avoid collision.
- At least one of the imaging sections 12101 to 12104 may be an infrared camera that detects infrared rays.
- the microcomputer 12051 can, for example, recognize a pedestrian by determining whether or not there is a pedestrian in imaged images of the imaging sections 12101 to 12104 .
- recognition of a pedestrian is, for example, performed by a procedure of extracting characteristic points in the imaged images of the imaging sections 12101 to 12104 as infrared cameras and a procedure of determining whether or not it is the pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object.
- the sound/image output section 12052 controls the display section 12062 so that a square contour line for emphasis is displayed so as to be superimposed on the recognized pedestrian.
- the sound/image output section 12052 may also control the display section 12062 so that an icon or the like representing the pedestrian is displayed at a desired position.
- the technology according to the present disclosure can be applied to the imaging section 12031 and the like in the configuration described above, for example.
- the imaging device 1 can be applied to the imaging section 12031 .
- the dynamic range of the imaging section 12031 is expanded.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
The present disclosure provides an imaging device capable of expanding a dynamic range. An imaging device according to one embodiment of the present disclosure includes: a photoelectric conversion element provided in each of a plurality of pixels; a first charge accumulation section connected to the photoelectric conversion element; a second charge accumulation section connected in parallel with the first charge accumulation section; a reset transistor that resets a potential of the second charge accumulation section; a transfer transistor disposed between the first charge accumulation section and the second charge accumulation section; a third charge accumulation section connected in parallel with the first charge accumulation section; and a first changeover switch that is disposed between the first charge accumulation section and the third charge accumulation section and switches whether or not to connect the third charge accumulation section to the photoelectric conversion element.
Description
- The present disclosure relates to an imaging device.
- An imaging device generally includes a photoelectric conversion element and a readout integrated circuit (ROIC) that reads out charges from the photoelectric conversion element. The readout integrated circuit is provided with a charge accumulation section that accumulates the charge read out from the photoelectric conversion element. The conversion efficiency indicating the voltage of the output signal per one electron depends on the capacitance of the charge accumulation section.
- Patent Document 1: WO 2019/155841
- In the imaging device, since the capacitance of the charge accumulation section is fixed, the conversion efficiency is also fixed. In this case, for example, the conversion efficiency cannot be changed according to the charge amount from the photoelectric conversion element, and thus it is difficult to expand the dynamic range.
- The present disclosure provides an imaging device capable of expanding a dynamic range.
- An imaging device according to one embodiment of the present disclosure includes: a photoelectric conversion element provided in each of a plurality of pixels; a first charge accumulation section connected to the photoelectric conversion element; a second charge accumulation section connected in parallel with the first charge accumulation section; a reset transistor that resets a potential of the second charge accumulation section; a transfer transistor disposed between the first charge accumulation section and the second charge accumulation section; a third charge accumulation section connected in parallel with the first charge accumulation section; and a first changeover switch that is disposed between the first charge accumulation section and the third charge accumulation section and switches whether or not to connect the third charge accumulation section to the photoelectric conversion element.
- Furthermore, the first changeover switch may switch whether or not to connect the third charge accumulation section to the photoelectric conversion element according to a light amount of light incident on the photoelectric conversion element.
- Furthermore, the first changeover switch may be in an off state in a case where the light amount is smaller than a reference value, and in an on state in a case where the light amount is larger than the reference value.
- Furthermore, a second changeover switch connected to each of the reset transistor and the second charge accumulation section; and
- a fourth charge accumulation section disposed between the reset transistor and the second changeover switch or connected in parallel with the second charge accumulation section via the second changeover switch may be further provided.
- Furthermore, the second changeover switch may switch whether or not to connect the fourth charge accumulation section to the second charge accumulation section in synchronization with the first changeover switch.
- Furthermore, an addition value of capacitances of the first charge accumulation section and the third charge accumulation section may be equal to an addition value of capacitances of the second charge accumulation section and the fourth charge accumulation section.
- Furthermore, the reset transistor, the transfer transistor, the first changeover switch, and the second changeover switch may be P-channel MOS transistors.
- Furthermore, a selection transistor that switches whether or not to output a pixel signal corresponding to an amount of charge accumulated in the second charge accumulation section may be further provided, and
- the selection transistor may be turned on again after the first changeover switch is maintained in an off state and the first changeover switch may be subsequently switched from the off state to an on state.
- Furthermore, a capacitance of the third charge accumulation section may be larger than a capacitance of the first charge accumulation section.
- Furthermore, among the plurality of pixels, a pixel in which the third charge accumulation section is connected to the photoelectric conversion element by the first changeover switch and a pixel in which the third charge accumulation section is not connected to the photoelectric conversion element by the first changeover switch may be mixed.
- Furthermore, an overflow gate transistor that discharges charge accumulated in the third charge accumulation section may be further provided.
- Furthermore, the reset transistor, the transfer transistor, and the first changeover switch may be P-channel MOS transistors.
- Furthermore, a third changeover switch that switches whether or not to connect the second charge accumulation sections respectively provided in adjacent pixels adjacent to each other among the plurality of pixels may be further provided.
- Furthermore, among the plurality of pixels, the third changeover switch may be always in an on state in a case where the adjacent pixels are added, and the third changeover switch may be always in an off state in a case where the adjacent pixels are not added.
-
FIG. 1 is a block diagram illustrating an example of a configuration of an imaging device according to a first embodiment. -
FIG. 2 is a circuit diagram illustrating a configuration of a pixel according to the first embodiment. -
FIG. 3 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the first embodiment. -
FIG. 4 is a timing chart illustrating another example of the readout operation of the pixel circuit according to the first embodiment. -
FIG. 5 is a circuit diagram illustrating a configuration of a pixel according to a first modification. -
FIG. 6 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the first modification. -
FIG. 7 is a circuit diagram illustrating a configuration of a pixel according to a second modification. -
FIG. 8 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the second modification. -
FIG. 9 is a circuit diagram illustrating a configuration of a pixel according to a third modification. -
FIG. 10A is a circuit diagram illustrating a configuration of a pixel according to a fourth modification. -
FIG. 10B is a circuit diagram illustrating the configuration of the pixel according to the fourth modification. -
FIG. 10C is a circuit diagram illustrating the configuration of the pixel according to the fourth modification. -
FIG. 11 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the fourth modification. -
FIG. 12A is a circuit diagram illustrating a configuration of a pixel according to a second embodiment. -
FIG. 12B is a circuit diagram illustrating the configuration of the pixel according to the second embodiment. -
FIG. 13 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the second embodiment. -
FIG. 14A is a circuit diagram illustrating a configuration of a pixel according to a fifth modification. -
FIG. 14B is a circuit diagram illustrating the configuration of the pixel according to the fifth modification. -
FIG. 15A is a circuit diagram illustrating a configuration of a pixel according to a sixth modification. -
FIG. 15B is a circuit diagram illustrating the configuration of the pixel according to the sixth modification. -
FIG. 16A is a circuit diagram illustrating a configuration of a pixel according to a seventh modification. -
FIG. 16B is a circuit diagram illustrating the configuration of the pixel according to the seventh modification. -
FIG. 17A is a circuit diagram illustrating a configuration of the pixel according to the sixth modification. -
FIG. 17B is a circuit diagram illustrating the configuration of the pixel according to the sixth modification. -
FIG. 18 is a functional block diagram illustrating an example of an electronic device. -
FIG. 19 is a block diagram illustrating an example of schematic configuration of an in-vivo information acquisition system. -
FIG. 20 is a diagram illustrating an example of a schematic configuration of an endoscopic surgical system. -
FIG. 21 is a block diagram depicting an example of schematic configuration of a vehicle control system. -
FIG. 22 is a block diagram illustrating an example of a schematic configuration of a vehicle control system. -
FIG. 23 is a diagram of assistance in explaining an example of installation positions of an outside-vehicle information detecting section and an imaging section. -
FIG. 1 is a block diagram illustrating an example of a configuration of an imaging device according to a first embodiment. Animaging device 1 illustrated inFIG. 1 is, for example, an infrared image sensor, and has sensitivity also to light having a wavelength of 800 nm or more, for example. In theimaging device 1, for example, arectangular pixel region 10P and anextra-pixel region 10B outside thepixel region 10P are provided. In theextra-pixel region 10B, a peripheral circuit for driving thepixel region 10P is provided. - In the
pixel region 10P of theimaging device 1, a plurality of pixels P is two-dimensionally disposed. The peripheral circuit provided in theextra-pixel region 10B includes, for example, arow scanning section 201, ahorizontal selection section 203, acolumn scanning section 204, and asystem control section 202. - The
row scanning section 201 includes a shift register, an address decoder, and the like, and is a pixel drive section that drives each pixel P, for example, in units of rows. The pixel signal output from each pixel P of the pixel row selectively scanned by therow scanning section 201 is supplied to thehorizontal selection section 203 through each of vertical signal lines Lsig. Thehorizontal selection section 203 includes an amplifier, a horizontal selection switch, and the like provided for each vertical signal line Lsig. - The
column scanning section 204 includes a shift register, an address decoder, and the like, and scans and concurrently drives each horizontal selection switch of thehorizontal selection section 203 in sequence. By the selective scanning by thecolumn scanning section 204, the pixel signal of each pixel transmitted through each of the vertical signal lines Lsig is sequentially output to the horizontal signal line. Thereafter, the pixel signal is input to a signal processing section (not illustrated) or the like through the horizontal signal line. - The
system control section 202 receives an externally supplied clock, data indicating an operation mode and the like, and further outputs data such as internal information of theimaging device 1. Thesystem control section 202 further includes a timing generator that generates various timing signals, and performs drive control of therow scanning section 201, thehorizontal selection section 203, thecolumn scanning section 204, and the like on the basis of the various timing signals generated by the timing generator. -
FIG. 2 is a circuit diagram illustrating a configuration of the pixel P. The pixel P includes aphotoelectric conversion element 10 and apixel circuit 20. Thephotoelectric conversion element 10 is disposed on one semiconductor substrate, and thepixel circuit 20 is disposed on another semiconductor substrate stacked under the one semiconductor substrate. - The
photoelectric conversion element 10 is, for example, a photodiode that photoelectrically converts incident light having a wavelength in an infrared region. In thephotoelectric conversion element 10, a photoelectric conversion film (not illustrated) is provided between an anode electrode and a cathode electrode. The photoelectric conversion film includes, for example, a compound semiconductor material such as a group III-V semiconductor, and photoelectrically converts light having a wavelength in a visible region to a short infrared region. - As the compound semiconductor material constituting the
photoelectric conversion element 10, for example, InGaAs (indium gallium arsenide), InAsSb (indium arsenide antimony), GaAsSb (gallium arsenide antimony), InAs (indium arsenide), InSb (indium antimony), HgCdTe (mercury cadmium telluride), and the like can be used. Furthermore, the photoelectric conversion film may include Ge (germanium). Furthermore, the photoelectric conversion film may include a semiconductor material having a Type II structure. - The
pixel circuit 20 is an example of a readout integrated circuit that reads out the charge of thephotoelectric conversion element 10. Specifically, thepixel circuit 20 includes a first charge accumulation section (SN1) 21, a transfer transistor (TRG) 22, a second charge accumulation section (FD1) 23, a reset transistor (RST) 24, an amplification transistor (AMP) 25, a selection transistor (SEL) 26, achangeover switch 27, and a thirdcharge accumulation section 28. - The first
charge accumulation section 21 includes a capacitor that temporarily holds the charge photoelectrically converted by thephotoelectric conversion element 10. One end of the firstcharge accumulation section 21 is connected to the anode electrode of thephotoelectric conversion element 10 and the source of thetransfer transistor 22. The other end of the firstcharge accumulation section 21 is grounded. - The drain of the
transfer transistor 22 is connected to the secondcharge accumulation section 23. That is, thetransfer transistor 22 is disposed between the firstcharge accumulation section 21 and the secondcharge accumulation section 23. Thetransfer transistor 22 is turned on and off on the basis of a transfer signal input from therow scanning section 201 to the gate. When thetransfer transistor 22 is turned on, the charge accumulated in the firstcharge accumulation section 21 is transferred to the secondcharge accumulation section 23 via thetransfer transistor 22. - The second
charge accumulation section 23 is a floating diffusion that accumulates the charge transferred from the firstcharge accumulation section 21. One end of the secondcharge accumulation section 23 is connected to the source of thereset transistor 24 and the gate of theamplification transistor 25 together with the drain of the transfer transistor. The other end of the secondcharge accumulation section 23 is grounded. - The drain of the
reset transistor 24 is connected to a power source VDD. Thereset transistor 24 is turned on and off on the basis of a reset signal input from therow scanning section 201 to the gate. When thereset transistor 24 is turned on, a reset potential VRST is applied to the secondcharge accumulation section 23. The reset potential VRST brings the potential of the secondcharge accumulation section 23 into an initial state (reset state). - In the
amplification transistor 25, the gate is connected to the secondcharge accumulation section 23, the drain is connected to the power source VDD, and the source is connected to the drain of theselection transistor 26. Theamplification transistor 25 configures a source follower circuit with a load metal oxide semiconductor (MOS) as a constant current source connected via a vertical signal line Lsig. Theamplification transistor 25 generates a pixel signal corresponding to the charge amount accumulated in the secondcharge accumulation section 23. - The source of the
selection transistor 26 is connected to the vertical signal line Lsig. Theselection transistor 26 is turned on and off on the basis of a selection signal input from therow scanning section 201 to the gate. When theselection transistor 26 is turned on, the pixel signal generated by theamplification transistor 25 is output to thehorizontal selection section 203 via the vertical signal line Lsig. - The
changeover switch 27 corresponds to a first changeover switch and includes an N-channel MOS transistor. In thechangeover switch 27, a drain is connected to one end of the firstcharge accumulation section 21, and a source is connected to one end of the thirdcharge accumulation section 28. That is, thechangeover switch 27 is disposed between the firstcharge accumulation section 21 and the thirdcharge accumulation section 28. Thechangeover switch 27 is turned on and off on the basis of a control signal input from therow scanning section 201 to the gate. When thechangeover switch 27 is turned on, the charge photoelectrically converted by thephotoelectric conversion element 10 is accumulated in the thirdcharge accumulation section 28 via thechangeover switch 27. - The third
charge accumulation section 28 includes a capacitor that temporarily holds the charge photoelectrically converted by thephotoelectric conversion element 10. The other end of the thirdcharge accumulation section 28 is grounded. - In the pixel P configured as described above, the conversion efficiency changes depending on whether or not the
changeover switch 27 connects the thirdcharge accumulation section 28 to thephotoelectric conversion element 10. When thechangeover switch 27 is turned on, the conversion efficiency decreases. Conversely, when thechangeover switch 27 is turned off, the conversion efficiency increases. - The following Formula (1) shows conversion efficiency (Low) when the
changeover switch 27 is turned on and conversion efficiency (Hi) when thechangeover switch 27 is turned off. In Formula (1), q represents the amount of charge of thephotoelectric conversion element 10. Furthermore, Csn1, Csn2, and Cfd1 are capacitances of the firstcharge accumulation section 21, the thirdcharge accumulation section 28, and the secondcharge accumulation section 23, respectively. -
- According to Formula (1), the conversion efficiency (Low) depends on the capacitance of each of the first
charge accumulation section 21, the secondcharge accumulation section 23, and the thirdcharge accumulation section 28. At this time, as the capacitance Csn2 of the thirdcharge accumulation section 28 increases, the saturation signal amount also increases. Therefore, the capacitance Csn2 of the thirdcharge accumulation section 28 is desirably larger than the capacitance Csn1 of the firstcharge accumulation section 21. - Furthermore, readout noise depends on the balance between the addition value of the capacitance Csn1 and the capacitance Csn2 and the capacitance Cfd1. For example, it is desirable that the addition value described above be equal to the capacitance Cfd1. Note that the readout noise is, for example, input-converted random noise (RN) in double delta sampling (DDS) driving, in other words, kT/C noise. Furthermore, as the capacitance Cfd1 increases, the amount of charge that can be accumulated in the second
charge accumulation section 23 increases. - With the configuration of the
pixel circuit 20 described above, at low illuminance when the amount of light incident on thephotoelectric conversion element 10 is smaller than the reference value, readout noise can be reduced by turning off thechangeover switch 27. On the other hand, at high illuminance when the amount of light incident on thephotoelectric conversion element 10 is higher than the reference value, the saturation signal amount can be increased by turning on thechangeover switch 27. - Hereinafter, a readout operation of the
pixel circuit 20 will be described. -
FIG. 3 is a timing chart illustrating an example of the readout operation of thepixel circuit 20. In this readout operation, the conversion efficiency (Low) is set. Therefore, the changeover switch 27 (SNG) is always in an on state regardless of a data phase (D phase) and a reset phase (P phase). - In the timing chart illustrated in
FIG. 3 , first, in the charge accumulation time from time t1 to time t2, the reset transistor 24 (RST), the transfer transistor 22 (TRG), and the selection transistor 26 (SEL) are in an off state. Therefore, the charge photoelectrically converted by thephotoelectric conversion element 10 is accumulated in the firstcharge accumulation section 21 and the thirdcharge accumulation section 28. - Next, within the time from time t2 to time t3, the
transfer transistor 22 is switched from the off state to the on state. As a result, the charges accumulated in the firstcharge accumulation section 21 and the thirdcharge accumulation section 28 are transferred to the secondcharge accumulation section 23 and amplified as a pixel signal by theamplification transistor 25. Subsequently, when thetransfer transistor 22 returns to the off state, theselection transistor 26 is turned on. As a result, the pixel signal is output to the vertical signal line Lsig. - Next, the
selection transistor 26 is turned off within the time from time t3 to time t4. Subsequently, thereset transistor 24, thetransfer transistor 22, and theselection transistor 26 are simultaneously switched from the off state to the on state. As a result, the charge remaining in the secondcharge accumulation section 23 is discharged via thereset transistor 24, so that the secondcharge accumulation section 23 is reset to the initial state. Thereafter, the above operation is repeated again. - Note that, in the readout operation set to the conversion efficiency (Hi), the changeover switch 27 (SNG) is always in the off state regardless of the data phase (D phase) and the reset phase (P phase). Furthermore, by adjusting the timing of the pulse of the signal input to the gate of each transistor, the
pixel region 10P can be driven by a global shutter method in which all the pixels P accumulate (expose) charges at the same timing, or can be driven by a rolling shutter method in which each pixel P accumulates charges at different timings. -
FIG. 4 is a timing chart illustrating another example of the readout operation of thepixel circuit 20. Here, differences from the timing chart illustrated inFIG. 3 will be described. In the timing chart illustrated inFIG. 3 , the charge set to the conversion efficiency (Hi) and the charge set to the conversion efficiency (Low) are read out at different times. - On the other hand, in the timing chart illustrated in
FIG. 4 , within the time from time t2 to time t3 after the charge accumulation time (time from time t1 to time t2), thechangeover switch 27 is maintained in the off state and theselection transistor 26 is turned on, so that the charge set to the conversion efficiency (Hi) is read out. - Next, within the time from time t3 to time t4, since the
selection transistor 26 is turned on again after thechangeover switch 27 is switched from the off state to the on state, the charge set to the conversion efficiency (Low) is read out. - Thereafter, within the time from time t4 to time t5, the charge remaining in the second
charge accumulation section 23 is discharged via thereset transistor 24, and thus the secondcharge accumulation section 23 is reset to the initial state. - According to the timing chart illustrated in
FIG. 4 , the charge set to the conversion efficiency (Hi) and the charge set to the conversion efficiency (Low) can be simultaneously read out by one exposure. Also, in this readout operation, the charge having overflowed from the firstcharge accumulation section 21 via thechangeover switch 27 is accumulated in the thirdcharge accumulation section 28, whereby the dynamic range can be expanded by one exposure. - According to the present embodiment described above, the
pixel circuit 20 is provided with thechangeover switch 27 and the thirdcharge accumulation section 28. Therefore, by switching on and off thechangeover switch 27 according to the illuminance condition, the capacitance of the entire charge accumulation section in thepixel circuit 20 changes. With this change, the conversion efficiency also changes, and thus the dynamic range can be expanded. - Note that, in the
pixel region 10P, the illuminance condition may be different for each pixel P in some cases. Therefore, in thepixel region 10P, a pixel P in which thechangeover switch 27 is turned on (set to the conversion efficiency (Low)) and a pixel P in which thechangeover switch 27 is turned off (set to the conversion efficiency (Hi)) may be mixed. In this case, the conversion rate can be optimized for each pixel P. - Some modifications of the first embodiment will be described below. Components similar to those of the first embodiment described above are denoted by the same reference numerals, and detailed description thereof will be omitted.
-
FIG. 5 is a circuit diagram illustrating a configuration of a pixel according to a first modification. Furthermore,FIG. 6 is a timing chart illustrating an example of the readout operation of the pixel circuit according to the first modification. - As illustrated in
FIG. 5 , apixel circuit 20 a according to the present modification further includes an overflow gate (OFG)transistor 29 in addition to the components of thepixel circuit 20 described above. Theoverflow gate transistor 29 includes an N-channel MOS transistor. In theoverflow gate transistor 29, the drain is connected to the source of thechangeover switch 27 and one end of the thirdcharge accumulation section 28, and the source is connected to the power source VDD. Theoverflow gate transistor 29 is turned on and off on the basis of a discharge signal input from therow scanning section 201 to the gate. - In the present modification, as illustrated in
FIG. 6 , theoverflow gate transistor 29 is switched from the off state to the on state until theselection transistor 26 is turned off and theselection transistor 26 is turned on within the time from time t2 to time t3. As a result, the charge remaining in the thirdcharge accumulation section 28 is discharged to the power source VDD via theoverflow gate transistor 29. - According to the present modification described above, since the
pixel circuit 20 a includes theoverflow gate transistor 29, the charge remaining in the thirdcharge accumulation section 28 can be discharged. As a result, blooming to other pixels P can be suppressed. -
FIG. 7 is a circuit diagram illustrating a configuration of a pixel according to a second modification. Furthermore,FIG. 8 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the second modification. - In the
pixel circuit 20 according to the first embodiment described above, as illustrated inFIG. 2 , all of thetransfer transistor 22, thereset transistor 24, and thechangeover switch 27 include N-channel MOS transistors. On the other hand, in thepixel circuit 20 b according to the present modification, as illustrated inFIG. 7 , all of thetransfer transistor 22, thereset transistor 24, and thechangeover switch 27 include P-channel MOS transistors. - As described above, in the present modification, the conductivity types of the
transfer transistor 22, thereset transistor 24, and thechangeover switch 27 are opposite to those of the first embodiment. Therefore, in the present modification, as illustrated inFIG. 8 , the signal level input to the gate of each transistor is opposite to that of the first embodiment. - According to the present modification described above, all of the
transfer transistor 22, thereset transistor 24, and thechangeover switch 27 include P-channel MOS transistors. Therefore, readout of signal charges from thephotoelectric conversion element 10 is readout of holes that are positive charges. In this case, as compared with a case where these transistors include N-channel MOS transistors, charges are less likely to overflow from the firstcharge accumulation section 21, the secondcharge accumulation section 23, and the thirdcharge accumulation section 28. Therefore, also in the present modification, blooming between the pixels P adjacent to each other can be suppressed. -
FIG. 9 is a circuit diagram illustrating a configuration of a pixel according to a third modification. Thepixel circuit 20 c illustrated inFIG. 9 is further provided with theoverflow gate transistor 29 described in the first modification in addition to the components of thepixel circuit 20 b of the second modification described above. However, in the present modification, theoverflow gate transistor 29 includes the P-channel MOS transistor. - The
pixel circuit 20 c according to the present modification has a configuration in which the first modification and the second modification are combined. Therefore, the charge remaining in the thirdcharge accumulation section 28 can be discharged by theoverflow gate transistor 29, and furthermore, the charge is less likely to overflow from the firstcharge accumulation section 21, the secondcharge accumulation section 23, and the thirdcharge accumulation section 28 by configuring the pixel transistor with the P-channel MOS transistor. - Therefore, according to the present modification, the blooming suppression effect can be enhanced.
-
FIGS. 10A, 10B, and 10C are circuit diagrams illustrating a configuration of a pixel according to a fourth modification. Furthermore,FIG. 11 is a timing chart illustrating an example of a readout operation of a pixel circuit according to the fourth modification. - In the present modification, as illustrated in
FIGS. 10A to 10C , thechangeover switch 30 is disposed between a pixel P11 and a pixel P12. The pixel P11 and the pixel P12 are commonly connected to the vertical signal line Lsig and are adjacent to each other. Thechangeover switch 30 corresponds to a third changeover switch and includes an N-channel MOS transistor. - In
FIG. 10A , the drain of thechangeover switch 30 is connected to the secondcharge accumulation section 23 of the pixel P11, and the source is connected to the secondcharge accumulation section 23 of the pixel P12. Thechangeover switch 30 is turned on and off on the basis of a signal input from therow scanning section 201 to the gate. - As illustrated in
FIG. 11 , for example, when the changeover switch 30 (FDX) is in the always-on state within the time from time t1 to time t4 (within the periods of the D phase and the P phase), the secondcharge accumulation section 23 is shared between the pixel P11 and the pixel P12, and accordingly, the charges accumulated in the secondcharge accumulation section 23 are added. As a result, the sensitivity and the saturation signal amount are doubled. On the other hand, when thechangeover switch 30 is always in the off state within the above time, since the secondcharge accumulation section 23 is not shared, the charges accumulated in each secondcharge accumulation section 23 are not added. - In
FIG. 10B , thechangeover switch 30 is disposed between the firstcharge accumulation section 21 of the pixel P11 and the firstcharge accumulation section 21 of the pixel P12. When thechangeover switch 30 is in the always-on state, the firstcharge accumulation section 21 of the pixel P11 is connected to the firstcharge accumulation section 21 of the pixel P12 via thechangeover switch 30. In this case, the firstcharge accumulation section 21 is shared between the pixel P11 and the pixel P12, whereby the charges accumulated in the firstcharge accumulation section 21 are added. As a result, the signal saturation amount is doubled. On the other hand, when thechangeover switch 30 is always in the off state within the above time, since the firstcharge accumulation section 21 is not shared, the charges accumulated in each firstcharge accumulation section 21 are not added. - In
FIG. 10C , thechangeover switch 30 is disposed between the thirdcharge accumulation section 28 of the pixel P11 and the thirdcharge accumulation section 28 of the pixel P12. When thechangeover switch 30 is in the always-on state, the thirdcharge accumulation section 28 of the pixel P11 is connected to the thirdcharge accumulation section 28 of the pixel P12 via thechangeover switch 30. In this case, the thirdcharge accumulation section 28 is shared between the pixel P11 and the pixel P12, whereby the charges accumulated in the thirdcharge accumulation section 28 are added. As a result, the signal saturation amount is doubled. On the other hand, when thechangeover switch 30 is always in the off state within the above time, since the thirdcharge accumulation section 28 is not shared, the charges accumulated in each thirdcharge accumulation section 28 are not added. - According to the present modification described above, the
changeover switch 30 can switch whether or not to connect the firstcharge accumulation section 21, the secondcharge accumulation section 23, or the thirdcharge accumulation section 28 disposed in the pixels adjacent to each other. As a result, the sensitivity and the saturation signal amount can be adjusted. Note that, in the present modification, thechangeover switch 30 is disposed between pixels adjacent in a direction parallel to the vertical signal line Lsig (vertical direction), but may be disposed between pixels adjacent in a direction orthogonal to the vertical signal line Lsig (horizontal direction). -
FIGS. 12A and 12B are circuit diagrams illustrating a configuration of a pixel according to a second embodiment. Components similar to those of the first embodiment described above are denoted by the same reference numerals, and detailed description thereof will be omitted. - A
pixel circuit 20 d illustrated inFIG. 12A and apixel circuit 20 e illustrated inFIG. 12B further include achangeover switch 31 and a fourthcharge accumulation section 32 in addition to the components of thepixel circuit 20 described in the first embodiment. Thechangeover switch 31 corresponds to a second changeover switch and includes an N-channel MOS transistor. The fourthcharge accumulation section 32 includes a capacitor. - In the
pixel circuit 20 d illustrated inFIG. 12A , in thechangeover switch 31, the drain is connected to the source of thereset transistor 24, and the source is connected to the secondcharge accumulation section 23. Thechangeover switch 31 is turned on and off on the basis of a signal input from therow scanning section 201 to the gate. One end of the fourthcharge accumulation section 32 is connected to the source of thereset transistor 24 and the drain of thechangeover switch 31, and the other end is grounded. When thetransfer transistor 22 and thechangeover switch 31 are turned on, the charge accumulated in the firstcharge accumulation section 21 is also transferred to the fourthcharge accumulation section 32 together with the secondcharge accumulation section 23. - On the other hand, in the
pixel circuit 20 e illustrated inFIG. 12B , the drain of thechangeover switch 31 is connected to the sources of the secondcharge accumulation section 23 and thereset transistor 24, and the gate of theamplification transistor 25. The source is connected to one end of the fourthcharge accumulation section 32. The other end of the fourthcharge accumulation section 32 is grounded. That is, the fourthcharge accumulation section 32 is connected to the secondcharge accumulation section 23 via thechangeover switch 31. - In the
pixel circuit 20 d and thepixel circuit 20 e configured as described above, the conversion efficiency is changed by thechangeover switch 27 switching whether or not to connect the thirdcharge accumulation section 28 to thephotoelectric conversion element 10 and thechangeover switch 31 switching whether or not to connect the fourthcharge accumulation section 32 to the secondcharge accumulation section 23. When thechangeover switch 27 is turned on, the conversion efficiency decreases. Conversely, when thechangeover switch 27 is turned off, the conversion efficiency increases. - The following Formula (2) shows conversion efficiency (Low) when the
changeover switch 27 and thechangeover switch 31 are turned on and conversion efficiency (Hi) when thechangeover switch 27 and thechangeover switch 31 are turned off. - In Formula (2), Cfd2 is the capacitance of the fourth
charge accumulation section 32. -
- According to Formula (2), the conversion efficiency (Low) depends on the capacitance of each of the first
charge accumulation section 21, the secondcharge accumulation section 23, the thirdcharge accumulation section 28, and the fourthcharge accumulation section 32. At this time, as the capacitance Cfd2 of the fourthcharge accumulation section 32 increases, the saturation signal amount also increases. Therefore, the capacitance Cfd2 of the fourthcharge accumulation section 32 is desirably larger than the capacitance Cfd1 of the secondcharge accumulation section 23. - Furthermore, the readout noise depends on a balance between a first addition value of the capacitance Csn1 and the capacitance Csn2 and a second addition value of the capacitance Cfd1 and the capacitance Cfd2. For example, the first addition value is desirably equal to the second addition value.
- In the present embodiment, at low illuminance when the amount of light incident on the
photoelectric conversion element 10 is smaller than the reference value, readout noise can be reduced by turning off thechangeover switch 27 and thechangeover switch 31. On the other hand, at high illuminance when the amount of light incident on thephotoelectric conversion element 10 is higher than the reference value, the saturation signal amount can be increased by turning on thechangeover switch 27 and thechangeover switch 31. -
FIG. 13 is a timing chart illustrating an example of the readout operation of thepixel circuits changeover switch 27 and thechangeover switch 31 are always in an off state regardless of the D phase and the P phase. In this manner, thechangeover switch 31 operates in synchronization with thechangeover switch 27. - Note that, also in the present embodiment, similarly to the first embodiment, the
reset transistor 24, thetransfer transistor 22, and theselection transistor 26 may be driven in the timing chart illustrated inFIG. 11 . In this case, the charge set to the conversion efficiency (Hi) and the charge set to the conversion efficiency (Low) can be simultaneously read out by one exposure. Also, in this readout operation, the charge having overflowed from the secondcharge accumulation section 23 via thechangeover switch 31 is accumulated in the fourthcharge accumulation section 32, whereby the dynamic range can be expanded by one exposure. - According to the present embodiment described above, the conversion efficiency can be switched by controlling on and off of the
changeover switch 31 in addition to thechangeover switch 27. Furthermore, by adjusting the balance between an SN capacitance of the firstcharge accumulation section 21 and the thirdcharge accumulation section 28 and an FD capacitance of the secondcharge accumulation section 23 and the fourthcharge accumulation section 32, imaging with a high SN ratio can be performed under wider illuminance conditions. - Some modifications of the second embodiment will be described below. Components similar to those of the second embodiment described above are denoted by the same reference numerals, and detailed description thereof will be omitted.
-
FIGS. 14A and 14B are circuit diagrams illustrating a configuration of a pixel according to a fifth modification. - A
pixel circuit 20 f illustrated inFIG. 14A and apixel circuit 20 g illustrated inFIG. 14B further include theoverflow gate transistor 29 similarly to the first modification described above. - In the present modification, the
overflow gate transistor 29 is driven in the timing chart illustrated inFIG. 6 , similarly to the first modification. That is, in the time from time t2 to time t3, theoverflow gate transistor 29 is switched from the off state to the on state until theselection transistor 26 is turned off and theselection transistor 26 is turned on. As a result, the charge remaining in the thirdcharge accumulation section 28 is discharged to the power source VDD via theoverflow gate transistor 29. At this time, thechangeover switch 31 is always in an off state together with thechangeover switch 27 in a case where the conversion efficiency (Hi) is set, and is always in an on state together with thechangeover switch 27 in a case where the conversion efficiency (Low) is set. - According to the present modification described above, since the
pixel circuits overflow gate transistor 29, the charge remaining in the thirdcharge accumulation section 28 can be discharged. As a result, blooming to other pixels P can be suppressed. -
FIGS. 15A and 15B are circuit diagrams illustrating a configuration of a pixel according to a sixth modification. Comparing apixel circuit 20 h illustrated inFIG. 15A with thepixel circuit 20 d illustrated inFIG. 12A , all of thetransfer transistor 22, thereset transistor 24, thechangeover switch 27, and thechangeover switch 31 include P-channel MOS transistors instead of N-channel MOS transistors. - Similarly, comparing a
pixel circuit 20 i illustrated inFIG. 15B with thepixel circuit 20 e illustrated inFIG. 12B , all of thetransfer transistor 22, thereset transistor 24, thechangeover switch 27, and thechangeover switch 31 include P-channel MOS transistors instead of N-channel MOS transistors. - As described above, in the present modification, the conductivity types of the
transfer transistor 22, thereset transistor 24, thechangeover switch 27, and thechangeover switch 31 are opposite to those of the second embodiment. Therefore, in the present modification, the signal level input to the gate of each transistor is opposite to that of the second embodiment. - According to the present modification described above, all of the
transfer transistor 22, thereset transistor 24, thechangeover switch 27, and thechangeover switch 31 include P-channel MOS transistors. Therefore, readout of signal charges from thephotoelectric conversion element 10 is readout of holes that are positive charges. Therefore, the charges are less likely to overflow from the firstcharge accumulation section 21, the secondcharge accumulation section 23, the thirdcharge accumulation section 28, and the fourthcharge accumulation section 32. Therefore, also in the present modification, blooming between adjacent pixels can be suppressed. -
FIGS. 16A and 16B are circuit diagrams illustrating a configuration of a pixel according to a seventh modification. Comparing apixel circuit 20 j illustrated inFIG. 16A with thepixel circuit 20 f illustrated inFIG. 14A , all of thetransfer transistor 22, thereset transistor 24, thechangeover switch 27, thechangeover switch 31, and theoverflow gate transistor 29 include P-channel MOS transistors instead of N-channel MOS transistors. - Similarly, comparing a
pixel circuit 20 k illustrated inFIG. 16B with thepixel circuit 20 g illustrated inFIG. 14B , all of thetransfer transistor 22, thereset transistor 24, thechangeover switch 27, thechangeover switch 31, and theoverflow gate transistor 29 include P-channel MOS transistors instead of N-channel MOS transistors. - The
pixel circuit 20 k according to the present modification has a configuration in which the fifth modification and the sixth modification are combined. Therefore, the charge remaining in the thirdcharge accumulation section 28 can be discharged by theoverflow gate transistor 29, and furthermore, the charge is less likely to overflow from the firstcharge accumulation section 21, the secondcharge accumulation section 23, and the thirdcharge accumulation section 28 by configuring the pixel transistor with the P-channel MOS transistor. - Therefore, according to the present modification, the blooming suppression effect can be enhanced.
-
FIGS. 17A and 17B are circuit diagrams illustrating a configuration of a pixel according to an eighth modification. In the present modification, similarly to the above-described fourth modification, thechangeover switch 30 is disposed between the pixel P21 and the pixel P22. The pixel P21 and the pixel P22 are commonly connected to the vertical signal line Lsig and are adjacent to each other. - In
FIG. 17A , each of the pixel P21 and the pixel P22 includes thepixel circuit 20 d illustrated inFIG. 12A . On the other hand, inFIG. 17B , each of the pixel P21 and the pixel P22 includes thepixel circuit 20 e illustrated inFIG. 12B . Furthermore, the drain of thechangeover switch 30 is connected between the secondcharge accumulation section 23 of the pixel P21 and thechangeover switch 31, and the source is connected between the secondcharge accumulation section 23 of the pixel P22 and thechangeover switch 31. - Also in the present modification, similarly to the fourth modification, in a case where the charges of the pixel P21 and the pixel P22 are added, the
changeover switch 31 is always turned on within the periods of the D phase and the P phase. Conversely, in a case where no charge is added, thechangeover switch 31 is always in an off state within the D-phase and P-phase periods. - Note that a connection form of the
changeover switch 30 is not limited to the arrangement illustrated inFIGS. 17A and 17B . Thechangeover switch 30 may switch the connection between the secondcharge accumulation sections 21 similarly toFIG. 10B , or may switch the connection between the thirdcharge accumulation sections 28 similarly to inFIG. 10C . - According to the present modification described above, the
changeover switch 30 can switch whether or not to connect the firstcharge accumulation section 21, the secondcharge accumulation section 23, or the thirdcharge accumulation section 28 disposed in the pixels adjacent to each other. As a result, the sensitivity and the saturation signal amount can be adjusted. Note that, in the present modification, thechangeover switch 30 is disposed between pixels adjacent in a direction parallel to the vertical signal line Lsig, but may be disposed between pixels adjacent in a direction orthogonal to the vertical signal line Lsig. - The
imaging device 1 described above can be applied to various types of electronic devices in addition to a camera capable of imaging an infrared region.FIG. 18 illustrates a schematic configuration of an electronic device 3 (camera) as an example. Theelectronic device 3 is, for example, a camera capable of capturing a still image or a moving image, and includes theimaging device 1, anoptical system 310, ashutter device 311, adrive section 313 that drives theimaging device 1 and theshutter device 311, and asignal processing section 312. - The
optical system 310 guides image light (incident light) from a subject to theimaging device 1. Theoptical system 310 may include a plurality of optical lenses. Theshutter device 311 controls a light irradiation period and a light shielding period for theimaging device 1. Thedrive section 313 controls a transfer operation of theimaging device 1 and a shutter operation of theshutter device 311. Thesignal processing section 312 performs various types of signal processing on a signal output from theimaging device 1. A video signal Dout after the signal processing is stored in a storage medium such as a memory or output to a monitor and the like. - Furthermore, the technology according to the present disclosure (present technology) can be applied to a wide variety of products. For example, the technology according to the present disclosure may be applied to an endoscopic surgical system.
-
FIG. 19 is a block diagram illustrating an example of a schematic configuration of a patient in-vivo information acquisition system using a capsule endoscope to which the technology according to the present disclosure (present technology) can be applied. - An in-vivo
information acquisition system 10001 includes acapsule endoscope 10100 and anexternal control device 10200. - The
capsule endoscope 10100 is swallowed by a patient at the time of examination. Thecapsule endoscope 10100 has an imaging function and a wireless communication function and, while moving inside an organ such as a stomach and an intestine by peristaltic movement or the like until it is naturally excreted from the patient, sequentially captures images inside the organ (hereinafter also referred to as in-vivo images) at predetermined intervals, and sequentially transmits information regarding the in-vivo images wirelessly to theexternal control device 10200 outside the body. - The
external control device 10200 centrally controls the operation of the in-vivoinformation acquisition system 10001. Furthermore, theexternal control device 10200 receives information regarding the in-vivo images transmitted from thecapsule endoscope 10100, and generates image data for displaying the in-vivo images on a display device (not illustrated) on the basis of the received information regarding the in-vivo images. - In this way, with the in-vivo
information acquisition system 10001, in-vivo images indicating the patient's internal conditions can be obtained continually from the time thecapsule endoscope 10100 is swallowed to the time thecapsule endoscope 10100 is excreted. - The configurations and functions of the
capsule endoscope 10100 and theexternal control device 10200 will be described in further detail. - The
capsule endoscope 10100 includes a capsule-shapedhousing 10101, and includes alight source section 10111, animaging section 10112, animage processing section 10113, awireless communication section 10114, apower supply section 10115, apower source section 10116, and acontrol section 10117 which are housed in the capsule-shapedhousing 10101. - The
light source section 10111 includes a light source such as a light emitting diode (LED) or the like, for example, and irradiates an imaging field of view of theimaging section 10112 with light. - The
imaging section 10112 includes an optical system including an imaging element and a plurality of lenses provided on a preceding stage of the imaging element. Reflected light (hereinafter referred to as observation light) of the light applied to body tissue to be observed is condensed by the optical system and is incident on the imaging element. In theimaging section 10112, in the imaging element, the observation light incident thereon is photoelectrically converted, and an image signal corresponding to the observation light is generated. The image signal generated by theimaging section 10112 is provided to theimage processing section 10113. - The
image processing section 10113 includes a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or the like, and performs various kinds of signal processing on the image signal generated by theimaging section 10112. Theimage processing section 10113 provides the image signal that has been subjected to signal processing to thewireless communication section 10114 as RAW data. - The
wireless communication section 10114 performs predetermined processing such as modulation processing or the like on the image signal that has been subjected to the signal processing by theimage processing section 10113, and transmits the image signal to theexternal control device 10200 via anantenna 10114A. Furthermore, thewireless communication section 10114 receives a control signal regarding drive control of thecapsule endoscope 10100 from theexternal control device 10200 via theantenna 10114A. Thewireless communication section 10114 provides the control signal received from theexternal control device 10200 to thecontrol section 10117. - The
power supply section 10115 includes an antenna coil for power reception, a power regeneration circuit for regenerating power from current generated in the antenna coil, a booster circuit and the like. In thepower supply section 10115, the principle of what is called contactless charging is used to generate power. - The
power source section 10116 includes a secondary battery, and accumulates the power generated by thepower supply section 10115. Although arrows or the like indicating the destination to which power from thepower source section 10116 is supplied are not illustrated inFIG. 19 for preventing the illustration from being complex, power accumulated in thepower source section 10116 is supplied to thelight source section 10111, theimaging section 10112, theimage processing section 10113, thewireless communication section 10114, and thecontrol section 10117, and may be used to drive these sections. - The
control section 10117 includes a processor such as a CPU, and appropriately controls drives of thelight source section 10111, theimaging section 10112, theimage processing section 10113, thewireless communication section 10114, and thepower supply section 10115 in accordance with a control signal transmitted from theexternal control device 10200. - The
external control device 10200 includes a processor such as a CPU, a GPU, or the like, or a microcomputer or a control board or the like on which a processor and a storage element such as a memory are mounted in a mixed manner. Theexternal control device 10200 controls the operation of thecapsule endoscope 10100 by transmitting the control signal to thecontrol section 10117 of thecapsule endoscope 10100 via anantenna 10200A. In thecapsule endoscope 10100, for example, an irradiation condition of the light to the observation target in thelight source section 10111 might be changed by the control signal from theexternal control device 10200. Furthermore, an imaging condition (for example, a frame rate, an exposure value and the like in the imaging section 10112) might be changed by the control signal from theexternal control device 10200. Furthermore, the contents of processing in theimage processing section 10113 and conditions for transmitting the image signal by the wireless communication section 10114 (for example, transmission interval, number of transmitted images, and the like) may be changed by the control signal from theexternal control device 10200. - Furthermore, the
external control device 10200 performs various types of image processing on the image signal transmitted from thecapsule endoscope 10100, and generates image data for displaying a captured in-vivo image on a display device. As the image processing, for example, various signal processing such as development processing (demosaic processing), image quality enhancement processing (band enhancement processing, super-resolution processing, noise reduction (NR) processing, and/or camera shake correction processing, and the like), and/or enlargement processing (electronic zoom processing) and the like can be performed. Theexternal control device 10200 controls driving of the display device to display the in-vivo image captured on the basis of the generated image data. Alternatively, theexternal control device 10200 may also cause a recording device (not illustrated) to record the generated image data, or cause a printing device (not illustrated) to make a printout of the generated image data. - An example of the in-vivo information acquisition system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the
imaging section 10112 among the above-described configurations. As a result, the dynamic range of theimaging section 10112 is expanded. - The technology of the present disclosure (present technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgical system.
-
FIG. 20 is a diagram illustrating an example of a schematic configuration of an endoscopic surgical system to which the technology according to the present disclosure (present technology) can be applied. -
FIG. 20 illustrates a state in which an operator (doctor) 11131 is performing surgery on apatient 11132 on apatient bed 11133 using an endoscopicsurgical system 11000. As illustrated, the endoscopicsurgical system 11000 includes anendoscope 11100, othersurgical tools 11110 such as apneumoperitoneum tube 11111 and anenergy device 11112, a supportingarm apparatus 11120 which supports theendoscope 11100 thereon, and acart 11200 on which various apparatus for endoscopic surgery are mounted. - The
endoscope 11100 includes alens barrel 11101 having a region of a predetermined length from a distal end thereof to be inserted into a body cavity of thepatient 11132, and acamera head 11102 connected to a proximal end of thelens barrel 11101. In the illustrated example, theendoscope 11100 configured as a so-called rigid scope having therigid lens barrel 11101 is illustrated, but theendoscope 11100 may be configured as a so-called flexible scope having a flexible lens barrel. - The
lens barrel 11101 has, at a distal end thereof, an opening in which an objective lens is fitted. Alight source device 11203 is connected to theendoscope 11100 such that light generated by thelight source device 11203 is introduced to a distal end of thelens barrel 11101 by a light guide extending in the inside of thelens barrel 11101 and is irradiated toward an observation target in a body cavity of thepatient 11132 through the objective lens. It is to be noted that theendoscope 11100 may be a forward-viewing endoscope or may be an oblique-viewing endoscope or a side-viewing endoscope. - An optical system and an imaging element are provided in the inside of the
camera head 11102 such that reflected light (observation light) from the observation target is condensed on the imaging element by the optical system. The observation light is photoelectrically converted by the imaging element to generate an electric signal corresponding to the observation light, namely, an image signal corresponding to an observation image. The image signal is transmitted as RAW data to a camera control unit (CCU) 11201. - The
CCU 11201 includes a central processing unit (CPU), a graphics processing unit (GPU), or the like, and comprehensively controls operation of theendoscope 11100 and adisplay device 11202. Moreover, theCCU 11201 receives an image signal from thecamera head 11102, and applies, on the image signal, various types of image processing for displaying an image based on the image signal, for example, development processing (demosaic processing) and the like. - The
display device 11202 displays thereon an image based on an image signal, for which the image processing has been performed by theCCU 11201, under the control of theCCU 11201. - The
light source device 11203 includes a light source such as, for example, a light emitting diode (LED) and supplies irradiation light upon imaging of a surgical site or the like to theendoscope 11100. - An
input device 11204 is an input interface for the endoscopicsurgical system 11000. A user can input various types of information and input instructions to the endoscopicsurgical system 11000 via theinput device 11204. For example, the user inputs an instruction or the like for changing imaging conditions (a type of irradiation light, a magnification, a focal length, and the like) by theendoscope 11100. - A treatment
tool control device 11205 controls driving of theenergy device 11112 for cautery or incision of a tissue, sealing of a blood vessel or the like. Apneumoperitoneum device 11206 feeds gas into a body cavity of thepatient 11132 through thepneumoperitoneum tube 11111 to inflate the body cavity in order to secure the field of view of theendoscope 11100 and secure the working space for the operator. Arecorder 11207 is an apparatus capable of recording various kinds of information relating to surgery. Aprinter 11208 is an apparatus capable of printing various kinds of information relating to surgery in various forms such as a text, an image, a graph, or the like. - Note that the
light source device 11203 that supplies theendoscope 11100 with the irradiation light at the time of imaging the surgical site can include, for example, an LED, a laser light source, or a white light source including a combination thereof. In a case where a white light source includes a combination of red, green, and blue (RGB) laser light sources, since the output intensity and the output timing can be controlled with a high degree of accuracy for each color (each wavelength), adjustment of the white balance of a captured image can be performed by thelight source device 11203. Furthermore, in this case, by irradiating the observation target with the laser light from each of the R, G, and B laser light sources in time division manner and controlling drive of the imaging element of thecamera head 11102 in synchronization with irradiation timing, it is possible to capture images corresponding to R, G, and B in time division manner. According to this method, a color image can be obtained even if color filters are not provided for the imaging element. - Furthermore, driving of the
light source device 11203 may be controlled so as to change the intensity of output light at every predetermined time interval. By controlling driving of the imaging element of thecamera head 11102 in synchronization with the timing of the change of the intensity of light to acquire images time-divisionally and synthesizing the images, an image of a high dynamic range free from underexposed blocked up shadows and overexposed highlights can be created. - Furthermore, the
light source device 11203 may be configured to be able to supply light having a predetermined wavelength band corresponding to special light observation. In special light observation, for example, by utilizing the wavelength dependency of absorption of light in a body tissue to irradiate light of a narrow band in comparison with irradiation light upon ordinary observation (namely, white light), narrow band observation (narrow band imaging) of imaging a predetermined tissue such as a blood vessel of a superficial portion of the mucous membrane or the like in a high contrast is performed. Alternatively, in special light observation, fluorescent observation for obtaining an image from fluorescent light generated by irradiation of excitation light may be performed. In fluorescent observation, it is possible, for example, to perform observation of fluorescent light from a body tissue by irradiating excitation light on the body tissue (autofluorescence observation) or to obtain a fluorescent light image by locally injecting a reagent such as indocyanine green (ICG) into a body tissue and irradiating excitation light corresponding to a fluorescent light wavelength of the reagent upon the body tissue. Thelight source device 11203 can be configured to supply such narrow-band light and/or excitation light suitable for special light observation as described above. -
FIG. 21 is a block diagram illustrating an example of a functional configuration of thecamera head 11102 and theCCU 11201 illustrated inFIG. 20 . - The
camera head 11102 includes alens unit 11401, animaging section 11402, adrive section 11403, acommunication section 11404, and a camerahead control section 11405. TheCCU 11201 includes acommunication section 11411, animage processing section 11412, and acontrol section 11413. Thecamera head 11102 and theCCU 11201 are communicatively connected to each other by atransmission cable 11400. - The
lens unit 11401 is an optical system provided at a connection to thelens barrel 11101. The observation light taken in from the distal end of thelens barrel 11101 is guided to thecamera head 11102 and is incident on thelens unit 11401. Thelens unit 11401 includes a combination of a plurality of lenses including a zoom lens and a focusing lens. - The number of the imaging elements included in the
imaging section 11402 may be one (a so-called single plate type) or plural (a so-called multi-plate type). In a case where theimaging section 11402 is of the multiple plate type, image signals corresponding to R, G, and B may be generated by the respective imaging elements, and a color image may be obtained by combining the generated image signals, for example. Theimaging section 11402 may also be configured so as to have a pair of imaging elements for acquiring respective image signals for the right eye and the left eye ready for three dimensional (3D) display. By the 3D display, theoperator 11131 may grasp a depth of the living tissue in the surgical site more accurately. Note that, in a case where theimaging section 11402 is of the multiple-plate type, a plurality of systems oflens units 11401 may be provided so as to correspond to the respective imaging elements. - Furthermore, the
imaging section 11402 may not necessarily be provided in thecamera head 11102. For example, theimaging section 11402 may be provided immediately behind the objective lens in the inside of thelens barrel 11101. - The
drive section 11403 includes an actuator and moves the zoom lens and the focusing lens of thelens unit 11401 by a predetermined distance along an optical axis under the control of the camerahead control section 11405. As a result, the magnification and the focal point of a captured image by theimaging section 11402 can be adjusted suitably. - The
communication section 11404 includes a communication device for transmitting and receiving various kinds of information to and from theCCU 11201. Thecommunication section 11404 transmits an image signal acquired from theimaging section 11402 as RAW data to theCCU 11201 through thetransmission cable 11400. - In addition, the
communication section 11404 receives a control signal for controlling driving of thecamera head 11102 from theCCU 11201 and supplies the control signal to the camerahead control section 11405. The control signal includes, for example, information regarding an imaging condition such as information specifying a frame rate of a captured image, information specifying an exposure value at the time of imaging, and/or information specifying the magnification and focal point of the captured image. - Note that the above imaging conditions such as the frame rate, exposure value, magnification, and focus described above may be appropriately specified by the user, or may be automatically set by the
control section 11413 of theCCU 11201 on the basis of the acquired image signal. In the latter case, an auto exposure (AE) function, an auto focus (AF) function and an auto white balance (AWB) function are incorporated in theendoscope 11100. - The camera
head control section 11405 controls driving of thecamera head 11102 on the basis of a control signal from theCCU 11201 received through thecommunication section 11404. - The
communication section 11411 includes a communication device for transmitting and receiving various types of information to and from thecamera head 11102. Thecommunication section 11411 receives an image signal transmitted thereto from thecamera head 11102 through thetransmission cable 11400. - Furthermore, the
communication section 11411 transmits, to thecamera head 11102, a control signal for controlling driving of thecamera head 11102. The image signal and the control signal can be transmitted by electrical communication, optical communication, or the like. - The
image processing section 11412 performs various types of image processing on the image signal being the RAW data transmitted from thecamera head 11102. - The
control section 11413 performs various control related to imaging of the surgical site or the like by theendoscope 11100 and display of a captured image obtained by the imaging of the surgical site or the like. For example, thecontrol section 11413 creates a control signal for controlling driving of thecamera head 11102. - Furthermore, the
control section 11413 controls, on the basis of an image signal for which image processes have been performed by theimage processing section 11412, thedisplay device 11202 to display a picked up image in which the surgical site or the like is imaged. Thereupon, thecontrol section 11413 may recognize various objects in the captured image using various image recognition technologies. For example, thecontrol section 11413 can recognize a surgical tool such as forceps, a particular living body region, bleeding, mist when theenergy device 11112 is used and the like by detecting the shape, color and the like of edges of objects included in a captured image. Thecontrol section 11413 may cause, when it controls thedisplay device 11202 to display a picked up image, various kinds of surgery supporting information to be displayed in an overlapping manner with an image of the surgical site using a result of the recognition. The surgery support information is superimposed to be displayed, and presented to theoperator 11131, so that it becomes possible to reduce the burden on theoperator 11131 and enable theoperator 11131 to reliably proceed with surgery. - The
transmission cable 11400 which connects thecamera head 11102 and theCCU 11201 to each other is an electric signal cable ready for communication of an electric signal, an optical fiber ready for optical communication or a composite cable ready for both of electrical and optical communications. - Here, in the illustrated example, the communication is performed by wire using the
transmission cable 11400, but the communication between thecamera head 11102 and theCCU 11201 may be performed wirelessly. - An example of the endoscopic surgical system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure may be applied to the
imaging section 11402 among the configurations described above. As a result, the dynamic range of theimaging section 11402 is expanded. - Note that, here, the endoscopic surgical system has been described as an example, but the technology according to the present disclosure may be applied to, for example, a microscopic surgical system or the like.
- The technology according to the present disclosure may also be achieved as a device mounted on any type of mobile body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a boat, a robot, and the like.
-
FIG. 22 is a block diagram depicting an example of schematic configuration of a vehicle control system as an example of a mobile body control system to which the technology according to an embodiment of the present disclosure can be applied. - The
vehicle control system 12000 includes a plurality of electronic control units connected to each other via acommunication network 12001. In the example depicted inFIG. 22 , thevehicle control system 12000 includes a drivingsystem control unit 12010, a bodysystem control unit 12020, an outside-vehicleinformation detecting unit 12030, an in-vehicleinformation detecting unit 12040, and anintegrated control unit 12050. In addition, amicrocomputer 12051, a sound/image output section 12052, and a vehicle-mounted network interface (I/F) 12053 are illustrated as a functional configuration of theintegrated control unit 12050. - The driving
system control unit 12010 controls the operation of devices related to the driving system of the vehicle in accordance with various kinds of programs. For example, the drivingsystem control unit 12010 functions as a control device for a driving force generating device for generating the driving force of the vehicle, such as an internal combustion engine, a driving motor, or the like, a driving force transmitting mechanism for transmitting the driving force to wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating the braking force of the vehicle, and the like. - The body
system control unit 12020 controls the operation of various kinds of devices provided to a vehicle body in accordance with various kinds of programs. For example, the bodysystem control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various kinds of lamps such as a headlamp, a backup lamp, a brake lamp, a turn signal, a fog lamp, or the like. In this case, radio waves transmitted from a mobile device as an alternative to a key or signals of various kinds of switches can be input to the bodysystem control unit 12020. The bodysystem control unit 12020 receives these input radio waves or signals, and controls a door lock device, the power window device, the lamps, or the like of the vehicle. - The outside-vehicle
information detecting unit 12030 detects information about the outside of the vehicle including thevehicle control system 12000. For example, the outside-vehicleinformation detecting unit 12030 is connected with animaging section 12031. The outside-vehicleinformation detecting unit 12030 makes theimaging section 12031 image an image of the outside of the vehicle, and receives the imaged image. On the basis of the received image, the outside-vehicleinformation detecting unit 12030 may perform processing of detecting an object such as a human, a vehicle, an obstacle, a sign, a character on a road surface, or the like, or processing of detecting a distance thereto. - The
imaging section 12031 is an optical sensor that receives light, and which outputs an electric signal corresponding to a received light amount of the light. Theimaging section 12031 can output the electric signal as an image, or can output the electric signal as information about a measured distance. In addition, the light received by theimaging section 12031 may be visible light, or may be invisible light such as infrared rays or the like. - The in-vehicle
information detecting unit 12040 detects information about the inside of the vehicle. The in-vehicleinformation detecting unit 12040 is, for example, connected with a driverstate detecting section 12041 that detects the state of a driver. The driverstate detecting section 12041, for example, includes a camera that images the driver. On the basis of detection information input from the driverstate detecting section 12041, the in-vehicleinformation detecting unit 12040 may calculate a degree of fatigue of the driver or a degree of concentration of the driver, or may determine whether the driver is dozing. - The
microcomputer 12051 can calculate a control target value for the driving force generating device, the steering mechanism, or the braking device on the basis of the information about the inside or outside of the vehicle which information is obtained by the outside-vehicleinformation detecting unit 12030 or the in-vehicleinformation detecting unit 12040, and output a control command to the drivingsystem control unit 12010. For example, themicrocomputer 12051 can perform cooperative control intended to implement functions of an advanced driver assistance system (ADAS) which functions include collision avoidance or shock mitigation for the vehicle, following driving based on a following distance, vehicle speed maintaining driving, a warning of collision of the vehicle, a warning of deviation of the vehicle from a lane, or the like. - In addition, the
microcomputer 12051 can perform cooperative control intended for automated driving, which makes the vehicle to travel automatedly without depending on the operation of the driver, or the like, by controlling the driving force generating device, the steering mechanism, the braking device, or the like on the basis of the information about the outside or inside of the vehicle which information is obtained by the outside-vehicleinformation detecting unit 12030 or the in-vehicleinformation detecting unit 12040. - In addition, the
microcomputer 12051 can output a control command to the bodysystem control unit 12020 on the basis of the information about the outside of the vehicle which information is obtained by the outside-vehicleinformation detecting unit 12030. For example, themicrocomputer 12051 can perform cooperative control intended to prevent a glare by controlling the headlamp so as to change from a high beam to a low beam, for example, in accordance with the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicleinformation detecting unit 12030. - The sound/
image output section 12052 transmits an output signal of at least one of a sound and an image to an output device capable of visually or auditorily notifying information to an occupant of the vehicle or the outside of the vehicle. In the example ofFIG. 11 , anaudio speaker 12061, adisplay section 12062, and aninstrument panel 12063 are illustrated as the output device. Thedisplay section 12062 may, for example, include at least one of an on-board display and a head-up display. -
FIG. 23 is a diagram depicting an example of the installation position of theimaging section 12031. - In
FIG. 23 , theimaging section 12031 includesimaging sections - The
imaging sections vehicle 12100 as well as a position on an upper portion of a windshield within the interior of the vehicle. Theimaging section 12101 provided to the front nose and theimaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle obtain mainly an image of the front of thevehicle 12100. Theimaging sections vehicle 12100. Theimaging section 12104 provided to the rear bumper or the back door obtains mainly an image of the rear of thevehicle 12100. Theimaging section 12105 provided to the upper portion of the windshield within the interior of the vehicle is used mainly to detect a preceding vehicle, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like. - Incidentally,
FIG. 23 depicts an example of photographing ranges of theimaging sections 12101 to 12104. Animaging range 12111 represents the imaging range of theimaging section 12101 provided to the front nose. Imaging ranges 1211212113 respectively represent the imaging ranges of theimaging sections imaging range 12114 represents the imaging range of theimaging section 12104 provided to the rear bumper or the back door. A bird's-eye image of thevehicle 12100 as viewed from above is obtained by superimposing image data imaged by theimaging sections 12101 to 12104, for example. - At least one of the
imaging sections 12101 to 12104 may have a function of obtaining distance information. For example, at least one of theimaging sections 12101 to 12104 may be a stereo camera constituted of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection. - For example, the
microcomputer 12051 can determine a distance to each three-dimensional object within the imaging ranges 12111 to 12114 and a temporal change in the distance (relative speed with respect to the vehicle 12100) on the basis of the distance information obtained from theimaging sections 12101 to 12104, and thereby extract, as a preceding vehicle, a nearest three-dimensional object in particular that is present on a traveling path of thevehicle 12100 and which travels in substantially the same direction as thevehicle 12100 at a predetermined speed (for example, equal to or more than 0 km/hour). Further, themicrocomputer 12051 can set a following distance to be maintained in front of a preceding vehicle in advance, and perform automatic brake control (including following stop control), automatic acceleration control (including following start control), or the like. It is thus possible to perform cooperative control intended for automated driving that makes the vehicle travel automatedly without depending on the operation of the driver or the like. - For example, the
microcomputer 12051 can classify three-dimensional object data on three-dimensional objects into three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a large-sized vehicle, a pedestrian, a utility pole, and other three-dimensional objects on the basis of the distance information obtained from theimaging sections 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data for automatic avoidance of an obstacle. For example, themicrocomputer 12051 identifies obstacles around thevehicle 12100 as obstacles that the driver of thevehicle 12100 can recognize visually and obstacles that are difficult for the driver of thevehicle 12100 to recognize visually. Then, themicrocomputer 12051 determines a collision risk indicating a risk of collision with each obstacle. In a situation in which the collision risk is equal to or higher than a set value and there is thus a possibility of collision, themicrocomputer 12051 outputs a warning to the driver via theaudio speaker 12061 or thedisplay section 12062, and performs forced deceleration or avoidance steering via the drivingsystem control unit 12010. Themicrocomputer 12051 can thereby assist in driving to avoid collision. - At least one of the
imaging sections 12101 to 12104 may be an infrared camera that detects infrared rays. Themicrocomputer 12051 can, for example, recognize a pedestrian by determining whether or not there is a pedestrian in imaged images of theimaging sections 12101 to 12104. Such recognition of a pedestrian is, for example, performed by a procedure of extracting characteristic points in the imaged images of theimaging sections 12101 to 12104 as infrared cameras and a procedure of determining whether or not it is the pedestrian by performing pattern matching processing on a series of characteristic points representing the contour of the object. When themicrocomputer 12051 determines that there is a pedestrian in the imaged images of theimaging sections 12101 to 12104, and thus recognizes the pedestrian, the sound/image output section 12052 controls thedisplay section 12062 so that a square contour line for emphasis is displayed so as to be superimposed on the recognized pedestrian. The sound/image output section 12052 may also control thedisplay section 12062 so that an icon or the like representing the pedestrian is displayed at a desired position. - An example of the vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the
imaging section 12031 and the like in the configuration described above, for example. Specifically, theimaging device 1 can be applied to theimaging section 12031. By applying the technology according to the present disclosure, the dynamic range of theimaging section 12031 is expanded. - Note that the present technology can have configurations as follows.
-
- (1) An imaging device including:
- a photoelectric conversion element provided in each of a plurality of pixels;
- a first charge accumulation section connected to the photoelectric conversion element;
- a second charge accumulation section connected in parallel with the first charge accumulation section;
- a reset transistor that resets a potential of the second charge accumulation section;
- a transfer transistor disposed between the first charge accumulation section and the second charge accumulation section;
- a third charge accumulation section connected in parallel with the first charge accumulation section; and
- a first changeover switch that is disposed between the first charge accumulation section and the third charge accumulation section and switches whether or not to connect the third charge accumulation section to the photoelectric conversion element.
- (2) The imaging device according to (1), in which the first changeover switch switches whether or not to connect the third charge accumulation section to the photoelectric conversion element according to a light amount of light incident on the photoelectric conversion element.
- (3) The imaging device according to (2), in which the first changeover switch is in an off state in a case where the light amount is smaller than a reference value, and in an on state in a case where the light amount is larger than the reference value.
- (4) The imaging device according to any one of (1) to (3), further including:
- a second changeover switch connected to each of the reset transistor and the second charge accumulation section; and
- a fourth charge accumulation section disposed between the reset transistor and the second changeover switch or connected in parallel with the second charge accumulation section via the second changeover switch.
- (5) The imaging device according to (4), in which the second changeover switch switches whether or not to connect the fourth charge accumulation section to the second charge accumulation section in synchronization with the first changeover switch.
- (6) The imaging device according to (4) or (5), in which an addition value of capacitances of the first charge accumulation section and the third charge accumulation section is equal to an addition value of capacitances of the second charge accumulation section and the fourth charge accumulation section.
- (7) The imaging device according to any one of (4) to (6), in which the reset transistor, the transfer transistor, the first changeover switch, and the second changeover switch are P-channel MOS transistors.
- (8) The imaging device according to any one of (1) to (7), further including
- a selection transistor that switches whether or not to output a pixel signal corresponding to an amount of charge accumulated in the second charge accumulation section,
- in which the selection transistor is turned on again after the first changeover switch is maintained in an off state and the first changeover switch is subsequently switched from the off state to an on state.
- (9) The imaging device according to any one of (1) to (8), in which a capacitance of the third charge accumulation section is larger than a capacitance of the first charge accumulation section.
- (10) The imaging device according to any one of (1) to (9), in which, among the plurality of pixels, a pixel in which the third charge accumulation section is connected to the photoelectric conversion element by the first changeover switch and a pixel in which the third charge accumulation section is not connected to the photoelectric conversion element by the first changeover switch are mixed.
- (11) The imaging device according to any one of (1) to (10), further including an overflow gate transistor that discharges charge accumulated in the third charge accumulation section.
- (12) The imaging device according to any one of (1) to (10), in which the reset transistor, the transfer transistor, and the first changeover switch are P-channel MOS transistors.
- (13) The imaging device according to any one of (1) to (10), further including a third changeover switch that switches whether or not to connect the second charge accumulation sections respectively provided in adjacent pixels adjacent to each other among the plurality of pixels.
- (14) The imaging device according to (13), in which, among the plurality of pixels, the third changeover switch is always in an on state in a case where the adjacent pixels are added, and the third changeover switch is always in an off state in a case where the adjacent pixels are not added.
-
-
- 1 Imaging device
- 10 Photoelectric conversion element
- 21 First charge accumulation section
- 22 Transfer transistor
- 23 Second charge accumulation section
- 24 Reset transistor
- 26 Selection transistor
- 27 Changeover switch
- 28 Third charge accumulation section
- 29 Overflow gate transistor
- 30 Changeover switch
- 31 Changeover switch
- 32 Fourth charge accumulation section
Claims (14)
1. An imaging device comprising:
a photoelectric conversion element provided in each of a plurality of pixels;
a first charge accumulation section connected to the photoelectric conversion element;
a second charge accumulation section connected in parallel with the first charge accumulation section;
a reset transistor that resets a potential of the second charge accumulation section;
a transfer transistor disposed between the first charge accumulation section and the second charge accumulation section;
a third charge accumulation section connected in parallel with the first charge accumulation section; and
a first changeover switch that is disposed between the first charge accumulation section and the third charge accumulation section and switches whether or not to connect the third charge accumulation section to the photoelectric conversion element.
2. The imaging device according to claim 1 , wherein the first changeover switch switches whether or not to connect the third charge accumulation section to the photoelectric conversion element according to a light amount of light incident on the photoelectric conversion element.
3. The imaging device according to claim 2 , wherein the first changeover switch is in an off state in a case where the light amount is smaller than a reference value, and in an on state in a case where the light amount is larger than the reference value.
4. The imaging device according to claim 1 , further comprising:
a second changeover switch connected to each of the reset transistor and the second charge accumulation section; and
a fourth charge accumulation section disposed between the reset transistor and the second changeover switch or connected in parallel with the second charge accumulation section via the second changeover switch.
5. The imaging device according to claim 4 , wherein the second changeover switch switches whether or not to connect the fourth charge accumulation section to the second charge accumulation section in synchronization with the first changeover switch.
6. The imaging device according to claim 4 , wherein an addition value of capacitances of the first charge accumulation section and the third charge accumulation section is equal to an addition value of capacitances of the second charge accumulation section and the fourth charge accumulation section.
7. The imaging device according to claim 4 , wherein the reset transistor, the transfer transistor, the first changeover switch, and the second changeover switch are P-channel MOS transistors.
8. The imaging device according to claim 1 , further comprising
a selection transistor that switches whether or not to output a pixel signal corresponding to an amount of charge accumulated in the second charge accumulation section, wherein
the selection transistor is turned on again after the first changeover switch is maintained in an off state and the first changeover switch is subsequently switched from the off state to an on state.
9. The imaging device according to claim 1 , wherein a capacitance of the third charge accumulation section is larger than a capacitance of the first charge accumulation section.
10. The imaging device according to claim 1 , wherein, among the plurality of pixels, a pixel in which the third charge accumulation section is connected to the photoelectric conversion element by the first changeover switch and a pixel in which the third charge accumulation section is not connected to the photoelectric conversion element by the first changeover switch are mixed.
11. The imaging device according to claim 1 , further comprising an overflow gate transistor that discharges charge accumulated in the third charge accumulation section.
12. The imaging device according to claim 1 , wherein the reset transistor, the transfer transistor, and the first changeover switch are P-channel MOS transistors.
13. The imaging device according to claim 1 , further comprising a third changeover switch that switches whether or not to connect the second charge accumulation sections respectively provided in adjacent pixels adjacent to each other among the plurality of pixels.
14. The imaging device according to claim 13 , wherein, among the plurality of pixels, the third changeover switch is always in an on state in a case where the adjacent pixels are added, and the third changeover switch is always in an off state in a case where the adjacent pixels are not added.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021-047710 | 2021-03-22 | ||
JP2021047710A JP2024075798A (en) | 2021-03-22 | 2021-03-22 | Imaging apparatus |
PCT/JP2022/003761 WO2022201874A1 (en) | 2021-03-22 | 2022-02-01 | Imaging device |
Publications (1)
Publication Number | Publication Date |
---|---|
US20240163585A1 true US20240163585A1 (en) | 2024-05-16 |
Family
ID=83396760
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/550,402 Pending US20240163585A1 (en) | 2021-03-22 | 2022-02-01 | Imaging device |
Country Status (3)
Country | Link |
---|---|
US (1) | US20240163585A1 (en) |
JP (1) | JP2024075798A (en) |
WO (1) | WO2022201874A1 (en) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3984814B2 (en) * | 2001-10-29 | 2007-10-03 | キヤノン株式会社 | Imaging device, radiation imaging apparatus using the imaging device, and radiation imaging system using the imaging device |
JP2018186398A (en) * | 2017-04-26 | 2018-11-22 | ソニーセミコンダクタソリューションズ株式会社 | Solid state image sensor, and electronic equipment |
WO2019155841A1 (en) * | 2018-02-07 | 2019-08-15 | ソニーセミコンダクタソリューションズ株式会社 | Solid-state image sensor and imaging device |
-
2021
- 2021-03-22 JP JP2021047710A patent/JP2024075798A/en active Pending
-
2022
- 2022-02-01 WO PCT/JP2022/003761 patent/WO2022201874A1/en active Application Filing
- 2022-02-01 US US18/550,402 patent/US20240163585A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
WO2022201874A1 (en) | 2022-09-29 |
JP2024075798A (en) | 2024-06-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3557863B1 (en) | Solid-state imaging element, and electronic device | |
EP3751840B1 (en) | Solid-state image sensor and imaging device | |
JP6968797B2 (en) | Image sensor | |
CN110537367B (en) | Solid-state imaging capturing device and electronic device | |
EP3833015A1 (en) | Solid-state imaging device | |
US12002825B2 (en) | Solid-state imaging device and electronic apparatus with improved sensitivity | |
US20230224602A1 (en) | Solid-state imaging device | |
US11457169B2 (en) | Solid-state imaging element and electronic equipment | |
CN112534579A (en) | Imaging device and electronic apparatus | |
US11218658B2 (en) | Solid-state imaging device, method of controlling the solid-state imaging device, and electronic apparatus to generate high dynamic range image | |
WO2017150168A1 (en) | Imaging element and electronic device | |
US20240163585A1 (en) | Imaging device | |
US11252355B2 (en) | Image pickup device and electronic device | |
US11252357B2 (en) | Image-capturing device and electronic apparatus for suppressing noise | |
US20240022839A1 (en) | Imaging device and imaging method | |
US20230412945A1 (en) | Image pickup apparatus | |
CN114556906A (en) | Imaging device and electronic apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SONY SEMICONDUCTOR SOLUTIONS CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OTSUKA, YUSUKE;OKUMURA, KENICHI;REEL/FRAME:064893/0049 Effective date: 20230821 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |