WO2023157489A1 - 固体撮像素子、撮像装置、および、固体撮像素子の制御方法 - Google Patents

固体撮像素子、撮像装置、および、固体撮像素子の制御方法 Download PDF

Info

Publication number
WO2023157489A1
WO2023157489A1 PCT/JP2022/048207 JP2022048207W WO2023157489A1 WO 2023157489 A1 WO2023157489 A1 WO 2023157489A1 JP 2022048207 W JP2022048207 W JP 2022048207W WO 2023157489 A1 WO2023157489 A1 WO 2023157489A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
pixel
metadata
circuit
stage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/048207
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
啓悟 中澤
凌平 川崎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Semiconductor Solutions Corp
Original Assignee
Sony Semiconductor Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Semiconductor Solutions Corp filed Critical Sony Semiconductor Solutions Corp
Priority to DE112022006684.9T priority Critical patent/DE112022006684T5/de
Priority to JP2024501006A priority patent/JPWO2023157489A1/ja
Priority to CN202280091761.5A priority patent/CN118872284A/zh
Priority to US18/838,491 priority patent/US20250150730A1/en
Publication of WO2023157489A1 publication Critical patent/WO2023157489A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/53—Control of the integration time
    • H04N25/531—Control of the integration time by controlling rolling shutters in CMOS SSIS
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/53—Control of the integration time
    • H04N25/532—Control of the integration time by controlling global shutters in CMOS SSIS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/65—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to reset noise, e.g. KTC noise related to CMOS structures by techniques other than CDS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/77—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
    • H04N25/771—Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components comprising storage means other than floating diffusion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/78—Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10—Recognition assisted with metadata

Definitions

  • This technology relates to solid-state imaging devices. More specifically, the present invention relates to a solid-state imaging device, an imaging apparatus, and a control method for a solid-state imaging device that performs exposure by a global shutter method.
  • the solid-state imaging device can process the next frame F2 using metadata generated from a certain frame F1.
  • the position of the subject may change between the frame F1 and the frame F2, and if the metadata of the frame F1 is used, appropriate image processing cannot be performed based on the recognition result. There is a risk.
  • This technology was created in view of this situation, and aims to perform appropriate image processing in a solid-state imaging device that processes frames using metadata.
  • a first aspect of the present technology includes a plurality of pixels each provided with a sample-and-hold circuit for holding a pixel signal; an analog-to-digital conversion unit for generating a first digital signal by analog-to-digital conversion processing on each of the signals, and generating a second digital signal by analog-to-digital conversion processing on each of the held pixel signals;
  • a solid-state imaging device comprising a metadata extraction unit for extracting predetermined metadata from a digital signal and an image processing unit for performing predetermined processing on the second digital signal using the metadata, and the control method. This brings about an effect that appropriate image processing is executed.
  • the metadata extracting unit performs recognition processing for recognizing a predetermined object on the frame in which the first digital signal is arranged, and outputs data indicating the result of the recognition processing to the May be generated as metadata. This brings about the effect of executing appropriate image processing based on the recognition regret.
  • the metadata may indicate a predetermined area
  • the image processing section may perform processing to replace a color in the area with a specific color. This brings about an effect that the predetermined object is hidden.
  • the metadata may indicate a predetermined area
  • the image processing section may perform mosaic processing on the area. This brings about an effect that the predetermined object is hidden.
  • the metadata may indicate a predetermined region
  • the image processing section may extract the region as the region of interest from the frame in which the second digital signal is arranged. This brings about the effect that the predetermined processing is performed on the region of interest.
  • each of the plurality of pixels includes a pre-stage circuit that generates a pixel signal, a sample-and-hold circuit that holds the pixel signal, and a sample-and-hold circuit that reads out the pixel signal from the sample-and-hold circuit and outputs the pixel signal.
  • a post-stage circuit may be provided. This brings about the effect that the pixel signal is held for each pixel.
  • the sample-and-hold circuit includes first and second capacitive elements, control for connecting one of the first and second capacitive elements to a predetermined post-stage node, and A selection circuit may also be provided which sequentially performs control for connecting the other of the second capacitive elements to the post-stage node. This brings about the effect of holding the reset level and signal level.
  • the selection circuit controls to connect one of the first and second capacitive elements to a predetermined post-stage node and connects both the first and second capacitive elements to the post-stage node. and the control of connecting the other of the first and second capacitive elements to the post-stage node may be sequentially performed. This brings about the effect of reducing noise.
  • each of the plurality of pixels includes a post-stage reset transistor that initializes the level of the post-stage node when both the first and second capacitive elements are disconnected from the post-stage node. may be further provided. This brings about the effect of reducing noise.
  • a first digital signal is generated by a plurality of pixels each provided with a sample-and-hold circuit for holding a pixel signal, and signal processing for each of the held pixel signals.
  • a signal processing unit for generating a second digital signal by performing the signal processing on each of the held pixel signals; a metadata extraction unit for extracting predetermined metadata from the first digital signal; and a recording unit configured to record frames in which the processed second digital signal is arranged.
  • FIG. 1 is a block diagram showing an example of 1 composition of an imaging device in a 1st embodiment of this art. It is a block diagram showing an example of 1 composition of a solid-state image sensing device in a 1st embodiment of this art. It is a circuit diagram showing a configuration example of a pixel in the first embodiment of the present technology. 1 is a block diagram showing one configuration example of a load MOS (Metal Oxide Semiconductor) circuit block and a column signal processing circuit according to the first embodiment of the present technology; FIG. 1 is a circuit diagram showing a configuration example of a pixel provided with two systems of post-stage circuits in the first embodiment of the present technology; FIG.
  • MOS Metal Oxide Semiconductor
  • FIG. 1 is a circuit diagram showing a configuration example of a pixel in which transistors in a sample-and-hold circuit are eliminated in the first embodiment of the present technology
  • FIG. 1 is a circuit diagram showing a configuration example of a charge domain pixel according to a first embodiment of the present technology
  • FIG. 1 is a block diagram showing a configuration example of a logic circuit according to a first embodiment of the present technology
  • FIG. It is a timing chart which shows an example of operation of a solid-state image sensing device in a 1st embodiment of this art.
  • FIG. 4 is a timing chart showing an example of operation of a solid-state imaging device in a comparative example; It is a figure showing an example of the 1st frame and a recognition result in a 1st embodiment of this art. It is a figure which shows an example of the frame of the 2nd time in 1st Embodiment of this technique, and a frame after a process. It is a flow chart which shows an example of operation of a solid-state image sensing device in a 1st embodiment of this art. It is a figure which shows an example of the frame of the 2nd time in the 1st modification of 1st Embodiment of this technique, and a frame after a process.
  • FIG. 14 is a diagram for explaining level variations due to reset feedthrough in the fourth embodiment of the present technology; It is a timing chart which shows an example of voltage control in a 4th embodiment of this art.
  • FIG. 14 is a timing chart showing an example of global shutter operation for odd frames according to the fifth embodiment of the present technology; FIG. It is a timing chart which shows an example of read-out operation
  • FIG. 14 is a timing chart showing an example of global shutter operation for even-numbered frames according to the fifth embodiment of the present technology; FIG.
  • FIG. 14 is a timing chart showing an example of rolling shutter operation according to the seventh embodiment of the present technology;
  • FIG. 8 is a block diagram which shows one structural example of the solid-state image sensor in 8th Embodiment of this technique.
  • FIG. 22 is a diagram for explaining effects in the eighth embodiment of the present technology; 1 is a block diagram showing a schematic configuration example of a vehicle control system; FIG. FIG. 4 is an explanatory diagram showing an example of an installation position of an imaging unit;
  • First Embodiment Example of Holding Pixel Signals and Performing Signal Processing Twice
  • Second embodiment an example in which a post-stage reset transistor is added, a pixel signal is held, and signal processing is performed twice
  • Third Embodiment Example of adding an ejection transistor, holding a pixel signal, and performing signal processing twice
  • Fourth Embodiment Example of Holding Pixel Signals and Performing Signal Processing Twice to Control Reset Power Supply Voltage
  • FIG. 1 is a block diagram showing a configuration example of an imaging device 100 according to the first embodiment of the present technology.
  • This imaging device 100 is a device for capturing image data, and includes an imaging lens 110 , a solid-state imaging device 200 , a recording section 120 and an imaging control section 130 .
  • As the imaging device 100 a digital camera or an electronic device (smartphone, personal computer, etc.) having an imaging function is assumed.
  • the solid-state imaging device 200 captures image data (frames) under the control of the imaging control section 130 .
  • the solid-state imaging device 200 supplies frames to the recording section 120 via the signal line 209 .
  • the imaging lens 110 collects light and guides it to the solid-state imaging device 200 .
  • the imaging control unit 130 controls the solid-state imaging device 200 to capture frames.
  • the imaging control unit 130 supplies imaging control signals including, for example, the vertical synchronization signal XVS to the solid-state imaging device 200 via the signal line 139 .
  • the recording unit 120 records frames.
  • the vertical synchronization signal XVS is a signal indicating the timing of imaging, and a periodic signal with a constant frequency (60 Hz, etc.) is used as the vertical synchronization signal XVS.
  • the imaging device 100 records frames
  • the frames may be transmitted to the outside of the imaging device 100.
  • an external interface is additionally provided for transmitting frames.
  • the imaging device 100 may further display frames.
  • a display section is further provided.
  • FIG. 2 is a block diagram showing a configuration example of the solid-state imaging device 200 according to the first embodiment of the present technology.
  • This solid-state imaging device 200 includes a vertical scanning circuit 211 , a pixel array section 220 , a timing control circuit 212 , a DAC (Digital to Analog Converter) 213 , a load MOS circuit block 250 and a column signal processing circuit 260 .
  • a plurality of pixels 300 are arranged in a two-dimensional grid in the pixel array section 220 .
  • each circuit in the solid-state imaging device 200 is provided on, for example, a single semiconductor chip.
  • a set of pixels 300 arranged in the horizontal direction is hereinafter referred to as a "row”, and a set of pixels 300 arranged in the direction perpendicular to the row is referred to as a "column”.
  • the timing control circuit 212 controls the operation timings of the vertical scanning circuit 211, the DAC 213, and the column signal processing circuit 260 in synchronization with the vertical synchronization signal XVS from the imaging control section .
  • the DAC 213 generates a sawtooth ramp signal by DA (Digital to Analog) conversion.
  • the DAC 213 supplies the generated ramp signal to the column signal processing circuit 260 .
  • the vertical scanning circuit 211 sequentially selects and drives rows to output analog pixel signals.
  • the pixel 300 photoelectrically converts incident light to generate an analog pixel signal. This pixel 300 supplies a pixel signal to the column signal processing circuit 260 via the load MOS circuit block 250 .
  • the load MOS circuit block 250 is provided with a MOS transistor for supplying a constant current for each column.
  • the column signal processing circuit 260 executes signal processing such as AD conversion processing and CDS processing on pixel signals for each column.
  • the column signal processing circuit 260 supplies the image data (frame) composed of the processed signal to the recording unit 120 .
  • FIG. 3 is a circuit diagram showing one configuration example of the pixel 300 according to the first embodiment of the present technology.
  • This pixel 300 comprises a front-stage circuit 310 , a sample-and-hold circuit 320 and a rear-stage circuit 350 .
  • Vertical signal lines 309 are wired in the pixel array section 220 for each column.
  • the pre-stage circuit 310 includes a photoelectric conversion element 311 , a transfer transistor 312 , an FD (Floating Diffusion) reset transistor 313 , an FD 314 , a pre-stage amplification transistor 315 and a current source transistor 316 .
  • the photoelectric conversion element 311 generates charges by photoelectric conversion.
  • the transfer transistor 312 transfers charges from the photoelectric conversion element 311 to the FD 314 according to the transfer signal trg from the vertical scanning circuit 211 .
  • the FD reset transistor 313 extracts electric charge from the FD 314 according to the FD reset signal rst from the vertical scanning circuit 211 and initializes it.
  • the FD 314 accumulates charges and generates a voltage corresponding to the amount of charges.
  • the front-stage amplification transistor 315 amplifies the voltage level of the FD 314 and outputs it to the front-stage node 319 .
  • the sources of the FD reset transistor 313 and the pre-amplification transistor 315 are connected to the power supply voltage VDD.
  • the current source transistor 316 is connected to the drain of the pre-amplification transistor 315 .
  • a predetermined bias voltage BIAS is supplied to the gate of the current source transistor 316 .
  • the sample hold circuit 320 includes capacitive elements 321 and 322 and a selection circuit 330 .
  • One end of each of capacitive elements 321 and 322 is commonly connected to previous stage node 319 , and the other end of each is connected to select circuit 330 .
  • the capacitive elements 321 and 322 are examples of the first and second capacitive elements described in the claims.
  • the selection circuit 330 includes selection transistors 331 and 332 .
  • the selection transistor 331 opens and closes the path between the capacitive element 321 and the subsequent node 340 according to the selection signal ⁇ r from the vertical scanning circuit 211 .
  • the selection transistor 332 opens and closes the path between the capacitive element 322 and the subsequent node 340 according to the selection signal ⁇ s from the vertical scanning circuit 211 .
  • the post-stage circuit 350 includes a post-stage amplification transistor 351 and a post-stage selection transistor 352 .
  • the rear-stage amplification transistor 351 amplifies the level of the rear-stage node 340 .
  • the post-stage selection transistor 352 outputs a signal of a level amplified by the post-stage amplification transistor 351 to the vertical signal line 309 as a pixel signal in accordance with the post-stage selection signal selb from the vertical scanning circuit 211 .
  • nMOS n-channel Metal Oxide Semiconductor
  • the vertical scanning circuit 211 supplies high-level FD reset signal rst and transfer signal trg to all pixels at the start of exposure. Thereby, the photoelectric conversion element 311 is initialized.
  • this control will be referred to as "PD reset”.
  • the vertical scanning circuit 211 supplies the high level FD reset signal rst over the pulse period while setting the selection signal ⁇ r to high level for all pixels.
  • the FD 314 is initialized, and the capacitive element 321 holds a level corresponding to the level of the FD 314 at that time.
  • This control is hereinafter referred to as "FD reset".
  • the level of the FD 314 at the time of FD reset and the level corresponding to that level are hereinafter collectively referred to as "P phase” or "reset level”. .
  • the vertical scanning circuit 211 supplies a high-level transfer signal trg over the pulse period while setting the selection signal ⁇ s to a high level for all pixels.
  • signal charges corresponding to the amount of exposure are transferred to the FD 314 , and a level corresponding to the level of the FD 314 at that time is held in the capacitive element 322 .
  • phase D phase D
  • signal level level
  • Exposure control that simultaneously starts and ends exposure for all pixels in this way is called a global shutter method.
  • the pre-stage circuits 310 of all pixels generate pixel signals (reset level and signal level).
  • the reset level of the pixel signal is held in the capacitor 321 and the signal level is held in the capacitor 322 .
  • the vertical scanning circuit 211 sequentially selects rows and sequentially outputs the reset level and signal level of the rows.
  • the vertical scanning circuit 211 supplies the high level selection signal ⁇ r for a predetermined period while setting the FD reset signal rst and the subsequent stage selection signal selb of the selected row to high level.
  • the capacitive element 321 is connected to the post-stage node 340, and the reset level is read.
  • the vertical scanning circuit 211 After reading the reset level, the vertical scanning circuit 211 supplies the high-level selection signal ⁇ s for a predetermined period while keeping the FD reset signal rst and the subsequent stage selection signal selb of the selected row at high level. Thereby, the capacitive element 322 is connected to the post-stage node 340, and the signal level is read.
  • the selection circuit 330 of the selected row sequentially performs control to connect the capacitive element 321 to the subsequent node 340 and control to connect the capacitive element 322 to the subsequent node 340 .
  • the post-stage circuit 350 of the selected row reads the pixel signal (reset level and signal level) from the sample-and-hold circuit 320 via the post-stage node 340 and outputs it to the vertical signal line 309 .
  • FIG. 4 is a block diagram showing one configuration example of the load MOS circuit block 250 and the column signal processing circuit 260 according to the first embodiment of the present technology.
  • a vertical signal line 309 is wired to the load MOS circuit block 250 for each column. Assuming that the number of columns is I (I is an integer), I vertical signal lines 309 are wired. A load MOS transistor 251 that supplies a constant current id2 is connected to each of the vertical signal lines 309 .
  • a signal processing unit 261 , a logic circuit 400 , and a high-speed interface 410 are arranged in the column signal processing circuit 260 .
  • the signal processing unit 261 includes multiple ADCs 262 .
  • ADC 262 is arranged for each column. Assuming that the number of columns is I, I ADCs 262 are arranged.
  • the ADC 262 uses the ramp signal Rmp from the DAC 213 to convert analog pixel signals from the corresponding columns into digital signals.
  • This ADC 262 supplies a digital signal to the logic circuit 400 .
  • the ADC 262 is a single-slope ADC that includes a comparator and a counter.
  • the counter in the ADC 262 performs down-counting when AD-converting the reset level, and performs up-counting when AD-converting the signal level.
  • CDS Correlated Double Sampling
  • the signal processing unit 261 performs signal processing including AD conversion processing and CDS processing.
  • the logic circuit 400 performs various signal processing other than the CDS processing on each digital signal for each column.
  • Logic circuit 400 provides frames of processed digital signals to high speed interface 410 .
  • a high-speed interface 410 outputs frames to the recording unit 120 .
  • circuit configuration of the pixel 300 is not limited to that illustrated in FIG. 3 as long as it can generate a pixel signal and sample and hold it.
  • the selection transistor 331 opens and closes the path between the capacitive element 321 and the post-circuit 350-1
  • the selection transistor 332 opens and closes the path between the capacitive element 322 and the post-circuit 350-2.
  • the post-stage circuit 350-1 includes a post-stage amplification transistor 351-1 and a post-stage selection transistor 352-1
  • the post-stage circuit 350-2 includes a post-stage amplification transistor 351-2 and a post-stage selection transistor 352-2.
  • two vertical signal lines are provided for each column, the post-stage circuit 350-1 outputs the pixel signal to the vertical signal line 309-1, and the post-stage circuit 350-2 outputs the pixel signal to the vertical signal line 309-2.
  • selection transistors 331 and 332 can be connected in series between the pre-stage circuit 310 and the post-stage circuit 350 .
  • the capacitive element 322 is connected to the connection node of the selection transistors 331 and 332
  • the capacitive element 321 is connected to the connection node of the selection transistor 331 and the post-stage circuit 350 .
  • the control method of this sample-and-hold circuit 320 is, for example, "Chen Xu et al., A Stacked Global-Shutter CMOS Imager with SC-Type Hybrid-GS Pixel and Self-Knee Point Calibration Single-Frame HDR and On-Chip Binarization Algorithm for Smart Vision Applications ISSCC2019.”
  • a sampling transistor 333 and a capacitive element 321 can be inserted in series between the pre-stage circuit 310 and the post-stage circuit 350, and the capacitive element 322 can be connected to their connection node.
  • a post-stage reset transistor 341 for initializing the post-stage node 340 is further provided.
  • the control method of this circuit is, for example, "Jae-kyu Lee, et al., A 2.1e-Temporal Noise and -105dB Parasitic Light Sensitivity Backside-Illuminated 2.3 ⁇ m-Pixel Voltage-Domain Global Shutter CMOS Image Sensor Using High-Capacity DRAM Capacitor Technology, ISSCC 2020.”
  • selection transistors 331-1 and 331-2 are inserted in series between the pre-stage circuit 310 and the post-stage circuit 350.
  • Serially connected select transistors 332-1 and 332-2 are inserted in parallel with the circuit composed of select transistors 331-1 and 331-2.
  • Capacitive element 321 is connected to a connection node between select transistors 331-1 and 331-2, and capacitive element 322 is connected to a connection node between select transistors 332-1 and 332-2.
  • FIG. 9 is a block diagram showing a configuration example of the logic circuit 400 according to the first embodiment of the present technology.
  • This logic circuit 400 includes a column interface 401 , an image processing section 402 , an ISP (Image Signal Processor) circuit 403 , a metadata extraction section 404 and a work memory 405 .
  • ISP Image Signal Processor
  • each of the pixels 300 described above holds a pixel signal in synchronization with the vertical synchronization signal XVS.
  • the signal processing unit 261 performs signal processing (AD conversion processing and CDS processing) for converting a pixel signal for each pixel into a digital signal after CDS processing twice in synchronization with the vertical synchronization signal XVS.
  • Each signal processing includes AD conversion processing for the reset level and AD conversion processing and CDS processing for the signal level, so four AD conversion processings and two CDS processings are executed in two signal processings. be.
  • the column interface 401 receives digital signals from the signal processing unit 261 and supplies image data (frames) in which they are arranged to the image processing unit 402 . Since signal processing is performed twice for each pixel in synchronization with the vertical synchronization signal XVS, two frames are generated for each period of the vertical synchronization signal XVS. These frames have the same data, but to distinguish them, the first frame is hereinafter referred to as "F1" and the second frame as "F2".
  • the image processing unit 402 performs various processes such as correction of defective pixels and dark correction on frames from the column interface 401 .
  • This image processing unit 402 processes the first frame F1 and supplies it to the ISP circuit 403 .
  • the ISP circuit 403 performs processing for reducing the amount of data on the frame F1 as necessary. When the processing load on the metadata extraction unit 404 is heavy, the ISP circuit 403 reduces the data amount. For example, downscaling and tone reduction are performed. Further, when the image from the signal processing unit 261 is a Bayer array image and the format of the input image to the metadata extraction unit 404 is a color image with three colors of R, G, and B for each pixel, the ISP circuit 403 Perform demosaic processing to convert the array image to a color image. The ISP circuit 403 supplies the processed frame F ⁇ b>1 to the metadata extraction unit 404 .
  • the metadata extraction unit 404 extracts predetermined metadata from the frame F1.
  • the metadata extraction unit 404 performs, for example, recognition processing for recognizing a predetermined object on the frame F1, and generates data indicating the processing result as metadata.
  • the metadata includes, for example, information as to whether or not the recognition target has been recognized, information as to the type, name, and attributes of the recognized object, information indicating the area of the object, and the like. Attributes include, for example, gender and age.
  • the metadata extraction unit 404 supplies the extracted metadata to the image processing unit 402 .
  • the work memory 405 holds data used in recognition processing. For example, recognition processing is executed by a neural network, and intermediate data exchanged between layers in the neural network is temporarily held in the work memory 405 . Also, weights used in the neural network are stored in advance in the work memory 405 .
  • the image processing unit 402 performs predetermined processing on the second frame F2 using the metadata extracted from the first frame F1. For example, for frame F2, a process of replacing the color of the object region indicated by the metadata with a specific color (eg, black) is performed.
  • the image processing unit 402 supplies the processed frame to the high-speed interface 410 as F2'.
  • each pixel 300 is provided with a sample-and-hold circuit that holds a pixel signal.
  • the signal processing unit 261 performs signal processing (AD conversion processing and CDS processing) on each pixel signal held in the pixel 300 in synchronization with the vertical synchronization signal XVS to generate a digital signal after CDS processing. times.
  • the frame in which the first digital signal is arranged is F1
  • the frame in which the second digital signal is arranged is F2.
  • the first digital signal is an example of the first digital signal described in the claims
  • the second digital signal is an example of the second digital signal described in the claims.
  • the metadata extraction unit 404 extracts metadata from the frame F1 and supplies it to the image processing unit 402.
  • the image processing unit 402 performs predetermined processing on the frame F2 using the metadata of the frame F1.
  • FIG. 10 is a timing chart showing an example of the operation of the solid-state imaging device 200 according to the first embodiment of the present technology.
  • the vertical scanning circuit 211 exposes all the pixels in the pixel array section 220 over the exposure period from timing T0 to T1. Also, at timing T1, the vertical synchronization signal XVS falls.
  • the signal processing unit 261 sequentially reads out the held pixel signals row by row, performs signal processing (AD conversion processing and CDS processing), and generates a frame F1.
  • the image processing unit 402 also performs image processing such as defective pixel correction on the frame F1, and supplies the processed image to the metadata extraction unit 404 via the ISP circuit 403 (not shown).
  • the metadata extraction unit 404 extracts metadata from the frame F1 and supplies it to the image processing unit 402 during the period from timing T2 to timing T3.
  • the signal processing unit 261 sequentially reads out the held pixel signals row by row, performs signal processing, and generates a frame F2. Since the signal processing is performed on the same pixel signals as the first time, the frame F2 has the same data as the frame F1.
  • the image processing unit 402 performs predetermined processing on the frame F2 using the metadata, and supplies the processed frame F2′ to the high-speed interface 410 .
  • the high speed interface 410 outputs the frame F2' to the outside.
  • the vertical synchronization signal XVS falls. Note that when capturing a plurality of frames in synchronization with the vertical synchronization signal XVS, all pixels are exposed within the exposure period from a predetermined timing to timing T4, and the next pixel signal is generated.
  • FIG. 11 is a timing chart showing an example of the operation of the solid-state imaging device 200 in the comparative example.
  • the vertical scanning circuit 211 exposes all the pixels in the pixel array section 220 over the exposure period from timings T0 to T1.
  • the signal processing unit 261 sequentially reads pixel signals row by row, performs signal processing (AD conversion processing and CDS processing), and generates a frame F1.
  • the image processing unit 402 also performs image processing such as defective pixel correction on the frame F1, and supplies the processed image to the metadata extraction unit 404 via the ISP circuit 403 (not shown).
  • the vertical scanning circuit 211 exposes all the pixels in the pixel array section 220 over the exposure period from timings T2 to T3. At timing T3, the vertical synchronizing signal XVS falls. Also, the metadata extraction unit 404 extracts metadata from the frame F1 and supplies it to the image processing unit 402 during the exposure period.
  • the signal processing unit 261 sequentially reads pixel signals row by row, performs signal processing, and generates a frame F2.
  • the pixel signals generated by the exposures at timings T0 to T1 are destroyed by the exposures at timings T2 to T3. Therefore, when a moving object is captured, the frame F2 has different data from the frame F1.
  • the image processing unit 402 performs predetermined processing on the frame F2 using the metadata, and supplies the processed frame F2′ to the high-speed interface 410.
  • the high speed interface 410 outputs the frame F2' to the outside.
  • the image processing unit 402 can perform processing using the metadata on an appropriate region, and can perform appropriate image processing on the frame F2'.
  • FIG. 12 is a diagram showing an example of the first frame and recognition results in the first embodiment of the present technology.
  • a is an example of the first frame 500
  • b in the same figure is a diagram showing an example of the recognition result.
  • a frame 500 includes subjects such as a person 501 and a building 502 .
  • the recognition target is a human being.
  • a person 501 is recognized, and information indicating a rectangular area 510 including the object is generated as metadata.
  • a thick dotted rectangular area 510 in the figure corresponds to the area indicated by the metadata.
  • FIG. 13 is a diagram showing an example of the second frame and the frame after processing in the first embodiment of the present technology.
  • a is an example of a second frame 510
  • b in the figure is an example of a frame 520 processed using metadata.
  • the second signal processing generates a frame 510 that is the same as the first frame 500 .
  • the area 521 indicated by the metadata is filled with a specific color (eg, black).
  • FIG. 14 is a flow chart showing an example of the operation of the solid-state imaging device 200 according to the first embodiment of the present technology. This operation is started, for example, when a predetermined application for capturing a frame is executed.
  • the vertical scanning circuit 211 in the solid-state imaging device 200 exposes all pixels simultaneously (step S901). For each pixel, the sample-and-hold circuit 320 within that pixel samples and holds the pixel signal (step S902).
  • the signal processing unit 261 performs signal processing (AD conversion processing and CDS processing) on each of the held pixel signals to generate the first frame (step S903).
  • the image processing unit 402 performs image processing on the frame (step S904), and the metadata extraction unit 404 extracts metadata from the processed frame (step S905).
  • the signal processing unit 261 performs signal processing on each of the held pixel signals to generate a second frame (step S906).
  • the image processing unit 402 refers to the metadata and determines whether or not the object has been recognized (step S907). When the object is recognized (step S907: Yes), the image processing unit 402 uses the metadata to perform image processing such as painting (step S908). On the other hand, if the target object is not recognized (step S907: No), the image processing unit 402 performs normal image processing such as defective pixel correction without using metadata (step S909). After step S908 or S909, the solid-state imaging device 200 finishes the operation for capturing the frame.
  • steps S901 to S909 are repeatedly executed in synchronization with the vertical synchronization signal XVS.
  • a pixel signal is held for each pixel, and metadata extracted from the first frame is used to generate a second frame that is the same as the first frame. Since the frame is processed, more appropriate image processing can be performed than the comparative example which does not hold the pixel signal.
  • the image processing unit 402 performs a process of filling the area indicated by the metadata with a specific color (eg, black) for the frame F2.
  • a specific color eg, black
  • the process using metadata is not limited to the process of filling in black or the like.
  • the solid-state imaging device 200 in the first modification of the first embodiment differs from the first embodiment in that the area indicated by the metadata is subjected to mosaic processing.
  • FIG. 15 is a diagram showing an example of the second frame and the frame after processing in the first modified example of the first embodiment of the present technology.
  • a is an example of a second frame 510
  • b in the figure is an example of a frame 520 processed using metadata.
  • the image processing unit 402 performs mosaic processing on the area 522 indicated by the metadata.
  • the mosaic processing is performed on the region 522 indicated by the metadata, so that the recognized object can be hidden by the mosaic.
  • the image processing unit 402 performs a process of filling the area indicated by the metadata with a specific color (eg, black) for the frame F2.
  • a specific color eg, black
  • the process using metadata is not limited to the process of filling in black or the like.
  • the solid-state imaging device 200 in the second modification of the first embodiment differs from the first embodiment in that the region indicated by metadata is extracted as a region of interest (ROI).
  • ROI region of interest
  • FIG. 16 is a diagram showing an example of the second frame and the region of interest in the second modified example of the first embodiment of the present technology.
  • a is an example of the second frame 510
  • b in the same figure is an example of the region of interest 530.
  • FIG. 16 is a diagram showing an example of the second frame and the region of interest in the second modified example of the first embodiment of the present technology.
  • a is an example of the second frame 510
  • b in the same figure is an example of the region of interest 530.
  • the image processing unit 402 extracts the region indicated by the metadata from the frame 510 as the region of interest 530 .
  • the metadata for example, detection results of suspicious behavior, possession of dangerous substances, and presence or absence of suspicious substances are used. Then, the image processing unit 402 performs various image processing such as defective pixel correction on the region of interest 530 .
  • FIGS. 13, 15 and 16 the entire human being is targeted for recognition, but only the face can also be targeted for recognition. Objects other than humans, such as vehicle license plates, can also be recognized.
  • the region indicated by the metadata is extracted as a region of interest (ROI). ) can be used to set the ROI.
  • ROI region of interest
  • the selection circuit 330 connects the capacitive elements 321 and 322 in order to the post-stage node 340, but this configuration may not sufficiently suppress noise.
  • the pixel 300 of the second embodiment differs from the first embodiment in that the latter node 340 is initialized when the selection circuit 330 disconnects the capacitive elements 321 and 322 from the latter node 340 .
  • FIG. 17 is a circuit diagram showing one configuration example of the pixel 300 according to the second embodiment of the present technology.
  • the pixel 300 of the second embodiment differs from the first embodiment in that it further includes a post-stage reset transistor 341 .
  • the post-stage reset transistor 341 initializes the level of the post-stage node 340 to a predetermined potential Vreg according to the post-stage reset signal rstb from the vertical scanning circuit 211 .
  • a potential different from the power supply potential VDD (for example, a potential lower than VDD) is set to the potential Vreg.
  • the vertical scanning circuit 211 After reading the reset level, the vertical scanning circuit 211 supplies the high-level post-stage reset signal rstb over the pulse period while keeping the FD reset signal rst and the post-stage selection signal selb of the selected row at high level. As a result, the level of the subsequent node 340 is initialized. At this time, both select transistor 331 and select transistor 332 are in an open state, and capacitive elements 321 and 322 are disconnected from subsequent node 340 .
  • the vertical scanning circuit 211 After initialization of the post-stage node 340, the vertical scanning circuit 211 supplies the high-level selection signal ⁇ s for a predetermined period while keeping the FD reset signal rst and the post-stage selection signal selb of the selected row at high level. Thereby, the capacitive element 322 is connected to the post-stage node 340, and the signal level is read.
  • the selection circuit 330 of the selected row performs control to connect the capacitive element 321 to the post-stage node 340, to disconnect the capacitive elements 321 and 322 from the post-node 340, and to connect the capacitive element 322 to the post-node 340. and control to connect to .
  • the post-stage reset transistor 341 in the selected row initializes the level of the post-stage node 340 .
  • the post-stage circuit 350 of the selected row sequentially reads out the reset level and the signal level from the capacitive elements 321 and 322 via the post-stage node 340 and outputs them to the vertical signal line 309 .
  • FIG. 18 is a timing chart showing an example of global shutter operation according to the second embodiment of the present technology.
  • the vertical scanning circuit 211 supplies high-level FD reset signal rst and transfer signal trg to all rows (in other words, all pixels) from timing T0 immediately before the start of exposure to timing T1 after the pulse period has elapsed. do. As a result, all pixels are PD-reset, and exposure is started simultaneously for all rows.
  • rst_[n] and trg_[n] in the same figure indicate the signals to the n-th row pixels of the N rows.
  • N is an integer indicating the total number of lines, and n is an integer from 1 to N.
  • the vertical scanning circuit 211 supplies the high-level FD reset signal rst over the pulse period while setting the post-stage reset signal rstb and the selection signal ⁇ r to high level in all pixels. .
  • all pixels are FD-reset, and the reset level is sample-held.
  • rstb_[n] and ⁇ r_[n] in the same figure indicate signals to pixels in the n-th row.
  • the vertical scanning circuit 211 returns the selection signal ⁇ r to low level.
  • the vertical scanning circuit 211 supplies the high-level transfer signal trg over the pulse period while setting the post-stage reset signal rstb and the selection signal ⁇ s to high level in all pixels. This samples and holds the signal level. Also, the level of the preceding node 319 drops from the reset level (VDD-Vgs) to the signal level (VDD-Vgs-Vsig).
  • VDD is the power supply voltage
  • Vsig is the net signal level obtained by the CDS process.
  • Vgs is the gate-to-source voltage of the pre-amplification transistor 315 .
  • ⁇ s_[n] in the figure indicates a signal to the n-th pixel.
  • the vertical scanning circuit 211 returns the selection signal ⁇ s to low level.
  • the vertical scanning circuit 211 controls the current source transistors 316 of all rows (all pixels) to supply the current id1.
  • id1_[n] in the figure indicates the current of the n-th pixel.
  • the current id1 needs to be on the order of several nanoamperes (nA) to several tens of nanoamperes (nA).
  • the load MOS transistors 251 of all columns are in the off state, and the current id2 is not supplied to the vertical signal line 309 .
  • FIG. 19 is a timing chart showing an example of read operation in the second embodiment of the present technology.
  • the vertical scanning circuit 211 sets the n-th row FD reset signal rst and the subsequent stage selection signal selb to high level.
  • the post-stage reset signal rstb for all rows is controlled to low level.
  • selb_[n] in the figure indicates a signal to the n-th row pixel.
  • the vertical scanning circuit 211 supplies a high-level selection signal ⁇ r to the n-th row over a period from timing T11 immediately after timing T10 to timing T13.
  • the potential of the post-stage node 340 becomes the reset level Vrst.
  • the DAC 213 gradually raises the ramp signal Rmp over the period from timing T12 to timing T13 after timing T11.
  • the ADC 262 compares the ramp signal Rmp with the level Vrst' of the vertical signal line 309, and counts the count value until the comparison result is inverted. As a result, the P-phase level (reset level) is read.
  • the vertical scanning circuit 211 supplies the high-level post-stage reset signal rstb to the n-th row over the pulse period from timing T14 immediately after timing T13. As a result, when a parasitic capacitance exists in the post-stage node 340, the history of the previous signal held in the parasitic capacitance can be erased.
  • the vertical scanning circuit 211 supplies a high-level selection signal ⁇ s to the n-th row over a period from timing T15 to timing T17 immediately after initialization of the subsequent node 340 .
  • the potential of the post-stage node 340 becomes the signal level Vsig.
  • the signal level was lower than the reset level, but at the time of reading, the signal level becomes higher than the reset level because the latter node 340 is used as a reference.
  • the difference between the reset level Vrst and the signal level Vsig corresponds to the net signal level after removing the FD reset noise and offset noise.
  • the DAC 213 gradually raises the ramp signal Rmp over a period from timing T16 to timing T17 after timing T15.
  • the ADC 262 compares the ramp signal Rmp with the level Vrst' of the vertical signal line 309, and counts the count value until the comparison result is inverted. As a result, the D-phase level (signal level) is read.
  • the vertical scanning circuit 211 controls the current source transistor 316 of the n-th row to be read over the period from timing T10 to timing T17 to supply the current id1. Further, the timing control circuit 212 controls the load MOS transistors 251 of all columns to supply the current id2 during the readout period of all rows.
  • the solid-state imaging device 200 reads the signal level after the reset level, the order is not limited to this. As illustrated in FIG. 20, the solid-state imaging device 200 can also read the reset level after the signal level. In this case, as illustrated in the figure, the vertical scanning circuit 211 supplies the high level selection signal ⁇ r after the high level selection signal ⁇ s. Also, in this case, it is necessary to reverse the slope of the ramp signal.
  • the rear-stage reset transistor 341 initializes the rear-stage node 340 when the selection circuit 330 disconnects the capacitive elements 321 and 322 from the rear-stage node 340 . Since capacitive elements 321 and 322 are separated, the level of reset noise due to their driving is a level corresponding to parasitic capacitance smaller than their capacities. This noise reduction can improve the image quality of the image data.
  • the signal is read while the pre-stage circuit 310 is connected to the pre-stage node 319, but in this configuration, noise from the pre-stage node 319 cannot be blocked during reading.
  • the pixel 300 of the first modified example of the second embodiment differs from the second embodiment in that a transistor is inserted between the pre-stage circuit 310 and the pre-stage node 319 .
  • FIG. 21 is a circuit diagram showing a configuration example of the pixel 300 in the first modified example of the second embodiment of the present technology.
  • a pixel 300 of the first modification of the second embodiment differs from the second embodiment in that it further includes a pre-stage reset transistor 323 and a pre-stage selection transistor 324 .
  • VDD1 is the power supply voltage of the pre-stage circuit 310 and the post-stage circuit 350 of the first modification of the second embodiment.
  • the pre-stage reset transistor 323 initializes the level of the pre-stage node 319 with the power supply voltage VDD2. It is desirable to set this power supply voltage VDD2 to a value that satisfies the following equation.
  • VDD2 VDD1-Vgs Equation 1
  • Vgs is the voltage between the gate and source of the preamplifying transistor 315 .
  • Equation 1 By setting a value that satisfies Equation 1, it is possible to reduce the potential fluctuation between the preceding node 319 and the succeeding node 340 when it is dark. This makes it possible to improve photo response non-uniformity (PRNU).
  • PRNU photo response non-uniformity
  • the front-stage selection transistor 324 opens and closes the path between the front-stage circuit 310 and the front-stage node 319 according to the front-stage selection signal sel from the vertical scanning circuit 211 .
  • FIG. 22 is a timing chart showing an example of global shutter operation in the first modified example of the second embodiment of the present technology.
  • the timing chart of the first modification of the second embodiment differs from that of the first embodiment in that the vertical scanning circuit 211 further supplies the previous stage reset signal rsta and the previous stage selection signal sel.
  • rsta_[n] and sel_[n] denote signals to pixels in the nth row.
  • the vertical scanning circuit 211 supplies a high-level pre-stage selection signal sel to all pixels from timing T2 immediately before the end of exposure to timing T5.
  • the previous stage reset signal rsta is controlled to a low level.
  • FIG. 23 is a timing chart showing an example of read operation in the first modified example of the second embodiment of the present technology.
  • the previous stage selection signal sel is controlled to a low level.
  • the pre-stage selection transistor 324 shifts to the open state, and the pre-stage node 319 is disconnected from the pre-stage circuit 310 .
  • noise from the preceding node 319 can be cut off during reading.
  • the vertical scanning circuit 211 supplies the high-level pre-stage reset signal rsta to the n-th row.
  • the vertical scanning circuit 211 controls the current source transistors 316 of all pixels to stop supplying the current id1.
  • Current id2 is supplied in the same manner as in the second embodiment.
  • control of the current id1 becomes simpler than in the second embodiment.
  • the pre-stage selection transistor 324 transitions to the open state during reading to disconnect the pre-stage circuit 310 from the pre-stage node 319 .
  • Noise from the circuit 310 can be blocked.
  • the circuits in the solid-state imaging device 200 are provided on a single semiconductor chip.
  • the solid-state imaging device 200 of the second modification of the second embodiment differs from the second embodiment in that the circuits in the solid-state imaging device 200 are distributed over two semiconductor chips.
  • FIG. 24 is a diagram showing an example of the layered structure of the solid-state imaging device 200 in the second modified example of the second embodiment of the present technology.
  • a solid-state imaging device 200 of a second modification of the second embodiment includes a lower pixel chip 202 and an upper pixel chip 201 stacked on the lower pixel chip 202 . These chips are electrically connected, for example, by Cu--Cu bonding. In addition to Cu--Cu bonding, vias and bumps can also be used for connection.
  • An upper pixel array section 221 is arranged in the upper pixel chip 201 .
  • a lower pixel array section 222 and a column signal processing circuit 260 are arranged in the lower pixel chip 202 .
  • Some of the pixels in the pixel array section 220 are arranged in the upper pixel array section 221 and the rest are arranged in the lower pixel array section 222 .
  • a vertical scanning circuit 211 , a timing control circuit 212 , a DAC 213 and a load MOS circuit block 250 are also arranged in the lower pixel chip 202 . These circuits are omitted in the figure.
  • the upper pixel chip 201 is manufactured by, for example, a process dedicated to pixels
  • the lower pixel chip 202 is manufactured by, for example, a CMOS (Complementary MOS) process.
  • CMOS Complementary MOS
  • FIG. 25 is a circuit diagram showing a configuration example of the pixel 300 in the second modified example of the second embodiment of the present technology.
  • the front-stage circuit 310 is arranged on the upper pixel chip 201
  • the other circuits and elements are arranged on the lower pixel chip 202 .
  • the current source transistor 316 can also be placed further on the lower pixel chip 202 .
  • the area of the pixel can be reduced and the pixel can be miniaturized. becomes easier.
  • the circuits and elements in the pixel 300 are distributed over two semiconductor chips, so that the pixel can be easily miniaturized. Become.
  • FIG. 26 is a diagram showing an example of the layered structure of the solid-state imaging device 200 according to the third modification of the second embodiment of the present technology.
  • a solid-state imaging device 200 of the third modification of the second embodiment includes an upper pixel chip 201 , a lower pixel chip 202 and a circuit chip 203 . These chips are stacked and electrically connected, for example, by Cu--Cu bonding. In addition to Cu--Cu bonding, vias and bumps can also be used for connection.
  • An upper pixel array section 221 is arranged in the upper pixel chip 201 .
  • a lower pixel array section 222 is arranged in the lower pixel chip 202 .
  • Some of the pixels in the pixel array section 220 are arranged in the upper pixel array section 221 and the rest are arranged in the lower pixel array section 222 .
  • a column signal processing circuit 260 In the circuit chip 203, a column signal processing circuit 260, a vertical scanning circuit 211, a timing control circuit 212, a DAC 213 and a load MOS circuit block 250 are arranged. Circuits other than the column signal processing circuit 260 are omitted in the figure.
  • the lower pixel chip 202 of the second layer can be manufactured by a dedicated process for capacitors and switches.
  • the circuits in the solid-state imaging device 200 are distributed over the three semiconductor chips, so that the circuits are distributed over the two semiconductor chips. Pixels can be further miniaturized as compared with the case where
  • the reset level is sampled and held within the exposure period, but in this configuration, the exposure period cannot be made shorter than the reset level sample and hold period.
  • the solid-state imaging device 200 of the third embodiment differs from that of the second embodiment in that the exposure period is made shorter by adding a transistor for discharging charges from the photoelectric conversion element.
  • FIG. 27 is a circuit diagram showing one configuration example of the pixel 300 according to the third embodiment of the present technology.
  • the pixel 300 of this third embodiment differs from that of the second embodiment in that it further includes a discharge transistor 317 in the pre-stage circuit 310 .
  • the discharge transistor 317 functions as an overflow drain that discharges charges from the photoelectric conversion element 311 according to the discharge signal ofg from the vertical scanning circuit 211 .
  • An nMOS transistor, for example, is used as the discharge transistor 317 .
  • blooming may occur when charges are transferred from the photoelectric conversion element 311 to the FD 314 for all pixels. Then, the potentials of the FD 314 and the previous stage node 319 drop when the FD is reset. Following this potential drop, currents for charging and discharging the capacitative elements 321 and 322 continue to be generated, and the IR drop of the power supply and ground changes from the steady state without blooming.
  • the discharge transistor 317 the charge of the photoelectric conversion element 311 is discharged to the overflow drain side. Therefore, the IR drop at the time of sampling and holding the reset level and the signal level is approximately the same, and streaking noise can be suppressed.
  • FIG. 28 is a timing chart showing an example of global shutter operation according to the third embodiment of the present technology.
  • the vertical scanning circuit 211 supplies the FD reset signal rst of high level to all the pixels for the pulse period while setting the discharge signal fg of all pixels to high level.
  • PD reset and FD reset are performed for all pixels.
  • the reset level is sample-held.
  • ?fg_[n] in the same figure indicates the signal to the pixel of the n-th row among the N rows.
  • the vertical scanning circuit 211 returns the discharge signal THERfg of all pixels to low level. Then, the vertical scanning circuit 211 supplies a high-level transfer signal trg to all pixels over a period from timing T2 immediately before the end of exposure to T3 at the end of exposure. This samples and holds the signal level.
  • both the transfer transistor 312 and the FD reset transistor 313 must be turned on at the start of exposure (that is, at PD reset).
  • the FD 314 must be reset at the same time when the PD is reset. Therefore, it is necessary to reset the FD again within the exposure period and sample and hold the reset level, and the exposure period cannot be shorter than the sample and hold period of the reset level.
  • a certain amount of waiting time is required until the voltage and current stabilize. A period is required.
  • the reset level can be sample-held by performing the FD reset before releasing the PD reset (starting exposure). As a result, the exposure period can be made shorter than the sample-and-hold period of the reset level.
  • the first to third modifications of the second embodiment can also be applied to the third embodiment.
  • the discharge transistor 317 that discharges the charge from the photoelectric conversion element 311 since the discharge transistor 317 that discharges the charge from the photoelectric conversion element 311 is provided, it is possible to perform the FD reset and sample and hold the reset level before the start of exposure. can. As a result, the exposure period can be made shorter than the sample-and-hold period of the reset level.
  • the FD 314 is initialized by the power supply voltage VDD.
  • PRNU sensitivity non-uniformity
  • the solid-state imaging device 200 of the fourth embodiment differs from the second embodiment in that PRNU is improved by lowering the power supply of the FD reset transistor 313 during reading.
  • FIG. 29 is a circuit diagram showing one configuration example of the pixel 300 according to the fourth embodiment of the present technology.
  • the pixel 300 of the fourth embodiment differs from the second embodiment in that the power supply of the FD reset transistor 313 is separated from the power supply voltage VDD of the pixel 300.
  • FIG. 29 is a circuit diagram showing one configuration example of the pixel 300 according to the fourth embodiment of the present technology.
  • the pixel 300 of the fourth embodiment differs from the second embodiment in that the power supply of the FD reset transistor 313 is separated from the power supply voltage VDD of the pixel 300.
  • the drain of the FD reset transistor 313 of the fourth embodiment is connected to the reset power supply voltage VRST.
  • This reset power supply voltage VRST is controlled by the timing control circuit 212, for example.
  • the potential of the FD 314 decreases due to the reset feedthrough of the FD reset transistor 313 at timing T0 immediately before the start of exposure, as illustrated in FIG. This fluctuation amount is assumed to be Vft.
  • the potential of the FD 314 changes from VDD to VDD-Vft at timing T0. Also, the potential of the previous stage node 319 during exposure is VDD-Vft-Vsig.
  • the FD reset transistor 313 is turned on during reading, and the FD 314 is fixed to the power supply voltage VDD. Due to the amount of variation Vft of FD 314, the potentials of pre-stage node 319 and post-stage node 340 in reading are shifted higher by about Vft. However, due to variations in the capacitance values of the capacitive elements 321 and 322 and parasitic capacitance, the amount of voltage to be shifted varies from pixel to pixel, resulting in deterioration of PRNU.
  • the transition amount of the subsequent node 340 when the preceding node 319 transitions by Vft is expressed by, for example, the following equation. ⁇ (Cs+ ⁇ Cs)/(Cs+ ⁇ Cs+Cp) ⁇ *Vft Equation 2
  • Cs is the capacitance value of the capacitive element 322 on the signal level side
  • ⁇ Cs is the variation of Cs
  • Cp is the capacitance value of the parasitic capacitance of the post-stage node 340 .
  • Equation 2 can be approximated by the following equation. ⁇ 1 ⁇ ( ⁇ Cs/Cs)*(Cp/Cs) ⁇ *Vft Equation 3
  • Equation 4 the variation of the post-stage node 340 can be expressed by the following equation. ⁇ ( ⁇ Cs/Cs)*(Cp/Cs) ⁇ *Vft Equation 4
  • FIG. 32 is a timing chart showing an example of voltage control in the fourth embodiment of the present technology.
  • the timing control circuit 212 controls the reset power supply voltage VRST to a value different from that during the exposure period during the row-by-row readout period after timing T9.
  • the timing control circuit 212 sets the reset power supply voltage VRST to the same value as the power supply voltage VDD.
  • the timing control circuit 212 reduces the reset power supply voltage VRST to VDD-Vft. That is, in the read period, the timing control circuit 212 reduces the reset power supply voltage VRST by an amount that substantially matches the variation Vft due to the reset feedthrough. With this control, the reset level of the FD 314 can be made uniform at the time of exposure and at the time of readout.
  • the timing control circuit 212 reduces the reset power supply voltage VRST by the fluctuation amount Vft due to the reset feedthrough at the time of reading. You can level up. This makes it possible to suppress deterioration of sensitivity non-uniformity (PRNU).
  • the signal level is read after the reset level for each frame.
  • sensitivity non-uniformity PRNU
  • PRNU sensitivity non-uniformity
  • the solid-state imaging device 200 of the fifth embodiment differs from the second embodiment in improving PRNU by exchanging the level held by the capacitive element 321 and the level held by the capacitative element 322 for each frame. Different from the form.
  • the solid-state imaging device 200 of the fifth embodiment continuously images a plurality of frames in synchronization with the vertical synchronization signal.
  • the odd-numbered frames are called “odd-numbered frames”, and the even-numbered frames are called “even-numbered frames”.
  • FIG. 33 is a timing chart showing an example of global shutter operation for odd frames in the fifth embodiment.
  • the pre-stage circuit 310 in the solid-state imaging device 200 causes the capacitive element 321 to hold the reset level by setting the selection signal ⁇ r and then the selection signal ⁇ s to high level, and then changes the signal level. It is held by the capacitor 322 .
  • FIG. 34 is a timing chart showing an example of the odd-numbered frame readout operation according to the fifth embodiment of the present technology.
  • the post-stage circuit 350 in the solid-state imaging device 200 sets the selection signal ⁇ r to the high level, then the selection signal ⁇ s, and reads the signal level after the reset level.
  • FIG. 35 is a timing chart showing an example of global shutter operation for even-numbered frames in the fifth embodiment.
  • the pre-stage circuit 310 in the solid-state imaging device 200 causes the capacitive element 322 to hold the reset level by setting the selection signal ⁇ s and then the selection signal ⁇ r to high level, and then changes the signal level. It is held in the capacitor 321 .
  • FIG. 36 is a timing chart showing an example of the even-numbered frame readout operation according to the fifth embodiment of the present technology.
  • the post-stage circuit 350 in the solid-state imaging device 200 sets the selection signal ⁇ s to the high level, then the selection signal ⁇ r, and reads the signal level after the reset level.
  • the levels held in the capacitive elements 321 and 322 are reversed between even-numbered frames and odd-numbered frames.
  • the polarity of the PRNU is also reversed between even and odd frames.
  • the post-stage column signal processing circuit 260 obtains the arithmetic mean of the odd-numbered frames and the even-numbered frames. This allows PRNUs with opposite polarities to cancel each other out.
  • This control is effective for capturing moving images and adding frames. In addition, it is possible to realize this by only changing the driving method without adding an element to the pixel 300 .
  • the level held in the capacitive element 321 and the level held in the capacitative element 322 are opposite between the odd frame and the even frame.
  • the polarity of PRNU can be reversed between frames. By adding these odd and even frames by the column signal processing circuit 260, deterioration of PRNU can be suppressed.
  • the column signal processing circuit 260 obtains the difference between the reset level and the signal level for each column.
  • the charge overflows from the photoelectric conversion element 311, which may cause a black spot phenomenon in which the brightness is lowered and the pixel is blackened.
  • the solid-state imaging device 200 of the sixth embodiment differs from that of the second embodiment in that whether or not the black spot phenomenon has occurred is determined for each pixel.
  • FIG. 37 is a circuit diagram showing one configuration example of the column signal processing circuit 260 according to the sixth embodiment of the present technology.
  • a plurality of ADCs 270 and a logic circuit 400 are arranged in the column signal processing circuit 260 of the sixth embodiment.
  • a plurality of CDS processing units 291 and a plurality of selectors 292 are arranged in the logic circuit 400 .
  • ADC 270, CDS processing unit 291 and selector 292 are provided for each column.
  • Other circuits (such as the metadata extractor 404) in the logic circuit 400 are omitted in FIG.
  • the ADC 270 also includes a comparator 280 and a counter 271 .
  • the comparator 280 compares the level of the vertical signal line 309 with the ramp signal Rmp from the DAC 213 and outputs the comparison result VCO.
  • a comparison result VCO is supplied to the counter 271 and the timing control circuit 212 .
  • Comparator 280 includes selector 281 , capacitive elements 282 and 283 , auto-zero switches 284 and 286 , and comparator 285 .
  • the selector 281 connects either the vertical signal line 309 of the corresponding column or the node of the predetermined reference voltage VREF to the non-inverting input terminal (+) of the comparator 285 according to the input-side selection signal selin, and the capacitive element 282. It connects through The input side selection signal selin is supplied from the timing control circuit 212 . Note that the selector 281 is an example of an input-side selector described in the claims.
  • the comparator 285 compares the levels of the non-inverting input terminal (+) and the inverting input terminal (-) and outputs the comparison result VCO to the counter 271 .
  • a ramp signal Rmp is input to the inverting input terminal (-) via the capacitive element 283 .
  • the auto-zero switch 284 short-circuits the non-inverting input terminal (+) and the output terminal of the comparison result VCO according to the auto-zero signal Az from the timing control circuit 212 .
  • the auto-zero switch 286 short-circuits the inverting input terminal (-) and the output terminal of the comparison result VCO according to the auto-zero signal Az.
  • the counter 271 counts the count value until the comparison result VCO is inverted, and outputs a digital signal CNT_out indicating the count value to the CDS processing section 291 .
  • the CDS processing unit 291 performs CDS processing on the digital signal CNT_out.
  • the CDS processing unit 291 calculates the difference between the digital signal CNT_out corresponding to the reset level and the digital signal CNT_out corresponding to the signal level, and outputs the difference as CDS_out to the selector 292 .
  • the selector 292 outputs either the CDS-processed digital signal CDS_out or the full-code digital signal FULL as the pixel data of the corresponding column according to the output-side selection signal selout from the timing control circuit 212 .
  • the selector 292 is an example of an output-side selector described in the claims.
  • FIG. 38 is a timing chart showing an example of global shutter operation according to the sixth embodiment of the present technology.
  • the method of controlling the transistors during the global shutter in the sixth embodiment is the same as in the second embodiment.
  • the dashed-dotted line in the figure shows the potential variation of the FD 314 when weak sunlight is incident so that the amount of overflowed charge is relatively small.
  • the dotted line in FIG. 3 indicates the potential fluctuation of the FD 314 when strong sunlight is incident so that the amount of overflowed charge is relatively large.
  • the reset level is lowered at timing T3 when the FD reset is completed, but the level is not lowered at this point.
  • the reset level drops completely at timing T3.
  • the signal level is the same as the reset level, and the potential difference between them is "0", so the digital signal after CDS processing is the same as in the dark state and darkens.
  • a phenomenon in which a pixel becomes black even when very high illuminance light such as sunlight is incident is called a black spot phenomenon or blooming.
  • the operating point of the pre-stage circuit 310 cannot be secured, and the current id1 of the current source transistor 316 fluctuates. Since the current source transistor 316 of each pixel is connected to a common power supply and ground, when the current fluctuates in one pixel, the IR drop fluctuation of that pixel affects the sample level of other pixels. end up A pixel where the black dot phenomenon occurs becomes an aggressor, and a pixel whose sample level changes due to that pixel becomes a victim. This results in streaking noise.
  • the black dot phenomenon is less likely to occur in a pixel with a black dot (blooming) because the overflowing charge is discarded to the drain transistor 317 side.
  • the discharge transistor 317 even if the discharge transistor 317 is provided, there is a possibility that part of the charge will flow to the FD 314, and the black spot phenomenon may not be eradicated.
  • the addition of the discharge transistor 317 has the disadvantage that the effective area/charge ratio for each pixel is reduced. Therefore, it is desirable to suppress the black spot phenomenon without using the discharge transistor 317 .
  • the first is adjustment of the clip level of the FD 314 .
  • the second method is to judge whether or not a black dot phenomenon has occurred during reading, and replace the output with a full code when the black dot phenomenon has occurred.
  • the high level of the FD reset signal rst (in other words, the gate of the FD reset transistor 313) in FIG.
  • the difference (ie amplitude) between these high and low levels is set to a value corresponding to the dynamic range.
  • the value is adjusted to a value obtained by adding a margin to that value.
  • the value corresponding to the dynamic range corresponds to the difference between the power supply voltage VDD and the potential of the FD 314 when the digital signal becomes full code.
  • the dynamic range changes depending on the analog gain of the ADC.
  • a low analog gain requires a large dynamic range, while a high analog gain requires a small dynamic range. Therefore, the gate voltage when the FD reset transistor 313 is turned off can be changed according to the analog gain.
  • FIG. 39 is a timing chart showing an example of read operation in the sixth embodiment of the present technology.
  • the selection signal ⁇ r becomes high level at the timing T11 immediately after the readout start timing T10
  • the potential of the vertical signal line 309 fluctuates in the pixel on which sunlight is incident.
  • the dashed-dotted line in FIG. 4 indicates the potential fluctuation of the vertical signal line 309 when weak sunlight is incident.
  • a dotted line in the figure indicates the potential fluctuation of the vertical signal line 309 when strong sunlight is incident.
  • the timing control circuit 212 supplies, for example, the input side selection signal selin of "0" to connect the comparator 285 to the vertical signal line 309. During this auto-zero period, the timing control circuit 212 performs auto-zero with the auto-zero signal Az.
  • the timing control circuit 212 supplies, for example, the input side selection signal selin of "1" within the determination period from timing T12 to timing T13.
  • the input side selection signal selin disconnects the comparator 285 from the vertical signal line 309 and connects it to the node of the reference voltage VREF.
  • This reference voltage VREF is set to the expected value of the level of the vertical signal line 309 when no blooming occurs.
  • Vrst corresponds to, for example, Vreg-Vgs2, where Vgs2 is the gate-source voltage of the rear-stage amplifying transistor 351 .
  • the DAC 213 reduces the level of the ramp signal Rmp from Vrmp_az to Vrmp_sun within the determination period.
  • the reset level Vrst of the vertical signal line 309 is substantially the same as the reference voltage VREF, and the potential of the inverting input terminal (+) of the comparator 285 is autozero. Not much different from time to time.
  • the comparison result VCO becomes high level.
  • the timing control circuit 212 can determine whether blooming has occurred based on whether the comparison result VCO becomes low level within the determination period.
  • the timing control circuit 212 connects the comparator 285 to the vertical signal line 309 after timing T13 after the determination period has elapsed. Further, after the P-phase settling period of timings T13 to T14 has passed, the P-phase is read out during the period of timings T14 to T15. After the D-phase settling period of timings T15 to T19 elapses, the D-phase is read out during the period of timings T19 to T20.
  • the timing control circuit 212 controls the selector 292 with the output side selection signal selout to output the digital signal CDS_out after the CDS processing as it is.
  • the timing control circuit 212 controls the selector 292 with the output side selection signal selout to output the full code FULL instead of the CDS-processed digital signal CDS_out. Thereby, the black spot phenomenon can be suppressed.
  • the timing control circuit 212 determines whether or not the black spot phenomenon has occurred based on the comparison result VCO, and outputs the full code when the black spot phenomenon has occurred. Since it is output, the black spot phenomenon can be suppressed.
  • the vertical scanning circuit 211 performs control (that is, global shutter operation) to simultaneously expose all rows (all pixels).
  • control that is, global shutter operation
  • the solid-state imaging device 200 of the seventh embodiment differs from that of the second embodiment in that it performs a rolling shutter operation during testing.
  • FIG. 40 is a timing chart showing an example of rolling shutter operation according to the seventh embodiment of the present technology.
  • the vertical scanning circuit 211 performs control to sequentially select a plurality of rows and start exposure.
  • the figure shows the exposure control of the n-th row.
  • the vertical scanning circuit 211 supplies the n-th row with the high-level post-stage selection signal selb, the selection signal ⁇ r, and the selection signal ⁇ s. Also, at the timing T0 of exposure start, the vertical scanning circuit 211 supplies the high-level FD reset signal rst and the post-stage reset signal rstb to the n-th row over the pulse period. The vertical scanning circuit 211 supplies the transfer signal trg to the n-th row at timing T1 when exposure ends.
  • the solid-state imaging device 200 can generate low-noise image data by the rolling shutter operation shown in FIG. Note that in the rolling shutter operation of FIG. 11, AD conversion twice and metadata extraction described in the first embodiment are not executed.
  • the solid-state imaging device 200 of the seventh embodiment performs a global shutter operation during normal imaging as in the second embodiment.
  • the vertical scanning circuit 211 performs control (that is, rolling shutter operation) to sequentially select a plurality of rows and start exposure. data can be generated.
  • the source of the source follower in the preceding stage (the amplifying transistor 315 in the preceding stage and the current source transistor 316) is connected to the power supply voltage VDD, and reading is performed row by row while the source follower is on. Ta.
  • the circuit noise of the source follower in the preceding stage propagates to the succeeding stage during readout in units of rows, and there is a possibility that the random noise increases.
  • the solid-state imaging device 200 of the eighth embodiment differs from that of the first embodiment in that noise is reduced by turning off the source follower in the preceding stage during readout.
  • FIG. 41 is a block diagram showing a configuration example of the solid-state imaging device 200 according to the eighth embodiment of the present technology.
  • the solid-state imaging device 200 of the eighth embodiment differs from that of the second embodiment in that a regulator 420 and a switching section 440 are further provided.
  • a plurality of effective pixels 301 and a predetermined number of dummy pixels 430 are arranged in the pixel array section 220 of the eighth embodiment.
  • the dummy pixels 430 are arranged around the area where the effective pixels 301 are arranged.
  • each of the dummy pixels 430 is supplied with the power supply voltage VDD
  • each of the effective pixels 301 is supplied with the power supply voltage VDD and the source voltage Vs.
  • a signal line for supplying the power supply voltage VDD to the effective pixels 301 is omitted in FIG.
  • the power supply voltage VDD is supplied from a pad 415 outside the solid-state imaging device 200 .
  • the regulator 420 generates a constant generation voltage V gen based on the input potential Vi from the dummy pixel 430 and supplies it to the switching section 440 .
  • the switching unit 440 selects either the power supply voltage VDD from the pad 415 or the generated voltage V gen from the regulator 420 and supplies it as the source voltage Vs to each column of the effective pixels 301 .
  • FIG. 42 is a circuit diagram showing one configuration example of the dummy pixel 430, the regulator 420, and the switching section 440 according to the eighth embodiment of the present technology.
  • a is a circuit diagram of the dummy pixel 430 and the regulator 420
  • b is a circuit diagram of the switching section 440 .
  • the dummy pixel 430 includes a reset transistor 431, an FD 432, an amplification transistor 433 and a current source transistor 434.
  • the reset transistor 431 initializes the FD 432 according to the reset signal RST from the vertical scanning circuit 211 .
  • the FD 432 accumulates charges and generates a voltage corresponding to the amount of charges.
  • the amplification transistor 433 amplifies the voltage level of the FD 432 and supplies it to the regulator 420 as an input voltage Vi.
  • the sources of the reset transistor 431 and the amplification transistor 433 are connected to the power supply voltage VDD.
  • Current source transistor 434 is connected to the drain of amplification transistor 433 . This current source transistor 434 supplies the current id1 under the control of the vertical scanning circuit 211 .
  • the regulator 420 includes a low-pass filter 421, a buffer amplifier 422 and a capacitive element 423.
  • the low-pass filter 421 passes, as an output voltage Vj, components of a low frequency band below a predetermined frequency in the signal of the input voltage Vi.
  • the output voltage Vj is input to the non-inverting input terminal (+) of the buffer amplifier 422 .
  • the inverting input terminal (-) of buffer amplifier 422 is connected to its output terminal.
  • the capacitive element 423 holds the voltage of the output terminal of the buffer amplifier 422 as Vgen .
  • This V gen is supplied to the switching section 440 .
  • the switching section 440 includes an inverter 441 and a plurality of switching circuits 442 .
  • a switching circuit 442 is arranged for each column of the effective pixels 301 .
  • the inverter 441 inverts the switching signal SW from the timing control circuit 212 . This inverter 441 supplies an inverted signal to each of the switching circuits 442 .
  • the switching circuit 442 selects either the power supply voltage VDD or the generated voltage V gen and supplies it to the corresponding column in the pixel array section 220 as the source voltage Vs.
  • the switching circuit 442 includes switches 443 and 444 .
  • the switch 443 opens and closes the path between the node of the power supply voltage VDD and the corresponding column according to the switching signal SW.
  • the switch 444 opens and closes the path between the node of the generated voltage V gen and the corresponding column according to the inverted signal of the switching signal SW.
  • FIG. 43 is a timing chart showing an example of operations of the dummy pixel 430 and the regulator 420 according to the eighth embodiment of the present technology.
  • the vertical scanning circuit 211 supplies a reset signal RST of high level (here, power supply voltage VDD) to each dummy pixel 430 .
  • the potential Vfd of the FD 432 within the dummy pixel 430 is initialized to the power supply voltage VDD. Then, when the reset signal RST becomes low level, it changes to VDD-Vft due to the reset feedthrough.
  • the input voltage Vi drops to VDD-Vgs-Vsig after reset.
  • Vj and Vgen become substantially constant voltages.
  • FIG. 44 is a circuit diagram showing one configuration example of the effective pixel 301 according to the eighth embodiment of the present technology.
  • the circuit configuration of the effective pixel 301 is the same as that of the pixel 300 of the second embodiment, except that the source of the preamplifying transistor 315 is supplied with the source voltage Vs from the switching unit 440 .
  • FIG. 45 is a timing chart showing an example of global shutter operation in the eighth embodiment of the present technology.
  • the switching unit 440 selects the power supply voltage VDD and supplies it as the source voltage Vs. Also, the voltage of the preceding node drops from VDD-Vgs-Vth to VDD-Vgs-Vsig at timing T4.
  • Vth is the threshold voltage of the transfer transistor 312 .
  • FIG. 46 is a timing chart showing an example of read operation in the eighth embodiment of the present technology.
  • the switching unit 440 selects the generated voltage V gen during reading and supplies it as the source voltage Vs. This generated voltage V gen is adjusted to VDD-Vgs-Vft. Further, in the eighth embodiment, the vertical scanning circuit 211 controls the current source transistors 316 of all rows (all pixels) to stop supplying the current id1.
  • FIG. 47 is a diagram for explaining the effects of the eighth embodiment of the present technology.
  • the source follower the front-stage amplification transistor 315 and the current source transistor 316
  • the circuit noise of the source follower in the preceding stage may propagate to the subsequent stage (the capacitive element, the source follower in the subsequent stage, and the ADC), increasing the readout noise.
  • kTC noise generated in pixels during global shutter operation is 450 ( ⁇ Vrms) as illustrated in the figure.
  • the noise generated in the source follower in the preceding stage (the amplifying transistor 315 in the preceding stage and the current source transistor 316) in reading for each row is 380 ( ⁇ Vrms).
  • the noise generated after the source follower in the latter stage is 160 ( ⁇ Vrms). Therefore, the total noise is 610 ( ⁇ Vrms).
  • the noise contribution of the preceding source follower in the total noise value is relatively large.
  • the source of the source follower in the preceding stage is supplied with an adjustable voltage (Vs) as described above.
  • Vs adjustable voltage
  • the switching unit 440 selects the power supply voltage VDD and supplies it as the source voltage Vs. After the exposure ends, the switching unit 440 switches the source voltage Vs to VDD-Vgs-Vft.
  • the timing control circuit 212 turns on the current source transistor 316 in the previous stage during the global shutter (exposure) operation, and turns it off after the end of the exposure.
  • the potentials of the front-stage nodes during global shutter operation and during readout for each row are uniform, and PRNU can be improved.
  • the source follower in the previous stage is turned off when reading out each row, the circuit noise of the source follower does not occur and becomes 0 ( ⁇ Vrms) as shown in FIG. Note that the front-stage amplifying transistor 315 of the front-stage source follower is in the ON state.
  • the technology (the present technology) according to the present disclosure can be applied to various products.
  • the technology according to the present disclosure can be realized as a device mounted on any type of moving body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility, airplanes, drones, ships, and robots. may
  • FIG. 48 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology according to the present disclosure can be applied.
  • a vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001.
  • vehicle control system 12000 includes drive system control unit 12010 , body system control unit 12020 , vehicle exterior information detection unit 12030 , vehicle interior information detection unit 12040 , and integrated control unit 12050 .
  • integrated control unit 12050 As the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio/image output unit 12052, and an in-vehicle network I/F (interface) 12053 are illustrated.
  • the drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs.
  • the driving system control unit 12010 includes a driving force generator for generating driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism to adjust and a brake device to generate braking force of the vehicle.
  • the body system control unit 12020 controls the operation of various devices equipped on the vehicle body according to various programs.
  • the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, winkers or fog lamps.
  • body system control unit 12020 can receive radio waves transmitted from a portable device that substitutes for a key or signals from various switches.
  • the body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, etc. of the vehicle.
  • the vehicle exterior information detection unit 12030 detects information outside the vehicle in which the vehicle control system 12000 is installed.
  • the vehicle exterior information detection unit 12030 is connected with an imaging section 12031 .
  • the vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image of the exterior of the vehicle, and receives the captured image.
  • the vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as people, vehicles, obstacles, signs, or characters on the road surface based on the received image.
  • the imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of received light.
  • the imaging unit 12031 can output the electric signal as an image, and can also output it as distance measurement information.
  • the light received by the imaging unit 12031 may be visible light or non-visible light such as infrared rays.
  • the in-vehicle information detection unit 12040 detects in-vehicle information.
  • the in-vehicle information detection unit 12040 is connected to, for example, a driver state detection section 12041 that detects the state of the driver.
  • the driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 detects the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing off.
  • the microcomputer 12051 calculates control target values for the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and controls the drive system control unit.
  • a control command can be output to 12010 .
  • the microcomputer 12051 realizes the functions of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane deviation warning. Cooperative control can be performed for the purpose of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane deviation warning. Cooperative control can be performed for the purpose of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle
  • the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, etc. based on the information about the vehicle surroundings acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, so that the driver's Cooperative control can be performed for the purpose of autonomous driving, etc., in which vehicles autonomously travel without depending on operation.
  • the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the information detection unit 12030 outside the vehicle.
  • the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the vehicle exterior information detection unit 12030, and performs cooperative control aimed at anti-glare such as switching from high beam to low beam. It can be carried out.
  • the audio/image output unit 12052 transmits at least one of audio and/or image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle.
  • an audio speaker 12061, a display section 12062 and an instrument panel 12063 are illustrated as output devices.
  • the display unit 12062 may include at least one of an on-board display and a head-up display, for example.
  • FIG. 49 is a diagram showing an example of the installation position of the imaging unit 12031.
  • the imaging unit 12031 has imaging units 12101, 12102, 12103, 12104, and 12105.
  • the imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose of the vehicle 12100, the side mirrors, the rear bumper, the back door, and the upper part of the windshield in the vehicle interior, for example.
  • An image pickup unit 12101 provided in the front nose and an image pickup unit 12105 provided above the windshield in the passenger compartment mainly acquire images in front of the vehicle 12100 .
  • Imaging units 12102 and 12103 provided in the side mirrors mainly acquire side images of the vehicle 12100 .
  • An imaging unit 12104 provided in the rear bumper or back door mainly acquires an image behind the vehicle 12100 .
  • the imaging unit 12105 provided above the windshield in the passenger compartment is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.
  • FIG. 49 shows an example of the imaging range of the imaging units 12101 to 12104.
  • the imaging range 12111 indicates the imaging range of the imaging unit 12101 provided in the front nose
  • the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided in the side mirrors, respectively
  • the imaging range 12114 The imaging range of an imaging unit 12104 provided in the rear bumper or back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.
  • At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information.
  • at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.
  • the microcomputer 12051 determines the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and changes in this distance over time (relative velocity with respect to the vehicle 12100). , it is possible to extract, as the preceding vehicle, the closest three-dimensional object on the course of the vehicle 12100, which runs at a predetermined speed (for example, 0 km/h or more) in substantially the same direction as the vehicle 12100. can. Furthermore, the microcomputer 12051 can set the inter-vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including following stop control) and automatic acceleration control (including following start control). In this way, cooperative control can be performed for the purpose of automatic driving in which the vehicle runs autonomously without relying on the operation of the driver.
  • automatic brake control including following stop control
  • automatic acceleration control including following start control
  • the microcomputer 12051 converts three-dimensional object data related to three-dimensional objects to other three-dimensional objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, and utility poles. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into those that are visible to the driver of the vehicle 12100 and those that are difficult to see. Then, the microcomputer 12051 judges the collision risk indicating the degree of danger of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, an audio speaker 12061 and a display unit 12062 are displayed. By outputting an alarm to the driver via the drive system control unit 12010 and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be performed.
  • At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays.
  • the microcomputer 12051 can recognize a pedestrian by determining whether or not the pedestrian exists in the captured images of the imaging units 12101 to 12104 .
  • recognition of a pedestrian is performed by, for example, a procedure for extracting feature points in images captured by the imaging units 12101 to 12104 as infrared cameras, and performing pattern matching processing on a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian.
  • the audio image output unit 12052 outputs a rectangular outline for emphasis to the recognized pedestrian. is superimposed on the display unit 12062 . Also, the audio/image output unit 12052 may control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.
  • the technology according to the present disclosure can be applied to the imaging unit 12031 among the configurations described above.
  • the imaging device 100 in FIG. 1 can be applied to the imaging unit 12031 .
  • the technology according to the present disclosure it is possible to obtain a captured image that is easier to see, thereby reducing driver fatigue.
  • the present technology can also have the following configuration.
  • the metadata extracting unit performs recognition processing for recognizing a predetermined object on the frame in which the first digital signal is arranged, and generates data indicating a result of the recognition processing as the metadata.
  • the metadata indicates a predetermined area;
  • the metadata indicates a predetermined area;
  • the metadata indicates a predetermined area;
  • the solid-state imaging device according to (1) or (2), wherein the image processing section extracts the region as a region of interest from a frame in which the second digital signals are arranged.
  • each of the plurality of pixels a pre-stage circuit that generates a pixel signal; a sample and hold circuit that holds the pixel signal;
  • the solid-state imaging device according to any one of (1) to (5), further comprising a post-stage circuit that reads the pixel signal from the sample-and-hold circuit and outputs the pixel signal.
  • the sample and hold circuit first and second capacitive elements; a selection circuit that sequentially performs control for connecting one of the first and second capacitive elements to a predetermined post-stage node and control for connecting the other of the first and second capacitive elements to the post-stage node.
  • the selection circuit controls to connect one of the first and second capacitive elements to a predetermined post-stage node, to disconnect both the first and second capacitive elements from the post-stage node, and to control the first and second capacitive elements to be disconnected from the post-stage node.
  • Each of the plurality of pixels further includes a post-stage reset transistor that initializes the level of the post-stage node when both the first and second capacitive elements are disconnected from the post-stage node (8 ) solid-state imaging device described.
  • (10) a plurality of pixels each provided with a sample-and-hold circuit for holding a pixel signal; a signal processing unit that generates a first digital signal by performing signal processing on each of the held pixel signals and generates a second digital signal by performing the signal processing on each of the held pixel signals; a metadata extraction unit for extracting predetermined metadata from the first digital signal; an image processing unit that performs predetermined image processing on the second digital signal using the metadata; and a recording unit that records frames in which the processed second digital signals are arranged.
  • the analog-to-digital converter generates a first digital signal by performing signal processing on a pixel signal held in each of a plurality of pixels each provided with a sample-and-hold circuit for holding the pixel signal, and a signal processing procedure for generating a second digital signal by said signal processing on each of said pixel signals; a metadata extraction procedure in which a metadata extraction unit extracts predetermined metadata from the first digital signal; and an image signal processing procedure in which an image processing unit performs predetermined image processing on the second digital signal using the metadata.
  • Imaging device 110 imaging lens 120 recording unit 130 imaging control unit 200 solid-state imaging device 201 upper pixel chip 202 lower pixel chip 203 circuit chip 211 vertical scanning circuit 212 timing control circuit 213 DAC 220 pixel array section 221 upper pixel array section 222 lower pixel array section 250 load MOS circuit block 251 load MOS transistor 260 column signal processing circuit 261 signal processing section 262, 270 ADC 271 counter 280 comparator 281, 292 selector 282, 283, 321, 322 capacitive element 284, 286 auto-zero switch 285 comparator 291 CDS processing unit 300 pixel 301 effective pixel 310 pre-stage circuit 311 photoelectric conversion element 312 transfer transistor 313 FD reset transistor 314 FD 315 Pre-stage amplification transistor 316 Current source transistor 317 Discharge transistor 320 Sample hold circuit 323 Pre-stage reset transistor 324 Pre-stage selection transistor 330 Selection circuit 331, 331-1, 331-2, 332, 332-1, 332-2 Selection transistor 333 Sampling transistor 341 post-stage

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
PCT/JP2022/048207 2022-02-21 2022-12-27 固体撮像素子、撮像装置、および、固体撮像素子の制御方法 Ceased WO2023157489A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
DE112022006684.9T DE112022006684T5 (de) 2022-02-21 2022-12-27 Festkörperbildgebungselement, bildgebungsvorrichtung und verfahren zum steuern eines festkörperbildgebungselements
JP2024501006A JPWO2023157489A1 (https=) 2022-02-21 2022-12-27
CN202280091761.5A CN118872284A (zh) 2022-02-21 2022-12-27 固态成像元件、成像装置和用于控制固态成像元件的方法
US18/838,491 US20250150730A1 (en) 2022-02-21 2022-12-27 Solid-state imaging element, imaging device, and method for controlling solid-state imaging element

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022024398 2022-02-21
JP2022-024398 2022-02-21

Publications (1)

Publication Number Publication Date
WO2023157489A1 true WO2023157489A1 (ja) 2023-08-24

Family

ID=87577959

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/048207 Ceased WO2023157489A1 (ja) 2022-02-21 2022-12-27 固体撮像素子、撮像装置、および、固体撮像素子の制御方法

Country Status (5)

Country Link
US (1) US20250150730A1 (https=)
JP (1) JPWO2023157489A1 (https=)
CN (1) CN118872284A (https=)
DE (1) DE112022006684T5 (https=)
WO (1) WO2023157489A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025105333A1 (ja) * 2023-11-17 2025-05-22 ソニーセミコンダクタソリューションズ株式会社 固体撮像装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12452555B2 (en) * 2021-07-26 2025-10-21 Sony Semiconductor Solutions Corporation Solid-state imaging element

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005287927A (ja) * 2004-04-02 2005-10-20 Konica Minolta Medical & Graphic Inc 画像処理装置、画像処理方法及び医用画像システム
WO2017099037A1 (ja) * 2015-12-09 2017-06-15 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
JP2018142954A (ja) * 2017-02-24 2018-09-13 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 自律走行のための映像処理方法、映像処理装置及び自律走行車両
JP2021033686A (ja) * 2019-08-26 2021-03-01 株式会社セルシス 画像領域抽出処理方法及び画像領域抽出処理プログラム
WO2021215105A1 (ja) * 2020-04-21 2021-10-28 ソニーセミコンダクタソリューションズ株式会社 固体撮像素子

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6959099B2 (en) * 2001-12-06 2005-10-25 Koninklijke Philips Electronics N.V. Method and apparatus for automatic face blurring
US20080180539A1 (en) * 2007-01-31 2008-07-31 Searete Llc, A Limited Liability Corporation Image anonymization
US9471852B1 (en) * 2015-11-11 2016-10-18 International Business Machines Corporation User-configurable settings for content obfuscation
JP6638852B1 (ja) 2018-08-31 2020-01-29 ソニー株式会社 撮像装置、撮像システム、撮像方法および撮像プログラム
US11277557B1 (en) * 2020-04-06 2022-03-15 The Government of the United States of America, as represented by the Secretary of Homeland Security Privacy-aware capture and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005287927A (ja) * 2004-04-02 2005-10-20 Konica Minolta Medical & Graphic Inc 画像処理装置、画像処理方法及び医用画像システム
WO2017099037A1 (ja) * 2015-12-09 2017-06-15 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
JP2018142954A (ja) * 2017-02-24 2018-09-13 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． 自律走行のための映像処理方法、映像処理装置及び自律走行車両
JP2021033686A (ja) * 2019-08-26 2021-03-01 株式会社セルシス 画像領域抽出処理方法及び画像領域抽出処理プログラム
WO2021215105A1 (ja) * 2020-04-21 2021-10-28 ソニーセミコンダクタソリューションズ株式会社 固体撮像素子

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025105333A1 (ja) * 2023-11-17 2025-05-22 ソニーセミコンダクタソリューションズ株式会社 固体撮像装置

Also Published As

Publication number Publication date
JPWO2023157489A1 (https=) 2023-08-24
CN118872284A (zh) 2024-10-29
DE112022006684T5 (de) 2025-01-02
US20250150730A1 (en) 2025-05-08

Similar Documents

Publication Publication Date Title
JP7668266B2 (ja) 固体撮像素子
WO2023062947A1 (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
US20240414451A1 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
US20240373143A1 (en) Solid-state imaging element, imaging device, and solid-state imaging element control method
WO2023062962A1 (ja) 固体撮像素子
US20240414452A1 (en) Solid-state imaging element, imaging device, and solid-state imaging element control method
US20250150730A1 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
US12452555B2 (en) Solid-state imaging element
US12368979B2 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
US20250184632A1 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
WO2024116605A1 (ja) 固体撮像素子、撮像装置、および、固体撮像素子の制御方法
WO2023166848A1 (ja) 撮像装置、画像処理装置および撮像装置の制御方法
US20250159383A1 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element
US20250380055A1 (en) Solid-state imaging element, imaging device, and method of controlling solid-state imaging element
US20250386117A1 (en) Solid-state imaging element, imaging device, and method of controlling solid-state imaging element
US20250175722A1 (en) Solid-state imaging element, imaging device, and method for controlling solid-state imaging element

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22927371

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024501006

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18838491

Country of ref document: US

Ref document number: 202280091761.5

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 112022006684

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22927371

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18838491

Country of ref document: US