JP2021197649A - Solid state imaging element and imaging apparatus - Google Patents

Abstract

To improve an image quality of image data in a solid state imaging element configured to detect the presence or absence of an address event.SOLUTION: A solid state imaging element includes a detection pixel and a normal pixel. In this solid state imaging element, the detection pixel detects whether or not an amount of variation of a photo-electric current generated by photoelectric conversion with respect incident light exceeds a predetermined threshold value, and outputs a charge in the photo-electric current as an output charge. The normal pixel generates a charge as a generated charge by photoelectric conversion with respect to the incident light, and outputs a pixel signal according to an amount of the output charge and the generated charge within a predetermined exposure period.SELECTED DRAWING: Figure 7

Description

The present technology relates to a solid-state image sensor. More specifically, the present invention relates to a solid-state image sensor that detects whether or not the amount of change in luminance exceeds a threshold value, and an image pickup device.

Conventionally, a synchronous solid-state image sensor that captures image data (frame) in synchronization with a synchronous signal such as a vertical synchronous signal has been used in an image pickup device or the like. With this general synchronous solid-state image sensor, image data can be acquired only every synchronization signal cycle (for example, 1/60 second), so faster processing can be performed in fields related to transportation and robots. It becomes difficult to respond when requested. Therefore, an asynchronous solid-state image sensor has been proposed in which a detection circuit for detecting in real time as an address event that the amount of change in the amount of light of the pixel exceeds a threshold value for each pixel address is provided for each pixel (for example). , Patent Document 1). Such a solid-state image sensor that detects an address event for each pixel is called a DVS (Dynamic Vision Sensor).

Special Table 2016-533140 Gazette

The asynchronous solid-state image sensor (that is, DVS) described above can generate and output data at a much higher speed than the synchronous solid-state image sensor. Further, in this asynchronous DVS, even if an attempt is made to further generate image data having higher image quality than the synchronous type by further adding synchronous type pixels, in a solid-state image sensor provided with both asynchronous type and synchronous type pixels. , It is difficult to improve the image quality of image data. This is because when both the synchronous type pixel and the asynchronous type pixel are arranged on the light receiving surface, the light receiving area of each is narrower and the brightness is insufficient as compared with the case where only one of them is arranged.

This technique was created in view of such a situation, and aims to improve the image quality of image data in a solid-state image sensor that detects the presence or absence of an address event.

The present technology has been made to solve the above-mentioned problems, and the first aspect thereof is to detect whether or not the amount of change in the optical current generated by photoelectric conversion with respect to the incident light exceeds a predetermined threshold value. Then, the detection pixel that outputs the charge in the photocurrent as the output charge and the charge generated as the generated charge by photoelectric conversion with respect to the incident light are generated according to the amount of the output charge and the generated charge within the predetermined exposure period. It is a solid-state imaging device including a normal pixel that outputs an electric charge signal. This has the effect of improving the image quality of the image data.

Further, in the first aspect, the normal pixel includes the first and second normal pixels, and the detection pixel is the first detection pixel connected to the first normal pixel and the second normal pixel. The first and second normal pixels may share a floating diffusion layer, including a second detection pixel connected to the normal pixel. This has the effect of reducing the circuit scale.

Further, in the first aspect, the normal pixel is a normal in-pixel photoelectric conversion element that generates the generated charge by photoelectric conversion with respect to the incident light, and a transfer that transfers the generated charge and the output charge to the floating diffusion layer. A transistor, a reset transistor that initializes the floating diffusion layer, an amplification transistor that amplifies the voltage of the floating diffusion layer, and a selection transistor that outputs a signal of the amplified voltage as a pixel signal according to a predetermined selection signal. The connection node between the normal in-pixel photoelectric conversion element and the transfer transistor may be connected to the detection pixel. This has the effect of generating a pixel signal in which the voltage corresponding to the amount of electric charge is amplified.

Further, in the first aspect, the normal pixel further includes a charge holding unit and an discharge transistor that discharges the generated charge and the output charge from the intra-pixel photoelectric conversion element, and the transfer transistor is the transfer transistor. A first transfer transistor that transfers the generated charge and the output charge from the normal intrapixel photoelectric conversion element and the detection pixel to the charge holding unit, and the stray diffusion layer that transfers the generated charge and the output charge from the charge holding unit. It may be provided with a second transfer transistor to transfer to. This has the effect of realizing a global shutter system.

Further, in the first aspect, an analog-to-digital converter that converts a pixel signal output from the normal pixel into a digital signal may be further provided. This has the effect of generating image data consisting of digital signals.

Further, on the first aspect, the normal pixel and a part of the detection pixel may be arranged on a predetermined light receiving chip, and the rest of the detection pixels may be arranged on a predetermined circuit chip. This has the effect of facilitating the miniaturization of pixels due to the laminated structure.

Further, in the first aspect, the detection pixel includes an intra-detection pixel photoelectric conversion element that generates the optical current by photoelectric conversion with respect to the incident light, a current-voltage conversion unit that converts the optical current into a voltage, and the above. A subtractor for obtaining the amount of change in voltage and a quantizer for detecting whether or not the amount of change exceeds the threshold are provided, and a part of the current-voltage conversion unit is arranged on the light receiving chip to obtain the current voltage. The rest of the conversion unit, the subtractor, and the quantizer may be arranged on the circuit chip. This has the effect of detecting the presence or absence of an address event.

Further, in the first aspect, the normal pixel generates a charge as a generated charge by photoelectric conversion with respect to the incident light, and outputs the output charge within a predetermined exposure period and a pixel signal corresponding to the generated charge. A pixel circuit and an analog-to-digital converter that converts the pixel signal into a digital signal may be provided. This has the effect of improving the reading speed.

Further, in the first aspect, a part of the normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip, and the rest of the normal pixel and the rest of the detection pixel are on a predetermined circuit chip. It may be arranged. This has the effect of facilitating the miniaturization of pixels due to the laminated structure.

Further, the second aspect of the present technology is detection that detects whether or not the amount of change in the optical current generated by photoelectric conversion with respect to the incident light exceeds a predetermined threshold value and outputs the electric charge in the optical current as an output charge. An array of pixels, a normal pixel that generates a charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal corresponding to the amount of the output charge and the generated charge within a predetermined exposure period, and the pixel signal. It is an image pickup apparatus including a recording unit for recording the image data. This has the effect of recording image data with improved image quality.

It is a block diagram which shows one configuration example of the image pickup apparatus in the 1st Embodiment of this technique. It is a figure which shows an example of the laminated structure of the solid-state image sensor in 1st Embodiment of this technique. It is a top view which shows one structural example of the light receiving chip in 1st Embodiment of this technique. FIG. 3 is a block diagram showing a configuration example of a circuit chip according to the first embodiment of the present technology. It is a figure which shows one configuration example of the lower pixel area in 1st Embodiment of this technique. It is a block diagram which shows one configuration example of a DVS pixel in 1st Embodiment of this technique. It is a circuit diagram which shows one structural example of a normal pixel and a current-voltage conversion part in 1st Embodiment of this technique. It is a circuit diagram which shows one configuration example of a normal pixel and a current-voltage conversion part in a comparative example. It is an example of the cross-sectional view of the normal pixel and the DVS pixel in the 1st Embodiment of this technique. It is an example of the potential diagram of a normal pixel and a DVS pixel in the 1st Embodiment of this technique. It is a circuit diagram which shows one structural example of a buffer, a subtractor and a quantizer in the 1st Embodiment of this technique. It is a circuit diagram which shows one configuration example of a pixel block in the 2nd Embodiment of this technique. It is an example of the plan view of the pixel block in the 2nd Embodiment of this technique. This is an example of a cross-sectional view of a solid-state image sensor according to a second embodiment of the present technology. It is an example of the layout of the element in the second embodiment of this technique. It is a circuit diagram which shows one configuration example of a normal pixel in the 3rd Embodiment of this technique. FIG. 3 is a block diagram showing a configuration example of a circuit chip according to a fourth embodiment of the present technology. It is a figure which shows one configuration example of the lower pixel area in 4th Embodiment of this technique. It is a block diagram which shows one configuration example of a normal pixel in 4th Embodiment of this technique. It is a circuit diagram which shows one configuration example of a normal pixel in 4th Embodiment of this technique. It is a block diagram which shows the schematic configuration example of a vehicle control system. It is explanatory drawing which shows an example of the installation position of the image pickup unit.

Hereinafter, embodiments for carrying out the present technology (hereinafter referred to as embodiments) will be described. The explanation will be given in the following order.
1. 1. The first embodiment (an example in which a DVS pixel outputs a charge to a normal pixel)
2. 2. The second embodiment (an example in which a DVS pixel outputs a charge to a normal pixel and a plurality of normal pixels share a floating diffusion layer).
3. 3. Third embodiment (an example in which a DVS pixel outputs a charge to a normal pixel and exposes it by a global shutter method)
4. Fourth embodiment (an example in which a DVS pixel outputs a charge to a normal pixel and an analog-digital converter is arranged in all the pixels)
5. Application example to mobile

<1. First Embodiment>
[Configuration example of image pickup device]
FIG. 1 is a block diagram showing a configuration example of an image pickup apparatus 100 according to a first embodiment of the present technology. The image pickup device 100 includes an image pickup lens 110, a solid-state image pickup element 200, a recording unit 120, and a control unit 130. As the image pickup apparatus 100, a camera mounted on an industrial robot, an in-vehicle camera, or the like is assumed.

The image pickup lens 110 collects the incident light and guides it to the solid-state image pickup element 200. The solid-state image sensor 200 captures image data by photoelectrically converting incident light. Further, the solid-state image sensor 200 detects the presence or absence of an address event, generates a detection signal, and performs predetermined signal processing on the detection signal. The solid-state image sensor 200 outputs the processed data together with the image data to the recording unit 120 via the signal line 209.

The recording unit 120 records data from the solid-state image sensor 200. The control unit 130 controls the solid-state image sensor 200 to capture image data.

[Structure example of solid-state image sensor]
FIG. 2 is a diagram showing an example of a laminated structure of the solid-state image pickup device 200 according to the first embodiment of the present technology. The solid-state image sensor 200 includes a circuit chip 202 and a light receiving chip 201 laminated on the circuit chip 202. These chips are electrically connected, for example, by Cu-Cu bonding. In addition to Cu-Cu bonding, it can also be connected by vias or bumps.

[Configuration example of light receiving chip]
FIG. 3 is a plan view showing a configuration example of the light receiving chip 201 according to the first embodiment of the present technology. Hereinafter, the light receiving chip 201 will be referred to as an upper chip, and the circuit chip 202 will be referred to as a lower chip. The light receiving chip 201 is provided with an upper pixel region 210. A plurality of pixel blocks 211 are arranged in a two-dimensional grid pattern in the upper pixel region 210.

A normal pixel 300 and a DVS pixel 400 are arranged in each of the pixel blocks 211. The normal pixel 300 generates an electric charge by photoelectric conversion of the incident light on the pixel block 211, and outputs a pixel signal according to the amount of the electric charge.

The DVS pixel 400 generates a photocurrent by photoelectric conversion of the incident light on the pixel block 211, and detects the presence or absence of an address event depending on whether or not the amount of change in the photocurrent exceeds a predetermined threshold value.

Here, the address event includes, for example, an on event indicating that the amount of increase in luminance exceeds the upper limit threshold value and an off event indicating that the amount of decrease in luminance is below the lower limit threshold value less than the upper limit threshold value. Further, the detection signal for each pixel includes an on-event detection signal indicating the presence or absence of an on-event and an off-event detection signal indicating the presence or absence of an off-event.

Further, a part of the DVS pixel 400 is arranged on the upper light receiving chip 201, and the rest is arranged on the lower circuit chip 202. In the DVS pixel 400, the upper circuit and the lower circuit are connected via the terminal 424. The terminal 424 is, for example, a terminal in which the upper Cu electrode and the lower Cu electrode are directly connected.

The DVS pixel 400 is an example of the detection pixels described in the claims.

[Circuit chip configuration example]
FIG. 4 is a block diagram showing a configuration example of the circuit chip 202 according to the first embodiment of the present technology. The circuit chip 202 includes a vertical scanning circuit 220, an output unit 230, a column ADC (Analog to Digital Converter) 240, and a load MOS (Metal-Oxide-Semiconductor) circuit 250. Further, the circuit chip 202 includes a lower pixel region 260, an X arbiter 270 and a Y arbiter 280.

The vertical scanning circuit 220 drives each of the normal pixel 300 and the DVS pixel 400 to output a pixel signal and a detection signal.

In the lower pixel area 260, a lower circuit in the corresponding DVS pixel 400 is arranged for each pixel block 211. Each of the DVS pixels 400 supplies a request to the Y arbiter 280 and the X arbiter 270 when an address event occurs, and supplies a detection signal to the output unit 230 based on the arbitration result thereof.

The Y arbiter 280 arbitrates the request from each of the rows of the DVS pixel 400 and returns a response based on the arbitration result. The X-arbiter 270 arbitrates the requests from each of the columns of the DVS pixel 400 and returns a response based on the arbitration result.

In the load MOS circuit 250, a load MOS current source connected to the vertical signal line of the row is usually arranged for each row of pixels 300.

In the column ADC 240, ADCs are usually arranged for each column of pixels 300. The ADC converts an analog pixel signal from the corresponding column into a digital signal and supplies it to the output unit 230.

The output unit 230 processes the digital signal from the column ADC 240 and the detection signal from the DVS pixel 400, and outputs the processing result to the recording unit 120. For example, the output unit 230 performs various signal processing such as CDS (Correlated Double Sampling) processing on the digital signal, and generates image data in which the processed signals are arranged. Further, the output unit 230 performs image recognition processing or the like on the detection signal.

FIG. 5 is a diagram showing a configuration example of the lower pixel region 260 according to the first embodiment of the present technology. In the lower pixel area 260, a lower circuit in the DVS pixel 400 is arranged for each pixel block 211. In the DVS pixel 400, the upper circuit and the lower circuit are connected via the terminal 424.

Further, the vertical scanning circuit 220 is provided with terminals 391 to 393 for each row. The vertical scan circuit 220 feeds the reset signal, the transfer signal and the selection signal to the corresponding rows via these terminals. Each of the terminals 391 to 393 is, for example, a terminal in which the upper Cu electrode and the lower Cu electrode are directly connected.

Further, in the load MOS circuit 250, terminals 394 are arranged for each row. The load MOS current source in the load MOS circuit 250 is connected to the vertical signal line of the corresponding row via the terminal 394. The terminal 394 is, for example, a terminal in which the upper Cu electrode and the lower Cu electrode are directly connected.

[Configuration example of DVS pixel]
FIG. 6 is a block diagram showing a configuration example of the DVS pixel 400 according to the first embodiment of the present technology. The DVS pixel 400 includes a photoelectric conversion element 410, a current-voltage conversion unit 420, a buffer 430, a subtractor 440, a quantizer 450, and a transfer unit 460.

Further, the current-voltage conversion unit 420 is usually connected to the pixel 300. Further, the normal pixel 300, the photoelectric conversion element 410, and a part of the current-voltage conversion unit 420 are arranged on the light receiving chip 201. On the other hand, the rest of the current-voltage conversion unit 420, the buffer 430, the subtractor 440, the quantizer 450, and the transfer unit 460 are arranged on the circuit chip 202.

The photoelectric conversion element 410 generates a photocurrent by photoelectric conversion with respect to incident light. For example, a photodiode is used as the photoelectric conversion element 410.

The current-voltage conversion unit 420 converts the photocurrent generated by the photoelectric conversion element 410 into a voltage. The current-voltage conversion unit 420 supplies a voltage to the buffer 430. Further, the electric charge in the photocurrent is output from the current-voltage conversion unit 420 to the normal pixel 300.

The buffer 430 outputs the voltage from the current-voltage conversion unit 420 to the subtractor 440. With this buffer 430, the driving force for driving the subsequent stage can be improved. Further, the buffer 430 can secure the noise isolation associated with the switching operation in the subsequent stage.

The subtractor 440 obtains the amount of change in voltage by the subtraction operation according to the drive signal from the vertical scanning circuit 220. The subtractor 440 supplies a differential signal indicating the amount of change to the quantizer 450.

The quantizer 450 compares the differential signal from the subtractor 440 with the threshold value. The quantizer 450 supplies the comparison result to the transfer unit 460 as a signal indicating the presence or absence of an address event.

The transfer unit 460 transfers the detection signal to the output unit 230. When an address event is detected, the transfer unit 460 supplies a request for transmitting a detection signal to the X arbiter 270 and the Y arbiter 280. Then, when the transfer unit 460 receives the response to the request from those arbiters, the transfer unit 460 supplies the detection signal to the output unit 230.

[Example of configuration of normal pixel and current-voltage converter]
FIG. 7 is a circuit diagram showing a configuration example of a normal pixel 300 and a current-voltage conversion unit 420 according to the first embodiment of the present technology.

The normal pixel 300 includes a photoelectric conversion element 311, a transfer transistor 312, a reset transistor 313, a stray diffusion layer 314, an amplification transistor 315, and a selection transistor 316. As the transfer transistor 312, the reset transistor 313, the amplification transistor 315, and the selection transistor 316, for example, an nMOS (n-channel Metal Oxide Semiconductor) transistor is used. Further, the connection node between the transfer transistor 312 and the photoelectric conversion element 410 is connected to the DVS pixel 400.

Further, the current-voltage conversion unit 420 includes nMOS transistors 421 and 422 and a pMOS (p-channel MOS) transistor 423. The nMOS transistor 421 and the photoelectric conversion element 410 are usually connected in series between the pixel 300 and the ground terminal. Further, the pMOS transistor 423 and the nMOS transistor 422 are connected in series between the power supply terminal and the ground terminal.

Further, the connection node of the nMOS transistor 421 and the photoelectric conversion element 410 is connected to the gate of the nMOS transistor 422. The connection node of the pMOS transistor 423 and the nMOS transistor 422 is connected to the gate of the nMOS transistor 421 and the buffer 430. A predetermined bias voltage Vlog is applied to the gate of the pMOS transistor 423.

Further, the photoelectric conversion element 410 and the nMOS transistors 421 and 422 are arranged on the light receiving chip 201, and the circuits and elements after the pMOS transistor 423 are arranged on the circuit chip 202. The drain of the pMOS transistor 423 is connected to the nMOS transistors 421 and 422 via the terminal 424.

The nMOS transistors 421 and 422 are connected to the power supply side, and such a circuit is called a source follower. These two source followers connected in a loop convert the photocurrent from the photoelectric conversion element 410 into its logarithmic voltage. Further, the pMOS transistor 423 supplies a constant current. Further, electric charges (electrons and the like) are output from the nMOS transistor 421 to the normal pixel 300.

In the normal pixel 300, the photoelectric conversion element 311 generates electric charges (electrons and the like) by photoelectric conversion. For example, a photodiode is used as the photoelectric conversion element 311. In this photoelectric conversion element 311, in addition to the electric charge generated by itself, the electric charge from the DVS pixel 400 is also accumulated.

The transfer transistor 312 transfers charges from the photoelectric conversion element 311 to the stray diffusion layer 314 according to the transfer signal TRG from the vertical scanning circuit 220.

The reset transistor 313 initializes the floating diffusion layer 314 according to the reset signal RST from the vertical scanning circuit 220. The floating diffusion layer 314 accumulates the transferred electric charge and generates a voltage according to the amount of the electric charge.

The amplification transistor 315 amplifies the voltage of the stray diffusion layer 314. The selection transistor 316 outputs an amplified voltage signal as a pixel signal to the vertical signal line 309 according to the selection signal SEL from the vertical scanning circuit 220.

At the start of exposure, the reset transistor 313 and the transfer transistor 312 are turned on, and the electric charge of the photoelectric conversion element 311 is extracted and initialized. Immediately before the end of exposure, the reset transistor 313 is turned on and the floating diffusion layer 314 is initialized. At the end of the exposure, the transfer transistor 312 is turned on to transfer the charge.

Further, in the load MOS circuit 250, a load MOS current source 251 is arranged for each row. The load MOS current source 251 is connected to the vertical signal line 309 in the corresponding row to supply a constant current.

Further, the signal line for transmitting the reset signal RST is connected to the vertical scanning circuit 220 via the terminal 391. The signal line for transmitting the transfer signal TRG is connected to the vertical scanning circuit 220 via the terminal 392. The signal line for transmitting the selection signal SEL is connected to the vertical scanning circuit 220 via the terminal 393. The vertical signal line 309 is connected to the load MOS circuit 250 via the terminal 394.

With the above configuration, the DVS pixel 400 generates a photocurrent by photoelectric conversion of the incident light on the pixel block 211, and detects whether or not the amount of change exceeds a predetermined threshold value. The electric charge in the photocurrent is usually output to the pixel 300. The electric charge output from the DVS pixel 400 is an example of the output electric charge described in the claims.

On the other hand, the normal pixel 300 generates an electric charge by photoelectric conversion of the incident light on the pixel block 211. The electric charge generated by the photoelectric conversion element 311 itself and the electric charge from the DVS pixel 400 are accumulated in the photoelectric conversion element 311 in the normal pixel 300. Then, the transfer transistor 312 in the normal pixel 300 transfers the electric charge from the photoelectric conversion element 311 to the floating diffusion layer 314 at the end of the exposure. This charge includes a charge generated by the photoelectric conversion element 311 itself and a charge from the photoelectric conversion element 410 in the DVS pixel 400. Therefore, the normal pixel 300 can generate a pixel signal according to the amount of electric charge generated by both the photoelectric conversion element 311 and the photoelectric conversion element 410 within the exposure period. The electric charge generated by the element 300 itself is an example of the generated electric charge described in the claims.

Further, at the time of exposure, the vertical scanning circuit 220 sequentially selects the rows of the normal pixels 300 and starts the exposure. Such exposure control is called a rolling shutter method. On the other hand, the exposure control that starts the exposure of all the pixels at the same time is called the global shutter method.

The photoelectric conversion element 311 in the normal pixel 300 is an example of the normal in-pixel photoelectric conversion element described in the claims. Further, the photoelectric conversion element 410 in the DVS pixel 400 is an example of the photoelectric conversion element in the detection pixel described in the claims.

Here, a comparative example in which the normal pixel 300 and the DVS pixel 400 are not connected is assumed.

FIG. 8 is a circuit diagram showing a configuration example of the normal pixel 300 and the current-voltage conversion unit 420 in the comparative example. In the comparative example, the drain of the nMOS transistor 421 in the current-voltage conversion unit 420 is not normally connected to the pixel 300, but is connected to the power supply terminal. In this configuration, the electric charge generated by the DVS pixel 400 is not normally output to the pixel 300 but is discharged to the power supply. Therefore, the normal pixel 300 in the comparative example generates a pixel signal according only to the amount of electric charge generated by itself during the exposure period.

On the other hand, as illustrated in FIG. 7, in the configuration in which the normal pixel 300 and the DVS pixel 400 are connected, the electric charge discharged to the power supply in the comparative example is output from the DVS pixel 400 to the normal pixel 300. To. Therefore, the normal pixel 300 can generate a pixel signal according to the amount of both the electric charge generated by itself and the electric charge generated by the DVS pixel 400. Thereby, the sensitivity can be improved as compared with the comparative example. By improving the sensitivity, the image quality of the image data is improved especially when the image is taken in a dark place.

FIG. 9 is an example of a cross-sectional view of a normal pixel 300 and a DVS pixel 400 according to the first embodiment of the present technology. The figure shows a cross-sectional view when the solid-state image sensor 200 is cut along the optical axis. A photoelectric conversion element 410 in a DVS pixel 400 and a photoelectric conversion element 311 in a normal pixel 300 are formed adjacent to each other in a substrate 500 formed of a p-type semiconductor or the like.

Further, the n-type semiconductor regions 501, 511, 513, 521, 513, 541, 551 and 553 are formed on the substrate 500. A predetermined negative voltage VNEG is applied to the n-type semiconductor region 501.

In the region between the n-type semiconductor regions 511 and 513, the gate 512 is arranged via an oxide film. The gate 512 is connected to the n-type semiconductor region 521, and the n-type semiconductor region 513 is connected to the ground terminal of the ground voltage VSS. The n-type semiconductor regions 511 and 513 and the gate 512 in FIG. 9 function as the nMOS transistor 422 in FIG. 7.

An n-type semiconductor region 521 is formed at one end of the photoelectric conversion element 410 in the DVS pixel 400. In the region between the n-type semiconductor region 521 and the photoelectric conversion element 311 in the normal pixel 300, a gate 522 is arranged via an oxide film. The gate 522 and the n-type semiconductor region 511 are connected to the pMOS transistor 423 and the buffer 430. The n-type semiconductor region 521 of FIG. 9, the photoelectric conversion element 311 and the gate 522 function as the nMOS transistor 421 of FIG.

An n-type semiconductor region 531 is formed at one end of the photoelectric conversion element 311. A gate 532 is arranged between the n-type semiconductor region 531 and the n-type semiconductor region 541 via an oxide film. A transfer signal TRG is input to the gate 532. The n-type semiconductor regions 531 and 541 and the gate 532 of FIG. 9 function as the transfer transistor 312 of FIG.

A gate 542 is arranged between the n-type semiconductor region 541 and the n-type semiconductor region 551 via an oxide film. A reset signal RST is input to the gate 542, and the n-type semiconductor region 551 is connected to the power supply terminal of the power supply voltage VDD. The n-type semiconductor regions 541 and 551 of FIG. 9 and the gate 542 function as the reset transistor 313 of FIG. 7.

A gate 552 is arranged between the n-type semiconductor region 551 and the n-type semiconductor region 553 via an oxide film. The gate 552 is connected to the n-type semiconductor region 541. The n-type semiconductor regions 551 and 553 of FIG. 9 and the gate 542 function as the amplification transistor 315 of FIG. The selection transistor 316 is omitted in FIG.

FIG. 10 is an example of a potential diagram of a normal pixel 300 and a DVS pixel 400 according to the first embodiment of the present technology. A potential barrier of negative voltage VSS, which is lower than the ground voltage VSS, is provided on the ground side of the photoelectric conversion element 410 in the DVS pixel 400. A potential barrier having a predetermined potential between the ground voltage VSS and the power supply voltage VDD is provided between the photoelectric conversion element 410 and the photoelectric conversion element 311 in the normal pixel 300. Of the charges stored in the photoelectric conversion element 410, the portion exceeding the potential barrier between the photoelectric conversion element 311 and the photoelectric conversion element 311 flows into the photoelectric conversion element 311 (in other words, is output to the photoelectric conversion element 311). In the figure, the gray part indicates the region where the charge is accumulated.

Further, the potential between the photoelectric conversion element 311 and the stray diffusion layer 314 is a predetermined potential between the ground voltage VSS and the power supply voltage VDD when the transfer transistor 312 is in the off state. On the other hand, when the transfer transistor 312 is in the ON state, it rises.

When the reset transistor 313 is in the off state, the potential barrier on the power supply side of the stray diffusion layer 314 is a predetermined potential between the ground voltage VSS and the power supply voltage VDD. On the other hand, when the reset transistor 313 is in the ON state, the voltage rises to the power supply voltage VDD.

FIG. 11 is a circuit diagram showing a configuration example of a buffer 430, a subtractor 440, and a quantizer 450 according to the first embodiment of the present technology.

The buffer 430 includes pMOS transistors 431 and 432. The pMOS transistors 431 and 432 are connected in series between the power supply terminal and the ground terminal. The gate of the pMOS transistor 432 on the ground side is connected to the current-voltage conversion unit 420, and the voltage Vp is input. A predetermined bias voltage Vbsf is applied to the gate of the pMOS transistor 431 on the power supply side. Further, the connection nodes of the pMOS transistors 431 and 432 are connected to the subtractor 440. By this connection, impedance conversion with respect to voltage Vp is performed.

The subtractor 440 includes capacitors 441 and 443, pMOS transistors 442 and 444, and an nMOS transistor 445.

The pMOS transistor 444 and the nMOS transistor 445 are connected in series between the power supply terminal and the ground terminal. These function as an inverter that inverts the input signal by using the gate of the pMOS transistor 444 as an input terminal and the connection node of the pMOS transistor 444 and the nMOS transistor 445 as an output terminal.

One end of the capacitor 441 is connected to the output terminal of the buffer 430, and the other end is connected to the input terminal of the inverter (that is, the gate of the pMOS transistor 444). The capacitor 443 is connected in parallel with the inverter. The pMOS transistor 442 opens and closes a path connecting both ends of the capacitor 443 according to a drive signal from the vertical scanning circuit 220.

When the pMOS transistor 442 is turned on, the voltage signal _Vinit is input to the buffer 430 side of the capacitor 441, and the opposite side becomes a virtual ground terminal. The potential of this virtual ground terminal is set to zero for convenience. At this time, the potential Q _init stored in the capacitor 441 is expressed by the following equation, where C1 is the capacitor of the capacitor 441. On the other hand, since both ends of the capacitor 443 are short-circuited, the accumulated charge becomes zero.
Q _init = C1 x V _init ... Equation 1

Next, considering the case where the pMOS transistor 442 is turned off and the voltage on the buffer 430 side of the capacitor 441 changes to become a V _{after , the charge Q after} stored in the capacitor 441 is expressed by the following equation. To.
Q _after = C1 x V _after ... Equation 2

On the other hand, the charge Q2 stored in the capacitor 443 is expressed by the following equation _{, where the output voltage is V out.}
Q2 = -C2 x V _out ... Equation 3

At this time, since the total charge amounts of the capacitors 441 and 443 do not change, the following equation holds.
Q _init = Q _after + Q2 ・ ・ ・ Equation 4

By substituting Equations 1 to 3 into Equation 4 and transforming it, the following equation is obtained.
V _out =-(C1 / C2) x (V _after- V _init ) ... Equation 5

Equation 5 represents the subtraction operation of the voltage signal, and the gain of the subtraction result is C1 / C2. Since it is usually desired to maximize the gain, it is preferable to design C1 to be large and C2 to be small. On the other hand, if C2 is too small, kTC noise may increase and noise characteristics may deteriorate. Therefore, the capacity reduction of C2 is limited to the range in which noise can be tolerated. Further, since the subtractor 440 is mounted on each DVS pixel 400, the capacities C1 and C2 have restrictions on the area.

By the subtraction operation described above, a differential signal indicating the amount of change in voltage Vp is generated.

The quantizer 450 includes pMOS transistors 451 and 453 and nMOS transistors 452 and 454.

The pMOS transistor 451 and the nMOS transistor 452 are connected in series between the power supply terminal and the ground terminal. The pMOS transistor 453 and the nMOS transistor 454 are also connected in series between the power supply terminal and the ground terminal. Further, the gates of the pMOS transistors 451 and 453 are connected to the output terminal of the subtractor 440. A bias voltage Von indicating an upper limit threshold value is applied to the gate of the nMOS transistor 452, and a bias voltage Voff indicating a lower limit threshold value is applied to the gate of the nMOS transistor 454.

The connection node of the pMOS transistor 451 and the nMOS transistor 452 is connected to the transfer unit 460, and the voltage of this connection node is output as an on-event detection signal VCH. The connection node of the pMOS transistor 453 and the nMOS transistor 454 is also connected to the transfer unit 460, and the voltage of this connection node is output as an off-event detection signal VCL. With such a connection, the quantizer 450 outputs a high-level on-event detection signal VCH when the differential signal exceeds the upper threshold, and low-level off-event detection when the differential signal falls below the lower threshold. Output the signal VCL.

The photoelectric conversion element 410 and a part of the current-voltage conversion unit 420 are arranged on the light receiving chip 201, and the circuit in the subsequent stage is arranged on the circuit chip 202. The circuit arranged on each chip has this configuration. Not limited to. For example, the photoelectric conversion element 410 and the entire current / voltage conversion unit 420 may be arranged on the light receiving chip 201, and the others may be arranged on the circuit chip 202. Further, the photoelectric conversion element 410, the current / voltage conversion unit 420 and the buffer 430 may be arranged on the light receiving chip 201, and the others may be arranged on the circuit chip 202. Further, the photoelectric conversion element 410, the current / voltage conversion unit 420, the buffer 430, and the capacitor 441 may be arranged on the light receiving chip 201, and the others may be arranged on the circuit chip 202. Further, the photoelectric conversion element 410, the current / voltage conversion unit 420, the buffer 430, the subtractor 440 and the quantizer 450 may be arranged on the light receiving chip 201, and the others may be arranged on the circuit chip 202.

As described above, according to the first embodiment of the present technology, the normal pixel 300 generates a pixel signal according to the amount of the electric charge from the DVS pixel 400 and the electric charge generated by the normal pixel 300 itself. The sensitivity can be improved as compared with the case where only the electric charge generated by itself is used. By improving the sensitivity, it is possible to improve the image quality of the image data, especially when the image is taken in a dark place.

<2. Second Embodiment>
In the first embodiment described above, four transistors such as a selection transistor 316 are usually arranged for each pixel 300, but in this configuration, the circuit scale may increase as the number of pixels increases. The solid-state image sensor 200 of the second embodiment is different from the first embodiment in that the floating diffusion layer 314 is shared by a plurality of normal pixels 300.

FIG. 12 is a circuit diagram showing a configuration example of the pixel block 211 according to the second embodiment of the present technology. The pixel block 211 of this second embodiment includes an FD shared block 305 and DVS pixels 400, 401, 402 and 403. The FD shared block 305 includes photoelectric conversion elements 311, 371, 373 and 375 and transfer transistors 312, 372, 374 and 376. Further, the FD shared block 305 includes a reset transistor 313, a stray diffusion layer 314, an amplification transistor 315 and a selection transistor 316. As the transfer transistors 372, 374 and 376, for example, nMOS transistors are used.

The connection configuration of the photoelectric conversion element 311, the transfer transistor 312, the reset transistor 313, the stray diffusion layer 314, the amplification transistor 315, and the selection transistor 316 is the same as that of the first embodiment. However, the transfer transistor 312 transfers the charge according to the transfer signal TRG1 from the vertical scanning circuit 220.

The transfer transistor 372 transfers charges from the photoelectric conversion element 371 to the stray diffusion layer 314 according to the transfer signal TRG2 from the vertical scanning circuit 220. The connection node of the transfer transistor 372 and the photoelectric conversion element 371 is connected to the DVS pixel 401.

The transfer transistor 374 transfers an electric charge from the photoelectric conversion element 373 to the stray diffusion layer 314 according to the transfer signal TRG3 from the vertical scanning circuit 220. The connection node of the transfer transistor 374 and the photoelectric conversion element 373 is connected to the DVS pixel 402.

The transfer transistor 376 transfers charges from the photoelectric conversion element 375 to the stray diffusion layer 314 according to the transfer signal TRG4 from the vertical scanning circuit 220. The connection node of the transfer transistor 376 and the photoelectric conversion element 375 is connected to the DVS pixel 403.

Further, the FD shared block 305 and a part of each of the DVS 400 to 403 are arranged on the light receiving chip 201. The rest of each of the DVS 400 to 403 is placed on the circuit chip 202.

With the above configuration, the FD shared block 305 functions as four normal pixels 300 that share the reset transistor 313, the stray diffusion layer 314, the amplification transistor 315 and the selection transistor 316. By sharing the floating diffusion layer 314 and the like, the circuit scale per pixel can be reduced.

Although the number of pixels sharing the floating diffusion layer 314 and the like is 4 pixels, the number of pixels is not limited to this configuration and may be 8 pixels or the like.

FIG. 13 is an example of a plan view of the pixel block 211 according to the second embodiment of the present technology. An FDTI (Front Deep Trench Isolation) 381 is formed around the pixel block 211. By this FDTI381, the pixel block 211 is separated from the peripheral elements. Further, the pixel block 211 is divided into four regions by RDTI (Rear Deep Trench Isolation) 382.

Further, a reset transistor 313, an amplification transistor 315, and a selection transistor 316 are arranged in the vicinity of the outer periphery of the pixel block 211. Further, a plurality of terminals 391 for connecting to the lower circuit are arranged.

Further, the optical axis is the Z axis, and a predetermined axis perpendicular to the Z axis is the X axis. The axis perpendicular to the X-axis and the Z-axis is defined as the Y-axis. The figure shows a plan view seen from the Z-axis direction, and the dotted line shows a line along the X-axis.

FIG. 14 is an example of a cross-sectional view of the solid-state image sensor 200 according to the second embodiment of the present technology. The figure shows a cross-sectional view when the solid-state image sensor 200 is cut along the dotted line (X-axis) of FIG.

Photoelectric conversion elements 311 and 373 are arranged under the plurality of microlenses 383. The photoelectric conversion element 311 and the photoelectric conversion element 373 are separated by the RDTI 382. Further, FDTI381 deeper than RDTI382 is formed on the left side of the photoelectric conversion element 410 and the right side of the photoelectric conversion element 412.

A wiring layer is formed under the photoelectric conversion elements 410, 311, 373, and 412, and the light receiving chip 201 and the circuit chip 202 are connected by a plurality of terminals 391.

FIG. 15 is an example of the layout of the elements in the second embodiment of the present technology. The figure shows the layout of the elements inside the FDTI381 of FIG.

The floating diffusion layer 314 is arranged in the central portion, and the photoelectric conversion elements 311, 371, 373 and 375 are arranged in 2 rows × 2 columns around the floating diffusion layer 314. Transfer transistors 312, 372, 374, and 376 are arranged between the photoelectric conversion elements 311, 371, 373, and 375 and the floating diffusion layer 314.

Further, on the upper left of the photoelectric conversion element 311 is arranged a photoelectric conversion element 410 having an L-shaped upside down shape. At the upper right of the photoelectric conversion element 371, a photoelectric conversion element 411 having an L-shape inverted vertically and horizontally is arranged. At the lower right of the photoelectric conversion element 373, a photoelectric conversion element 412 having an L-shaped inverted shape is arranged. An L-shaped photoelectric conversion element 413 is arranged at the lower left of the photoelectric conversion element 375.

As described above, according to the second embodiment of the present technology, since the plurality of normal pixels 300 share the floating diffusion layer 314, the circuit scale per pixel can be reduced as compared with the case where the floating diffusion layer 314 is not shared. ..

<3. Third Embodiment>
In the first embodiment described above, the vertical scanning circuit 220 is exposed by the rolling shutter method, but in this configuration, rolling shutter distortion may occur. The solid-state image sensor 200 of the third embodiment is different from the first embodiment in that exposure is performed by a global shutter method.

FIG. 16 is a circuit diagram showing a configuration example of a normal pixel 300 according to a third embodiment of the present technology. The normal pixel 300 of the third embodiment is different from the first embodiment in that it further includes an emission transistor 317, a transfer transistor 318, and a charge holding unit 319. For example, an nMOS transistor is used as the emission transistor 317 and the transfer transistor 318.

The discharge transistor 317 discharges electric charges from the photoelectric conversion element 311 according to the discharge signal OFD from the vertical scanning circuit 220. As described above, the discharged charges include the charges generated by the photoelectric conversion element 311 itself and the charges from the DVS pixel 400.

The transfer transistor 318 transfers charges from the photoelectric conversion element 311 and the DVS pixel 400 to the charge holding unit 319 according to the transfer signal TRGa from the vertical scanning circuit 220. The charge holding unit 319 holds the transferred charge and functions as an analog memory.

Further, the transfer transistor 312 of the third embodiment transfers charges from the charge holding unit 319 to the stray diffusion layer 314 according to the transfer signal TRGb from the vertical scanning circuit 220.

The transfer transistor 318 is an example of the first transfer transistor described in the claims, and the transfer transistor 312 is an example of the second transfer transistor described in the claims.

The vertical scanning circuit 220 of the third embodiment exposes by a global shutter method. The vertical scanning circuit 220 starts exposure for all pixels at the same time, and at the end of exposure, the charge holding unit 319 holds charges for all pixels. Then, the vertical scanning circuit 220 selects rows in order to perform initialization of the floating diffusion layer 314 and charge transfer to the floating diffusion layer 314 for the rows. The column ADC 240 performs AD conversion on the selected row.

As illustrated in the figure, by having the charge holding unit 319 hold the charge, the vertical scanning circuit 220 can start the exposure for all the pixels at the same time.

It should be noted that the second embodiment sharing the floating diffusion layer 314 can also be applied to the third embodiment.

As described above, according to the third embodiment of the present technology, since the charge holding unit 319 normally holds the charge for each pixel 300, the vertical scanning circuit 220 can perform exposure by the global shutter method. This makes it possible to suppress rolling shutter distortion and improve the image quality of image data.

<4. Fourth Embodiment>
In the first embodiment described above, the column ADC 240 performs AD conversion row by row, but in this configuration, the speed of AD conversion may be insufficient. The solid-state image sensor 200 of the fourth embodiment is different from the first embodiment in that ADCs are arranged in all the pixels.

FIG. 17 is a block diagram showing a configuration example of the circuit chip 202 according to the fourth embodiment of the present technology. The circuit chip 202 of the fourth embodiment does not include the column ADC 240 and the load MOS circuit 250, but includes a DAC 291, a plurality of time code generators 292, a pixel drive circuit 293, and a timing generation circuit 294.

Further, in the circuit chip 202 of the fourth embodiment, the X arbiter 270 and the Y arbiter 280 are arranged in the arbiter 275.

The DAC 291 generates an analog reference signal that changes in a slope shape by DA (Digital to Analog) conversion. The DAC 291 supplies a reference signal to the lower pixel region 260.

The time code generation unit 292 generates a time code. This time code indicates the time within the period in which the reference signal changes in a slope shape. The time code generation unit 292 supplies the generated time code to the lower pixel region 260.

The pixel drive circuit 293 usually drives a circuit in the pixel 300 to generate an analog pixel signal.

The timing generation circuit 294 controls the operation timing of the vertical scanning circuit 220 and the output unit 230 in synchronization with the vertical synchronization signal.

FIG. 18 is a diagram showing a configuration example of the lower pixel region 260 according to the fourth embodiment of the present technology. A plurality of pixel blocks 211 and a plurality of time code transfer units 261 are arranged in the lower pixel region 260 of the fourth embodiment. The time code transfer unit 261 is arranged for each time code generation unit 292.

A normal pixel 300 and a DVS pixel 400 are arranged in each of the pixel blocks 211. Unlike the first embodiment in which all the circuits in the normal pixel 300 are arranged on the upper side, in the fourth embodiment, a part of the circuits in the normal pixel 300 is arranged on the upper side and the rest are arranged on the lower side. Be placed.

The time code transfer unit 261 transfers the time code from the corresponding time code generation unit 292. The time code transfer unit 261 transfers the time code from the corresponding time code generation unit 292 to the normal pixel 300, and transfers the time code from the normal pixel 300 to the output unit 230 as a digital pixel signal.

FIG. 19 is a block diagram showing a configuration example of a normal pixel 300 according to a fourth embodiment of the present technology.

The normal pixel 300 includes a pixel circuit 310 and an ADC 320. The ADC 320 also includes a comparison circuit 321 and a data storage unit 360. The comparison circuit 321 includes a differential input circuit 330, a voltage conversion circuit 340, and a positive feedback circuit 350.

The pixel circuit 310 generates a pixel signal SIG according to the control of the pixel drive circuit 293 and supplies it to the ADC 320.

The ADC 320 AD-converts the pixel signal SIG into a digital signal. The differential input circuit 330 in the ADC 320 compares the reference signal REF from the DAC 291 with the pixel signal SIG from the pixel circuit 310. The differential input circuit 330 supplies an output signal indicating the comparison result to the voltage conversion circuit 340.

The voltage conversion circuit 340 converts the voltage of the output signal from the differential input circuit 330 and outputs it to the positive feedback circuit 350.

The positive feedback circuit 350 adds a part of the output to the input (output signal) and outputs the output signal VCO to the data storage unit 360.

The data storage unit 360 holds the time code when the output signal VCO is inverted. The data storage unit 360 outputs the time code as a digital signal to the time code transfer unit 261 according to the control of the vertical scanning circuit 220.

FIG. 20 is a circuit diagram showing a configuration example of a normal pixel 300 according to a fourth embodiment of the present technology. The pixel circuit 310 includes a reset transistor 313, a floating diffusion layer 314, a transfer transistor 312, a photoelectric conversion element 311 and an emission transistor 317. The connection node of the transfer transistor 312 and the photoelectric conversion element 311 is connected to the DVS pixel 400 as in the first embodiment.

The discharge transistor 317 discharges the electric charge according to the discharge signal OFG from the pixel drive circuit 293. The transfer transistor 312 transfers the electric charge according to the transfer signal TX from the pixel drive circuit 293. The reset transistor 313 initializes the floating diffusion layer 314 according to the reset signal RST from the pixel drive circuit 293.

The differential input circuit 330 includes pMOS transistors 331, 334 and 336, and nMOS transistors 332, 333 and 335.

The nMOS transistors 332 and 335 form a differential pair, and the source of these transistors is commonly connected to the drain of the nMOS transistor 333. Further, the drain of the nMOS transistor 332 is connected to the drain of the pMOS transistor 331 and the gate of the pMOS transistors 331 and 334. The drain of the nMOS transistor 335 is connected to the drain of the pMOS transistor 334, the gate of the pMOS transistor 336, and the drain of the reset transistor 313. Further, a reference signal REF is input to the gate of the nMOS transistor 332.

A predetermined bias voltage Vb is applied to the gate of the nMOS transistor 333, and a predetermined ground voltage is applied to the source of the nMOS transistor 333.

The pMOS transistors 331 and 334 form a current mirror circuit. A power supply voltage VDD1 is applied to the sources of the pMOS transistors 331, 334 and 336. This power supply voltage VDD1 is higher than the power supply voltage VDD2. Further, the drain of the pMOS transistor 336 is connected to the voltage conversion circuit 340.

The voltage conversion circuit 340 includes an nMOS transistor 341. A predetermined bias voltage VBIAS is applied to the gate of the nMOS transistor 341. Further, the drain of the nMOS transistor 341 is connected to the drain of the pMOS transistor 336, and the source is connected to the positive feedback circuit 350.

The positive feedback circuit 350 includes pMOS transistors 351 and 352 and 354 and nMOS transistors 353 and 355. The pMOS transistors 351 and 352 and the nMOS transistor 353 are connected in series between the power supply voltage VDD2 and the ground voltage VSS. The pMOS transistor 354 and the nMOS transistor 355 are also connected in series between the power supply voltage VDD2 and the ground voltage VSS.

The initialization signal INI from the vertical scanning circuit 220 is input to the gates of the pMOS transistor 351 and the nMOS transistor 353. The connection nodes of the pMOS transistor 352 and the nMOS transistor 353 are connected to the voltage conversion circuit 340. The gates of the pMOS transistor 354 and the nMOS transistor 355 are commonly connected to the connection nodes of the pMOS transistor 352 and the nMOS transistor 353. The gate of the pMOS transistor 352 is connected to the connection node of the pMOS transistor 354 and the nMOS transistor 355. Further, the voltage of the connection node of the pMOS transistor 354 and the nMOS transistor 355 is output to the data storage unit 360 as an output signal VCO.

Further, the pixel circuit 310 in the normal pixel 300 and the nMOS transistors 332, 333, and 335 are arranged on the light receiving chip 201. The pMOS transistors 331, 334 and 336 and the circuit after the voltage conversion circuit 340 are arranged on the circuit chip 202.

As illustrated in FIGS. 18 to 20, since the ADC 320 is usually arranged for each pixel 300, the solid-state image sensor 200 can perform AD conversion for all the pixels at the same time. As a result, the speed of AD conversion can be improved as compared with the first embodiment in which AD conversion is performed row by row. The details of the circuits illustrated in FIGS. 18 to 20 are described in, for example, International Publication No. 2016/136448.

As described above, according to the fourth embodiment of the present technology, since the ADC 320 is usually arranged for each pixel 300, AD conversion can be performed for all the pixels at the same time. Thereby, the speed of AD conversion can be improved.

<5. Application example to mobile>
The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device mounted on a moving body of any kind such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, and a robot. You may.

FIG. 21 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technique according to the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via the communication network 12001. In the example shown in FIG. 21, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside information detection unit 12030, an in-vehicle information detection unit 12040, and an integrated control unit 12050. Further, as a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio image output unit 12052, and an in-vehicle network I / F (interface) 12053 are shown.

The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 has a driving force generator for generating a driving force of a vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism for adjusting and a braking device for generating braking force of the vehicle.

The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, turn signals or fog lamps. In this case, the body system control unit 12020 may be input with radio waves transmitted from a portable device that substitutes for the key or signals of various switches. The body system control unit 12020 receives inputs of these radio waves or signals and controls a vehicle door lock device, a power window device, a lamp, and the like.

The vehicle outside information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, the image pickup unit 12031 is connected to the vehicle outside information detection unit 12030. The vehicle outside information detection unit 12030 causes the image pickup unit 12031 to capture an image of the outside of the vehicle and receives the captured image. The out-of-vehicle information detection unit 12030 may perform object detection processing or distance detection processing such as a person, a vehicle, an obstacle, a sign, or a character on the road surface based on the received image.

The image pickup unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of the light received. The image pickup unit 12031 can output an electric signal as an image or can output it as distance measurement information. Further, the light received by the image pickup unit 12031 may be visible light or invisible light such as infrared light.

The in-vehicle information detection unit 12040 detects information in the vehicle. For example, a driver state detection unit 12041 that detects the driver's state is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether or not the driver has fallen asleep.

The microcomputer 12051 calculates the control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit. A control command can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane deviation warning, and the like. It is possible to perform cooperative control for the purpose of.

Further, the microcomputer 12051 controls the driving force generating device, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform coordinated control for the purpose of automatic driving that runs autonomously without depending on the operation.

Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the vehicle outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the outside information detection unit 12030, and performs cooperative control for the purpose of anti-glare such as switching the high beam to the low beam. It can be carried out.

The audio-image output unit 12052 transmits an output signal of at least one of audio and an image to an output device capable of visually or audibly notifying information to the passenger or the outside of the vehicle. In the example of FIG. 21, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

FIG. 22 is a diagram showing an example of the installation position of the image pickup unit 12031.

In FIG. 22, as the image pickup unit 12031, the image pickup unit 12101, 12102, 12103, 12104, 12105 is provided.

The image pickup units 12101, 12102, 12103, 12104, 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumpers, back doors, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The image pickup unit 12101 provided in the front nose and the image pickup section 12105 provided in the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The image pickup units 12102 and 12103 provided in the side mirror mainly acquire images of the side of the vehicle 12100. The image pickup unit 12104 provided in the rear bumper or the back door mainly acquires an image of the rear of the vehicle 12100. The image pickup unit 12105 provided on the upper part of the windshield in the vehicle interior is mainly used for detecting a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

Note that FIG. 22 shows an example of the photographing range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, the imaging ranges 12112 and 12113 indicate the imaging range of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and the imaging range 12114 indicates the imaging range. The imaging range of the imaging unit 12104 provided on the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 can be obtained.

At least one of the image pickup units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image pickup units 12101 to 12104 may be a stereo camera including a plurality of image pickup elements, or may be an image pickup element having pixels for phase difference detection.

For example, the microcomputer 12051 has a distance to each three-dimensional object in the image pickup range 12111 to 12114 based on the distance information obtained from the image pickup unit 12101 to 12104, and a temporal change of this distance (relative speed with respect to the vehicle 12100). By obtaining can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in advance in front of the preceding vehicle, and can perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform coordinated control for the purpose of automatic driving or the like in which the vehicle travels autonomously without depending on the operation of the driver.

For example, the microcomputer 12051 converts three-dimensional object data related to a three-dimensional object into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, electric poles, and other three-dimensional objects based on the distance information obtained from the image pickup units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the image pickup units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured image of the imaging unit 12101 to 12104. Such pedestrian recognition is, for example, a procedure for extracting feature points in an image captured by an image pickup unit 12101 to 12104 as an infrared camera, and pattern matching processing is performed on a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. It is done by the procedure to determine. When the microcomputer 12051 determines that a pedestrian is present in the captured image of the image pickup unit 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 determines the square contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to superimpose and display. Further, the audio image output unit 12052 may control the display unit 12062 so as to display an icon or the like indicating a pedestrian at a desired position.

The example of the vehicle control system to which the technique according to the present disclosure can be applied has been described above. The technique according to the present disclosure can be applied to, for example, the image pickup unit 12031 among the configurations described above. Specifically, for example, the image pickup apparatus 100 of FIG. 1 can be applied to the image pickup unit 12031. By applying the technique according to the present disclosure to the image pickup unit 12031, the sensitivity can be improved and a photographed image that is easier to see can be obtained, so that the driver's fatigue can be reduced.

It should be noted that the above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the matters specifying the invention within the scope of claims have a corresponding relationship with each other. Similarly, the matters specifying the invention within the scope of claims and the matters in the embodiment of the present technology having the same name have a corresponding relationship with each other. However, the present technology is not limited to the embodiment, and can be embodied by applying various modifications to the embodiment without departing from the gist thereof.

It should be noted that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can have the following configurations.
(1) A detection pixel that detects whether or not the amount of change in the photocurrent generated by photoelectric conversion with respect to incident light exceeds a predetermined threshold value and outputs the charge in the photocurrent as an output charge.
A solid-state image sensor including a normal pixel that generates a charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal according to the amount of the output charge and the generated charge within a predetermined exposure period.
(2) The normal pixel includes the first and second normal pixels, and includes the first and second normal pixels.
The detection pixel includes a first detection pixel connected to the first normal pixel and a second detection pixel connected to the second normal pixel.
The solid-state image pickup device according to (1) above, wherein the first and second ordinary pixels share a floating diffusion layer.
(3) The normal pixel is
A normal in-pixel photoelectric conversion element that generates the generated charge by photoelectric conversion with respect to the incident light, and
A transfer transistor that transfers the generated charge and the output charge to the floating diffusion layer,
A reset transistor that initializes the floating diffusion layer,
An amplification transistor that amplifies the voltage of the stray diffusion layer,
A selection transistor that outputs a signal of the amplified voltage as the pixel signal according to a predetermined selection signal is provided.
The solid-state image sensor according to (1), wherein the connection node between the normal intra-pixel photoelectric conversion element and the transfer transistor is connected to the detection pixel.
(4) The normal pixel is
Charge holder and
Further, an emission transistor for discharging the generated charge and the output charge from the ordinary in-pixel photoelectric conversion element is provided.
The transfer transistor is
A first transfer transistor that transfers the generated charge and the output charge from the normal in-pixel photoelectric conversion element and the detection pixel to the charge holding unit.
The solid-state image pickup device according to (3) above, comprising a second transfer transistor that transfers the generated charge and the output charge from the charge holding unit to the floating diffusion layer.
(5) The solid-state image sensor according to any one of (1) to (4), further comprising an analog-to-digital converter that converts a pixel signal output from the normal pixel into a digital signal.
(6) The normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip, and the normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip.
The solid-state image pickup device according to any one of (1) to (5) above, wherein the rest of the detection pixels are arranged on a predetermined circuit chip.
(7) The detection pixel is
An intra-detection pixel photoelectric conversion element that generates the photocurrent by photoelectric conversion with respect to the incident light,
A current-voltage converter that converts the photocurrent into a voltage,
A subtractor for obtaining the amount of change in voltage and
A quantizer for detecting whether or not the amount of change exceeds the threshold value is provided.
A part of the current-voltage conversion unit is arranged on the light receiving chip.
The solid-state image sensor according to (6), wherein the rest of the current-voltage conversion unit, the subtractor, and the quantizer are arranged on the circuit chip.
(8) The normal pixel is
A pixel circuit that generates an electric charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal corresponding to the output charge and the generated charge within a predetermined exposure period.
The solid-state image pickup device according to (1) above, comprising an analog-to-digital converter that converts the pixel signal into a digital signal.
(9) A part of the normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip.
The solid-state image pickup device according to (7), wherein the rest of the normal pixels and the rest of the detection pixels are arranged on a predetermined circuit chip.
(10) A detection pixel that detects whether or not the amount of change in the photocurrent generated by photoelectric conversion with respect to incident light exceeds a predetermined threshold value and outputs the charge in the photocurrent as an output charge.
A normal pixel that generates an electric charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal corresponding to the amount of the output charge and the generated charge within a predetermined exposure period.
An image pickup apparatus including a recording unit for recording image data in which the pixel signals are arranged.

100 Image sensor 110 Image lens 120 Recording unit 130 Control unit 200 Solid-state image sensor 201 Light receiving chip 202 Circuit chip 210 Upper pixel area 211 Pixel block 220 Vertical scanning circuit 230 Output unit 240 Column ADC
250 Load MOS circuit 251 Load MOS current source 260 Lower pixel area 261 Time code transfer unit 270 X Arbiter 275 Arbiter 280 Y Arbiter 291 DAC
292 Time code generator 293 Pixel drive circuit 294 Timing generation circuit 300 Normal pixel 305 FD shared block 310 Pixel circuit 311, 371, 373, 375, 410-413 Photoelectric conversion element 312, 318, 372, 374, 376 Transfer transistor 313 Reset Transistor 314 Floating diffusion layer 315 Amplification transistor 316 Selective transistor 317 Ejection transistor 319 Charge holder 320 ADC
321 Comparison circuit 330 Differential input circuit 331, 334, 336, 351, 352, 354, 423, 431, 432, 442, 444, 451 and 453 pMOS transistors 332, 333, 335, 341, 353, 355, 421, 422 445, 452, 454 nMOS transistor 340 Voltage conversion circuit 350 Positive feedback circuit 360 Data storage unit 381 FDTI (Front Deep Trench Isolation)
382 RDTI (Rear Deep Trench Isolation)
383 Microlens 391-394, 424 Terminals 400-403 DVS Pixel 420 Current / Voltage Converter 430 Buffer 440 Subtractor 441, 443 Capacitor 450 Quantizer 460 Transfer Unit 500 Board 501, 511, 513, 521, 541, 541,551 552 n-type semiconductor region 512, 522, 532, 542, 552 Gate 12031 Imaging unit

Claims (10)

A detection pixel that detects whether or not the amount of change in the photocurrent generated by photoelectric conversion with respect to incident light exceeds a predetermined threshold value and outputs the charge in the photocurrent as an output charge.
A solid-state image sensor including a normal pixel that generates a charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal according to the amount of the output charge and the generated charge within a predetermined exposure period. The normal pixel includes the first and second normal pixels, and includes the first and second normal pixels.
The detection pixel includes a first detection pixel connected to the first normal pixel and a second detection pixel connected to the second normal pixel.
The solid-state image sensor according to claim 1, wherein the first and second ordinary pixels share a floating diffusion layer. The normal pixel is
A normal in-pixel photoelectric conversion element that generates the generated charge by photoelectric conversion with respect to the incident light, and
A transfer transistor that transfers the generated charge and the output charge to the floating diffusion layer,
A reset transistor that initializes the floating diffusion layer,
An amplification transistor that amplifies the voltage of the stray diffusion layer,
A selection transistor that outputs a signal of the amplified voltage as the pixel signal according to a predetermined selection signal is provided.
The solid-state image sensor according to claim 1, wherein the connection node between the normal intra-pixel photoelectric conversion element and the transfer transistor is connected to the detection pixel. The normal pixel is
Charge holder and
Further, an emission transistor for discharging the generated charge and the output charge from the ordinary in-pixel photoelectric conversion element is provided.
The transfer transistor is
A first transfer transistor that transfers the generated charge and the output charge from the ordinary intrapixel photoelectric conversion element to the charge holding unit, and
The solid-state image pickup device according to claim 3, further comprising a second transfer transistor that transfers the generated charge and the output charge from the charge holding unit to the floating diffusion layer. The solid-state image sensor according to claim 1, further comprising an analog-to-digital converter that converts a pixel signal output from the normal pixel into a digital signal. The normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip, and the normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip.
The solid-state image sensor according to claim 1, wherein the rest of the detection pixels are arranged on a predetermined circuit chip. The detection pixel is
An intra-detection pixel photoelectric conversion element that generates the photocurrent by photoelectric conversion with respect to the incident light,
A current-voltage converter that converts the photocurrent into a voltage,
A subtractor for obtaining the amount of change in voltage and
A quantizer for detecting whether or not the amount of change exceeds the threshold value is provided.
A part of the current-voltage conversion unit is arranged on the light receiving chip.
The solid-state image sensor according to claim 6, wherein the rest of the current-voltage conversion unit, the subtractor, and the quantizer are arranged on the circuit chip. The normal pixel is
A pixel circuit that generates an electric charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal corresponding to the output charge and the generated charge within a predetermined exposure period.
The solid-state image sensor according to claim 1, further comprising an analog-to-digital converter that converts the pixel signal into a digital signal. A part of the normal pixel and a part of the detection pixel are arranged on a predetermined light receiving chip.
The solid-state image sensor according to claim 8, wherein the rest of the normal pixels and the rest of the detection pixels are arranged on a predetermined circuit chip. A detection pixel that detects whether or not the amount of change in the photocurrent generated by photoelectric conversion with respect to incident light exceeds a predetermined threshold value and outputs the charge in the photocurrent as an output charge.
A normal pixel that generates an electric charge as a generated charge by photoelectric conversion with respect to the incident light and outputs a pixel signal corresponding to the amount of the output charge and the generated charge within a predetermined exposure period.
An image pickup apparatus including a recording unit for recording image data in which the pixel signals are arranged.

Images