JP7398880B2 - Solid-state imaging device and imaging device - Google Patents

Description

The present disclosure relates to a solid-state imaging device and an imaging device.

2. Description of the Related Art Synchronous solid-state imaging devices that capture image data (frames) in synchronization with a synchronization signal such as a vertical synchronization signal have conventionally been used in imaging devices and the like. This general synchronous solid-state imaging device can only acquire image data every synchronous signal period (for example, 1/60 seconds), so it is useful for faster processing in fields such as transportation and robots. It becomes difficult to respond when requested. Therefore, an asynchronous solid-state imaging device has been proposed in which each pixel is provided with a detection circuit that detects in real time that the amount of received light exceeds a threshold value as an address event. An asynchronous solid-state imaging device that detects an address event for each pixel is also called a DVS (Dynamic Vision Sensor).

Furthermore, in recent years, DVS has been developed that acquires a gradation image in addition to detecting an address event.

Special Publication No. 2017-535999

As a DVS that acquires gradation images along with address event detection, a detection circuit is placed not for each pixel but for each pixel block, and while detecting events for each pixel block, gradation is acquired for each pixel. A method is proposed. However, with this method, it is necessary to perform both event detection and tone acquisition in a time-sharing manner using the same pixel, so in scenes that change rapidly, the time between event detection and tone acquisition is There is a possibility that a gradation image of the subject to be photographed cannot be obtained due to the misalignment.

Therefore, the present disclosure proposes a solid-state imaging device and an imaging device that can reduce the time lag between event detection and gradation acquisition.

In order to solve the above problems, a solid-state imaging device according to one embodiment of the present disclosure includes a pixel array section including a plurality of pixel blocks arranged in a matrix, and a pixel array section that includes a plurality of pixel blocks in which an address event is fired. a drive circuit that generates a pixel signal in the detected first pixel block; each of the pixel blocks includes a first photoelectric conversion element that generates a charge according to the amount of incident light; a detection unit that detects firing of the address event based on electric charge; a second photoelectric conversion element that generates electric charge according to the amount of incident light; and a pixel that generates a pixel signal based on the electric charge generated in the second photoelectric conversion element. A circuit.

1 is a block diagram showing a schematic configuration example of an imaging device according to a first embodiment. FIG. 1 is a diagram showing an example of a stacked structure of a solid-state imaging device according to a first embodiment; FIG. 1 is a block diagram showing a schematic configuration example of a solid-state imaging device according to a first embodiment. FIG. FIG. 2 is a block diagram showing a schematic configuration example of a pixel block according to the first embodiment. 4 is a diagram showing an example of a stacked structure when the pixel block shown in FIG. 4 is applied to the stacked chip shown in FIG. 3. FIG. FIG. 3 is a plan view showing an example of the planar layout of pixel blocks in the pixel array section according to the first embodiment. FIG. 2 is a circuit diagram showing an example of a circuit configuration of a grayscale pixel according to the first embodiment. FIG. 2 is a circuit diagram showing an example of the circuit configuration of an event pixel according to the first embodiment. 1 is a block diagram showing a schematic configuration example of an address event detection circuit according to a first embodiment. FIG. FIG. 2 is a circuit diagram showing a schematic configuration example of a current-voltage converter according to the first embodiment. FIG. 3 is a circuit diagram showing another schematic configuration example of the current-voltage converter according to the first embodiment. FIG. 2 is a circuit diagram showing a schematic configuration example of a subtracter and a quantizer according to the first embodiment. FIG. 2 is a circuit diagram showing a schematic configuration example of a transfer unit according to the first embodiment. FIG. 2 is a block diagram showing a schematic configuration example of a column ADC according to the first embodiment. FIG. 2 is a block diagram showing a schematic configuration example of an AD conversion section according to the first embodiment. 1 is a block diagram showing a schematic configuration example of a control circuit according to a first embodiment; FIG. 1 is a flowchart illustrating a schematic operation example of the solid-state imaging device according to the first embodiment. FIG. 7 is a circuit diagram showing an example of a circuit configuration of a pixel block according to a first modification of the first embodiment. FIG. 2 is a block diagram illustrating a schematic configuration example of a solid-state imaging device according to a second modification of the first embodiment. FIG. 7 is a block diagram showing a schematic configuration example of a pixel block according to a second modification of the first embodiment. FIG. 3 is a block diagram showing a schematic configuration example of an AD conversion section according to a second embodiment. FIG. 2 is a block diagram showing a schematic configuration example of a control circuit according to a second embodiment. FIG. 7 is a diagram for explaining an example of readout control during pixel signal readout according to the second embodiment. FIG. 7 is a plan view showing a layout example of a part of a pixel array section and a column ADC according to a first example of a third embodiment. FIG. 7 is a plan view showing a partial layout example of a pixel array section and a column ADC according to a second example of the third embodiment. FIG. 7 is a plan view showing a layout example of a part of a pixel array section and a column ADC according to a third example of the third embodiment. FIG. 7 is a block diagram showing a schematic configuration example of a solid-state imaging device according to a fourth embodiment. FIG. 7 is a block diagram showing a schematic configuration example of a Y arbiter according to a fourth embodiment. FIG. 7 is a block diagram showing a schematic configuration example of an event processing unit according to a fourth embodiment. FIG. 7 is a block diagram showing a schematic configuration example of a gradation pixel control section according to a fourth embodiment. 12 is a flowchart illustrating an example of an event detection operation according to a fifth embodiment. 12 is a flowchart illustrating an example of a periodic read operation according to the fifth embodiment. 12 is a flowchart illustrating an example of gradation image data updating operation according to the fifth embodiment. 12 is a timing chart showing an example of the operation of the solid-state imaging device according to the fifth embodiment. 35 is a timing chart for explaining updating of gradation values focusing on the pixel block in the second row in FIG. 34. FIG. FIG. 7 is a block diagram showing a schematic configuration example of an event processing unit according to a sixth embodiment. 12 is a timing chart showing an example of the operation of the solid-state imaging device according to the sixth embodiment. FIG. 7 is a block diagram showing a schematic configuration example of a pixel block according to a seventh embodiment. 12 is a timing chart showing an example of a pixel signal readout operation according to a seventh embodiment. 12 is a timing chart showing an example of a pixel signal readout operation according to a modification of the seventh embodiment. FIG. 7 is a schematic diagram showing an example of a schematic configuration of a pixel block according to an eighth embodiment. FIG. 12 is a schematic diagram illustrating a schematic configuration example of a pixel block according to a modification of the eighth embodiment. FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 2 is an explanatory diagram showing an example of installation positions of an outside-vehicle information detection section and an imaging section.

An embodiment of the present disclosure will be described in detail below based on the drawings. In the following embodiments, the same parts are given the same reference numerals and redundant explanations will be omitted.

Further, the present disclosure will be described according to the order of items shown below.
1. First Embodiment 1.1 Configuration Example of Imaging Device 1.2 Layered Configuration Example of Solid-State Imaging Device 1.3 Schematic Configuration Example of Solid-State Imaging Device 1.4 Configuration Example of Pixel Block 1.4.1 Lamination of Pixel Block Configuration examples 1.4.2 Planar layout example of pixel blocks in the pixel array section 1.4.3 Circuit configuration example of gradation pixels 1.4.4 Circuit configuration example of event pixels 1.4.5 Address event detection circuit example Functional example 1.4.6 Configuration example of address event detection circuit
1.4.6.1 Configuration example of current-voltage converter
1.4.6.1.1 Modification example of current-voltage conversion unit 1.4.7 Configuration example of subtracter and quantizer 1.4.8 Configuration example of transfer unit 1.4.9 Configuration example of column ADC
1.4.9.1 Configuration example of AD conversion section
1.4.9.2 Configuration example of control circuit 1.5 Operation example of solid-state imaging device 1.6 Actions and effects 1.7 First modification 1.8 Second modification 2. Second embodiment 2.1 Configuration example of AD conversion section 2.2 Configuration example of control circuit 2.3 Example of switching control during pixel signal readout 2.4 Actions and effects 3. Third embodiment 3.1 First example 3.2 Second example 3.3 Third example 4. Fourth embodiment 4.1 Schematic configuration example of solid-state imaging device 4.2 Schematic configuration example of Y arbiter 4.3 Schematic configuration example of event processing unit 4.4 Schematic configuration example of gradation pixel control unit 4.5 Effect・Effect 5. Fifth Embodiment 5.1 Operation example of solid-state imaging device 5.1.1 Event detection operation example 5.1.2 Periodic readout operation example 5.2 Gradation image data update operation example 5.2.1 Flowchart 5 .2.2 Timing chart 5.3 Action/effect 6. Sixth embodiment 6.1 Schematic configuration example of event processing section 6.2 Example of gradation image data update operation 6.3 Actions and effects 7. Seventh embodiment 7.1 Example of pixel block configuration 7.2 Example of pixel signal readout operation 7.3 Actions/effects 7.4 Modifications 8. Eighth embodiment 8.1 Modification 9. Example of application to mobile objects

1. First Embodiment First, a first embodiment will be described in detail with reference to the drawings.

1.1 Configuration Example of Imaging Device FIG. 1 is a block diagram showing a schematic configuration example of an imaging device according to the first embodiment. As shown in FIG. 1, the imaging device 100 includes an optical system 110, a solid-state imaging device 200, a recording section 120, a control section 130, and an external interface (I/F) 140. As the imaging device 100, a camera mounted on an industrial robot, a vehicle-mounted camera, etc. are assumed.

The optical system 110 includes, for example, a lens and the like, and forms an image of the incident light on the light receiving surface of the solid-state imaging device 200.

The solid-state imaging device 200 captures image data by photoelectrically converting incident light while detecting whether or not an address event is fired. A detection result indicating whether or not an address event has been fired (hereinafter referred to as event detection data) and image data of a luminance value according to the amount of incident light (hereinafter referred to as gradation image data) are inputted into the recording unit 120, for example. Alternatively, the data may be output to an external host 150 or the like via the external I/F 140.

The external I/F 140 is, for example, a communication network compliant with any standard such as wireless LAN (Local Area Network), wired LAN, CAN (Controller Area Network), LIN (Local Interconnect Network), or FlexRay (registered trademark). The communication adapter may be a communication adapter for establishing communication with an external host 150 via the external host 150 .

Here, the host 150 may be, for example, an ECU (Engine Control Unit) installed in the car or the like when the imaging device 100 is installed in the car or the like. Furthermore, when the imaging device 100 is mounted on an autonomous mobile body such as an autonomous mobile robot such as a domestic pet robot, a robot vacuum cleaner, an unmanned aerial vehicle, or a tracking transport robot, the host 150 controls the autonomous mobile body. It may be a control device etc. that performs In addition, the host 150 may be, for example, an information processing device such as a personal computer.

The recording unit 120 is configured with, for example, a nonvolatile memory such as a flash memory, and records event detection data, gradation image data, and other various data input from the solid-state imaging device 200.

The control unit 130 is configured with an information processing device such as a CPU (Central Processing Unit), and controls the solid-state imaging device 200 to obtain event detection data and gradation image data.

1.2 Example of Laminated Structure of Solid-State Imaging Device FIG. 2 is a diagram showing an example of the layered structure of the solid-state imaging device according to the first embodiment. As shown in FIG. 2, the solid-state imaging device 200 has a stacked chip structure in which a light receiving chip 201 and a detection chip 202 are stacked one above the other. For example, so-called direct bonding can be used to bond the light receiving chip 201 and the detection chip 202, in which the respective bonding surfaces are flattened and the two are bonded together using electronic force. However, the present invention is not limited to this, and it is also possible to use, for example, so-called Cu-Cu bonding, in which electrode pads made of copper (Cu) formed on mutual bonding surfaces are bonded together, or other methods such as bump bonding. .

Further, the light receiving chip 201 and the detection chip 202 are electrically connected, for example, via a connecting portion such as a TSV (Through-Silicon Via) that penetrates the semiconductor substrate. Connections using TSVs include, for example, the so-called twin TSV method in which two TSVs, the TSV provided on the light-receiving chip 201 and the TSV provided from the light-receiving chip 201 to the detection chip 202, are connected on the outer surface of the chip, and the light-receiving method It is possible to adopt a so-called shared TSV method, in which a TSV passing through the chip 201 to the detection chip 202 connects the two.

However, when Cu--Cu bonding or bump bonding is used to bond the light-receiving chip 201 and the detection chip 202, the two are electrically connected via the Cu--Cu bonding portion or the bump bonding portion.

1.3 Schematic Configuration Example of Solid-State Imaging Device FIG. 3 is a block diagram showing a schematic configuration example of the solid-state imaging device according to the first embodiment. As shown in FIG. 3, the solid-state imaging device 200 includes a drive circuit 211, a signal processing section 212, a Y arbiter (arbitration section) 213, a column ADC (conversion section) 220, an event encoder 250, and a pixel array section. 300.

The pixel array section 300 has a configuration in which a plurality of pixel blocks 310 are arranged in a two-dimensional grid (also referred to as a matrix). Hereinafter, a set of pixel blocks arranged in the horizontal direction will be referred to as a "row", and a set of pixel blocks arranged in the direction perpendicular to the row will be referred to as a "column". The position of each pixel block 310 in the pixel array section 300 in the row direction is specified by an X address, and the position in the column direction is specified by a Y address.

Each pixel block 310 photoelectrically converts incident light to generate an analog pixel signal with a voltage value corresponding to the amount of incident light. Furthermore, the pixel block 310 detects whether or not an address event is fired based on whether the amount of change in the amount of incident light exceeds a predetermined threshold.

The pixel block 310 that has detected the firing of the address event outputs a request to the Y arbiter 213. Further, upon receiving a response to the request from the Y arbiter, the pixel block 310 transmits a detection signal indicating the address event detection result to the drive circuit 211 and column ADC 220.

The Y arbiter 213 arbitrates requests from the pixel blocks 310 to determine the readout order for the rows to which the pixel blocks 310 that are the source of the request belong, and based on the determined readout order, A response is returned to all pixel blocks 310 included in the row to which each pixel block 310 belongs. Note that in the following description, the act of arbitrating requests and determining the read order is referred to as "arbitrating the read order."

The drive circuit 211 drives each of the pixel blocks 310 that output the detection signal, thereby transmitting a pixel signal having a voltage value corresponding to the amount of light incident on the photoelectric conversion element 321 to the vertical signal line 308 to which each of the pixel blocks 310 is connected. Make it appear.

The column ADC 220 reads out pixel signals in parallel columns by converting analog pixel signals appearing on the vertical signal lines 308 of each column into digital pixel signals for each row. The column ADC 220 then supplies the read digital pixel signal to the signal processing unit 212.

The signal processing unit 212 performs predetermined signal processing such as CDS (Correlated Double Sampling) processing on the pixel signals from the column ADC 220, and outputs gradation image data consisting of the pixel signals after the signal processing to the outside. .

The event encoder 250 generates data indicating in which pixel block 310 an on event has occurred and in which pixel block 310 an off event has occurred, for each row in the pixel array section 300. For example, when the event encoder 250 receives a request from a certain pixel block 310, it informs that an on event or an off event has occurred in this pixel block 310, and an X address and an Event detection data including the Y address is generated.

At this time, the event encoder 250 also includes information (time stamp) regarding the time when firing of the on event or off event was detected in the event detection data. Then, the event encoder 250 outputs the generated event detection data to the outside.

1.4 Configuration Example of Pixel Block FIG. 4 is a block diagram showing a schematic configuration example of a pixel block according to the first embodiment. As shown in FIG. 4, the pixel block 310 includes a gradation pixel 320 for generating a pixel signal that is gradation information, an event pixel 330 for detecting whether or not an address event is fired, and The address event detection circuit (detection unit) 400 detects whether or not an address event is fired based on the photocurrent of the address event detection circuit (detection unit).

1.4.1 Example of Laminated Structure of Pixel Block FIG. 5 is a diagram showing an example of the layered structure when the pixel block shown in FIG. 4 is applied to the layered chip shown in FIG. 3. As shown in FIG. 5, of the pixel block 310, for example, the gradation pixels 320 and the event pixels 330 are arranged on the light receiving chip 201, and the address event detection circuit 400 is arranged on the detection chip 202.

However, the present invention is not limited to this, and various modifications can be made, such as, for example, disposing a part of the circuit configuration of the gradation pixel 320 on the detection chip 202.

1.4.2 Example of Planar Layout of Pixel Blocks in Pixel Array Section FIG. 6 is a plan view showing an example of the planar layout of pixel blocks in the pixel array section according to the first embodiment. As shown in FIG. 6, the pixel array unit 300 includes a plurality of pixel blocks 310 arranged in rows and columns. Further, in the pixel array section 300, detection signal lines 306 and 307, a vertical signal line 308, and an enable signal line 309 are wired for each column along the column direction. Each pixel block 310 is connected to detection signal lines 306 and 307, a vertical signal line 308, and an enable signal line 309 in the corresponding column.

1.4.3 Example of circuit configuration of gradation pixel FIG. 7 is a circuit diagram showing an example of the circuit configuration of the gradation pixel 320 according to the first embodiment. As shown in FIG. 7, the gradation pixel 320 includes a photoelectric conversion element 321, a transfer transistor 322, a floating diffusion layer 323, a reset transistor 324, an amplification transistor 325, and a selection transistor 326. An analog signal with a corresponding voltage is generated as a pixel signal Vsig. The configuration of the gradation pixel 320 other than the photoelectric conversion element 321 is also referred to as a pixel circuit. The transfer transistor, reset transistor 324, amplification transistor 325, and selection transistor 326 may be, for example, N-type MOS (Metal-Oxide-Semiconductor) transistors.

The photoelectric conversion element (second photoelectric conversion element) 321 is composed of, for example, a photodiode, and generates charges by photoelectrically converting incident light. The transfer transistor 322 transfers charges from the photoelectric conversion element 321 to the floating diffusion layer 323 in accordance with the transfer signal TRG from the drive circuit 211.

The floating diffusion layer 323 is a charge storage section that generates a voltage according to the amount of accumulated charge. The reset transistor 324 discharges (initializes) the charges in the floating diffusion layer 323 in accordance with the reset signal RST from the drive circuit 211. The amplification transistor 325 amplifies the voltage of the floating diffusion layer 323. The selection transistor 326 causes the amplified voltage signal to appear on the vertical signal line 308 as a pixel signal Vsig in accordance with the selection signal SEL from the drive circuit 211. The pixel signal Vsig appearing on the vertical signal line 308 is read out by the column ADC 220, for example, and converted into a digital pixel signal.

1.4.4 Example of circuit configuration of event pixel FIG. 8 is a circuit diagram showing an example of circuit configuration of the event pixel according to the first embodiment. As shown in FIG. 8, the event pixel 330 includes a photoelectric conversion element 331.

Like the photoelectric conversion element 321, the photoelectric conversion element (first photoelectric conversion element) 331 is configured with, for example, a photodiode, and generates charges by photoelectrically converting incident light. Charges generated by photoelectric conversion by the photoelectric conversion element 331 are supplied to the address event detection circuit 400 as a photocurrent.

1.4.5 Functional Example of Address Event Detection Circuit In addition, the address event detection circuit 400 shown in FIG. , detect whether or not an address event is fired. This address event includes, for example, an on event indicating that the amount of change in photocurrent according to the amount of incident light exceeds an upper threshold, and an off event indicating that the amount of change falls below a lower threshold. In other words, an address event is detected when the amount of change in the amount of incident light is outside a predetermined range from the lower limit to the upper limit. Further, the address event detection signal includes, for example, one bit indicating the detection result of an on event and one bit indicating the detection result of an off event. Note that the address event detection circuit 400 can also detect only on-events.

Address event detection circuit 400 transmits a request to Y arbiter 213 to request transmission of a detection signal when an address event occurs. Then, upon receiving a response to the request from the Y arbiter 213, the address event detection circuit 400 transmits detection signals DET+ and DET- to the drive circuit 211 and column ADC 220. Here, the detection signal DET+ is a signal indicating the detection result of the presence or absence of an on-event, and is transmitted to the column ADC 220 via the detection signal line 306, for example. Further, the detection signal DET- is a signal indicating the detection result of the presence or absence of an off event, and is transmitted to the column ADC 220 via the detection signal line 307, for example.

Further, the address event detection circuit 400 enables the column enable signal ColEN in synchronization with the selection signal SEL, and transmits the signal to the column ADC 220 via the enable signal line 309. Here, the column enable signal ColEN is a signal for enabling or disabling AD (Analog to Digital) conversion for pixel signals of the corresponding column.

When an address event is detected in a certain row, the drive circuit 211 drives that row using a selection signal SEL or the like. Each of the pixel blocks 310 in the driven row causes a pixel signal Vsig to appear on the vertical signal line 308. The pixel signal Vsig appearing on the vertical signal line 308 is read out by the column ADC 220 and converted into a digital pixel signal.

Furthermore, the pixel block 310 that has detected an address event among the driven rows transmits a column enable signal ColEN set to enable to the column ADC 220. On the other hand, the column enable signal ColEN of the pixel block 310 in which no address event has been detected is set to disabled.

1.4.6 Configuration Example of Address Event Detection Circuit FIG. 9 is a block diagram showing a schematic configuration example of the address event detection circuit according to the first embodiment. As shown in FIG. 9, the address event detection circuit 400 includes a current-voltage converter 410, a buffer 420, a subtracter 430, a quantizer 440, and a transfer unit 450.

The current-voltage converter 410 converts the photocurrent from the event pixel 330 into a logarithmic voltage signal. The current-voltage converter 410 then supplies the voltage signal to the buffer 420.

Buffer 420 outputs the voltage signal from current-voltage converter 410 to subtracter 430. This buffer 420 can improve the driving force for driving the subsequent stage. Furthermore, the buffer 420 can ensure isolation of noise associated with subsequent switching operations.

Subtractor 430 reduces the level of the voltage signal from buffer 420 according to the row drive signal from drive circuit 211. The subtracter 430 then supplies the reduced voltage signal to the quantizer 440.

The quantizer 440 quantizes the voltage signal from the subtracter 430 into a digital signal and outputs it to the transfer unit 450 as a detection signal.

The transfer unit 450 transfers the detection signal from the quantizer 440 to the signal processing unit 212 and the like. The transfer unit 450 sends a request to the Y arbiter 213 and the event encoder 250 to request transmission of a detection signal when an address event is detected. Then, upon receiving a response to the request from the Y arbiter 213, the transfer unit 450 supplies detection signals DET+ and DET- to the drive circuit 211 and column ADC 220. Further, when the selection signal SEL is transmitted, the transfer unit 450 transmits the enabled column enable signal ColEN to the column ADC 220.

1.4.6.1 Configuration Example of Current-Voltage Conversion Unit FIG. 10 is a circuit diagram showing a schematic configuration example of the current-voltage conversion unit according to the first embodiment. As shown in FIG. 10, the current-voltage converter 410 includes an LG (LoG) transistor 411, an amplification transistor 413, and a load MOS transistor 412. For the LG transistor 411 and the amplification transistor 413, for example, an N-type MOS transistor can be used. On the other hand, the load MOS transistor 412 is a constant current circuit, and a P-type MOS transistor can be used for this.

The source of the LG transistor 411 is connected to the cathode of the photoelectric conversion element 331 in the event pixel 330, and the drain is connected to the power supply terminal. Load MOS transistor 412 and amplification transistor 413 are connected in series between a power supply terminal and a ground terminal. Further, a connection point between the load MOS transistor 412 and the amplification transistor 413 is connected to the gate of the LG transistor 411 and the input terminal of the buffer 420. Further, a predetermined bias voltage Vbias is applied to the gate of the load MOS transistor 412.

The drains of the LG transistor 411 and the amplification transistor 413 are connected to the power supply side, and such a circuit is called a source follower. These two source followers connected in a loop convert the photocurrent from the photoelectric conversion element 331 into a logarithmic voltage signal. Further, the load MOS transistor 412 supplies a constant current to the amplification transistor 413.

Note that in the configuration shown in FIG. 10, for example, the LG transistor 411 and the amplification transistor 413 may be arranged in the light receiving chip 201 shown in FIG.

1.4.6.1.1 Modification of current-voltage converter Note that instead of the source follower type current-voltage converter 410 as illustrated in FIG. 10, a gain boost type as illustrated in FIG. It is also possible to use the current-voltage converter 410A.

As shown in FIG. 11, in the current-voltage converter 410A, the source of the LG transistor 411 and the gate of the amplification transistor 413 are connected to, for example, the cathode of the photoelectric conversion element 331 in the event pixel 330. Further, the drain of the LG transistor 411 is connected to the source of the LG transistor 414 and the gate of the amplification transistor 413, for example. The drain of the LG transistor 414 is connected to the power supply terminal VDD, for example.

Further, for example, the source of the amplification transistor 415 is connected to the gate of the LG transistor 411 and the drain of the amplification transistor 413. The drain of the amplification transistor 415 is connected to the power supply terminal VDD via the load MOS transistor 412, for example.

Even in such a configuration, the photocurrent from the photoelectric conversion element 331 is converted into a logarithmic voltage signal corresponding to the amount of charge. Note that the LG transistors 411 and 414 and the amplification transistors 413 and 415 may each be configured with, for example, an N-type MOS transistor.

Note that in the configuration shown in FIG. 11, for example, the LG transistors 411 and 414 and the amplification transistors 413 and 415 may be arranged in the light receiving chip 201 shown in FIG.

1.4.7 Configuration Example of Subtractor and Quantizer FIG. 12 is a circuit diagram showing a schematic configuration example of the subtracter and quantizer according to the first embodiment. As shown in FIG. 12, subtracter 430 includes capacitors 431 and 433, an inverter 432, and a switch 434. The quantizer 440 also includes comparators 441 and 442.

One end of capacitor 431 is connected to the output terminal of buffer 420, and the other end is connected to the input terminal of inverter 432. Capacitor 433 is connected in parallel to inverter 432. Switch 434 opens and closes a path connecting both ends of capacitor 433 in accordance with auto-zero signal AZ from drive circuit 211.

Inverter 432 inverts the voltage signal input via capacitor 431. This inverter 432 outputs the inverted signal to the non-inverting input terminal (+) of the comparator 441.

When the switch 434 is turned on, the voltage signal Vinit is input to the buffer 420 side of the capacitor 431, and the opposite side becomes a virtual ground terminal. For convenience, the potential of this virtual ground terminal is set to zero. At this time, the potential Qinit stored in the capacitor 431 is expressed by the following equation (1), assuming that the capacitance of the capacitor 431 is C1. On the other hand, since both ends of the capacitor 433 are short-circuited, the accumulated charge becomes zero.
Qinit=C1×Vinit...(1)

Next, considering the case where the switch 434 is turned off and the voltage on the buffer 420 side of the capacitor 431 changes to become Vafter, the charge Qafter accumulated in the capacitor 431 is expressed by the following equation (2). Ru.
Qafter=C1×Vafter...(2)

On the other hand, the charge Q2 accumulated in the capacitor 433 is expressed by the following equation (3), where the output voltage is Vout.
Q2=-C2×Vout...(3)

At this time, since the total charge amount of capacitors 431 and 433 does not change, the following equation (4) holds true.
Qinit=Qafter+Q2...(4)

When formula (4) is transformed by substituting formulas (1) to (3), the following formula (5) is obtained.
Vout=-(C1/C2)×(Vafter-Vinit)...(5)

Equation (5) represents the subtraction operation of the voltage signal, and the gain of the subtraction result is C1/C2. Since it is usually desired to maximize the gain, it is preferable to design C1 large and C2 small. On the other hand, if C2 is too small, kTC noise may increase and noise characteristics may deteriorate, so the reduction in the capacity of C2 is limited to a range where noise can be tolerated. Further, since the address event detection circuit 400 including the subtracter 430 is installed in each pixel block, there are area restrictions on the capacitors C1 and C2. Taking these into consideration, the values of capacitances C1 and C2 are determined.

Comparator 441 compares the voltage signal from subtracter 430 with the upper limit voltage Vbon applied to the inverting input terminal (-). Here, the upper limit voltage Vbon is a voltage indicating the upper limit threshold. The comparator 441 outputs the comparison result COMP+ to the transfer unit 450. The comparator 441 outputs a high-level comparison result COMP+ when an on-event occurs, and outputs a low-level comparison result COMP+ when no on-event occurs.

Comparator 442 compares the voltage signal from subtracter 430 with the lower limit voltage Vboff applied to the inverting input terminal (-). Here, the lower limit voltage Vboff is a voltage indicating the lower limit threshold. The comparator 442 outputs the comparison result COMP- to the transfer unit 450. The comparator 442 outputs a high level comparison result COMP- when an off event occurs, and outputs a low level comparison result COMP- when there is no off event.

1.4.8 Configuration Example of Transfer Unit FIG. 13 is a circuit diagram showing a schematic configuration example of the transfer unit according to the first embodiment. As shown in FIG. 13, the transfer unit 450 includes AND (logical product) gates 451 and 453, an OR (logical sum) gate 452, and flip-flops 454 and 455.

The AND gate 451 outputs the logical product of the comparison result COMP+ of the quantizer 440 and the response AckY from the Y arbiter 213 to the column ADC 220 as a detection signal DET+. This AND gate 451 outputs a high level detection signal DET+ when an on event occurs, and outputs a low level detection signal DET+ when there is no on event.

The OR gate 452 outputs the logical sum of the comparison results COMP+ and COMP- of the quantizer 440 to the Y arbiter 213 as a request ReqY. The OR gate 452 outputs a high level request ReqY when an address event occurs, and outputs a low level request ReqY when there is no address event. Further, the inverted value of the request ReqY is input to the input terminal D of the flip-flop 454.

The AND gate 453 outputs the logical product of the comparison result COMP- of the quantizer 440 and the response AckY from the Y arbiter 213 to the column ADC 220 as a detection signal DET-. This AND gate 453 outputs a high level detection signal DET- when an off event occurs, and outputs a low level detection signal DET- when there is no off event.

Flip-flop 454 holds the inverted value of request ReqY in synchronization with response AckY. Then, the flip-flop 454 outputs the held value to the input terminal D of the flip-flop 455 as an internal signal ColEN'.

Flip-flop 455 holds internal signal ColEN' in synchronization with selection signal SEL from drive circuit 211. Then, the flip-flop 455 outputs the held value to the column ADC 220 as a column enable signal ColEN.

1.4.9 Configuration Example of Column ADC FIG. 14 is a block diagram showing a schematic configuration example of the column ADC according to the first embodiment. As shown in FIG. 14, in the column ADC 220, one AD conversion section 230 is arranged for each column in the pixel array section 300, for example. However, it is not an essential configuration to provide one-to-one AD converters 230 for each column, and for example, one AD converter 230 may be arranged for two or more columns.

The AD converter 230 converts analog pixel signals appearing on the vertical signal lines 308 of each column into digital pixel signals.

1.4.9.1 Configuration Example of AD Conversion Unit FIG. 15 is a block diagram showing a schematic configuration example of the AD conversion unit according to the first embodiment. As shown in FIG. 15, the AD converter 230 includes an ADC 232 and a control circuit 240.

The ADC 232 converts the pixel signal Vsig into a digital pixel signal Dout. This ADC 232 includes a comparator 233 and a counter 234.

Comparator 233 compares a predetermined reference signal RMP and pixel signal Vsig according to comparator enable signal CompEN from control circuit 240. As the reference signal RMP, for example, a ramp signal that changes in a slope shape or a step shape can be used. Further, the comparator enable signal CompEN is a signal for enabling or disabling the comparison operation of the comparator 233. Comparator 233 supplies the comparison result VCO to counter 234 .

The counter 234 counts the count value in synchronization with the clock signal CLK over a period until the comparison result VCO is inverted, according to the counter enable signal CntEN from the control circuit 240. Counter enable signal CntEN is a signal for enabling or disabling the counting operation of counter 234. This counter 234 outputs a digital pixel signal Dout indicating a count value to the signal processing section 212.

Control circuit 240 controls multiplexer 231 and ADC 232 according to column enable signal ColEN. Details of the control content will be described later.

Further, the detection signals DET+ and DET- outputted from each pixel block 310 are outputted to the signal processing section 212 via the AD conversion section 230.

Note that although a single slope ADC including a comparator 233 and a counter 234 is used as the ADC 232, the configuration is not limited to this. For example, a delta-sigma type ADC can also be used as the ADC 232.

1.4.9.2 Configuration Example of Control Circuit FIG. 16 is a block diagram showing a schematic configuration example of the control circuit according to the first embodiment. As shown in FIG. 16, the control circuit 240 includes an OR (logical sum) gate 241, a level shifter 242, and an AND (logical product) gate 243.

The OR gate 241 outputs the logical sum of the column enable signal ColEN and the extra enable signal ExtEN to the level shifter 242 and the AND gate 243. The extra enable signal ExtEN is a signal that instructs to enable AD conversion regardless of the presence or absence of an address event, and is set according to a user operation or the like. For example, when enabling, the extra enable signal ExtEN is set to a high level, and when disabling, it is set to a low level.

Level shifter 242 converts the voltage of the output signal of OR gate 241. Then, the level shifter 242 supplies the converted signal to the comparator 233 in the ADC 232 as a comparator enable signal CompEN, for example, in accordance with the block control signal Crtl2. The block control signal Crtl2 is a signal for disabling the comparator 233 regardless of the presence or absence of an address event. For example, regardless of the presence or absence of an address event, the block control signal Crtl2 is set to a low level when the comparator 233 is disabled, and otherwise set to a high level.

The AND gate 243 outputs the logical product of the output signal of the OR gate 241 and the block control signal Crtl1 to the counter 234 as a counter enable signal CntEN. The block control signal Crtl1 is a signal for invalidating the counter 234 regardless of the presence or absence of an address event. For example, regardless of the presence or absence of an address event, the block control signal Crtl1 is set to a low level when the counter 234 is disabled, and otherwise set to a high level.

1.5 Operation example of solid-state imaging device FIG. 17 is a flowchart showing a schematic operation example of the solid-state imaging device according to the first embodiment. This operation is started, for example, when an application for detecting and imaging an address event is executed.

As shown in FIG. 17, the solid-state imaging device 200 starts detecting whether an address event has occurred (step S101), and determines whether an address event has occurred (step S102). Event pixel 330 is used to detect firing of an address event. If firing of an address event is not detected (NO in step S102), the operation proceeds to step S105.

On the other hand, if the firing of the address event is detected (YES in step S102), the event encoder 250 outputs event detection data for the pixel block 310 in which the firing of the address event was detected (step S103).

Next, the column ADC 220 reads pixel signals from all the pixel blocks 310 included in the row to which the pixel block 310 in which firing of the address event has been detected belongs (step S104). Gradation pixels 320 are used to read out pixel signals. Further, pixel signals for one row are read out in parallel (column parallel) from all pixel blocks 310 included in the row to which the pixel block 310 in which the firing of the address event has been detected belongs. After that, the operation proceeds to step S105.

In step S105, the solid-state imaging device 200 determines whether or not to end this operation, and if it ends (YES in step S105), ends this operation. On the other hand, if the process does not end (NO in step S105), the process returns to step S101, and the subsequent operations are executed.

1.6 Effects and Effects As described above, according to the first embodiment, pixel signals are read out in column parallel from all the pixel blocks 310 included in the row to which the pixel block 310 in which firing of an address event has been detected belongs. It will be done. This makes it possible to omit the step of identifying each pixel block 310 in which an address event has fired and reading them out individually, thereby reducing the time difference between detecting the firing of an address event and reading out the pixel signal (gradation). It becomes possible to do so.

Further, according to the first embodiment, since it is possible to omit the X arbiter that arbitrates the readout order for the pixel block 310 in which the firing of an address event is detected in the column direction, the circuit of the solid-state imaging device 200 can be omitted. It is also possible to simplify the configuration and reduce the size.

Furthermore, in this embodiment, in one pixel block 310, a pixel for event detection (event pixel 330) and a pixel for obtaining gradation (gradation pixel 320) are provided separately, and each is independently controlled. Therefore, it is possible to eliminate dead time from detection of firing of an address event to reading out a pixel signal (grayscale), and to perform event detection and grayscale acquisition simultaneously.

1.7 First Modified Example In this embodiment, the event pixel 330 and the gradation pixel 320 each include separate photoelectric conversion elements 331 or 321, but in this embodiment, such a configuration For example, the present invention is not limited to this, and various modifications may be made, such as a configuration in which the event pixel 330 and the gradation pixel 320 share one photoelectric conversion element.

Note that when the event pixel 330 and the gradation pixel 320 share one photoelectric conversion element, as shown in FIG. The configuration is such that the circuit configuration other than the photoelectric conversion element 321 in the gradation pixel 320 is connected.

Further, in driving the configuration shown in FIG. 18, an OFG (OverFlow Gate) transistor 332 is first turned on for monitoring an address event, and when firing of an address event is detected in that state, the OFG transistor 332 is turned off. At the same time, by turning on the transfer transistor 322, the charges generated in the photoelectric conversion element 341 are transferred to the floating diffusion layer 323 of the gradation pixel 320. Note that the pixel signal readout operation at that time is similar to the operation described above, and therefore detailed explanation will be omitted here.

1.8 Second Modification Further, FIG. 19 is a block diagram showing a schematic configuration example of a solid-state imaging device according to a second modification of the first embodiment. Further, FIG. 20 is a block diagram showing a schematic configuration example of a pixel block according to the second modification of the first embodiment.

In the first embodiment described above, the case where the address event detection circuit 400 is provided in each pixel block 310 is illustrated, but the configuration is not limited to this, and for example, as shown in FIG. 19, each pixel block 310A It is also possible to replace the address event detection circuit 400 with an address event detection section 400A configured of a common address event detection circuit 400 for each row.

With this configuration, as shown in FIG. 20, it is possible to omit the address event detection circuit 400 from each pixel block 310A, so it is possible to further downsize the solid-state imaging device 200.

2. Second Embodiment Next, a second embodiment will be described in detail with reference to the drawings. Note that, in this embodiment, the same configurations and operations as those of the above-described embodiments will be cited, and redundant explanations will be omitted.

In the first embodiment described above, a so-called 1-column 1-ADC configuration in which one AD converter 230 is provided for each column is illustrated, but the configuration is not limited to such a configuration. It is also possible to configure so that one AD conversion section 230 is shared. Therefore, in the second embodiment, a case where one AD converter 230 is shared by two or more columns will be explained by giving an example.

The configurations of the imaging device and the solid-state imaging device according to this embodiment may be, for example, the same as the imaging device 100 and the solid-state imaging device 200 illustrated in the first embodiment. However, in this embodiment, the AD converter 230 is replaced with an AD converter 530, which will be described later.

2.1 Configuration Example of AD Conversion Unit FIG. 21 is a block diagram showing a schematic configuration example of the AD conversion unit according to the second embodiment. As shown in FIG. 21, the AD conversion unit 530 has a configuration similar to that of the AD conversion unit 230 illustrated in FIG. 15, except that the control circuit 240 is replaced with a control circuit 540, and a multiplexer 531 is added. In this description, two columns corresponding to the AD converter 530 are assumed to be a 2m-1 (m is an integer from 1 to M) column and a 2m column.

According to the control signal from the control circuit 540, the multiplexer 531 selects one of the pixel signal Vsig2m-1 of the 2m-1 column and the pixel signal Vsig2m of the 2m column and outputs it to the ADC 232 as a pixel signal VsigSEL. A switching signal SW and a multiplexer enable signal MuxEN are input to the multiplexer 531 as control signals.

Similar to the ADC 232 in FIG. 15, the ADC 232 includes a comparator 233 and a counter 234, and converts the pixel signal VsigSEL into a digital pixel signal Dout.

However, the comparator 233 compares the predetermined reference signal RMP and the pixel signal VsigSEL according to the comparator enable signal CompEN from the control circuit 540.

Control circuit 540 controls multiplexer 531 and ADC 232 according to column enable signals ColEN2m-1 and ColEN2m for columns 2m-1 and 2m, respectively. Details of the control content will be described later.

Furthermore, the detection signals DET+ and DET− of each column are output to the signal processing unit 212 via the AD conversion unit 530.

Note that although a single slope ADC including a comparator 233 and a counter 234 is used as the ADC 232, the configuration is not limited to this. For example, a delta-sigma type ADC can also be used as the ADC 232.

2.2 Configuration Example of Control Circuit FIG. 22 is a block diagram showing a schematic configuration example of the control circuit 540 according to the second embodiment. As shown in FIG. 22, the control circuit 540 has the same configuration as the control circuit 240 illustrated in FIG. 16, and further includes a demultiplexer 544 and a switching control section 545.

Demultiplexer 544 distributes the output signal of level shifter 242 to multiplexer 531 and comparator 233 according to block control signal Crtl2. The block control signal Crtl2 is a signal for disabling at least one of the multiplexer 531 and the comparator 233, regardless of the presence or absence of an address event.

For example, when disabling only the multiplexer 531, regardless of the presence or absence of an address event, "10" in binary is set to the block control signal Crtl2. At this time, the output signal of the level shifter 242 is output to the comparator 233 as a comparator enable signal CompEN. When only the comparator 233 is disabled, "01" in binary is set to the block control signal Crtl2. At this time, the output signal of level shifter 242 is output to multiplexer 531 as multiplexer enable signal MuxEN. Moreover, when disabling both the multiplexer 531 and the comparator 233, "00" is set, and in other cases, "11" is set. When "11" is set, the output signal of the level shifter 242 is output to both the multiplexer 531 and the comparator 233.

The switching control unit 545 switches the pixel signal output by the multiplexer 531 based on the column enable signals ColEN2m-1 and ColEN2m. For example, if only one of the columns is enabled, the switching control unit 545 causes the multiplexer 531 to select the pixel signal of the enabled column using the switching signal SW. Further, when enable is set for both columns, the switching control unit 545 causes the multiplexer 531 to select the pixel signal of one column using the switching signal SW, and then selects the pixel signal of the other column.

2.3 Example of switching control when reading out pixel signals FIG. 23 is a diagram for explaining an example of readout control when reading out pixel signals according to the second embodiment. Note that in this embodiment as well, as in the first embodiment, pixel signals are read out from all pixel blocks 310 included in the row to which the pixel block 310 in which firing of the address event has been detected belongs. The control is executed when firing of an address event is detected in at least one pixel block 310 among the pixel blocks 310 included in the row to which the pixel blocks 310 in columns 2m-1 and 2m belong.

As shown in FIG. 23, when firing of an address event is detected in at least one of the pixel blocks 310 in the 2m-1 column and the pixel blocks 310 in the 2m column, the control circuit 540 uses the switching signal SW to, for example, The multiplexer 531 first selects the pixel blocks 310 in the 2m-1 column, and then causes the multiplexer 531 to select the pixel blocks 310 in the 2m column. At this time, the control circuit 540 enables the ADC 232 over the AD conversion period of the 2m-1 column and the 2m column.

Note that when both the 2m-1 column and the 2m column are disabled, the control circuit 540 sets the ADC 232 to disabled.

2.4 Actions and Effects As described above, by adopting a configuration in which two or more rows share one AD converter 230, it is possible to reduce the number of AD converters 230, so that the solid-state imaging device 200 Further downsizing is possible.

Other configurations, operations, and effects may be the same as those of the embodiments described above, so detailed explanations will be omitted here.

3. Third Embodiment In addition, in the above-mentioned embodiment, the case where one ADC 232 is associated with one or more columns is illustrated, but the configuration is not limited to this, and for example, one ADC 232 is associated with one column. Various modifications can be made, such as associating a plurality of ADCs 232 with each other. Hereinafter, some of the modifications will be explained by giving specific examples.

3.1 First Example FIG. 24 is a plan view showing a partial layout example of a pixel array section and a column ADC according to a first example of the third embodiment. As shown in FIG. 24, in the pixel array section 300 according to the first example, two ADCs 232 are associated with one column.

The pixel blocks 310 of 2n (n is an integer from 1 to N) rows are connected to one AD conversion section 230 via signal lines 306 to 309, and the number of rows is 2N (N is an integer). The pixel blocks 310 in the row are connected to the other AD converter 230 via different signal lines 306 to 309.

With this configuration, when reading multiple rows, it is possible to read out odd and even rows in parallel, thereby reducing the time difference between address event firing detection and pixel signal (gradation) readout. It becomes possible to further reduce the amount.

3.2 Second Example FIG. 25 is a plan view showing a partial layout example of a pixel array section and a column ADC according to a second example of the third embodiment. As shown in FIG. 25, in the second example, two ADCs 232 are arranged with a pixel array section 300 in between, in the same configuration as the first example.

In this way, by dividing the column ADC 220 into two and arranging the divided column ADC 220 at positions sandwiching the pixel array section 300, it is possible to reduce the circuit scale per column ADC 220.

3.3 Third Example FIG. 26 is a plan view showing a partial layout example of a pixel array section and a column ADC according to a third example of the third embodiment. As shown in FIG. 26, in the third example, the number of columns is 4M, and 4m columns and 4m-2 columns are connected to the column ADC 220 arranged above the pixel array section 300, and 4m-1 columns and The 4m-3 columns are connected to the column ADC 220 located below.

In the lower column ADC 220, an AD converter 230 is arranged for every K columns out of a total of 2M columns connected. When K is "2", M AD converters 530 are arranged. Note that the configuration of each AD converter 530 according to the third example may be the same as the AD converter 530 according to the second embodiment.

Similarly, AD converters 530 are arranged every two columns in the upper column ADC 220.

As described above, according to the third example, one AD conversion section 530 is shared by multiple columns, and the column ADC 220 is further divided into two and arranged at positions sandwiching the pixel array section 300. It becomes possible to reduce the circuit scale of the column ADC 220 as a whole, and also to reduce the circuit scale of each column ADC 220.

4. Fourth Embodiment Next, a fourth embodiment will be described in detail with reference to the drawings. Note that, in the following description, the same configurations and operations as those in the above-described embodiments will be cited and redundant descriptions will be omitted.

The configuration of the imaging device according to this embodiment may be the same as, for example, the imaging device 100 illustrated in the first embodiment. However, in this embodiment, the solid-state imaging device 200 is replaced with a solid-state imaging device 600, which will be described later.

4.1 Schematic configuration example of solid-state imaging device FIG. 27 is a block diagram showing a schematic configuration example of a solid-state imaging device according to the fourth embodiment. As shown in FIG. 27, the solid-state imaging device 600 has the same configuration as the solid-state imaging device 200 illustrated in FIG. .

The Y arbiter 601 has the same functions as the Y arbiter 213 in the first embodiment, as well as the functions of the drive circuit 211 in the first embodiment. Therefore, when the firing of an address event is detected in one or more pixel blocks 310 in the pixel array section 300, the Y arbiter 601 arbitrates the readout order for the rows to which each of the pixel blocks 310 in which firing of the address event has been detected belongs. Then, each row is driven in accordance with the arbitrated read order. As a result, pixel signals are read out in column parallel from each row to which the pixel block 310 in which the firing of the address event has been detected belongs.

4.2 Schematic Configuration Example of Y Arbiter FIG. 28 is a block diagram showing a schematic configuration example of the Y arbiter according to the fourth embodiment. As shown in FIG. 28, the Y arbiter 601 includes an event processing section 620 and a gradation pixel control section 610.

When receiving requests ReqY from a plurality of pixel blocks 310 belonging to different rows, the event processing unit 620 arbitrates the reading order for the rows and returns a response AckY according to the arbitration result to all the pixel blocks 310 belonging to the row. . In response, each pixel block 310 that has received the response AckY transmits a detection signal to the column ADC 220.

Further, the event processing unit 620 inputs the arbitrated readout order to the gradation pixel control unit 610. The gradation pixel control unit 610 drives the rows according to the input readout order. As a result, in all the pixel blocks 310 included in the driven row, a pixel signal having a voltage value corresponding to the amount of light incident on the photoelectric conversion element 321 appears on the vertical signal line 308.

The column ADC 220 reads out the pixel signals appearing on each vertical signal line 308 in column parallel, thereby reading out the pixel signals of one row at once.

4.3 Schematic Configuration Example of Event Processing Unit FIG. 29 is a block diagram showing a schematic configuration example of the event processing unit according to the fourth embodiment. As shown in FIG. 29, the event processing section 620 includes an address specifying section 621, a latch circuit 622, and a driver 623.

The latch circuit 622 is provided for each row and temporarily holds the request ReqY input from the pixel block 310. The latch circuit 622 then inputs the held request ReqY to the address specifying unit 621 in synchronization with the input clock CLK.

Based on the input request ReqY, the address specifying unit 621 specifies the Y address of the row to which the pixel block 310 that is the transmission source of the request ReqY belongs, and sends a response AckY to the driver 623 corresponding to the specified Y address. Output.

The driver 623 to which the response AckY has been input inputs the input response AckY to all pixel blocks 310 included in the row corresponding to the Y address.

4.4 Schematic configuration example of gradation pixel control unit FIG. 30 is a block diagram showing a schematic configuration example of a gradation pixel control unit according to the fourth embodiment. As shown in FIG. 30, the gradation pixel control section 610 includes an address generation section 611 and a driver 612.

The address generation unit 611 specifies the Y address of the pixel block 310 that is the transmission source of the detection signal, and inputs the specified Y address to the driver 612 in synchronization with the clock CLK.

The driver 612 inputs a reset signal RST, a transfer signal TRG, and a selection signal SEL as appropriate to all pixel blocks 310 included in the row of the Y address input from the address generation unit 611, thereby changing the pixels of the row. Drive all blocks 310.

4.5 Actions and Effects As described above, according to the present embodiment, it is possible to omit the drive circuit 211, so it is possible to reduce the circuit scale of the solid-state imaging device 600 and achieve miniaturization. Become.

Other configurations, operations, and effects may be the same as those of the embodiments described above, so detailed explanations will be omitted here.

5. Fifth Embodiment In the embodiment described above, when firing of an address event is detected in a certain pixel block 310, pixel signals are read out in column parallel from all the pixel blocks 310 included in the row to which the pixel block 310 belongs. An example is given below. On the other hand, in the fifth embodiment, pixel signals are periodically read out from all or some of the pixel blocks 310 regardless of the firing of an address event, and image data (hereinafter referred to as gradation level A case where image data (referred to as image data) is updated with event detection data will be explained using an example.

The configurations of the imaging device and solid-state imaging device according to this embodiment may be, for example, the same as those of the imaging device 100 and the solid-state imaging device 200, 200A, or 600 illustrated in the above-described embodiment. In the following description, a case based on the fourth embodiment will be exemplified. However, the base embodiment is not limited to the fourth embodiment, and other embodiments are also possible.

5.1 Operation example of solid-state imaging device In the present embodiment, the solid-state imaging device 200 performs an address event detection operation in which the firing of an address event is detected asynchronously, and a cycle in which gradation image data is periodically acquired from the pixel block 310. A specific read operation is performed.

5.1.1 Example of Event Detection Operation FIG. 31 is a flowchart illustrating an example of event detection operation according to the fifth embodiment. This operation is started, for example, when an application for detecting and imaging an address event is executed.

As shown in FIG. 31, the solid-state imaging device 200 starts detecting whether an address event has occurred (step S701), and determines whether an address event has occurred (step S702). Event pixel 330 is used to detect firing of an address event. If firing of an address event is not detected (NO in step S702), the operation proceeds to step S704.

On the other hand, if firing of the address event is detected (YES in step S702), the event encoder 250 outputs event detection data for the pixel block 310 in which firing of the address event was detected (step S703), and then the main Operation proceeds to step S704. Note that the event detection data read in step S703 is stored in the recording unit 120 or transmitted to the host 150 via the external I/F 140.

In step S704, the solid-state imaging device 200 determines whether or not to end this operation, and if it is to end (YES in step S704), ends this operation. On the other hand, if the process does not end (NO in step S704), the process returns to step S701, and the subsequent operations are executed.

5.1.2 Example of Periodic Read Operation FIG. 32 is a flowchart showing an example of the periodic read operation according to the fifth embodiment. Similar to the event detection operation, this operation is started, for example, when an application for detecting and imaging an address event is executed.

As shown in FIG. 32, the solid-state imaging device 200 starts measuring elapsed time (step S721), and waits until a predetermined time has elapsed (NO in step S722). Thereafter, when a predetermined period of time has elapsed (YES in step S722), the solid-state imaging device 200 causes the gradation pixel control unit 610 of the Y arbiter 601 to read out pixel signals from all pixel blocks 310 (step S723), and steps Proceed to S724. Note that the pixel signals read out in step S723 are stored in the recording unit 120 as gradation image data, or are transmitted to the host 150 via the external I/F 140.

In step S724, the solid-state imaging device 200 determines whether or not to end this operation, and if it ends (YES in step S724), ends this operation. On the other hand, if the process does not end (NO in step S724), the counter etc. that measure the elapsed time are reset (step S725), and then the process returns to step S722 and the subsequent operations are executed.

As described above, the gradation image data read out in the periodic readout operation is sequentially updated using the event detection data output in the event detection operation (gradation image data update operation). This gradation image data updating operation may be executed, for example, by the signal processing unit 212 within the solid-state imaging device 200, or may be executed by an external control unit 130, host 150, or the like.

5.2 Example of Gradation Image Data Update Operation Next, the tone image data update operation according to the fifth embodiment will be described in detail with reference to the drawings.

5.2.1 Flowchart FIG. 33 is a flowchart showing an example of the gradation image data update operation according to the fifth embodiment. Note that in this description, an example will be given of a case where the host 150 executes a gradation image data update operation.

As shown in FIG. 33, when the host 150 receives gradation image data from the solid-state imaging device 200 (step S301), the host 150 stores the input gradation image data in a predetermined memory (step S302).

Next, the host 150 determines whether event detection data has been input from the solid-state imaging device 200 within a predetermined time (step S303), and if not (NO in step S303), the process advances to step S308.

On the other hand, if event detection data is input (YES in step S303), the host 150 stores the input event detection data in a predetermined memory (step S304).

Subsequently, the host 150 determines whether the input event detection data indicates an on event or an off event (step S305), and if the input event detection data indicates an on event (YES in step S305), the event detection data The gradation value of the pixel is increased by adding a predetermined value to the gradation value (also referred to as pixel value) of the pixel specified from the move on.

Further, if the address event indicated by the input event detection data is not an on event, that is, an off event (NO in step S305), the host 150 selects a pixel specified from the X address and Y address included in the event detection data. By subtracting a predetermined value from the gradation value (also referred to as pixel value) of the pixel, the gradation value of the pixel is decreased (step S307), and the process proceeds to step S308.

In step S308, it is determined whether a predetermined time has elapsed since the input of the previous gradation image data, and if the predetermined time has not elapsed (NO in step S308), the host 150 returns to step S303. , execute the following operations. On the other hand, if the predetermined time has elapsed (YES in step S308), the host 150 determines whether or not to end this operation (step S309). If it is to end (YES in step S309), the host 150 finish. On the other hand, if the process does not end (NO in step S309), the host 150 returns to step S301, inputs the next gradation image data, and executes the subsequent operations. Note that the predetermined time may be the acquisition cycle of gradation image data in the solid-state imaging device 200, that is, the frame rate.

5.2.2 Timing Chart FIG. 34 is a timing chart showing an example of the operation of the solid-state imaging device according to the fifth embodiment. Note that FIG. 34 shows an example of the operation of the pixel block 310 in a certain column. Further, FIG. 35 is a timing chart for explaining updating of tone values focusing on the pixel block in the second row in FIG. 34.

First, as shown in FIG. 34, in this embodiment, in synchronization with the frame synchronization signal A signal read operation is performed.

On the other hand, in addition to the above-described periodic reset operation and readout operation for the gradation pixel 320, presence or absence of firing of an address event using the event pixel 330 is detected asynchronously.

Here, when paying attention to the pixel block 310 in the second row in FIG. 34, as shown in FIG. Until the pixel signal is read out from the gradation pixel 320, that is, during the period from timing t1 to t2, each time an address event is detected in the event pixel 330, it is determined whether the detected address event is an on event or not. It is increased or decreased by a predetermined value depending on whether it is an off event or not.

Similarly, in the period from the next timing t2 to t3, the gradation value based on the pixel signal read from the gradation pixel 320 at the timing t2 is determined by the address event at the event pixel 330 during the period from timing t2 to t3. Each time the address event is detected, it is increased or decreased by a predetermined value depending on whether the detected address event is an on event or an off event.

5.3 Effects/Effects In general, the time required for event detection is shorter than the time required for pixel signal readout because it does not require an accumulation period or transfer period like the pixel signal readout operation. is high. Therefore, as in the present embodiment, solid-state imaging It becomes possible to increase the time resolution of the gradation image data read out from the device 200, in other words, it becomes possible to increase the frame rate.

Furthermore, by accumulating periodically acquired gradation image data and asynchronously acquired event detection data in chronological order, it is possible to generate gradation images between frames after the fact. is also possible.

Other configurations, operations, and effects may be the same as those of the embodiments described above, so detailed explanations will be omitted here.

6. Sixth Embodiment In the fifth embodiment described above, pixel signals are periodically read out from all or some of the pixel blocks 310, regardless of the firing of an address event, and thereby the gray scale image data is read out. An example is shown in which the data is updated using event detection data. However, for a pixel block 310 in which firing of an address event is not detected during a certain period, it is highly likely that no change occurs in the gradation value due to the pixel signal read from the gradation pixel 320.

Therefore, in the sixth embodiment, when reading out periodic pixel signals from the pixel block 310, for the pixel block 310 in which firing of an address event was not detected during the immediately preceding period, pixels from the grayscale pixel 320 The case where signal reading is not performed will be explained by giving an example.

The configurations of the imaging device and the solid-state imaging device according to the present embodiment are similar to the fifth embodiment, and are, for example, the same as the imaging device 100 and the solid-state imaging device 200, 200A, or 600 exemplified in the above-described embodiment. good. However, in this embodiment, the event processing section 620 illustrated in FIG. 29 is replaced with an event processing section 720, which will be described later. Note that in the following description, a case based on the fourth embodiment will be exemplified, but the base embodiment is not limited to the fourth embodiment and may be other embodiments.

6.1 Schematic Configuration Example of Event Processing Unit FIG. 36 is a block diagram showing a schematic configuration example of the event processing unit according to the sixth embodiment. As shown in FIG. 36, the event processing section 720 has the same configuration as the event processing section 620 illustrated in FIG. 29, and further includes an address storage section 721.

In this embodiment, the address specifying unit 621 specifies the X address and Y address of the pixel block 310 that is the source of the request ReqY based on the input request ReqY, and specifies the address corresponding to the specified X address and Y address. A response AckY is output to the driver 623 that responds.

The driver 623 to which the response AckY has been input inputs the input response AckY to the pixel block 310 specified by the X address and the Y address.

Further, the address storage section 721 temporarily stores the X address and Y address (address information) specified by the address specifying section 621. Thereafter, the address storage section 721 inputs the held X address and Y address to the address generation section 611 of the gradation pixel control section 610 in synchronization with the frame synchronization signal XVS.

The address generation unit 611 of the gradation pixel control unit 610 inputs the X address and Y address input from the address storage unit 721 to the driver 612 in synchronization with the clock CLS. Then, the driver 612 appropriately inputs the reset signal RST, transfer signal TRG, and selection signal SEL to the pixel block 310 specified by the X address and Y address input from the address generation unit 611, thereby Activate block 310.

6.2 Example of Gradation Image Data Update Operation FIG. 37 is a timing chart showing an example of the operation of the solid-state imaging device according to the sixth embodiment. Note that, similarly to FIG. 34, FIG. 37 shows an example of the operation of the pixel block 310 in a certain column.

As shown in FIG. 37, in the present embodiment, for the pixel block 310 in which firing of an address event was not detected in the immediately preceding period T1, the reset operation of the gradation pixel 320 is performed in the next period T1. The pixel signal readout operation is not performed.

To explain this by focusing on the pixel block 310 in the first row and the pixel block 310 in the second row, in the period from timing t10 to t11, an address event is fired in the event pixel 330 of the pixel block 310 in the first row. is not detected. In this case, since the address storage unit 721 does not hold the X address and Y address of the pixel block 310 in the first row, in the next cycle (timing t12 to t13), the gradation pixel in the pixel block 310 in the first row Reset and read operations for 320 have not been performed.

On the other hand, regarding the pixel block 310 in the second row, since one or more address event firings have been detected during the period from timing t10 to t11, the pixel block 310 in the second row A reset operation and a read operation are being performed on the grayscale pixels 320 of the pixel block 310.

6.3 Actions and Effects As described above, according to the present embodiment, for the pixel block 310 in which firing of an address event was not detected during the immediately preceding period, pixel signals from the grayscale pixels 320 are not read out. Omitted. This makes it possible to simplify the periodic pixel signal readout operation, thereby making it possible to improve the operating speed and reduce power consumption of the solid-state imaging device 600.

Other configurations, operations, and effects may be the same as those of the embodiments described above, so detailed explanations will be omitted here.

7. Seventh Embodiment In the embodiment described above, the case where the gradation value of each pixel in gradation image data is updated based on the address event detected between frames was exemplified. On the other hand, in the seventh embodiment, a pixel signal is asynchronously read out from the grayscale pixel 320 of the pixel block 310 in which firing of an address event has been detected, and the readout pixel value is used to periodically read out the grayscale pixel 320. The case of updating tone image data will be explained by giving an example.

The configurations of the imaging device and the solid-state imaging device according to the present embodiment are similar to the fifth embodiment, and are, for example, the same as the imaging device 100 and the solid-state imaging device 200, 200A, or 600 exemplified in the above-described embodiment. good. However, in this embodiment, the pixel block 310 illustrated in FIG. 4 is replaced with a pixel block 810, which will be described later. Note that in the following description, a case based on the fourth embodiment will be exemplified, but the base embodiment is not limited to the fourth embodiment and may be other embodiments.

7.1 Configuration Example of Pixel Block FIG. 38 is a block diagram showing a schematic configuration example of a pixel block according to the seventh embodiment. As shown in FIG. 38, a pixel block 810 has, for example, the same configuration as the pixel block 310 illustrated in FIG. 4, but the tone pixels 320 further include a memory 801.

The memory 801 is a charge storage unit that temporarily holds charges generated in the photoelectric conversion element 321, and may be configured using, for example, a capacitive element formed on the same semiconductor substrate as the photoelectric conversion element 321.

Charges generated in the photoelectric conversion element 321 according to the amount of incident light are temporarily transferred to the memory 801 and held there. Thereafter, the charge held in the memory 801 is transferred to the floating diffusion layer 323 by a read operation for the gradation pixel 320, and then an operation similar to a normal read operation is performed.

7.2 Example of Pixel Signal Readout Operation FIG. 39 is a timing chart showing an example of the pixel signal readout operation according to the seventh embodiment. Note that FIG. 39 shows an example of the operation of the pixel block 810 in a certain column.

As shown in FIG. 39, in this embodiment, charges are transferred from the photoelectric conversion elements 321 in the gradation pixels 320 of each pixel block 810 to the memory 801 in synchronization with the frame synchronization signal XVS. After that, for example, the pixel signal readout operation is performed in order from the pixel block 810 in the first row to the pixel block 810 in the last row.

Note that the event detection operation may be the same as in the embodiment described above.

7.3 Actions and Effects As described above, by temporarily retaining the charge generated in the photoelectric conversion element 321 of the gradation pixel 320 in the memory 801, the shutter operation (reset operation) of all the pixel blocks 810 is performed. This makes it possible to realize a so-called global shutter operation in which the functions (corresponding to the above) are executed simultaneously.

Other configurations, operations, and effects may be the same as those of the embodiments described above, so detailed explanations will be omitted here.

7.4 Modification Note that for the pixel signal readout operation using the memory 801 according to the present embodiment, the pixel block 310 in which firing of an address event is not detected during a certain period illustrated in the sixth embodiment Regarding (810), it is also possible to combine a configuration in which reading out the pixel signal for the grayscale pixel 320 is omitted.

In that case, as illustrated in FIG. 40, for the pixel block 810 in which firing of an address event was not detected in the immediately preceding period T1, the reset operation of the grayscale pixel 320 and the pixel block 810 in the next period T1 are performed. The signal read operation is not executed. This makes it possible to simplify the periodic readout operation of pixel signals, thereby making it possible to improve the operating speed and reduce power consumption of the solid-state imaging device 600.

8. Eighth Embodiment In an eighth embodiment, modifications of the pixel blocks according to the above-described embodiments will be described by citing some examples. Note that the following description is based on the pixel block described using FIGS. 4 and 5 in the first embodiment, but the base pixel block is not limited to this, and may be based on pixels according to other embodiments. It may be a block.

Due to recent advances in process technology, the gradation pixels 320 are becoming increasingly finer. Therefore, when the gray scale pixel 320 and the event pixel 330 are combined as in the embodiment described above, an address event detection circuit for detecting whether or not an address event is fired from the gray scale pixel 320 and the event pixel 330. The difference in pitch (or size) from 400 becomes large.

Here, in the embodiment described above, for example, in the stacked chip illustrated in FIG. Gradation pixels 320 and event pixels 330 belonging to the pixel block 310 may be arranged.

Therefore, it is conceivable to add the gradation pixel 320 to the surplus area on the light receiving chip 201 caused by the size difference between the gradation pixel 320 and the address event detection circuit 400. In that case, a plurality of tone pixels 320 will belong to one pixel block 310.

However, if a plurality of gradation pixels 320 are made to correspond to one event pixel 330, there is a possibility that the sensitivity to firing of an address event will be reduced.

For example, in a distance measurement method using structured light (hereinafter referred to as structured light method), it is necessary to improve the positional accuracy of each dot by making the event pixels 330 finer to obtain the center of gravity of the dot. There is.

On the other hand, in the structured light method, by including an on/off code in the time direction in the emitted structured light dots, that is, by blinking each dot in a different pattern, the structured light is It becomes possible to specify which dot in the light it is, and it becomes possible to greatly simplify signal processing in distance measurement.

Therefore, in this embodiment, a plurality of event pixels 330 are arranged in a scattered manner within one pixel block, and the sum of their currents is received by one address event detection circuit 400, thereby increasing the sensitivity to the firing of an address event. The configuration of a pixel block that makes it possible to accurately obtain the center of gravity of a structured light dot without dropping the dots will be explained using an example.

FIG. 41 is a schematic diagram showing a schematic configuration example of a pixel block according to the eighth embodiment. In addition, in FIG. 41, the white squares in the light receiving chip 201 indicate the gradation pixels 320, and the hatched squares indicate the event pixels 330.

As shown in FIG. 41, a pixel block 910 according to the present embodiment includes one address event detection circuit 400, four event pixels 330, and 32 grayscale pixels 320.

A total of 36 pixels, including the event pixel 330 and the gradation pixel 320, are arranged in a 6×6 matrix. For example, if the size of the event pixel 330 and the size of the gradation pixel 320 are the same, and the size is a square with a side of 1.5 μm (micrometers), the pixel array 911 in a 6×6 matrix is as follows. This is a rectangular area of 6 μm square. In that case, the size of the address event detection circuit 400 in the detection chip 202 is preferably a rectangular area of 6 μm square.

Furthermore, in the pixel array 911 in each pixel block 910, the event pixels 330 are scattered at equal intervals (for example, every two pixels in the vertical and horizontal directions). In this way, by scattering the event pixels 330 at equal intervals, it is possible to accurately determine the center of gravity of the structured light dots.

Four event pixels 330 of the same pixel block 910 are connected to the same address event detection circuit 400. In this way, the address event detection circuit 400 receives the sum of currents from a plurality of (four in this example) event pixels 330, thereby accurately detecting the center of gravity of the structured light dot without reducing sensitivity to address event firing. It becomes possible to ask well.

As described above, according to the present embodiment, a plurality of event pixels 330 are arranged in a scattered manner within one pixel block 910, and the sum of their currents is received by one address event detection circuit 400. It becomes possible to accurately obtain the center of gravity of a structured light dot without reducing sensitivity to address event firing.

8.1 Modification In the eighth embodiment, by interspersing a plurality of event pixels 330 in one pixel block 910, the center of gravity of a structured light dot can be adjusted without reducing the sensitivity to the firing of an address event. Although the case where accurate determination is given is illustrated, the configuration is not limited to this.

For example, the size of the light receiving area of the event pixel 330 included in one pixel block 1010 may be increased. For example, as in a modification of the eighth embodiment shown in FIG. 42, the size of the light receiving area of one event pixel 330 may be the same size as the light receiving area of 2×2 gradation pixels 320. . In that case, the event pixel 330 will be arranged using a 2×2 pixel area in the 6×6 pixel array 1011.

Even with this configuration, it is possible to expand the light receiving area of the event pixel 330 and improve the sensitivity to address event firing, so the center of gravity of the structured light dot can be adjusted without reducing the sensitivity to address event firing. It becomes possible to obtain the data with high accuracy.

Furthermore, in the pixel array 911 shown in FIG. 41, the size of the light receiving area of each event pixel 330 may be increased as illustrated in FIG. 42.

9. Example of Application to Mobile Object The technology according to the present disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as a car, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility, airplane, drone, ship, robot, etc. You can.

FIG. 43 is a block diagram illustrating a schematic configuration example of a vehicle control system, which is an example of a mobile body control system to which the technology according to the present disclosure can be applied.

Vehicle control system 12000 includes a plurality of electronic control units connected via communication network 12001. In the example shown in FIG. 43, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an inside vehicle information detection unit 12040, and an integrated control unit 12050. Further, as the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio/image output section 12052, and an in-vehicle network I/F (interface) 12053 are illustrated.

The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 includes a drive force generation device such as an internal combustion engine or a drive motor that generates drive force for the vehicle, a drive force transmission mechanism that transmits the drive force to wheels, and a drive force transmission mechanism that controls the steering angle of the vehicle. It functions as a control device for a steering mechanism to adjust and a braking device to generate braking force for the vehicle.

The body system control unit 12020 controls the operations of various devices installed in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a headlamp, a back lamp, a brake lamp, a turn signal, or a fog lamp. In this case, radio waves transmitted from a portable device that replaces a key or signals from various switches may be input to the body control unit 12020. The body system control unit 12020 receives input of these radio waves or signals, and controls the door lock device, power window device, lamp, etc. of the vehicle.

External information detection unit 12030 detects information external to the vehicle in which vehicle control system 12000 is mounted. For example, an imaging section 12031 is connected to the outside-vehicle information detection unit 12030. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image of the exterior of the vehicle, and receives the captured image. The external information detection unit 12030 may perform object detection processing such as a person, car, obstacle, sign, or text on the road surface or distance detection processing based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of received light. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or non-visible light such as infrared rays.

The in-vehicle information detection unit 12040 detects in-vehicle information. For example, a driver condition detection section 12041 that detects the condition of the driver is connected to the in-vehicle information detection unit 12040. The driver condition detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 detects the degree of fatigue or concentration of the driver based on the detection information input from the driver condition detection unit 12041. It may be calculated, or it may be determined whether the driver is falling asleep.

The microcomputer 12051 calculates control target values for the driving force generation device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, Control commands can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions, including vehicle collision avoidance or impact mitigation, following distance based on vehicle distance, vehicle speed maintenance, vehicle collision warning, vehicle lane departure warning, etc. It is possible to perform cooperative control for the purpose of

In addition, the microcomputer 12051 controls the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform cooperative control for the purpose of autonomous driving, etc., which does not rely on operation.

Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or oncoming vehicle detected by the vehicle exterior information detection unit 12030, and performs cooperative control for the purpose of preventing glare, such as switching from high beam to low beam. It can be carried out.

The audio image output unit 12052 transmits an output signal of at least one of audio and image to an output device that can visually or audibly notify information to a passenger of the vehicle or to the outside of the vehicle. In the example of FIG. 43, an audio speaker 12061, a display section 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

FIG. 44 is a diagram showing an example of the installation position of the imaging section 12031.

In FIG. 44, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. An imaging unit 12101 provided in the front nose and an imaging unit 12105 provided above the windshield inside the vehicle mainly acquire images in front of the vehicle 12100. Imaging units 12102 and 12103 provided in the side mirrors mainly capture images of the sides of the vehicle 12100. An imaging unit 12104 provided in the rear bumper or back door mainly captures images of the rear of the vehicle 12100. The imaging unit 12105 provided above the windshield inside the vehicle is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

Note that FIG. 44 shows an example of the imaging range of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging section 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate imaging ranges of imaging sections 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 shows the imaging range of imaging section 12101 provided on the front nose. The imaging range of the imaging unit 12104 provided in the rear bumper or back door is shown. For example, by overlapping the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 is obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of image sensors, or may be an image sensor having pixels for phase difference detection.

For example, the microcomputer 12051 determines the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the temporal change in this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. In particular, by determining the three-dimensional object that is closest to the vehicle 12100 on its path and that is traveling at a predetermined speed (for example, 0 km/h or more) in approximately the same direction as the vehicle 12100, it is possible to extract the three-dimensional object as the preceding vehicle. can. Furthermore, the microcomputer 12051 can set an inter-vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform cooperative control for the purpose of autonomous driving, etc., in which the vehicle travels autonomously without depending on the driver's operation.

For example, the microcomputer 12051 transfers three-dimensional object data to other three-dimensional objects such as two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, and utility poles based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the degree of risk of collision with each obstacle, and when the collision risk exceeds a set value and there is a possibility of a collision, the microcomputer 12051 transmits information via the audio speaker 12061 and the display unit 12062. By outputting a warning to the driver and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether the pedestrian is present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition involves, for example, a procedure for extracting feature points in images captured by the imaging units 12101 to 12104 as infrared cameras, and a pattern matching process is performed on a series of feature points indicating the outline of an object to determine whether it is a pedestrian or not. This is done by a procedure that determines the When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 creates a rectangular outline for emphasis on the recognized pedestrian. The display unit 12062 is controlled to display the . Furthermore, the audio image output unit 12052 may control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the imaging unit 12031 among the configurations described above. Specifically, the imaging device 100 in FIG. 1 can be applied to the imaging unit 12031. By applying the technology according to the present disclosure to the imaging unit 12031, the power consumption of the imaging unit 12031 can be reduced, and therefore the power consumption of the entire vehicle control system can be reduced.

Note that the above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the matters specifying the invention in the claims have a corresponding relationship, respectively. Similarly, the matters specifying the invention in the claims and the matters in the embodiments of the present technology having the same names have a corresponding relationship, respectively. However, the present technology is not limited to the embodiments, and can be realized by making various modifications to the embodiments without departing from the gist thereof.

Further, the processing procedure described in the above embodiment may be regarded as a method having a series of these procedures, or as a program for causing a computer to execute this series of procedures, or a recording medium that stores the program. You can. As this recording medium, for example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disc), a memory card, a Blu-ray Disc (Blu-ray (registered trademark) Disc), etc. can be used.

Note that the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

Note that the present technology can also have the following configuration.
(1)
a pixel array section including a plurality of pixel blocks arranged in a matrix;
a drive circuit that generates a pixel signal in a first pixel block in which firing of an address event is detected among the plurality of pixel blocks;
Equipped with
Each of the pixel blocks is
a first photoelectric conversion element that generates a charge according to the amount of incident light;
a detection unit that detects firing of the address event based on the charge generated in the first photoelectric conversion element;
a second photoelectric conversion element that generates a charge according to the amount of incident light;
a pixel circuit that generates a pixel signal based on the charge generated in the second photoelectric conversion element;
A solid-state imaging device comprising:
(2)
The solid-state imaging device according to (1), wherein the drive circuit generates a pixel signal for each of the plurality of second pixel blocks included in the row to which the first pixel block belongs.
(3)
The solid-state imaging device according to (2), further comprising a conversion unit that reads out the pixel signals generated by each of the plurality of second pixel blocks in parallel.
(4)
When a plurality of the first pixel blocks exist and at least one of the plurality of first pixel blocks belongs to a different row, the readout order for each of the rows to which one or more of the first pixel blocks belongs is determined. The solid-state imaging device according to (2) or (3), further comprising an arbitration unit that makes a decision.
(5)
The solid-state imaging device according to (4), wherein the arbitration section includes the drive circuit.
(6)
The first pixel block outputs a request to the arbitration unit to arbitrate the readout order for the row to which the first pixel block belongs,
The arbitration unit includes a plurality of latch circuits that are provided on a one-to-one basis for each row and temporarily hold the requests input from the corresponding rows,
Each of the latch circuits inputs the held request to the determination unit in synchronization with a clock input from the outside,
The solid-state imaging device according to (4) or (5), wherein the arbitration unit determines the read order based on the request input via the latch circuit.
(7)
The solid-state imaging device according to any one of (1) to (6), wherein the drive circuit causes at least one third pixel block of the plurality of pixel blocks to generate the pixel signal at a predetermined period. .
(8)
When a plurality of the first pixel blocks exist and at least one of the plurality of first pixel blocks belongs to a different row, reading for each of the plurality of rows to which one or more of the first pixel blocks belongs. further comprising an arbitration section that determines the order;
The arbitration unit includes an address storage unit that stores address information that specifies a position in the pixel array unit of the first pixel block that detected the address event within a predetermined period,
In (7) above, the drive circuit generates the pixel signal at the predetermined period by using the first pixel block specified by the address information stored in the address storage section as the third pixel block. The solid-state imaging device described.
(9)
A gradation value indicated by a pixel signal read out from the third pixel block at the predetermined period based on the number of address events detected in the third pixel block within the period specified by the predetermined period. The solid-state imaging device according to (7) or (8), further comprising a signal processing unit that increases or decreases the amount of the signal.
(10)
Each of the pixel blocks further includes a memory that temporarily holds charges generated in the second photoelectric conversion element,
The drive circuit transmits a pixel signal to the first pixel block based on the charge held in the memory of the first pixel block when the first pixel block detects firing of the address event. The solid-state imaging device according to (1) above.
(11)
When a plurality of the first pixel blocks exist and at least one of the plurality of first pixel blocks belongs to a different row, reading for each of the plurality of rows to which one or more of the first pixel blocks belongs. further comprising an arbitration section that determines the order;
The arbitration unit includes an address storage unit that stores address information that specifies a position in the pixel array unit of the first pixel block that detected the address event within a predetermined period,
The solid-state imaging device according to (10), wherein the drive circuit causes the first pixel block specified by the address information stored in the address storage section to generate a pixel signal at a predetermined cycle.
(12)
Each of the pixel blocks includes a plurality of the first photoelectric conversion elements,
The solid-state imaging device according to any one of (1) to (11), wherein the plurality of first photoelectric conversion elements are connected to the detection section.
(13)
Each of the pixel blocks further includes a plurality of the second photoelectric conversion elements,
The plurality of first photoelectric conversion elements and the plurality of second photoelectric conversion elements form a matrix arrangement,
The solid-state imaging device according to (12), wherein the plurality of first photoelectric conversion elements are scattered at equal intervals in the matrix arrangement.
(14)
The solid-state imaging device according to any one of (1) to (13), wherein the size of the light receiving area of the first photoelectric conversion element is larger than the size of the light receiving area of the second photoelectric conversion element.
(15)
a solid-state imaging device;
an optical system that images incident light on a light receiving surface of the solid-state imaging device;
a recording unit that stores image data acquired by the solid-state imaging device;
Equipped with
The solid-state imaging device includes:
a pixel array section including a plurality of pixel blocks arranged in a matrix;
a drive circuit that generates a pixel signal in a first pixel block in which firing of an address event is detected among the plurality of pixel blocks;
Equipped with
Each of the pixel blocks is
a first photoelectric conversion element that generates a charge according to the amount of incident light;
a detection unit that detects firing of the address event based on the charge generated in the first photoelectric conversion element;
a second photoelectric conversion element that generates a charge according to the amount of incident light;
a pixel circuit that generates a pixel signal based on the charge generated in the second photoelectric conversion element;
An imaging device comprising:

100 Imaging device 110 Optical system 120 Recording unit 130 Control unit 140 External I/F
150 host 200, 200A, 600 solid-state imaging device 201 light receiving chip 202 detection chip 211 drive circuit 212 signal processing unit 213, 601 Y arbiter 220 column ADC
230, 530 AD conversion section 233 Comparator 234 Counter 240, 540 Control circuit 241 OR gate 242 Level shifter 243 AND gate 250 Event encoder 300 Pixel array section 306, 307 Detection signal line 308 Vertical signal line 309 Enable signal line 310, 310A, 810 , 910, 1010 Pixel block 320 Gradation pixel 321, 331, 341 Photoelectric conversion element 322 Transfer transistor 323 Floating diffusion layer 324 Reset transistor 325 Amplification transistor 326 Selection transistor 330 Event pixel 332 OFG transistor 400 Address event detection circuit 400A Address event detection section 410, 410A Current-voltage conversion unit 411, 414 LG transistor 412 Load MOS transistor 413, 415 Amplification transistor 420 Buffer 430 Subtractor 431, 433 Capacitor 432 Inverter 434 Switch 440 Quantizer 441, 442 Comparator 450 Transfer unit 451, 453 AND gate 452 OR gate 454, 455 Flip-flop 531 Multiplexer 545 Switching control unit 544 Demultiplexer 610 Gradation pixel control unit 611 Address generation unit 612 Driver 620, 720 Event processing unit 621 Address identification unit 622 Latch circuit 623 Driver 721 Address storage unit 801 Memory 911, 1011 pixel array

Claims (12)

a pixel array section including a plurality of pixel blocks arranged in a matrix;
a drive circuit that generates a pixel signal in a first pixel block in which firing of an address event is detected among the plurality of pixel blocks;
Equipped with
Each of the pixel blocks is
a first photoelectric conversion element that generates a charge according to the amount of incident light;
a detection unit that detects firing of the address event based on the charge generated in the first photoelectric conversion element;
a second photoelectric conversion element that generates a charge according to the amount of incident light;
a pixel circuit that generates a pixel signal based on the charge generated in the second photoelectric conversion element;
Equipped with
The drive circuit generates a pixel signal for each of the plurality of second pixel blocks included in the row to which the first pixel block belongs,
Solid-state imaging device. The solid-state imaging device according to claim 1 , further comprising a conversion unit that reads out the pixel signals generated by each of the plurality of second pixel blocks in parallel. When a plurality of the first pixel blocks exist and at least one of the plurality of first pixel blocks belongs to a different row, the readout order for each of the rows to which one or more of the first pixel blocks belongs is determined. The solid-state imaging device according to claim 1 or 2, further comprising an arbitration unit that makes a decision. The solid-state imaging device according to claim 3 , wherein the arbitration section includes the drive circuit. The first pixel block outputs a request to the arbitration unit to arbitrate the readout order for the row to which the first pixel block belongs,
The arbitration unit includes a plurality of latch circuits that are provided on a one-to-one basis for each row and temporarily hold the requests input from the corresponding rows,
Each of the latch circuits inputs the held request to the arbitration unit in synchronization with a clock input from the outside,
The solid-state imaging device according to claim 3 , wherein the arbitration unit determines the read order based on the request input via the latch circuit. The solid-state imaging device according to claim 1, wherein the drive circuit causes at least one third pixel block of the plurality of pixel blocks to generate the pixel signal at a predetermined period. When a plurality of the first pixel blocks exist and at least one of the plurality of first pixel blocks belongs to a different row, reading for each of the plurality of rows to which one or more of the first pixel blocks belongs. further comprising an arbitration section that determines the order;
The arbitration unit includes an address storage unit that stores address information that specifies a position in the pixel array unit of the first pixel block that detected the address event within a predetermined period,
The drive circuit sets a plurality of second pixel blocks included in a row to which the first pixel block specified by the address information stored in the address storage section belongs as the third pixel block, and performs the predetermined period. The solid-state imaging device according to claim 6 , wherein the pixel signal is generated. A gradation value indicated by a pixel signal read out from the third pixel block at the predetermined period based on the number of address events detected in the third pixel block within the period specified by the predetermined period. The solid-state imaging device according to claim 6 or 7, further comprising a signal processing section that increases or decreases the signal. Each of the pixel blocks includes a plurality of the first photoelectric conversion elements,
The solid-state imaging device according to claim 1, wherein the plurality of first photoelectric conversion elements are connected to the detection section. Each of the pixel blocks further includes a plurality of the second photoelectric conversion elements,
The plurality of first photoelectric conversion elements and the plurality of second photoelectric conversion elements form a matrix arrangement,
The solid-state imaging device according to claim 9 , wherein the plurality of first photoelectric conversion elements are scattered at equal intervals in the matrix arrangement. The solid-state imaging device according to claim 1, wherein the size of the light receiving area of the first photoelectric conversion element is larger than the size of the light receiving area of the second photoelectric conversion element. a solid-state imaging device;
an optical system that images incident light on a light receiving surface of the solid-state imaging device;
a recording unit that stores image data acquired by the solid-state imaging device;
Equipped with
The solid-state imaging device includes:
a pixel array section including a plurality of pixel blocks arranged in a matrix;
a drive circuit that generates a pixel signal in a first pixel block in which firing of an address event is detected among the plurality of pixel blocks;
Equipped with
Each of the pixel blocks is
a first photoelectric conversion element that generates a charge according to the amount of incident light;
a detection unit that detects firing of the address event based on the charge generated in the first photoelectric conversion element;
a second photoelectric conversion element that generates a charge according to the amount of incident light;
a pixel circuit that generates a pixel signal based on the charge generated in the second photoelectric conversion element;
Equipped with
The drive circuit generates a pixel signal for each of the plurality of second pixel blocks included in the row to which the first pixel block belongs,
Imaging device.

Images