JP2023162463A - Imaging device and imaging method - Google Patents

Abstract

To provide an imaging device and an imaging method that enable reading an event at a higher speed while suppressing an increase in circuit size.SOLUTION: An imaging device includes: a signal holding circuit that holds a first event signal indicating whether or not a luminance signal based on luminance has exceeded a first threshold value; and an arithmetic circuit that, on the basis of the first event signal held in the signal holding circuit and a second event signal indicating whether or not the luminance signal has exceeded a second threshold value different from the first threshold value, generates an event signal indicating whether or not the luminance signal has exceeded the first threshold value and/or the second threshold value.SELECTED DRAWING: Figure 12

Description

The present disclosure relates to an imaging device and an imaging method.

Synchronous solid-state imaging devices that capture image data (frames) in synchronization with a synchronization signal such as a vertical synchronization signal are used in imaging devices and the like. This general synchronous solid-state image sensor can only acquire image data every synchronous signal period (for example, 1/60 seconds), so it is useful for faster processing in fields such as transportation and robots. It becomes difficult to respond when requested. Therefore, an asynchronous solid-state image sensor has been proposed in which each pixel is provided with a detection circuit that detects, in real time, as an address event, that the amount of light at that pixel exceeds a threshold value for each pixel address. A solid-state image sensor that detects an address event for each pixel in this way is called a DVS (Dynamic Vision Sensor). It can generate and output data much faster than synchronous solid-state image sensors. Therefore, for example, in the transportation field, it is possible to perform image recognition processing for people and obstacles at high speed, thereby improving safety.

JP 2017-50853 Publication

On the other hand, the address event includes an on event indicating that the amount of increase in brightness exceeds the upper limit threshold, and an off event indicating that the amount of decrease in brightness falls below the lower limit threshold that is less than the upper limit threshold. Therefore, a signal holding circuit that holds a signal indicating the presence or absence of an on event and a signal holding circuit that holds a signal indicating the presence or absence of an off event are required, which increases the circuit scale. In addition, in order to suppress the increase in circuit scale, if the signal holding circuit is shared between on event and off event, the signal holding circuit will generate an on event signal indicating the presence or absence of an on event, and an off event signal indicating the presence or absence of an off event. It becomes necessary to read out the signal twice in order.

Therefore, the present disclosure provides an imaging device and an imaging method that can read events at higher speed while suppressing an increase in circuit scale.

In order to solve the above problems, according to the present disclosure, a signal holding circuit that holds a first event signal indicating whether a brightness signal based on brightness exceeds a first threshold;
Based on a second event signal indicating whether the brightness signal exceeds a second threshold different from the first threshold and the first event signal held in the signal holding circuit, the brightness signal an arithmetic circuit that generates an event signal indicating whether at least one of the threshold and the second threshold has been exceeded;
An imaging device is provided.

the arithmetic circuit holds the event signal in the signal holding circuit;
a transfer unit that transfers the event signal held in the signal holding circuit;
Further provision may be made.

The first event signal and the second event signal are sequentially input to the arithmetic circuit,
The device may have a first mode in which the first event signal and the second event signal are sequentially output to the signal holding circuit in response to input of a control signal.

The first event signal and the second event signal are sequentially input to the arithmetic circuit,
a first event signal outputting the first event signal to the signal holding circuit in response to input of a control signal, and generating the event signal based on the first event signal and the second event signal held in the signal holding circuit; It may have two modes.

The first event signal and the second event signal are signals containing information of at least one of a true value and a false value,
The arithmetic circuit may include an OR circuit and perform an OR operation on the first event signal and the second event signal.

The first event signal and the second event signal are sequentially input to the arithmetic circuit,
The arithmetic circuit includes the OR circuit and a multiplexer,
The multiplexer outputs the first event signal to the signal holding circuit according to a control signal,
The logical sum circuit may perform a logical sum operation of the first event signal held in the signal holding circuit and the second event signal.

The multiplexer may output the result of the pre-rational sum operation to the signal holding circuit according to the pre-control signal.

The first event signal and the second event signal are sequentially input to the arithmetic circuit,
A false value is initially set in the signal holding circuit,
The logical sum circuit holds a logical sum operation of the false value and the first event signal in the signal holding circuit, and then performs a logical sum operation of the held result of the logical sum operation and the second event signal. It's okay.

The arithmetic circuit may further include an AND circuit.

The first event signal and the second event signal are sequentially input to the arithmetic circuit,
A false value is set at one end of the AND circuit,
The OR circuit holds the OR operation of the output signal of the AND circuit and the first event signal in the signal holding circuit,
Next, a true value is set at the one end of the AND circuit, and the result of the held OR operation, the result of the AND of the true value, and the second event signal are input to the OR circuit. is,
The OR circuit may perform an OR operation between the held result of the OR operation, the result of the AND operation of the true value, and the second event signal.

The arithmetic circuit may include an equivalent circuit of at least one of the OR circuit and the AND circuit.

Within the same pixel,
a photoelectric conversion element that outputs a signal according to the luminance;
a current-voltage converter that logarithmically converts the signal;
A difference circuit may be configured to generate the first event signal and the second event signal based on the signal logarithmically converted by the current-voltage converter.

In the same pixel,
The signal holding circuit and the arithmetic circuit may be further configured.

a plurality of pixels having a photoelectric conversion unit that outputs a signal according to brightness, and a current-voltage conversion unit that logarithmically converts the signal;
a difference circuit that sequentially generates the first event signal and the second event signal based on the logarithmically converted signals in each of the current-voltage converters;
a plurality of the signal holding circuits respectively corresponding to the plurality of pixels;
The plurality of signal holding circuits and the arithmetic circuit may process the first event signal and the second event signal generated in the order.

According to the present disclosure, a signal holding step of holding a first event signal indicating whether a brightness signal based on brightness exceeds a first threshold value in a signal holding circuit;
Based on a second event signal indicating whether the brightness signal exceeds a second threshold different from the first threshold and the first event signal held in the signal holding circuit, the brightness signal an arithmetic processing step of generating an event signal indicating whether at least one of the threshold and the second threshold has been exceeded;
An imaging method is provided, comprising:

FIG. 1 is a block diagram illustrating an example of the configuration of an imaging device to which the technology according to the present disclosure is applied. FIG. 1 is a block diagram showing an example of the configuration of an imaging device according to a first configuration example. FIG. 2 is a block diagram showing an example of the configuration of a pixel array section. FIG. 2 is a circuit diagram showing an example of a pixel circuit configuration. FIG. 3 is a block diagram showing a configuration example of an address event detection section. FIG. 7 is a diagram showing a first comparative example with a two-latch configuration. FIG. 7 is a diagram showing a second comparative example with a two-latch configuration. The figure which shows the 3rd comparative example by 1 latch structure. 6 is a diagram showing an example of reading in the configuration example shown in FIG. 5. FIG. FIG. 3 is a circuit diagram showing an example of the configuration of a current-voltage conversion section in an address event detection section. FIG. 3 is a circuit diagram showing an example of the configuration of a subtracter and a quantizer in an address event detection section. FIG. 1 is a block diagram illustrating an example of the configuration of a scan-type imaging device. FIG. 2 is an exploded perspective view schematically showing a stacked chip structure of the imaging device. FIG. 2 is a block diagram illustrating an example of a configuration of a column processing section of an imaging device according to a first configuration example. FIG. 3 is a circuit diagram showing a first configuration example of an arithmetic circuit and a signal holding circuit in an address event detection section. The figure which shows the truth table of a multiplexer. A diagram showing a truth table of an arithmetic circuit. FIG. 3 is a circuit diagram showing a second configuration example of an arithmetic circuit and a signal holding circuit. A diagram showing a truth table of an OR circuit. FIG. 7 is a diagram showing a truth table of an arithmetic circuit in a second configuration example. The figure which shows the equivalent circuit example of the arithmetic circuit in the 2nd example of a structure. A diagram showing an equivalent circuit of a NAND circuit based on De Morgan's theorem. A diagram showing an equivalent circuit of NOR based on De Morgan's theorem. 19 is a diagram showing an example of an equivalent circuit of the arithmetic circuit shown in FIG. 18. FIG. 21 is a diagram showing an example of an equivalent circuit of the arithmetic circuit shown in FIG. 20. FIG. FIG. 7 is a circuit diagram showing a third configuration example of an arithmetic circuit and a signal holding circuit. FIG. 7 is a diagram showing a truth table of an OR circuit in a third configuration example. FIG. 7 is a block diagram showing an example of the configuration of a pixel array section according to a second embodiment. FIG. 3 is a block diagram showing an example of the configuration of a detection unit according to a second embodiment. FIG. 7 is a block diagram showing an example of the configuration of a pixel array section and a column processing section according to a second embodiment. FIG. 7 is a block diagram showing an example of the configuration of a pixel array section according to a third embodiment. FIG. 3 is a block diagram showing an example of the configuration of an arithmetic circuit for a pixel block. FIG. 1 is a block diagram illustrating a schematic configuration example of a vehicle control system, which is an example of a mobile body control system to which the technology according to the present disclosure can be applied. The figure which shows the example of the installation position of an imaging part and an information detection part outside a vehicle.

Embodiments of an imaging device and an imaging method will be described below with reference to the drawings. Although the main components of the imaging device will be mainly described below, the imaging device may include components and functions that are not shown or explained. The following description does not exclude components or features not shown or described.

FIG. 1 is a block diagram illustrating an example of the system configuration of an imaging system to which the technology according to the present disclosure is applied.

As shown in FIG. 1, an imaging system 10 to which the technology according to the present disclosure is applied includes an imaging lens 11, an imaging device 20, a recording section 12, and a control section 13. This imaging system 10 is an example of an electronic device according to the present disclosure, and examples of the electronic device include a camera system mounted on an industrial robot, a vehicle-mounted camera system, and the like.

In the imaging system 10 configured as described above, the imaging lens 11 takes in incident light from a subject and forms an image on the imaging surface of the imaging device 20 . The imaging device 20 photoelectrically converts incident light taken in by the imaging lens 11 on a pixel-by-pixel basis to obtain imaging data. As this imaging device 20, an imaging device of the present disclosure, which will be described later, is used.

The imaging device 20 performs predetermined signal processing such as image recognition processing on the captured image data, and outputs the processing results and an address event detection signal (hereinafter simply referred to as "detection signal") to be described later. data indicating that there is) is output to the recording unit 12. A method of generating the address event detection signal will be described later. The recording unit 12 stores data supplied from the imaging device 20 via the signal line 14. The control unit 13 is configured by, for example, a microcomputer, and controls the imaging operation in the imaging device 20.

[Imaging device according to first configuration example (arbiter method)]
FIG. 2 is a block diagram illustrating an example of the configuration of an imaging device according to a first configuration example used as the imaging device 20 in the imaging system 10 to which the technology according to the present disclosure is applied.

As shown in FIG. 2, the imaging device 20 according to the first configuration example as an imaging device of the present disclosure is an asynchronous imaging device called DVS, which includes a pixel array section 21, a driving section 22, an arbiter section (arbitration section ) 23, a column processing section 24, and a signal processing section 25.

In the imaging device 20 having the above configuration, a plurality of pixels 30 are two-dimensionally arranged in a matrix (array) in the pixel array section 21. Vertical signal lines VSL (Vertical Signal Lin), which will be described later, are wired for each pixel column in this matrix-like pixel arrangement.

Each of the plurality of pixels 30 generates, as a pixel signal, an analog signal of a voltage corresponding to the photocurrent. Furthermore, each of the plurality of pixels 30 detects the presence or absence of an address event based on whether the amount of change in photocurrent exceeds a predetermined threshold. Then, when an address event occurs, the pixel 30 outputs a request to the arbiter section 23.

The driving unit 22 drives each of the plurality of pixels 30 to output a pixel signal generated by each pixel 30 to the column processing unit 24.

The arbiter unit 23 arbitrates requests from each of the plurality of pixels 30 and transmits a response to the pixel 30 based on the arbitration result. The pixel 30 that has received the response from the arbiter section 23 supplies a detection signal (address event detection signal) indicating the detection result to the drive section 22 and the signal processing section 25 . Regarding the reading of detection signals from the pixels 30, it is also possible to read out a plurality of lines.

The column processing section 24 is composed of, for example, an analog-to-digital converter, and performs processing for converting analog pixel signals output from the pixels 30 of that column into digital signals for each pixel column of the pixel array section 21. The column processing section 24 then supplies the digital signal after analog-to-digital conversion to the signal processing section 25.

The signal processing unit 25 performs predetermined signal processing such as CDS (Correlated Double Sampling) processing and image recognition processing on the digital signal supplied from the column processing unit 24 . Then, the signal processing section 25 supplies data indicating the processing result and the detection signal supplied from the arbiter section 23 to the recording section 12 (see FIG. 1) via the signal line 14.

[Example of configuration of pixel array section]
FIG. 3 is a block diagram showing an example of the configuration of the pixel array section 21. As shown in FIG.

In a pixel array section 21 in which a plurality of pixels 30 are two-dimensionally arranged in a matrix, each of the plurality of pixels 30 has a light receiving section 31, a pixel signal generating section 32, and an address event detecting section 33. ing.

In the pixel 30 having the above configuration, the light receiving section 31 photoelectrically converts incident light to generate a photocurrent. Then, the light receiving section 31 supplies the photocurrent generated by photoelectric conversion to either the pixel signal generating section 32 or the address event detecting section 33 under the control of the driving section 22 (see FIG. 2).

The pixel signal generation unit 32 generates a voltage signal corresponding to the photocurrent supplied from the light receiving unit 31 as a pixel signal SIG, and transmits the generated pixel signal SIG to the column processing unit 24 ( (see Figure 2).

The address event detection section 33 detects the presence or absence of an address event based on whether the amount of change in photocurrent from each of the light receiving sections 31 exceeds a predetermined threshold. The address event includes, for example, an on event indicating that the amount of change in photocurrent exceeds an upper threshold, and an off event indicating that the amount of change falls below a lower threshold. In addition, the address event detection unit 33 according to the present embodiment may receive, for example, a 1-bit on-event signal indicating whether an on-event has occurred, a 1-bit off-event signal indicating whether an off-event has occurred, or a 1-bit off-event signal indicating whether an off-event has occurred. It is possible to generate a 1-bit event signal indicating whether either an on event or an off event has occurred. For example, the on-event signal, off-event signal, and event signal indicate a true value (eg, 1) or a false value (eg, 0). For example, the on-event signal indicates a true value when an on-event occurs, and indicates a false value when no on-event occurs. Similarly, the off-event signal indicates a true value when an off-event occurs, and indicates a false value when no off-event occurs. Similarly, the event signal indicates a true value when at least one of an on event and an off event occurs, and indicates a false value when neither an on event nor an off event occurs.

When an address event occurs, the address event detection section 33 supplies the arbiter section 23 (see FIG. 2) with a request for transmitting an address event detection signal. Then, upon receiving a response to the request from the arbiter section 23, the address event detection section 33 supplies an address event detection signal to the drive section 22 and the signal processing section 25.

[Example of pixel circuit configuration]
FIG. 4 is a circuit diagram showing an example of the circuit configuration of the pixel 30. As described above, each of the plurality of pixels 30 includes a light receiving section 31, a pixel signal generating section 32, and an address event detecting section 33.

In the pixel 30 having the above configuration, the light receiving section 31 includes a light receiving element (photoelectric conversion element) 311, a transistor 312, and a transistor 313. As the transistors 312 and 313, for example, N-type MOS (Metal Oxide Semiconductor) transistors are used. Transistor 312 and transistor 313 are connected in series with each other.

The light receiving element 311 is connected between the common connection node _N1 of the transistor 312 and the transistor 313 and the ground, and photoelectrically converts incident light to generate an amount of charge corresponding to the amount of the incident light.

A transfer signal TRG is supplied to the gate electrode of the transistor 312 from the drive section 22 shown in FIG. The transistor 312 supplies the charge photoelectrically converted by the light receiving element 311 to the pixel signal generation section 32 in response to the transfer signal TRG.

A control signal OFG is supplied from the drive section 22 to the gate electrode of the transistor 313. The transistor 313 supplies the electric signal generated by the light receiving element 311 to the address event detection section 33 in response to the control signal OFG. The electrical signal supplied to the address event detection section 33 is a photocurrent made of electric charge.

The pixel signal generation section 32 has a configuration including a reset transistor 321, an amplification transistor 322, a selection transistor 323, and a floating diffusion layer 324. As the reset transistor 321, the amplification transistor 322, and the selection transistor 323, for example, N-type MOS transistors are used.

The pixel signal generation section 32 is supplied with charge photoelectrically converted by the light receiving element 311 from the light receiving section 31 by the transistor 312 . Charges supplied from the light receiving section 31 are accumulated in the floating diffusion layer 324. The floating diffusion layer 324 generates a voltage signal whose voltage value corresponds to the amount of accumulated charge. That is, the floating diffusion layer 324 converts charge into voltage.

The reset transistor 321 is connected between the power supply line of the power supply voltage V _DD and the floating diffusion layer 324 . A reset signal RST is supplied from the drive unit 22 to the gate electrode of the reset transistor 321 . The reset transistor 321 initializes (resets) the amount of charge in the floating diffusion layer 324 in response to the reset signal RST.

The amplification transistor 322 is connected in series with the selection transistor 323 between the power supply line of the power supply voltage _VDD and the vertical signal line VSL. The amplification transistor 322 amplifies the voltage signal subjected to charge-voltage conversion in the floating diffusion layer 324.

A selection signal SEL is supplied from the driving section 22 to the gate electrode of the selection transistor 323. In response to the selection signal SEL, the selection transistor 323 outputs the voltage signal amplified by the amplification transistor 322 as a pixel signal SIG to the column processing unit 24 (see FIG. 2) via the vertical signal line VSL.

In the imaging device 20 having the pixel array section 21 in which the pixels 30 having the above configuration are two-dimensionally arranged, when the control section 13 shown in FIG. By supplying a control signal OFG to the transistor 313 of No. 31, the transistor 313 is driven to supply a photocurrent to the address event detection section 33.

When an address event is detected in a certain pixel 30, the driving section 22 turns off the transistor 313 of that pixel 30 and stops supplying photocurrent to the address event detecting section 33. Next, the driving unit 22 drives the transistor 312 by supplying the transfer signal TRG to the transistor 312 to transfer the charge photoelectrically converted by the light receiving element 311 to the floating diffusion layer 324.

In this way, the imaging device 20 having the pixel array unit 21 in which the pixels 30 having the above configuration are two-dimensionally arranged outputs only the pixel signal of the pixel 30 in which an address event has been detected to the column processing unit 24. Thereby, the power consumption of the imaging device 20 and the amount of image processing can be reduced compared to the case where pixel signals of all pixels are output regardless of the presence or absence of an address event.

Note that the configuration of the pixel 30 illustrated here is one example, and the configuration is not limited to this example. For example, a pixel configuration that does not include the pixel signal generation section 32 may be used. In the case of this pixel configuration, the transistor 313 may be omitted in the light receiving portion 31, and the transistor 312 may have the function of the transistor 313.

[Configuration example of address event detection unit]
FIG. 5 is a block diagram showing an example of the configuration of the address event detection section 33. As shown in FIG. As shown in FIG. 5, the address event detection section 33 according to this configuration example includes a current-voltage conversion section 331, a buffer 332, a subtracter 333, a quantizer 334, an arithmetic circuit 335, a signal holding circuit 336, a transfer section 337, The configuration also includes a control circuit 338.

The current-voltage conversion unit 331 converts the photocurrent from the light receiving unit 31 of the pixel 30 into a logarithmic voltage signal. The current-voltage converter 331 supplies the converted voltage signal to the buffer 332. The buffer 332 buffers the voltage signal supplied from the current-voltage converter 331 and supplies it to the subtracter 333 .

A row drive signal is supplied to the subtracter 333 from the drive section 22 . Subtractor 333 reduces the level of the voltage signal supplied from buffer 332 according to the row drive signal. The subtracter 333 then supplies the level-reduced voltage signal to the quantizer 334. The quantizer 334 quantizes the voltage signal supplied from the subtracter 333 into a digital signal and outputs it to the arithmetic circuit 335 as an address event detection signal. For example, the quantizer 334 sequentially supplies the on-event signal and the off-event signal to the arithmetic circuit 335.

The arithmetic circuit 335 performs a logical operation based on the on-event signal and off-event signal sequentially supplied from the quantizer 334, and generates an event signal. For example, the arithmetic circuit 335 supplies the signal holding circuit 336 with the value of the first supplied signal of the on-event signal and the off-event signal. Next, the arithmetic circuit 335 determines whether one of the value of the next supplied signal of the on-event signal and the off-event signal and the value of the first supplied signal held by the signal holding circuit 336 is true. If both are false values, a signal indicating the true value is supplied to the signal holding circuit 336, and if both are false values, a signal indicating the false value is supplied to the signal holding circuit 336. In this way, the arithmetic circuit 335 outputs an event signal indicating a true value if at least one of the on-event signal and the off-event signal has a true value, and outputs an event signal indicating a false value if both of them have a false value. Output.

The signal holding circuit 336 is provided between the arithmetic circuit 335 and the transfer section 337, and accumulates the arithmetic result of the arithmetic circuit 335 based on the sample signal supplied from the control circuit 338. The signal holding circuit 336 may be a sampling circuit such as a switch, plastic, or capacitor, or may be a digital memory circuit such as a latch or a flip-flop. The signal holding circuit 336 according to the present embodiment has a so-called 1-latch (Latc) configuration, in which one signal holding circuit 336 is provided for an on-event signal and an off-event signal. Therefore, the area of the signal holding circuit 336 can be made smaller than in the so-called two-latch (Latc) configuration in which the signal holding circuit 336 is configured for each of the on-event signal and the off-event signal.

The transfer unit 337 transfers the address event detection signal supplied from the quantizer 334 to the arbiter unit 23 and the like. The transfer unit 337 supplies the arbiter unit 23 with a request for transmitting an address event detection signal when an address event is detected. When the transfer unit 337 receives a response to the request from the arbiter unit 23, it supplies an address event detection signal to the drive unit 22 and the signal processing unit 25.

The control circuit 338 supplies a predetermined threshold voltage V _th to the inverting (-) input terminal of the comparator 3341 . The threshold voltage V _th supplied from the control circuit 338 to the comparator 3341 has a voltage value that differs in time division. For example, the control circuit 338 controls the threshold voltage V th1 corresponding to an on event indicating that the amount of change in photocurrent exceeds an upper threshold, and the threshold voltage V _th1 corresponding to an off event indicating that the amount of change falls below a lower threshold. The corresponding threshold voltages V _th2 are supplied at different timings. This allows one comparator 3341 to generate an on-event signal and an off-event signal.

Here, the number of reads from the transfer unit 337 will be explained while comparing with the two-latch configuration.
FIG. 6A is a diagram showing a first comparative example with a two-latch configuration. The horizontal axis shows time. For example, there is a case where there is a comparator that generates an off-event signal and a comparator that generates an on-event signal. As shown in FIG. 6A, in signal reading using a two-latch configuration having two signal holding circuits 336, an off-event signal is stored in the first signal holding circuit 336, and an on-event signal is stored in the second signal holding circuit 336. , the signal is transferred by the transfer unit 337 corresponding to each. As can be seen from this, in the two-latch configuration, reading can be performed once for the first signal holding circuit 336 and the second signal holding circuit 336. Note that in the configuration of FIG. 6A, if an event signal including information on the presence or absence of an event is required, it is necessary to generate it in a circuit other than the address event detection unit 33 (see FIG. 4) after reading.

FIG. 6B is a diagram showing a second comparative example with a two-latch configuration. The horizontal axis shows time. For example, there is a case where there is one comparator that generates an off-event signal and one comparator that generates an on-event signal. As shown in FIG. 6B, in signal reading using a two-latch configuration having two signal holding circuits 336, an off-event signal is first stored in the first signal holding circuit 336, and then an on-event signal is stored in the second signal holding circuit 336. The signals are stored in the respective transfer units 337 and transferred by the corresponding transfer units 337. As can be seen from these, in the two-latch configuration, it is possible to read the first signal holding circuit 336 and the second signal holding circuit 336 once. Note that in the configuration of FIG. 6B, if an event signal including information on the presence or absence of an event is required, it will be necessary to generate it in a post-read circuit other than the address event detection unit 33 (see FIG. 4).

FIG. 6C is a diagram showing a third comparative example using a one-latch configuration. The horizontal axis shows time. For example, there is a case where there is one comparator that generates an off-event signal and one comparator that generates an on-event signal. As shown in FIG. 6C, in signal reading using a one-latch configuration having one signal holding circuit 336, an off event signal is first stored in the first signal holding circuit 336, and then read out. Next, the on-event signal is stored in the second signal holding circuit 336 and read. As can be seen from this, in the one-latch configuration according to the third comparative example, the number of reads is two. Note that in the configuration of FIG. 6C, if an event signal including information on the presence or absence of an event is required, it will be necessary to generate it in a post-read circuit other than the address event detection unit 33 (see FIG. 4).

FIG. 6D is a diagram showing an example of reading in the configuration example shown in FIG. 5. FIG. The horizontal axis shows time. As shown in FIG. 6D, the arithmetic circuit 335 first stores one of the off-event signal and the on-event signal in the first signal holding circuit 336. Next, the arithmetic circuit 335 generates an event signal by a logical operation using the other of the off-event signal and the on-event signal and the signal stored in the first signal holding circuit 336. to be memorized. Then, the event signal is read. In this manner, by providing the arithmetic circuit 335, it becomes possible to read the event signal including information on the presence or absence of an event only once. Note that the circuit configuration example shown in FIG. 5 can also handle a case where an off-event signal including information on the presence or absence of an off-event and an on-event signal containing information on the presence or absence of an on-event are required. In this case, as in the read example of FIG. 6C, first the off-event signal is stored in the first signal holding circuit 336 and read out (Read), and then the on-event signal is stored in the second signal holding circuit 336 and read out (Read). Read), it is possible to output an off-event signal and an on-event signal from the address event detection section 33 (see FIG. 4). Note that in this embodiment, the off-event signal is generated first and then the on-event signal is generated, but the present invention is not limited to this. For example, the on-event signal may be generated first, and then the off-event signal may be generated.

Next, a configuration example of the current-voltage conversion unit 331, subtracter 333, and quantizer 334 in the address event detection unit 33 will be described. Note that details of the arithmetic circuit 335 will be described later.

(Example of configuration of current-voltage converter)
FIG. 7 is a circuit diagram showing an example of the configuration of the current-voltage conversion section 331 in the address event detection section 33. As shown in FIG. 7, the current-voltage converter 331 according to this example has a circuit configuration including an N-type transistor 3311, a P-type transistor 3312, and an N-type transistor 3313. For example, MOS transistors are used as these transistors 3311 to 3313.

The N-type transistor 3311 is connected between the power supply line of the power supply voltage V _DD and the signal input line 3314 . The P-type transistor 3312 and the N-type transistor 3313 are connected in series between the power supply line of the power supply voltage _VDD and the ground. The common connection node _N2 of the P-type transistor 3312 and the N-type transistor 3313 is connected to the gate electrode of the N-type transistor 3311 and the input terminal of the buffer 332 shown in FIG.

A predetermined bias voltage V _bias is applied to the gate electrode of the P-type transistor 3312. As a result, the P-type transistor 3312 supplies a constant current to the N-type transistor 3313. A photocurrent is input from the light receiving section 31 to the gate electrode of the N-type transistor 3313 through a signal input line 3314.

The drain electrodes of the N-type transistor 3311 and the N-type transistor 3313 are connected to the power supply side, and such a circuit is called a source follower. These two source followers connected in a loop convert the photocurrent from the light receiving section 31 into a logarithmic voltage signal.

(Example of configuration of subtracter and quantizer)
FIG. 8 is a circuit diagram showing an example of the configuration of the subtracter 333 and the quantizer 334 in the address event detection section 33.

The subtracter 333 according to this example has a configuration including a capacitive element 3331, an inverter circuit 3332, a capacitive element 3333, and a switch element 3334.

One end of the capacitive element 3331 is connected to the output terminal of the buffer 332 shown in FIG. 5, and the other end is connected to the input terminal of the inverter circuit 3332. Capacitive element 3333 is connected in parallel to inverter circuit 3332. Switch element 3334 is connected between both ends of capacitive element 3333. A row drive signal is supplied from the drive section 22 to the switch element 3334 as its opening/closing control signal. The switch element 3334 opens and closes a path connecting both ends of the capacitive element 3333 in accordance with the row drive signal. The inverter circuit 3332 inverts the polarity of the voltage signal input via the capacitive element 3331.

In the subtracter 333 having the above configuration, when the switch element 3334 is turned on (closed), the voltage signal V _init is input to the terminal on the buffer 332 side of the capacitive element 3331, and the terminal on the opposite side is a virtual ground terminal. becomes. For convenience, the potential of this virtual ground terminal is set to zero. At this time, the charge Q _init accumulated in the capacitive element 3331 is expressed by the following equation (1), assuming that the capacitance value of the capacitive element 3331 is C ₁ . On the other hand, since both ends of the capacitive element 3333 are short-circuited, the accumulated charge becomes zero.
Q _init =C ₁ ×V _init ...(1)

Next, considering the case where the switch element 3334 is turned off (open) and the voltage at the terminal on the buffer 332 side of the capacitive element 3331 changes to V _after , the charge Q _after accumulated in the capacitive element 3331 is , is expressed by the following equation (2).
Q _after =C ₁ ×V _after ...(2)

On the other hand, the charge Q ₂ accumulated in the capacitive element 3333 is expressed by the following equation (3), where the capacitance value of the capacitive element 3333 is C ₂ and the output voltage is V _out .
Q ₂ =-C ₂ ×V _out ...(3)

At this time, since the total charge amount of the capacitor 3331 and the capacitor 3333 does not change, the following equation (4) holds true.
Q _init =Q _after +Q ₂ ...(4)

When formula (4) is transformed by substituting formulas (1) to (3), the following formula (5) is obtained.
V _out =-(C ₁ /C ₂ )×(V _after -V _init ) (5)

Equation (5) represents the subtraction operation of the voltage signal, and the gain of the subtraction result is C ₁ /C ₂ . Since it is usually desired to maximize the gain, it is preferable to design C ₁ to be large and C ₂ to be small. On the other hand, if C ₂ is too small, kTC noise may increase and noise characteristics may deteriorate, so the reduction in the capacity of C ₂ is limited to a range where noise can be tolerated. Furthermore, since the address event detection unit 33 including the subtracter 333 is mounted for each pixel 30, the capacitor 3331 and the capacitor 3333 have area limitations. Taking these into consideration, the capacitance values C ₁ and C ₂ of the capacitive elements 3331 and 3333 are determined.

In FIG. 8, the quantizer 334 has a comparator 3341. The comparator 3341 has the output signal of the inverter circuit 3332, ie, the voltage signal from the subtracter 430, as a non-inverting (+) input, and the predetermined threshold voltage V _th as an inverting (-) input. The comparator 3341 then compares the voltage signal from the subtracter 430 with a predetermined threshold voltage V _th and outputs a signal indicating the comparison result to the transfer unit 337 as an address event detection signal.

[Imaging device according to second configuration example (scan method)]
The imaging device 20 according to the first configuration example described above is an asynchronous imaging device that reads events using an asynchronous readout method. However, the event readout method is not limited to an asynchronous readout method, but may be a synchronous readout method. The imaging device to which the synchronous readout method is applied is a scanning imaging device, which is the same as a normal imaging device that captures images at a predetermined frame rate.

FIG. 9 is a block diagram illustrating an example of the configuration of an imaging device according to a second configuration example, that is, a scan-type imaging device, which is used as the imaging device 20 in the imaging system 10 to which the technology according to the present disclosure is applied. .

As shown in FIG. 9, an imaging device 20 according to a second configuration example as an imaging device of the present disclosure includes a pixel array section 21, a driving section 22, a signal processing section 25, a readout area selection section 27, and a signal generation section. 28.

The pixel array section 21 includes a plurality of pixels 30. The plurality of pixels 30 output output signals in response to a selection signal from the readout area selection section 27. Each of the plurality of pixels 30 may be configured to include a quantizer within the pixel, as shown in FIG. 3, for example. The plurality of pixels 30 output output signals corresponding to the amount of change in light intensity. The plurality of pixels 30 may be two-dimensionally arranged in a matrix, as shown in FIG.

The driving section 22 drives each of the plurality of pixels 30 and outputs a pixel signal generated by each pixel 30 to the signal processing section 25. Note that the driving section 22 and the signal processing section 25 are circuit sections for acquiring gradation information. Therefore, when acquiring only event information, the driving section 22 and the signal processing section 25 may be omitted.

The readout area selection section 27 selects a part of the plurality of pixels 30 included in the pixel array section 21. For example, the readout area selection unit 27 selects one or more of the rows included in the two-dimensional matrix structure corresponding to the pixel array unit 21. The readout area selection unit 27 sequentially selects one or more rows according to a preset cycle. Further, the readout area selection unit 27 may determine the selection area in response to a request from each pixel 30 of the pixel array unit 21.

The signal generation unit 28 generates an event signal corresponding to an active pixel that has detected an event among the selected pixels, based on the output signal of the pixel selected by the readout area selection unit 27. An event is an event in which the intensity of light changes. An active pixel is a pixel in which the amount of change in light intensity corresponding to the output signal exceeds or falls below a preset threshold. For example, the signal generation unit 28 compares the output signal of a pixel with a reference signal, detects an active pixel that outputs an output signal when it is larger or smaller than the reference signal, and generates an event signal corresponding to the active pixel. .

The signal generation section 28 can be configured to include, for example, a column selection circuit that arbitrates signals input to the signal generation section 28. Furthermore, the signal generation unit 28 may be configured to output not only information on active pixels that have detected an event, but also information on inactive pixels that have not detected an event.

The signal generation unit 28 outputs address information and time stamp information (for example, (X, Y, T)) of the active pixel that detected the event through the output line 15. However, the data output from the signal generation unit 28 may be not only address information and time stamp information, but also frame format information (for example, (0, 0, 1, 0, ...)). .

[Example of chip structure configuration]
As the chip (semiconductor integrated circuit) structure of the imaging device 20 according to the first configuration example or the second configuration example described above, for example, a stacked chip structure can be adopted. FIG. 10 is an exploded perspective view schematically showing the stacked chip structure of the imaging device 20. As shown in FIG.

As shown in FIG. 10, a stacked chip structure, so-called stacked structure, is a structure in which at least two chips, a first chip, a light receiving chip 201, and a second chip, a detection chip 202, are stacked. It becomes. In the circuit configuration of the pixel 30 shown in FIG. 4, each of the light receiving elements 311 is arranged on the light receiving chip 201, and all the elements other than the light receiving element 311 and the elements of other circuit parts of the pixel 30 are arranged on the detection chip. 202. The light-receiving chip 201 and the detection chip 202 are electrically connected via a connection portion such as a via (VIA), a Cu--Cu junction, or a bump.

Note that here, a configuration example is illustrated in which the light receiving element 311 is arranged on the light receiving chip 201, and elements other than the light receiving element 311 and elements of other circuit parts of the pixel 30 are arranged on the detection chip 202. It is not limited to.

For example, in the circuit configuration of the pixel 30 shown in FIG. 3, each element of the light receiving section 31 is arranged on the light receiving chip 201, and elements other than the light receiving section 31 and elements of other circuit parts of the pixel 30 are arranged on the detection chip 202. It can be configured to do this. Further, each element of the light receiving section 31, the reset transistor 321, and the floating diffusion layer 324 of the pixel signal generating section 32 may be arranged in the light receiving chip 201, and the other elements may be arranged in the detection chip 202. . Furthermore, a part of the elements constituting the address event detection section 33 can be arranged in the light receiving chip 201 together with each element of the light receiving section 31.

[Example of configuration of column processing section]
FIG. 11 is a block diagram illustrating an example of the configuration of the column processing section 24 of the imaging device 20 according to the first configuration example. As shown in FIG. 11, the column processing section 24 according to this example has a configuration including a plurality of analog-to-digital converters (ADCs) 241 arranged for each pixel column of the pixel array section 21.

Note that although a configuration example in which the analog-to-digital converters 241 are arranged in a one-to-one correspondence with the pixel columns of the pixel array section 21 is illustrated here, the configuration is not limited to this example. For example, the analog-to-digital converter 241 may be arranged in units of a plurality of pixel columns, and the analog-to-digital converter 241 may be configured to perform time-sharing processing among the plurality of pixel columns.

The analog-to-digital converter 241 converts the analog pixel signal SIG supplied via the vertical signal line VSL into a digital signal having a larger number of bits than the address event detection signal described above. For example, if the address event detection signal is 2 bits, the pixel signal is converted to a digital signal of 3 bits or more (16 bits, etc.). The analog-to-digital converter 241 supplies the digital signal generated by analog-to-digital conversion to the signal processing section 25.

[First configuration example of arithmetic circuit and signal holding circuit]
Here, a detailed configuration example of the arithmetic circuit 335 will be described based on FIG. 12. FIG. 12 is a circuit diagram showing a first configuration example of the arithmetic circuit 335 and signal holding circuit 336 in the address event detection section 33.

The arithmetic circuit 335 according to this example includes a logical sum circuit (OR circuit) 335a and a multiplexer 335b. One input terminal of the two input terminals of the OR circuit 335a is connected to the output terminal of the quantizer 334, and the other input terminal is connected to the output terminal of the signal holding circuit 336. The output terminal of the OR circuit 335a is connected to the input terminal of the multiplexer 335b. The arithmetic circuit 335 according to this example has a first mode in which an on-event signal (first event signal) and an off-event signal (second event signal) are sequentially output to the signal holding circuit 336 in response to input of a control signal. Further, the arithmetic circuit 335 according to the present example outputs an on-event signal to the signal holding circuit 336 in response to the input of the control signal, and generates an event signal based on the on-event signal and off-event signal held in the signal holding circuit 336. It has a second mode that generates.

One input terminal of the two input terminals of the multiplexer 335b is connected to the output terminal of the OR circuit 335a, and the other input terminal is connected to the output terminal of the quantizer 334, as described above. Further, the output terminal of the multiplexer 335b is connected to the input terminal of the signal holding circuit 336. Furthermore, a selection control terminal of multiplexer 335b is connected to control circuit 338.

The input terminal of the signal holding circuit 336 is connected to the output terminal of the multiplexer 335b, as described above. The output terminal of the signal holding circuit 336 is connected to the output terminal of the transfer section 337.

With the circuit configuration shown in FIG. 12, the OR circuit 335a outputs a true value if the output signal S1 of the quantizer 334 or the output signal S4 of the signal holding circuit 336 is a true value. On the other hand, if the output signal S1 of the quantizer 334 and the output signal S4 of the signal holding circuit 336 are both false values, the OR circuit 335a outputs a false value. In this embodiment, a true value is shown as "1", a false value is shown as "0", and a case where either "1" or "0" is acceptable is shown as "-".

FIG. 13 is a diagram showing a truth table of multiplexer 335b. As shown in FIG. 13, when the selection control signal E0 is 0, the output signal S1 of the quantizer 334 is output as the output signal S3 regardless of the output signal S2 of the OR circuit 335a. That is, when the selection control signal E0 is set to 0, the output signal S1 of the quantizer 334 can be supplied as is to the signal holding circuit 336. Thereby, it is possible to perform the first mode of processing. More specifically, first, one of the off-event signal and the on-event signal is stored in the first signal holding circuit 336 and read out (Read), and then the other of the off-event signal and the on-event signal is stored in the second signal holding circuit 336. By storing and reading the signal in the signal holding circuit 336, it becomes possible to output the off-event signal and the on-event signal from the address event detection section 33 (see FIG. 4).

On the other hand, when the selection control signal E0 is 1, the output signal S2 of the OR circuit 335a is output as the output signal S3, regardless of the output signal S1 of the quantizer 334.

FIG. 14 is a diagram showing a truth table of the arithmetic circuit 335 in the second mode. In FIG. 14, first, at time t1, the quantizer 334 generates an on-event signal as the signal S1, and then at time t2, the quantizer 334 generates an off-event signal as the signal S1. It shows. Note that in this embodiment, an on-event signal is generated and then an off-event signal is generated, but as described above, the present invention is not limited to this. For example, an off-event signal may be generated and then an on-event signal may be generated. In this case as well, it is possible to perform the same processing as when an on-event signal is generated and then an off-event signal is generated.

First, a case will be described in which the on-event signal (see column S1) at time t1 is 1 and the off-event signal (see column S1) at time t2 is 1. At time t1 when the on-event signal is input, the selection control signal E0 is input as 0 by the control circuit 338.

The output signal S2 of the OR circuit 335a is 1 at time t1 because the signal S1 is 1 even if the initial value of the signal holding circuit 336 is "1" or "0". In this case, as shown in FIG. 10, since the selection control signal E0 is 0, the multiplexer 335b selects the output signal S1 of the quantizer 334 as the output signal S3 regardless of the output signal S2 of the OR circuit 335a. Output. That is, at time t1, even if the initial value of the signal holding circuit 336 is "1" or "0", the value 1 of the on-event signal is held in the signal holding circuit 336 as the output signal S3 of the multiplexer 335b. . As a result, the value of the signal holding circuit 336 is held as 1 at time t1.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the output signal S4, which is the initial value of the signal holding circuit 336, is 1, and the output signal S1 of the quantizer 334 is 1, so the output signal S2 of the OR circuit 335a becomes 1. Further, at time t2, the selection control signal E0 is inputted as 1 by the control circuit 338. In this case, as shown in FIG. 10, the output signal S2 of the OR circuit 335a is output as the output signal S3, regardless of the output signal S1 of the quantizer 334. That is, the output signal S3 is 1, and the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. As a result, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 1 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 0 and the off-event signal (see column S1) at time t2 is 1. At time t1 when the on-event signal is input, the selection control signal E0 is input as 0 by the control circuit 338.

At time t1, the signal S1 is 0, and the signal S4, which is the initial value of the signal holding circuit 336, is "1" or "0". Therefore, the output signal S2 of the OR circuit 335a is "1" or "0". In this case, as shown in FIG. 10, the multiplexer 335b outputs the output signal S1 of the quantizer 334 as the output signal S3, regardless of the output signal S2 of the OR circuit 335a. That is, at time t1, even if the initial value of the signal holding circuit 336 is "1" or "0", 0, which is the value of the on-event signal, is held in the signal holding circuit 336 as the output signal S3 of the multiplexer 335b. . As a result, the value of the signal holding circuit 336 is held as 0 at time t1.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the signal S4, which is the initial value of the signal holding circuit 336, is 0, and the output signal S1 of the quantizer 334 is 1, so the output signal S2 of the OR circuit 335a becomes 1. Further, at time t2, the selection control signal E0 is inputted as 1 by the control circuit 338. In this case, as shown in FIG. 10, the output signal S2 of the OR circuit 335a is output as the output signal S3, regardless of the output signal S1 of the quantizer 334. That is, the signal S3 is 1, and the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. As a result, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 1 and the off-event signal (see column S1) at time t2 is 0. At time t1 when the on-event signal is input, the signal S1 is 1, the signal S2 is 1, the signal S3 is 1, and the signal S4 is 1, as described above.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the initial signal S4 of the signal holding circuit 336 is 1, and the output signal S1 of the quantizer 334 is 0, so the output signal S2 of the OR circuit 335a becomes 1. Further, at time t2, the selection control signal E0 is inputted as 1 by the control circuit 338. In this case, as shown in FIG. 10, the output signal S2 of the OR circuit 335a is output as the output signal S3, regardless of the output signal S1 of the quantizer 334. That is, the output signal S3 is 1, and the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. As a result, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 1 and the off-event signal is 0, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 0 and the off-event signal (see column S1) at time t2 is 0. At time t1 when the on-event signal is input, as described above, the signal S1 is 0, the signal S2 is "1" or "0", the signal S3 is 0, and the signal S4 is 0.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the output signal S4, which is the initial value of the signal holding circuit 336, is 0, and the output signal S1 of the quantizer 334 is 0, so the output signal S2 of the OR circuit 335a becomes 0. Further, at time t2, the selection control signal E0 is inputted as 1 by the control circuit 338. In this case, as shown in FIG. 10, the output signal S2 of the OR circuit 335a is output as the output signal S3, regardless of the output signal S1 of the quantizer 334. That is, the output signal S3 is 0, and the output signal S4 of the signal holding circuit 336 is held at 0 at time t2. As a result, the transfer unit 317 outputs 0 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 0, the transfer unit 337 outputs 0.

[Second configuration example of arithmetic circuit and signal holding circuit]
FIG. 15 is a circuit diagram showing a second configuration example of the arithmetic circuit 335 and signal holding circuit 336 in the address event detection section 33.

The arithmetic circuit 335 according to this example includes an OR circuit 335a and an AND circuit (AND circuit) 335c. One of the two input terminals of the OR circuit 335a is connected to the output terminal of the quantizer 334, and the other input terminal is connected to the output terminal of the AND circuit 335c. The output terminal of the OR circuit 335a is connected to the input terminal of the signal holding circuit 336.

One input terminal of the two input terminals of the AND circuit 335c is connected to the control circuit 338. The other input terminal of the AND circuit 335c is connected to the output terminal of the signal holding circuit 336. Furthermore, the output terminal of the AND circuit 335c is connected to one input terminal of the OR circuit 335a, as described above.

The input terminal of the signal holding circuit 336 is connected to the output terminal of the OR circuit 335a, as described above. The output terminal of the signal holding circuit 336 is connected to the output terminal of the transfer section 337.

FIG. 16 is a diagram showing a truth table of the OR circuit 335a. With the circuit configuration shown in FIG. 16, the OR circuit 335a outputs a true value if the output signal S1 of the quantizer 334 or the output signal S5 of the AND circuit 335c is a true value. On the other hand, if the output signal S1 of the quantizer 334 and the output signal S5 of the AND circuit 335c are both false values, the OR circuit 335a outputs a false value. In this case, the output signal S5 of the AND circuit 335c is always 0 when the selection control signal E0 is 0.

Thereby, it is possible to perform the first mode of processing. More specifically, when the selection control signal E0 is set to 0, the OR circuit 335a can supply the output signal S1 of the quantizer 334 to the signal holding circuit 336 as it is. As a result, first, one of the off-event signal and the on-event signal is stored in the first signal holding circuit 336 and read out (Read), and then the other of the off-event signal and the on-event signal is stored in the second signal holding circuit 336. 336 and read (Read), it becomes possible to output an off-event signal and an on-event signal from the address event detection unit 33 (see FIG. 4).

On the other hand, FIG. 17 is a diagram showing a truth table of the arithmetic circuit 335 in the second configuration example in the second mode. In FIG. 17, first, at time t1, the quantizer 334 generates an on-event signal as the signal S1, and then at time t2, the quantizer 334 generates an off-event signal as the signal S1. It shows.

First, a case will be described in which the on-event signal (see column S1) at time t1 is 1 and the off-event signal (see column S1) at time t2 is 1. At time t1 when the on-event signal is input, the selection control signal E0 is input as 0 by the control circuit 338.

Since the selection control signal E0 is 0, the output signal S5 of the AND circuit 335c becomes 0 even if the initial value of the signal holding circuit 336 is "1" or "0". As a result, the output signal S2 of the OR circuit 335a outputs 1 since the output signal S1 of the quantizer 334 can be 1. Therefore, the value of the signal holding circuit 336 is held as 1 at time t1.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the output signal S4, which is the initial value of the signal holding circuit 336, is 1, so the output signal S5 of the AND circuit 335c becomes 1. As a result, the output signal S2 of the OR circuit 335a becomes 1 regardless of the value of the output signal S1 of the quantizer 334. Therefore, the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. Then, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 1 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 0 and the off-event signal (see column S1) at time t2 is 1. At time t1 when the on-event signal is input, the selection control signal E0 is input as 0 by the control circuit 338.

Since the selection control signal E0 is 0, the output signal S5 of the AND circuit 335c becomes 0 even if the initial value of the signal holding circuit 336 is "1" or "0". As a result, the output signal S2 of the OR circuit 335a outputs 0 because the output signal S1 of the quantizer 334 can be 0. Therefore, the value of the signal holding circuit 336 is held as 0 at time t1.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the output signal S4, which is the initial value of the signal holding circuit 336, is 0, so the output signal S5 of the AND circuit 335c becomes 0. On the other hand, since the value of the output signal S1 of the quantizer 334 is 1, the output signal S2 of the OR circuit 335a is 1. As a result, the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. Then, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 1 and the off-event signal (see column S1) at time t2 is 0. At time t1 when the on-event signal is input, the selection control signal E0 is input as 0 by the control circuit 338.

As described above, the output signal S5 of the AND circuit 335c becomes 0, the output signal S2 of the OR circuit 335a becomes 1, and the value of the signal holding circuit 336 is held as 1 at time t1.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the output signal S4, which is the initial value of the signal holding circuit 336, is 1, so the output signal S5 of the AND circuit 335c becomes 1. Therefore, regardless of the value of the output signal S1 of the quantizer 334, the output signal S2 of the OR circuit 335a becomes 1. As a result, the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. Then, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 1 and the off-event signal is 0, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 0 and the off-event signal (see column S1) at time t2 is 0. At time t1 when the on-event signal is input, the selection control signal E0 is input as 0 by the control circuit 338.

As described above, the output signal S5 of the AND circuit 335c becomes 0, the output signal S2 of the OR circuit 335a becomes 0, and the value of the signal holding circuit 336 is held as 0 at time t1.

Subsequently, at time t2 when the off-event signal is input, the selection control signal E0 is input as 1 by the control circuit 338. At time t2, the output signal S4, which is the initial value of the signal holding circuit 336, is 0, so the output signal S5 of the AND circuit 335c becomes 0. On the other hand, since the value of the output signal S1 of the quantizer 334 is 0, the output signal S2 of the OR circuit 335a is 0. As a result, the output signal S4 of the signal holding circuit 336 is held as 0 at time t2. Then, the transfer unit 317 outputs 0 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 0, the transfer unit 337 outputs 0.

Here, an equivalent circuit of the arithmetic circuit 335 in the second configuration example will be described. FIG. 18 is a diagram showing an example of an equivalent circuit of the arithmetic circuit 335 in the second configuration example. As shown in FIG. 18, the logical AND circuit 335c is equivalent to the negative logical product circuit 33d and the NOT circuit 335i. Not circuits 335e and 335f are connected between the quantizer 334 and the OR circuit 335a, and Not circuits 335g and 335h are connected between the OR circuit 335a and the signal holding circuit 336.

FIG. 19A is a diagram showing an equivalent circuit of a NAND circuit based on De Morgan's theorem. As shown in FIG. 19A, the NAND circuit is equivalent to a combination of a Not circuit and an OR circuit.

FIG. 19B is a diagram showing an equivalent circuit of NOR based on De Morgan's theorem. As shown in FIG. 19B, the NOR is equivalent to a combination of a Not circuit and an AND circuit.

FIG. 20 is a diagram showing an example of an equivalent circuit of the arithmetic circuit 335 shown in FIG. 18. As shown in FIG. 18, the combination of the Not circuits 335i, 335f and the OR circuit 335a is equivalent to the NOT circuit as described above. When the NOT circuit 335g is further combined with the NAND circuit, it becomes equivalent to the AND circuit 335j.

FIG. 21 is a diagram showing an example of an equivalent circuit of the arithmetic circuit 335 shown in FIG. 20. As shown in FIG. 21, the combination of the AND circuit 335j and the NOT circuit 335h shown in FIG. 20 is equivalent to the NOT circuit 335k. In this way, the arithmetic circuit 335 may be configured with various equivalent circuits depending on the circuit situation.

(Third configuration example of arithmetic circuit and signal holding circuit)
FIG. 22 is a circuit diagram showing a third configuration example of the arithmetic circuit 335 and signal holding circuit 336 in the address event detection section 33. The third configuration example is an example in which only the second mode of processing is possible.

The arithmetic circuit 335 according to this example includes an OR circuit 335a. One input terminal of the two input terminals of the OR circuit 335a is connected to the output terminal of the quantizer 334, and the other input terminal is connected to the output terminal of the signal holding circuit 336. Further, an output terminal of the OR circuit 335a is connected to a signal holding circuit 336.

The input terminal of the signal holding circuit 336 is connected to the output terminal of the OR circuit 335a, as described above. The output terminal of the signal holding circuit 336 is connected to the output terminal of the transfer section 337.

FIG. 23 is a diagram showing a truth table of the OR circuit 335a in the third configuration example. In FIG. 23, first, at time t1, the quantizer 334 generates an on-event signal as the signal S1, and then at time t2, the quantizer 334 generates an off-event signal as the signal S1. It shows.

First, a case will be described in which the on-event signal (see column S1) at time t1 is 1 and the off-event signal (see column S1) at time t2 is 1. The control circuit 338 sets the initial value of the signal holding circuit 336 to zero. In this case, the output signal S2 of the OR circuit 335a is 1 because the output signal S1 of the quantizer 334 is 1. Therefore, the value of the signal holding circuit 336 is held as 1 at time t1.

Subsequently, at time t2 when the off-event signal is input, since the output signal S4, which is the initial value of the signal holding circuit 336, is 1, the output signal of the OR circuit 335a is independent of the value of the output signal S1 of the quantizer 334. The output signal S2 becomes 1. As a result, the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. Then, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 1 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 0 and the off-event signal (see column S1) at time t2 is 1. The control circuit 338 sets the initial value of the signal holding circuit 336 to zero. As a result, the output signal S2 of the OR circuit 335a is 0 because the output signal S1 of the quantizer 334 is 0. Therefore, the value of the signal holding circuit 336 is held as 0 at time t1.

Subsequently, at time t2 when the off-event signal is input, the output signal S4, which is the initial value of the signal holding circuit 336, is 0. On the other hand, since the value of the output signal S1 of the quantizer 334 is 1, the output signal S2 of the circuit 335a is 1. As a result, the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. Then, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 1 and the off-event signal (see column S1) at time t2 is 0. The control circuit 338 sets the initial value of the signal holding circuit 336 to zero. As a result, the output signal S2 of the OR circuit 335a is 1 since the output signal S1 of the quantizer 334 is 1. Therefore, the value of the signal holding circuit 336 is held as 1 at time t1.

Subsequently, at time t2 when the off-event signal is input, the output signal S4, which is the initial value of the signal holding circuit 336, is 1. On the other hand, since the value of the output signal S1 of the quantizer 334 is 0, the output signal S2 of the OR circuit 335a is 1. As a result, the output signal S4 of the signal holding circuit 336 is held as 1 at time t2. Then, the transfer unit 317 outputs 1 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 1, the transfer unit 337 outputs 1.

Next, a case will be described in which the on-event signal (see column S1) at time t1 is 0 and the off-event signal (see column S1) at time t2 is 0. The control circuit 338 sets the initial value of the signal holding circuit 336 to zero. As a result, the output signal S2 of the OR circuit 335a is 0 because the output signal S1 of the quantizer 334 is 0. Therefore, the value of the signal holding circuit 336 is held as 0 at time t1.

Subsequently, at time t2 when the off-event signal is input, the output signal S4, which is the initial value of the signal holding circuit 336, is 0. On the other hand, since the value of the output signal S1 of the quantizer 334 is 0, the output signal S2 of the OR circuit 335a is 0. As a result, the output signal S4 of the signal holding circuit 336 is held as 0 at time t2. Then, the transfer unit 317 outputs 0 as an event signal. In this way, when the on-event signal is 0 and the off-event signal is 0, the transfer unit 337 outputs 0.

As described above, according to the present embodiment, the signal holding circuit 336 holds the value of the first supplied signal of the on-event signal and the off-event signal, and the arithmetic circuit 335 holds the value of the first supplied signal of the on-event signal and the off-event signal. If one of the value of the next supplied signal of the off-event signals and the value of the first supplied signal held by the signal holding circuit 336 is a true value, the signal indicating the true value is output as a signal. If both are false values, a signal indicating a false value is supplied to the signal retention circuit 336. Thereby, even if there is only one signal holding circuit 336 corresponding to the on-event signal and the off-event signal, it is possible to output the event signal with one readout.

(Second embodiment)
The imaging device 100 according to the second embodiment is different from the imaging device 100 according to the first embodiment in that the column processing section includes a quantizer 334, an arithmetic circuit 335, and a signal holding circuit 336. Below, differences from the imaging device 100 according to the first embodiment will be explained.

FIG. 24 is a block diagram showing an example of the configuration of the pixel array section 21 according to the second embodiment. As shown in FIG. 24, each of the plurality of pixels 30a has a light receiving section 31, a pixel signal generating section 32, and a detecting section 33a. The configuration of the pixel 30 of the pixel array section 21 according to the first embodiment is different in that a detection section 33a is included instead of the address event detection section 33 (see FIG. 3).

FIG. 25 is a block diagram showing an example of the configuration of the detection section 33a according to the second embodiment. As shown in FIG. 25, the detection section 33a includes a current-voltage conversion section 331, a buffer 332, and a subtracter 333. That is, it differs from the address event detection section 33 (see FIG. 3) in that it does not include a quantizer 334, an arithmetic circuit 335, a signal holding circuit 336, a transfer section 337, and a control circuit 338.

FIG. 26 is a block diagram showing an example of the configuration of the pixel array section 21 and the column processing section 26 according to the second embodiment. As shown in FIG. 26, for each of the plurality of pixels 30a arranged in a column in the pixel array section 21, a quantizer 334, an arithmetic circuit 335, a signal holding circuit 336, a transfer section 337, and a A control circuit 338 is provided. With such a configuration, the column processing unit 26 according to the second embodiment supplies the quantizer 334 with voltage signals after level reduction, which are sequentially supplied from each of the subtracters 333 of the plurality of pixels 30a arranged in a column. do. The arithmetic circuit 335 performs logical operations based on the on-event signal and off-event signal sequentially supplied from the quantizer 334, and sequentially generates event signals.

As described above, according to the present embodiment, the quantizer 334, the arithmetic circuit 335, the signal holding circuit 336, the transfer unit 337, and the control circuit are commonly used for the plurality of pixels 30a arranged in a column. 338. This makes it possible to further reduce the scale of the circuit configuration of the imaging device 100.

(Third embodiment)
The imaging device 100 according to the third embodiment differs from the imaging device 100 according to the first embodiment in that each pixel block includes a quantizer 334 and an arithmetic circuit 335. Below, differences from the imaging device 100 according to the first embodiment will be explained.

FIG. 27 is a block diagram showing an example of the configuration of the pixel array section 21 according to the third embodiment. As shown in FIG. 27, the pixel array section 21 has a plurality of pixel blocks 300a. In the pixel block 300a, a plurality of pixels 30a (see FIG. 25) are arranged in a two-dimensional matrix.

FIG. 28 is a block diagram showing an example of the configuration of the arithmetic circuit 335 for the pixel block 300a. As shown in FIG. 28, N pixels 30a (see FIG. 25) are arranged in a two-dimensional matrix in the pixel block 300a. Further, one quantizer 334 is connected to the pixel block 300a, and one arithmetic circuit 335 is further connected to the quantizer 334. The arithmetic circuit 335 includes an OR circuit 335a, a multiplexer 335k, and N AND circuits 335L. Furthermore, N signal holding circuits 336 are connected to the OR circuit 335a. Further, N corresponding transfer sections 337 are connected to each of the N signal holding circuits 336.

With such a configuration, voltage signals whose level has been lowered and which are sequentially supplied from each of the subtracters 333 of the plurality of pixels 30a arranged in a matrix in the pixel block 300a are supplied to the quantizer 334. When a voltage signal is supplied from the first pixel 30a, the control circuit 338 first causes the quantizer 334 to generate an on-event signal, and sets all control inputs E1 to EN to 0. This causes the signal holding circuit 336 corresponding to the first pixel 30a to store the value of the on-event signal corresponding to the first pixel 30a.

Next, the control circuit 338 causes the quantizer 334 to generate an off event signal when the voltage signal is supplied from the first pixel 30a, sets the control input E1 to 1, and controls all the remaining control inputs E2 to Let EN be 0. As a result, the OR circuit 335a determines that one of the value of the next supplied signal of the on-event signal and the off-event signal and the value of the first supplied signal held by the signal holding circuit 336. If it is a true value, a signal indicating the true value is supplied to the corresponding signal holding circuit 336, and if both are false values, a signal indicating the false value is supplied to the corresponding signal holding circuit 336.

Similarly, when the voltage signal is supplied from the Nth pixel 30a, the control circuit 338 first causes the quantizer 334 to generate an on-event signal, and sets all control inputs E1 to EN to 0. As a result, the value of the on-event signal corresponding to the N-th pixel 30a is stored in the signal holding circuit 336 corresponding to the N-th pixel 30a.

Next, the control circuit 338 causes the quantizer 334 to generate an off event signal when the voltage signal is supplied from the Nth pixel 30a, sets the control input EN to 1, and controls all the remaining control inputs E1 to Set EN-1 to 0. As a result, the OR circuit 335a determines that one of the value of the next supplied signal of the on-event signal and the off-event signal and the value of the first supplied signal held by the signal holding circuit 336. If it is a true value, a signal indicating the true value is supplied to the signal holding circuit 336 corresponding to the Nth pixel 30a, and if both are false values, a signal indicating the false value is supplied to the signal holding circuit 336 corresponding to the Nth pixel 30a. Supplied to circuit 336. Through such processing, an event signal corresponding to the Nth pixel 30a is generated and held in the corresponding signal holding circuit 336. By sequentially performing such processing from the first pixel 30a to the Nth pixel 30a, event signals corresponding to the first to Nth pixels 30a can be held in the corresponding holding circuits 336, respectively. Thereby, the transfer unit 337 corresponding to the signal holding circuit 336 can output the event signal by reading it once.

As explained above, the quantizer 334 and the arithmetic circuit 335 are commonly configured for the plurality of pixels 30a arranged in a matrix. As a result, according to the present embodiment, it is possible to have the same effect as the imaging device 100 according to the first embodiment, and to further suppress the scale of the circuit configuration of the imaging device 100.

<Example of application of technology related to this disclosure>
The technology according to the present disclosure can be applied to various products. More specific application examples will be described below. For example, the technology according to the present disclosure can be applied to any type of transportation such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility vehicle, an airplane, a drone, a ship, a robot, a construction machine, an agricultural machine (tractor), etc. It may also be realized as a distance measuring device mounted on the body.

[Mobile object]
FIG. 29 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied. Vehicle control system 7000 includes multiple electronic control units connected via communication network 7010. In the example shown in FIG. 29, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside vehicle information detection unit 7400, an inside vehicle information detection unit 7500, and an integrated control unit 7600. . The communication network 7010 that connects these multiple control units is compliant with any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark). did It may be an in-vehicle communication network.

Each control unit includes a microcomputer that performs calculation processing according to various programs, a storage unit that stores programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various devices to be controlled. Equipped with Each control unit is equipped with a network I/F for communicating with other control units via the communication network 7010, and also communicates with devices or sensors inside and outside the vehicle through wired or wireless communication. A communication I/F is provided for communication. In FIG. 27, the functional configuration of the integrated control unit 7600 includes a microcomputer 7610, a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning section 7640, a beacon receiving section 7650, an in-vehicle device I/F 7660, an audio image output section 7670, An in-vehicle network I/F 7680 and a storage unit 7690 are illustrated. The other control units similarly include a microcomputer, a communication I/F, a storage section, and the like.

Drive system control unit 7100 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 includes a drive force generation device such as an internal combustion engine or a drive motor that generates drive force for the vehicle, a drive force transmission mechanism that transmits the drive force to wheels, and a drive force transmission mechanism that controls the steering angle of the vehicle. It functions as a control device for a steering mechanism to adjust and a braking device to generate braking force for the vehicle. The drive system control unit 7100 may have a function as a control device such as an ABS (Antilock Brake System) or an ESC (Electronic Stability Control).

A vehicle state detection section 7110 is connected to the drive system control unit 7100. The vehicle state detection unit 7110 includes, for example, a gyro sensor that detects the angular velocity of the axial rotation movement of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or an operation amount of an accelerator pedal, an operation amount of a brake pedal, or a steering wheel. At least one sensor for detecting angle, engine rotational speed, wheel rotational speed, etc. is included. The drive system control unit 7100 performs arithmetic processing using signals input from the vehicle state detection section 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, and the like.

The body system control unit 7200 controls the operations of various devices installed in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a headlamp, a back lamp, a brake lamp, a turn signal, or a fog lamp. In this case, radio waves transmitted from a portable device that replaces a key or signals from various switches may be input to the body control unit 7200. The body system control unit 7200 receives input of these radio waves or signals, and controls the door lock device, power window device, lamp, etc. of the vehicle.

Battery control unit 7300 controls secondary battery 7310, which is a power supply source for the drive motor, according to various programs. For example, information such as battery temperature, battery output voltage, or remaining battery capacity is input to the battery control unit 7300 from a battery device including a secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals, and controls the temperature adjustment of the secondary battery 7310 or the cooling device provided in the battery device.

External information detection unit 7400 detects information external to the vehicle in which vehicle control system 7000 is mounted. For example, at least one of an imaging section 7410 and an external information detection section 7420 is connected to the vehicle exterior information detection unit 7400. The imaging unit 7410 includes at least one of a ToF (Time of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The vehicle external information detection unit 7420 includes, for example, an environmental sensor for detecting the current weather or weather, or a sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000. At least one of the surrounding information detection sensors is included.

The environmental sensor may be, for example, at least one of a raindrop sensor that detects rainy weather, a fog sensor that detects fog, a sunlight sensor that detects the degree of sunlight, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging section 7410 and the vehicle external information detection section 7420 may be provided as independent sensors or devices, or may be provided as a device in which a plurality of sensors or devices are integrated.

Here, FIG. 30 shows an example of the installation positions of the imaging section 7410 and the vehicle external information detection section 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle 7900. An imaging unit 7910 provided in the front nose and an imaging unit 7918 provided above the windshield inside the vehicle mainly acquire images in front of the vehicle 7900. Imaging units 7912 and 7914 provided in the side mirrors mainly capture images of the sides of the vehicle 7900. An imaging unit 7916 provided in the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided above the windshield inside the vehicle is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

Note that FIG. 30 shows an example of the imaging range of each of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of imaging unit 7910 provided on the front nose, imaging ranges b and c indicate imaging ranges of imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d is The imaging range of an imaging unit 7916 provided in the rear bumper or back door is shown. For example, by superimposing image data captured by imaging units 7910, 7912, 7914, and 7916, an overhead image of vehicle 7900 viewed from above can be obtained.

The vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided at the front, rear, side, corner, and upper part of the windshield inside the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. The vehicle exterior information detection units 7920, 7926, and 7930 provided at the front nose, rear bumper, back door, and upper part of the windshield inside the vehicle interior of the vehicle 7900 may be, for example, LIDAR devices. These external information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, and the like.

Returning to FIG. 29, the explanation will be continued. The vehicle exterior information detection unit 7400 causes the imaging unit 7410 to capture an image of the exterior of the vehicle, and receives the captured image data. Further, the vehicle exterior information detection unit 7400 receives detection information from the vehicle exterior information detection section 7420 to which it is connected. When the external information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the external information detection unit 7400 transmits ultrasonic waves, electromagnetic waves, etc., and receives information on the received reflected waves. The external information detection unit 7400 may perform object detection processing such as a person, car, obstacle, sign, or text on the road surface or distance detection processing based on the received information. The external information detection unit 7400 may perform environment recognition processing to recognize rain, fog, road surface conditions, etc. based on the received information. The vehicle exterior information detection unit 7400 may calculate the distance to the object outside the vehicle based on the received information.

Further, the external information detection unit 7400 may perform image recognition processing or distance detection processing for recognizing people, cars, obstacles, signs, characters on the road surface, etc., based on the received image data. The outside-vehicle information detection unit 7400 performs processing such as distortion correction or alignment on the received image data, and also synthesizes image data captured by different imaging units 7410 to generate an overhead image or a panoramic image. Good too. The outside-vehicle information detection unit 7400 may perform viewpoint conversion processing using image data captured by different imaging units 7410.

The in-vehicle information detection unit 7500 detects in-vehicle information. For example, a driver condition detection section 7510 that detects the condition of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that images the driver, a biosensor that detects biometric information of the driver, a microphone that collects audio inside the vehicle, or the like. The biosensor is provided, for example, on a seat surface or a steering wheel, and detects biometric information of a passenger sitting on a seat or a driver holding a steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or determine whether the driver is dozing off. You may. The in-vehicle information detection unit 7500 may perform processing such as noise canceling processing on the collected audio signal.

Integrated control unit 7600 controls overall operations within vehicle control system 7000 according to various programs. An input section 7800 is connected to the integrated control unit 7600. The input unit 7800 is realized by, for example, a device such as a touch panel, a button, a microphone, a switch, or a lever that can be inputted by the passenger. The integrated control unit 7600 may be input with data obtained by voice recognition of voice input through a microphone. The input unit 7800 may be, for example, a remote control device using infrared rays or other radio waves, or an externally connected device such as a mobile phone or a PDA (Personal Digital Assistant) that is compatible with the operation of the vehicle control system 7000. It's okay. The input unit 7800 may be, for example, a camera, in which case the passenger can input information using gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by a passenger may be input. Further, the input section 7800 may include, for example, an input control circuit that generates an input signal based on information input by a passenger or the like using the input section 7800 described above and outputs it to the integrated control unit 7600. By operating this input unit 7800, a passenger or the like inputs various data to the vehicle control system 7000 and instructs processing operations.

The storage unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. Further, the storage unit 7690 may be realized by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.

The general-purpose communication I/F 7620 is a general-purpose communication I/F that mediates communication with various devices existing in the external environment 7750. The general-purpose communication I/F 7620 supports cellular communication protocols such as GSM (registered trademark) (Global System of Mobile communications), WiMAX, LTE (Long Term Evolution), or LTE-A (LTE-Advanced), or wireless LTE. AN (Wi-Fi Other wireless communication protocols may be implemented, such as Bluetooth®. The general-purpose communication I/F 7620 connects to a device (for example, an application server or a control server) existing on an external network (for example, the Internet, a cloud network, or an operator-specific network) via a base station or an access point, for example. You may. In addition, the general-purpose communication I/F 7620 uses, for example, P2P (Peer To Peer) technology to communicate with a terminal located near the vehicle (for example, a driver, a pedestrian, a store terminal, or an MTC (Machine Type Communication) terminal). You can also connect it with

The dedicated communication I/F 7630 is a communication I/F that supports communication protocols developed for use in vehicles. The dedicated communication I/F 7630 supports, for example, WAVE (Wireless Access in Vehicle Environment), which is a combination of lower layer IEEE802.11p and upper layer IEEE1609, and DSRC (Dedicated Short Range Co., Ltd.). standard protocols such as communications) or cellular communication protocols. May be implemented. The dedicated communication I/F 7630 is typically used for vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestria communication. n ) communications, a concept that includes one or more of the following:

The positioning unit 7640 performs positioning by receiving, for example, a GNSS signal from a GNSS (Global Navigation Satellite System) satellite (for example, a GPS signal from a GPS (Global Positioning System) satellite), and determines the latitude, longitude, and altitude of the vehicle. Generate location information including. Note that the positioning unit 7640 may specify the current location by exchanging signals with a wireless access point, or may acquire location information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

The beacon receiving unit 7650 receives, for example, radio waves or electromagnetic waves transmitted from a wireless station installed on a road, and obtains information such as the current location, traffic congestion, road closure, or required time. Note that the function of the beacon receiving unit 7650 may be included in the dedicated communication I/F 7630 described above.

The in-vehicle device I/F 7660 is a communication interface that mediates connections between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I/F 7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). The in-vehicle device I/F 7660 also connects to USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or MHL (Mobile A wired connection such as a High-definition Link) may also be established. In-vehicle equipment 7760 may include, for example, at least one of a mobile device or wearable device owned by a passenger, or an information device carried into or attached to the vehicle. Further, the in-vehicle device 7760 may include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I/F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

In-vehicle network I/F 7680 is an interface that mediates communication between microcomputer 7610 and communication network 7010. The in-vehicle network I/F 7680 transmits and receives signals and the like in accordance with a predetermined protocol supported by the communication network 7010.

The microcomputer 7610 of the integrated control unit 7600 communicates via at least one of a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning section 7640, a beacon reception section 7650, an in-vehicle device I/F 7660, and an in-vehicle network I/F 7680. The vehicle control system 7000 is controlled according to various programs based on the information obtained. For example, the microcomputer 7610 calculates a control target value for a driving force generating device, a steering mechanism, or a braking device based on acquired information inside and outside the vehicle, and outputs a control command to the drive system control unit 7100. Good too. For example, the microcomputer 7610 realizes ADAS (Advanced Driver Assistance System) functions, including vehicle collision avoidance or impact mitigation, following distance based on vehicle distance, vehicle speed maintenance, vehicle collision warning, vehicle lane departure warning, etc. Coordination control may be performed for the purpose of In addition, the microcomputer 7610 controls the driving force generating device, steering mechanism, braking device, etc. based on the acquired information about the surroundings of the vehicle, so that the microcomputer 7610 can drive the vehicle autonomously without depending on the driver's operation. Cooperative control for the purpose of driving etc. may also be performed.

The microcomputer 7610 acquires information through at least one of a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning section 7640, a beacon reception section 7650, an in-vehicle device I/F 7660, and an in-vehicle network I/F 7680. Based on this, three-dimensional distance information between the vehicle and surrounding objects such as structures and people may be generated, and local map information including surrounding information of the current position of the vehicle may be generated. Furthermore, the microcomputer 7610 may predict dangers such as a vehicle collision, a pedestrian approaching, or entering a closed road, based on the acquired information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or lighting a warning lamp.

The audio image output unit 7670 transmits an output signal of at least one of audio and image to an output device that can visually or audibly notify information to a passenger of the vehicle or to the outside of the vehicle. In the example of FIG. 29, an audio speaker 7710, a display section 7720, and an instrument panel 7730 are illustrated as output devices. Display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display section 7720 may have an AR (Augmented Reality) display function. The output device may be other devices other than these devices, such as headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, or a lamp. When the output device is a display device, the display device displays results obtained from various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, graphs, etc. Show it visually. Further, when the output device is an audio output device, the audio output device converts an audio signal consisting of reproduced audio data or acoustic data into an analog signal and audibly outputs the analog signal.

Note that in the example shown in FIG. 29, at least two control units connected via the communication network 7010 may be integrated as one control unit. Alternatively, each control unit may be composed of a plurality of control units. Furthermore, vehicle control system 7000 may include another control unit not shown. Further, in the above description, some or all of the functions performed by one of the control units may be provided to another control unit. In other words, as long as information is transmitted and received via the communication network 7010, predetermined arithmetic processing may be performed by any one of the control units. Similarly, sensors or devices connected to any control unit may be connected to other control units, and multiple control units may send and receive detection information to and from each other via communication network 7010. .

An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. Among the configurations described above, the technology according to the present disclosure includes, for example, imaging units 7910, 7912, 7914, 7916, and 7918, external information detection units 7920, 7922, 7924, 7926, 7928, and 7930, and driver state detection. 7510, etc. Specifically, the imaging system 10 of FIG. 1 having the imaging device of the present disclosure can be applied to these imaging units and detection units. By applying the technology according to the present disclosure, the effects of noise events such as sensor noise can be alleviated and the occurrence of a true event can be detected reliably and quickly, thereby realizing safe vehicle driving. It becomes possible to do so.

Note that the present technology can have the following configuration.

(1) a signal holding circuit that holds a first event signal indicating whether a brightness signal based on brightness exceeds a first threshold;
Based on a second event signal indicating whether the brightness signal exceeds a second threshold different from the first threshold and the first event signal held in the signal holding circuit, the brightness signal an arithmetic circuit that generates an event signal indicating whether at least one of the threshold and the second threshold has been exceeded;
An imaging device comprising:

(2) the arithmetic circuit holds the event signal in the signal holding circuit;
a transfer unit that transfers the event signal held in the signal holding circuit;
The imaging device according to (1), further comprising:

(3) the first event signal and the second event signal are sequentially input to the arithmetic circuit;
The imaging device according to (1) or (2), which has a first mode in which the first event signal and the second event signal are sequentially output to the signal holding circuit in response to input of a control signal.

(4) the first event signal and the second event signal are sequentially input to the arithmetic circuit;
a first event signal outputting the first event signal to the signal holding circuit in response to input of a control signal, and generating the event signal based on the first event signal and the second event signal held in the signal holding circuit; The imaging device according to any one of (1) to (3), having two modes.

(5) The first event signal and the second event signal are signals containing at least one of true value and false value information,
The imaging device according to any one of (1) to (4), wherein the arithmetic circuit has an OR circuit and performs an OR operation on the first event signal and the second event signal.

(6) the first event signal and the second event signal are sequentially input to the arithmetic circuit;
The arithmetic circuit includes the OR circuit and a multiplexer,
The multiplexer outputs the first event signal to the signal holding circuit according to a control signal,
The imaging device according to (5), wherein the logical sum circuit performs a logical sum operation of the first event signal held in the signal holding circuit and the second event signal.

(7) The imaging device according to (6), wherein the multiplexer outputs the result of the pre-rational sum operation to the signal holding circuit according to the pre-control signal.

(8) The first event signal and the second event signal are sequentially input to the arithmetic circuit,
A false value is initially set in the signal holding circuit,
The logical sum circuit holds the logical sum operation of the false value and the first event signal in the signal holding circuit, and then performs the logical sum operation of the held result of the logical sum operation and the second event signal. , (5).

(9) The imaging device according to (5), wherein the arithmetic circuit further includes an AND circuit.

(10) The first event signal and the second event signal are sequentially input to the arithmetic circuit,
A false value is set at one end of the AND circuit,
The OR circuit holds the OR operation of the output signal of the AND circuit and the first event signal in the signal holding circuit,
Next, a true value is set at the one end of the AND circuit, and the result of the held OR operation, the result of the AND of the true value, and the second event signal are input to the OR circuit. is,
The imaging device according to (9), wherein the OR circuit performs an OR operation between the held result of the OR operation, the result of the AND operation of the true value, and the second event signal.

(11) The imaging device according to (9), wherein the arithmetic circuit is configured with an equivalent circuit of at least one of the OR circuit and the AND circuit.

(12) Within the same pixel,
a photoelectric conversion element that outputs a signal according to the luminance;
a current-voltage converter that logarithmically converts the signal;
The imaging according to any one of (1) to (11), further comprising a differential circuit that generates the first event signal and the second event signal based on the signal logarithmically converted by the current-voltage converter. Device.

(13) In the same pixel,
The imaging device according to (12), further comprising the signal holding circuit and the arithmetic circuit.

(14) a plurality of pixels including a photoelectric conversion unit that outputs a signal according to brightness and a current-voltage conversion unit that logarithmically converts the signal;
a difference circuit that sequentially generates the first event signal and the second event signal based on the logarithmically converted signals in each of the current-voltage converters;
a plurality of the signal holding circuits respectively corresponding to the plurality of pixels;
The method according to any one of (1) to (11), wherein the plurality of signal holding circuits and the arithmetic circuit process the first event signal and the second event signal that are generated in the order. Imaging device.

(15) a signal holding step of holding in a signal holding circuit a first event signal indicating whether the brightness signal based on the brightness exceeds the first threshold;
Based on a second event signal indicating whether the brightness signal exceeds a second threshold different from the first threshold and the first event signal held in the signal holding circuit, the brightness signal an arithmetic processing step of generating an event signal indicating whether at least one of the threshold and the second threshold has been exceeded;
An imaging method comprising:

Aspects of the present disclosure are not limited to the individual embodiments described above, and include various modifications that can be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the contents described above. That is, various additions, changes, and partial deletions are possible without departing from the conceptual idea and spirit of the present disclosure derived from the content defined in the claims and equivalents thereof.

30: Pixel array unit, 31: Light receiving unit (photoelectric conversion element), 100: Imaging device, 331: Current-voltage conversion unit (logarithmic conversion circuit), 334: Quantizer (difference circuit), 335: Arithmetic circuit, 335a: Arithmetic circuit, 335b: multiplexer, 335c: AND circuit, 336: signal holding circuit, 337: transfer unit, 338: control circuit.

Claims (15)

a signal holding circuit that holds a first event signal indicating whether a brightness signal based on brightness exceeds a first threshold;
Based on a second event signal indicating whether the brightness signal exceeds a second threshold different from the first threshold and the first event signal held in the signal holding circuit, the brightness signal an arithmetic circuit that generates an event signal indicating whether at least one of the threshold and the second threshold has been exceeded;
An imaging device comprising: the arithmetic circuit holds the event signal in the signal holding circuit;
a transfer unit that transfers the event signal held in the signal holding circuit;
The imaging device according to claim 1, further comprising: The first event signal and the second event signal are sequentially input to the arithmetic circuit,
The imaging device according to claim 1 or 2, having a first mode in which the first event signal and the second event signal are sequentially output to the signal holding circuit in response to input of a control signal. The first event signal and the second event signal are sequentially input to the arithmetic circuit,
a first event signal outputting the first event signal to the signal holding circuit in response to input of a control signal, and generating the event signal based on the first event signal and the second event signal held in the signal holding circuit; The imaging device according to claim 3, having two modes. The first event signal and the second event signal are signals containing information of at least one of a true value and a false value,
The imaging device according to claim 4, wherein the arithmetic operation circuit includes an OR circuit, and performs an OR operation on the first event signal and the second event signal. The first event signal and the second event signal are sequentially input to the arithmetic circuit,
The arithmetic circuit includes the OR circuit and a multiplexer,
The multiplexer outputs the first event signal to the signal holding circuit according to a control signal,
The imaging device according to claim 5, wherein the logical sum circuit performs a logical sum operation of the first event signal held in the signal holding circuit and the second event signal. The imaging device according to claim 6, wherein the multiplexer outputs the result of the pre-rational sum operation to the signal holding circuit according to the pre-control signal. The first event signal and the second event signal are sequentially input to the arithmetic circuit,
A false value is initially set in the signal holding circuit,
The logical sum circuit holds the logical sum operation of the false value and the first event signal in the signal holding circuit, and then performs the logical sum operation of the held result of the logical sum operation and the second event signal. , an imaging device according to claim 5. The imaging device according to claim 5, wherein the arithmetic circuit further includes an AND circuit. The first event signal and the second event signal are sequentially input to the arithmetic circuit,
A false value is set at one end of the AND circuit,
The OR circuit holds the OR operation of the output signal of the AND circuit and the first event signal in the signal holding circuit,
Next, a true value is set at the one end of the AND circuit, and the result of the held OR operation, the result of the AND of the true value, and the second event signal are input to the OR circuit. is,
The imaging device according to claim 9, wherein the OR circuit performs an OR operation between the held result of the OR operation, the result of the AND operation of the true value, and the second event signal. The imaging device according to claim 9, wherein the arithmetic circuit is configured with an equivalent circuit of at least one of the OR circuit and the AND circuit. Within the same pixel,
a photoelectric conversion element that outputs a signal according to the luminance;
a current-voltage converter that logarithmically converts the signal;
The imaging device according to claim 1, further comprising: a differential circuit that generates the first event signal and the second event signal based on the signal logarithmically converted by the current-voltage converter. In the same pixel,
The imaging device according to claim 12, further comprising the signal holding circuit and the arithmetic circuit. a plurality of pixels having a photoelectric conversion unit that outputs a signal according to brightness, and a current-voltage conversion unit that logarithmically converts the signal;
a difference circuit that sequentially generates the first event signal and the second event signal based on the logarithmically converted signals in each of the current-voltage converters;
a plurality of the signal holding circuits respectively corresponding to the plurality of pixels;
The imaging device according to claim 1, wherein the plurality of signal holding circuits and the arithmetic circuit perform processing on the first event signal and the second event signal that are generated in the order. a signal holding step of holding in a signal holding circuit a first event signal indicating whether the brightness signal based on the brightness exceeds a first threshold;
Based on a second event signal indicating whether the brightness signal exceeds a second threshold different from the first threshold and the first event signal held in the signal holding circuit, the brightness signal an arithmetic processing step of generating an event signal indicating whether at least one of the threshold and the second threshold has been exceeded;
An imaging method comprising:

Images