JP2020141362A - Photodetector - Google Patents

Abstract

To obtain a photodetector which can execute imaging operation while ensuring a dynamic range.SOLUTION: A photodetector includes a first pixel having a first photoelectric conversion element generating first received charges, first and second charge storage parts for storing the first received charges, a first switch for connecting the first photoelectric conversion element with the first charge storage part, and a second switch for connecting the first photoelectric conversion element with the second charge storage part, a first exposure control section performing first set treatment for setting a first time length of a first exposure period where the first photoelectric conversion element generates the first received charges to be stored in the first charge storage part, on the basis of a first charge amount of the first received charges stored in the first charge storage part, and a second time length of a second exposure period generating the first received charges, and a first drive part for controlling operation of the first and second switches on the basis of results of first setting processing.SELECTED DRAWING: Figure 9

Description

The present disclosure relates to a photodetector capable of detecting light.

When measuring the distance to the object to be measured, the TOF (Time Of Flight) method is often used. In this TOF method, light is emitted and the reflected light reflected by the object to be measured is detected. Then, in the TOF method, the distance to the object to be measured is measured by measuring the time difference between the timing at which the light is emitted and the timing at which the reflected light is detected (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2013-0764645

By the way, in the photodetector, there may be a case where it is desired to perform an imaging operation in addition to the distance measuring operation for measuring the distance to the object to be measured. It is desirable to secure a dynamic range in the imaging operation.

It is desirable to provide a photodetector capable of performing imaging operations while ensuring a dynamic range.

The photodetector according to the embodiment of the present disclosure includes a first pixel, a first exposure control unit, and a first drive unit. The first pixel is a first photoelectric conversion element capable of performing a photoelectric conversion operation for generating a first received light charge based on light, and a first charge storage unit capable of accumulating a first received light charge. And a second charge storage unit, a first switch that can connect the first photoelectric conversion element to the first charge storage unit by turning on, and a first photoelectric conversion element by turning on. Has a second switch that can be connected to the second charge storage unit. In the first exposure control unit, the first photoelectric conversion element is stored in the first charge storage unit based on the first charge amount of the first received charge stored in the first charge storage unit. The first time length of the first exposure period for generating the first received light charge, and the second exposure for which the first photoelectric conversion element generates the first received light charge accumulated in the second charge storage unit. It is possible to perform the first setting process for setting the second time length of the period. The first drive unit is configured to be able to control the operation of the first switch and the second switch based on the result of the first setting process.

In the photodetector according to the embodiment of the present disclosure, the first photoelectric conversion element generates the first received charge, and the first switch is turned on, so that the first received charge is generated. It is accumulated in the charge storage unit of 1, and when the second switch is turned on, the first received charge is accumulated in the second charge storage unit. Based on the first charge amount of the first received charge accumulated in the first charge storage unit, the first time length of the first exposure period and the second time length of the second exposure period are set. The first setting process for setting is performed. The first exposure period is a period in which the first photoelectric conversion element generates the first received charge accumulated in the first charge storage unit, and the second exposure period is a period in which the first photoelectric conversion element generates the first received charge. This is the period for generating the first received charge accumulated in the second charge storage unit. Then, the operation of the first switch and the second switch is controlled based on the result of the first setting process.

It is a block diagram which shows one structural example of the photodetector which concerns on one Embodiment of this disclosure. It is a block diagram which shows one structural example of the light detection part shown in FIG. It is a circuit diagram which shows one configuration example of the pixel shown in FIG. It is sectional drawing which shows an example of the main part sectional structure of the pixel shown in FIG. It is sectional drawing which shows one structural example of the light detection apparatus shown in FIG. It is a timing diagram which shows an example of the distance measuring operation in the light detection apparatus shown in FIG. It is a timing diagram which shows an example of the image pickup operation in the photodetector shown in FIG. It is a timing waveform diagram which shows an example of the distance measuring operation in the photodetector shown in FIG. It is another timing waveform diagram which shows an example of the distance measuring operation in the photodetector shown in FIG. It is a timing waveform diagram which shows an example of the imaging operation in the photodetector shown in FIG. It is explanatory drawing which shows one operation state of the pixel shown in FIG. It is explanatory drawing which shows the other operation state of the pixel shown in FIG. It is explanatory drawing which shows the other operation state of the pixel shown in FIG. It is explanatory drawing which shows the other operation state of the pixel shown in FIG. It is explanatory drawing which shows the other operation state of the pixel shown in FIG. It is a flowchart which shows an example of the imaging operation in the photodetector shown in FIG. It is a timing waveform diagram which shows an example of the imaging operation in the photodetector which concerns on the modification. It is explanatory drawing which shows one operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is a flowchart which shows an example of the image pickup operation in the photodetector which concerns on another modification. It is a circuit diagram which shows one composition example of the pixel which concerns on another modification. It is a circuit diagram which shows one composition example of the pixel which concerns on another modification. It is a block diagram which shows one configuration example of a pixel array and a drive part which concerns on another modification. It is explanatory drawing which shows one structural example of the light detection part which concerns on other modification. It is a circuit diagram which shows one structural example of a pixel and a circuit shown in FIG. It is a block diagram which shows an example of the schematic structure of a vehicle control system. It is explanatory drawing which shows an example of the installation position of the vehicle exterior information detection unit and the image pickup unit.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The explanation will be given in the following order.
1. 1. Embodiment 2. Application example to moving body

<1. Embodiment>
[Configuration example]
FIG. 1 shows a configuration example of a photodetector (photodetector 1) according to an embodiment. The photodetector 1 has two operation modes M (distance measuring operation mode MD and imaging operation mode MI). The distance measuring operation mode MD is an operation mode for performing a distance measuring operation for measuring the distance to the object 100 by an indirect method, and the imaging operation mode MI is a mode for performing an imaging operation for imaging the object 100. The photodetector 1 includes a light emitting unit 11, an optical system 12, a photodetector 20, and a control unit 13.

The light emitting unit 11 is configured to emit an optical pulse L0 toward the object 100 in the distance measuring operation mode MD. Based on the instruction from the control unit 13, the light emitting unit 11 emits the light pulse L0 by performing a light emitting operation in which light emission and non-light emission are alternately repeated. The light emitting unit 11 has, for example, a light source that emits infrared light. This light source is configured by using, for example, a laser light source or an LED (Light Emitting Diode).

The optical system 12 includes a lens that forms an image on the light receiving surface S of the photodetector 20. In the distance measuring operation mode MD, a light pulse (reflected light pulse L1) emitted from the light emitting unit 11 and reflected by the object 100 is incident on the optical system 12. Further, in the imaging operation mode MI, light from the subject is incident on the optical system 12.

The light detection unit 20 is configured to generate an image PIC by detecting light based on an instruction from the control unit 13. Specifically, the photodetector 20 generates an image PIC (distance image PIC1) by receiving the reflected light pulse L1 in the distance measuring operation mode MD. Each of the plurality of pixel values included in the distance image PIC1 indicates a value for the distance D to the object 100. Further, the photodetector 20 generates an image PIC (image PIC2) by receiving light L2 in the image pickup operation mode MI. Each of the plurality of pixel values included in the captured image PIC2 indicates the amount of received light. Then, the light detection unit 20 outputs the generated image PIC as an image signal DATA.

The control unit 13 controls the operation of the photodetector 1 by supplying control signals to the light emitting unit 11 and the photodetector 20 and controlling their operations. The control unit 13 has an operation mode setting unit 14. The operation mode setting unit 14 sets the operation mode M to one of the distance measurement operation mode MD and the imaging operation mode MI based on the instruction included in the control signal CTL supplied from the outside. The control unit 13 controls the operations of the light emitting unit 11 and the light detection unit 20 according to the operation mode M set by the operation mode setting unit 14.

FIG. 2 shows a configuration example of the photodetector 20. The light detection unit 20 includes a pixel array 21, a drive unit 22, a reading unit 24, and a processing unit 26.

The pixel array 21 has a plurality of pixels P arranged in a matrix. Each pixel P is designed to output pixel signals SIGA and SIGB.

FIG. 3 shows a configuration example of the pixel P. FIG. 4 shows an example of a configuration of a main part of the pixel P. The pixel array 21 includes a plurality of control line TGLA, a plurality of control line TGLB, a plurality of control line RSLA, a plurality of control line RSLB, a plurality of control line SELL, a plurality of signal lines SGLA, and a plurality of signals. It has a line SGLB. The control line TGLA is provided so as to extend in the horizontal direction (horizontal direction in FIGS. 2 and 3), and a control signal STGLA is applied to the control line TGLA by the drive unit 22. The control line TGLB is provided so as to extend in the horizontal direction, and a control signal STGLB is applied to the control line TGLB by the drive unit 22. The control line RSLA is provided so as to extend in the horizontal direction, and a control signal SRSLA is applied to the control line RSLA by the drive unit 22. The control line RSLB is provided so as to extend in the horizontal direction, and a control signal SRSLB is applied to the control line RSLB by the drive unit 22. The control line SELL is provided so as to extend in the horizontal direction, and a control signal SSELL is applied to the control line SELL by the drive unit 22. The signal line SGLA is provided so as to extend in the vertical direction (vertical direction in FIGS. 2 and 3), and transmits the pixel signal SIGA to the reading unit 24. The signal line SGLB is provided so as to extend in the vertical direction, and transmits the pixel signal SIGB to the reading unit 24.

The pixel P has a photodiode PD, transistors TGA and TGB, floating diffusion FDA and FDB, transistors RSTA and RSTB, transistors AMPA and AMPB, and transistors SELA and SELB. Transistors TGA, TGB, RSTA, RSTB, AMPA, AMPB, SELA, SELB are N-type MOS (Metal Oxide Semiconductor) transistors in this example.

The photodiode PD is a photoelectric conversion element that generates an electric charge according to the amount of light received. The anode of the photodiode PD is grounded, and the cathode is connected to the sources of the transistors TGA and TGB. In this example, the amount of charge that can be stored in the photodiode PD is smaller than the amount of charge that can be stored in the floating diffusion FDA and the amount of charge that can be stored in the floating diffusion FDB.

The gate of the transistor TGA is connected to the control line TGLA, the source is connected to the cathode of the photodiode PD and the source of the transistor TGB, and the drain is connected to the floating diffusion FDA, the source of the transistor RSTA, and the gate of the transistor AMPA. The floating diffusion FDA is configured to accumulate the charge supplied from the photodiode PD via the transistor TGA, for example, using a diffusion layer formed on the surface of the semiconductor substrate 9, as shown in FIG. It is composed. In FIG. 3, the floating diffusion FDA is shown using a capacitive element symbol. In this example, the amount of charge that can be stored in the floating diffusion FDA is larger than the amount of charge that can be stored in the photodiode PD. The gate of the transistor RSTA is connected to the control line RSLA, the drain is supplied with a power supply voltage VDD, and the source is connected to the floating diffusion FDA, the drain of the transistor TGA, and the gate of the transistor AMPA. The gate of the transistor AMPA is connected to the floating diffusion FDA, the drain of the transistor TGA, and the source of the transistor RSTA, the drain is supplied with the power supply voltage VDD, and the source is connected to the drain of the transistor SELA. The gate of the transistor SELA is connected to the control line SELL, the drain is connected to the source of the transistor AMPA, and the source is connected to the signal line SGLA. Hereinafter, the circuit including the photodiode PD, the floating diffusion FDA, and the transistors RSTA, AMPA, and SELA will also be referred to as tap A.

The gate of the transistor TGB is connected to the control line TGLB, the source is connected to the cathode of the photodiode PD and the source of the transistor TGA, and the drain is connected to the floating diffusion FDB, the source of the transistor RSTB, and the gate of the transistor AMPB. The floating diffusion FDB is configured to store the charge supplied from the photodiode PD via the transistor TGB, for example, using a diffusion layer formed on the surface of the semiconductor substrate 9, as shown in FIG. It is composed. In FIG. 3, similar to the floating diffusion FDA, the floating diffusion FDB is shown using the symbol of the capacitive element. In this example, the amount of charge that can be stored in the floating diffusion FDB is larger than the amount of charge that can be stored in the photodiode PD. The gate of the transistor RSTB is connected to the control line RSLB, the drain is supplied with the power supply voltage VDD, and the source is connected to the floating diffusion FDB, the drain of the transistor TGB, and the gate of the transistor AMPB. The gate of the transistor AMPB is connected to the floating diffusion FDB, the drain of the transistor TGB, and the source of the transistor RSTB, the drain is supplied with the power supply voltage VDD, and the source is connected to the drain of the transistor SELB. The gate of the transistor SELB is connected to the control line SELL, the drain is connected to the source of the transistor AMPB, and the source is connected to the signal line SGLB. Hereinafter, the circuit including the photodiode PD, the floating diffusion FDB, and the transistors RSTB, AMPB, and SELB is also referred to as tap B.

With this configuration, in the pixel P, the floating diffusion FDA is reset when the transistor RSTA is turned on, and the floating diffusion FDB is reset when the transistor RSTB is turned on. Then, when any one of the transistors TGA and TGB is turned on, the electric charge generated by the photodiode PD is selectively accumulated in the floating diffusion FDA and the floating diffusion FDB. Then, when the transistors SELA and SELB are turned on, the pixel P outputs a pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA, and at the same time, the amount of electric charge accumulated in the floating diffusion FDB is increased. The corresponding pixel signal SIGB is output.

The drive unit 22 (FIG. 2) is configured to drive a plurality of pixels P based on an instruction from the processing unit 26. Specifically, the drive unit 22 applies the control signal STGLA to the control line TGLA, applies the control signal STGLB to the control line TGLB, and applies the control signal SRSLA to the control line RSLA for control. The control signal SRSLB is applied to the line RSLB, and the control signal SSELL is applied to the control line SELL. The drive unit 22 has a signal generation unit 23. The signal generation unit 23 is configured to generate control signals STGLA and STGLB. The drive unit 22 applies a single control signal STGLA generated by the signal generation unit 23 to the plurality of control lines TGLA, and applies a single control signal STGLB generated by the signal generation unit 23 to the plurality of control lines TGLB. It is designed to do.

The reading unit 24 performs AD (Analog to Digital) conversion based on the pixel signal SIG (pixel signal SIGA, SIGB) supplied from the pixel array 21 via the signal line SGL (signal line SGLA, SGLB). It is configured to generate the image signal DATA0.

The reading unit 24 has a plurality of AD conversion units 25 and a bus wiring BUS. The plurality of AD conversion units 25 are provided corresponding to the plurality of signal lines SGL (signal lines SGLA, SGLB), respectively. The AD conversion unit 25 is configured to generate a digital code CODE by performing AD conversion based on the voltage of the pixel signal SIG supplied via the corresponding signal line SGL. Then, the AD conversion unit 25 supplies the digital code CODE to the bus wiring BUS.

The bus wiring BUS has a plurality of wirings and is configured to transmit the digital code CODE output from the AD conversion unit 25. Using this bus wiring BUS, the reading unit 24 sequentially transfers a plurality of digital code CODEs supplied from the plurality of AD conversion units 25 to the processing unit 26 as an image signal DATA0.

The processing unit 26 is configured to control the operation of the light detection unit 20 by supplying a control signal to the drive unit 22 and the reading unit 24 based on an instruction from the control unit 13. The processing unit 26 has an exposure control unit 27 and a pixel value calculation unit 28.

In the imaging operation mode MI, the exposure control unit 27 determines the length of the period during which the photodiode PD generates the electric charge accumulated in the floating diffusion FDA (exposure period T2A) and the electric charge accumulated in the floating diffusion FDB by the photodiode PD. Is configured to set the time length of the period (exposure period T2B) for generating the. Specifically, the exposure control unit 27 sets, for example, the ratio of the time length of the exposure period T2A to the time length of the exposure period T2B (exposure time ratio RT). The exposure time ratio RT is, in this example, the time length of the exposure period T2B divided by the time length of the exposure period T2A. Then, the processing unit 26 supplies the drive unit 22 with information about the time lengths of the exposure periods T2A and T2B set by the exposure control unit 27.

The pixel value calculation unit 28 is configured to calculate the pixel value of each of the plurality of pixels P based on the digital code CODE related to the plurality of pixels P in the pixel array 21. In the distance measuring operation mode MD, the pixel value indicates a value for the distance D to the object 100. The processing unit 26 generates a distance image PIC1 using the pixel values of a plurality of pixels P in the pixel array 21, and outputs the distance image PIC1 as an image signal DATA. Further, in the imaging operation mode MI, the pixel value indicates the amount of received light. The processing unit 26 generates an captured image PIC2 using the pixel values of a plurality of pixels P in the pixel array 21, and outputs the captured image PIC2 as an image signal DATA.

FIG. 5 shows a configuration example of the photodetector 1. A light emitting unit 11, a light detecting unit 20, and a holder 91 are arranged on the substrate of the substrate 90. The holder 91 is provided with openings 91A and 91B. The light emitting unit 11 is arranged on the substrate of the substrate 90 at a position corresponding to the opening 91A, and the light detecting unit 20 is arranged at a position on the substrate of the substrate 90 corresponding to the opening 91B. The holder 91 holds the cover glass 93. The cover glass 93 is configured to protect the light emitting unit 11 and the light detecting unit 20 from dust and an external atmosphere. The cover glass 93 is provided with a light diffusing film 94 and an infrared filter 95. The light diffusing film 94 is provided in a region of the cover glass 93 corresponding to the light emitting portion 11, and is configured to diffuse the light pulse L0. The infrared filter 95 is provided in a region of the cover glass 93 corresponding to the photodetector 20, and is configured to transmit infrared light. A lens holder 92 is provided in the opening 91B of the holder 91. The lens holder 92 holds the lenses 12A and 12B. The lenses 12A and 12B constitute an optical system 12.

Here, the photodiode PD corresponds to a specific example of the "first photoelectric conversion element" in the present disclosure. The floating diffusion FDB corresponds to a specific example of the "first charge storage unit" in the present disclosure. The floating diffusion FDA corresponds to a specific example of the "second charge storage unit" in the present disclosure. The transistor TGB corresponds to a specific example of the "first switch" in the present disclosure. The transistor TGA corresponds to a specific example of the "second switch" in the present disclosure. The transistors AMPB and SELB correspond to a specific example of the "first output unit" in the present disclosure. The transistors AMPA and SELA correspond to a specific example of the "second output unit" in the present disclosure. The exposure control unit 27 corresponds to a specific example of the "first exposure control unit" in the present disclosure. The exposure period T2B corresponds to a specific example of the "first exposure period" in the present disclosure. The exposure period T2A corresponds to a specific example of the "second exposure period" in the present disclosure. The exposure time ratio RT corresponds to a specific example of the "first exposure time ratio" in the present disclosure. The drive unit 22 corresponds to a specific example of the "first drive unit" in the present disclosure. The pixel value calculation unit 28 corresponds to a specific example of the “first pixel value calculation unit” in the present disclosure.

[Action and action]
Subsequently, the operation and operation of the photodetector 1 of the present embodiment will be described.

(Overview of overall operation)
First, an outline of the overall operation of the photodetector 1 will be described with reference to FIGS. When performing a distance measuring operation for measuring the distance to the object 100, the photodetector 1 operates in the distance measuring operation mode MD. In this distance measuring operation mode MD, the photodetector 1 emits the light pulse L0 by performing the light emitting operation OP1 and receives the reflected light pulse L1 reflected by the object 100 by performing the exposure operation OP2. Then, charges are selectively accumulated in the floating diffusion FDA and FDB in each of the plurality of pixels P. Then, the photodetector 1 performs AD conversion based on the pixel signal SIG supplied from the plurality of pixels P via the signal line SGL by performing the read operation OP3, and generates the image signal DATA0. Then, the photodetector 1 generates a distance image PIC1 based on the image signal DATA0, and outputs the distance image PIC1 as an image signal DATA.

Further, when performing an imaging operation for imaging the object 100, the photodetector 1 operates in the imaging operation mode MI. In this imaging operation mode MI, the photodetector 1 receives the light L2 by performing the exposure operation OP2, and selectively accumulates charges in the floating diffusion FDA and FDB in each of the plurality of pixels P. Then, the photodetector 1 performs AD conversion based on the pixel signal SIG supplied from the plurality of pixels P via the signal line SGL by performing the read operation OP3, and generates the image signal DATA0. Then, the photodetector 1 generates the captured image PIC2 based on the image signal DATA0, and outputs the captured image PIC2 as the image signal DATA.

(Detailed operation)
6A and 6B show an example of the operation in the photodetector 1, FIG. 6A shows the distance measuring operation in the distance measuring operation mode MD, and FIG. 6B shows the imaging operation in the imaging operation mode MI. In FIGS. 6A and 6B, the upper end indicates the uppermost portion of the pixel array 21, and the lower end indicates the lowermost portion of the pixel array 21.

In the distance measuring operation mode MD, as shown in FIG. 6A, the photodetector 1 performs the light emitting operation OP1 and the exposure operation OP2 during the period of timings t1 to t2. Specifically, the control unit 13 controls the operation of the light emitting unit 11, and the light emitting unit 11 emits the light pulse L0 by performing the light emitting operation OP1 that alternately repeats light emission and non-light emission. Further, the control unit 13 controls the operation of the light detection unit 20, and the drive unit 22 of the light detection unit 20 drives a plurality of pixels P in the pixel array 21. The plurality of pixels P receive the reflected light pulse L1 reflected by the object 100. Then, the photodetector 1 performs the read operation OP3 during the period of timings t2 to t3. Specifically, the drive unit 22 sequentially drives a plurality of pixels P in the pixel array 21 in pixel line units, and the plurality of pixels P sequentially drive a pixel signal SIG and a signal line SGL (signal line SGLA, SGLB). It is supplied to the reading unit 24 via. Then, the reading unit 24 generates the image signal DATA0 by performing AD conversion based on the pixel signal SIG. The photodetector 1 alternately repeats the light emission operation OP1 and the exposure operation OP2, and the read operation OP3. The processing unit 26 generates a distance image PIC1 in which each pixel value indicates a value for the distance D based on the image signal DATA0.

In the imaging operation mode MI, the photodetector 1 performs the exposure operation OP2 during the period from timing t4 to t5, as shown in FIG. 6B. Specifically, the control unit 13 controls the operation of the light detection unit 20, and the drive unit 22 of the light detection unit 20 drives a plurality of pixels P in the pixel array 21. Then, the photodetector 1 performs the read operation OP3 during the period of timings t5 to t6. Specifically, the drive unit 22 sequentially drives a plurality of pixels P in the pixel array 21 in pixel line units, and the plurality of pixels P sequentially drive a pixel signal SIG and a signal line SGL (signal line SGLA, SGLB). It is supplied to the reading unit 24 via. Then, the reading unit 24 generates the image signal DATA0 by performing AD conversion based on the pixel signal SIG. The photodetector 1 alternately repeats the exposure operation OP2 and the read operation OP3. The processing unit 26 generates an image PIC2 in which each pixel value indicates the amount of light received, based on the image signal DATA0.

(About distance measurement operation)
Next, the distance measuring operation for measuring the distance to the object 100 in the distance measuring operation mode MD will be described in detail. In the following, a certain pixel P (pixel P1) among the plurality of pixels P will be focused on, and the operation of the focused pixel P1 will be described in detail by taking as an example.

FIG. 7 shows an example of the distance measuring operation of the light detection device 1. FIG. 7A shows the waveform of the optical pulse L0 emitted by the light emitting unit 11, and FIG. 7B shows the waveforms of the control signals SRSLA and SRSLB. (C) shows the waveform of the voltage VFDA in the floating diffusion FDA, (D) shows the waveform of the voltage VFDB in the floating diffusion FDB, (E) shows the waveform of the control signal STGLA, and (F) shows the control signal. The waveform of STGLB is shown, (G) shows the operation of the AD conversion unit 25 (AD conversion unit 25A) connected to the signal line SGLA related to the pixel P1, and (H) is connected to the signal line SGLB related to the pixel P1. The operation of the AD conversion unit 25 (AD conversion unit 25B) is shown.

First, the photodetector 1 performs a light emitting operation OP1 and an exposure operation OP2. In this exposure operation OP2, the drive unit 22 lowers the voltage of the control signal SSELL. As a result, in the pixel P1, both the transistors SELA and SELB are turned off, and the pixel P1 is electrically separated from the signal lines SGLA and SGLB. The floating diffusion FDA and FDB of the pixel P1 selectively store the electric charge generated by the photodiode PD. Then, the photodetector 1 performs the read operation OP3. In this read operation OP3, the drive unit 22 raises the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, both the transistors SELA and SELB are turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs the pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs the pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB. This operation will be described in detail below.

In the period before the timing t11, the drive unit 22 raises the voltage of the control signals SRSLA and SRSLB to a high level (FIG. 7 (B)). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned on, and the voltages VFDA and VFDB in the floating diffusion FDA and FDB are set to the power supply voltage VDD (FIGS. 7 (C) and 7 (D)). In this way, the floating diffusion FDA and FDB are reset.

Then, at the timing t11, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a high level to a low level (FIG. 7B). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. Further, at this timing t11, the light emitting unit 11 starts a light emitting operation OP1 that alternately repeats light emission and non-light emission (FIG. 7 (A)). Then, the drive unit 22 starts generating the control signals STGLA and STGLB at this timing t11 (FIGS. 7 (E) and 7 (F)). The control signals STGLA and STGLB are signals that alternately repeat high level and low level. The phase of the control signal STGLB lags the phase of the control signal STGLA by 180 degrees. As shown in FIGS. 7A and 7E, the frequency of the optical pulse L0 is the same as the frequency of the control signal STGLA, and the phase of the optical pulse L0 is the same as the phase of the control signal STGLA. That is, the control unit 13 controls the operations of the light emitting unit 11 and the light detection unit 20 so that the light pulse L0 and the control signals STGLA and STGLB are synchronized.

In this way, the exposure period T1 starts at this timing t11. In the exposure period T1, in the pixel P1, the photodiode PD generates an electric charge based on the reflected light pulse L1 corresponding to the light pulse L0. The transistor TGA is turned on and off based on the control signal STGLA, and the transistor TGB is turned on and off based on the control signal STGLB. That is, either one of the transistors TGA and TGB is turned on. As a result, the electric charge generated by the photodiode PD is selectively accumulated in the floating diffusion FDA and the floating diffusion FDB.

8A and 8B show an operation example of the pixel P1, in which FIG. 8A shows the waveform of the optical pulse L0, FIG. 8B shows the waveform of the reflected light pulse L1, and FIG. 8C shows the waveform of the control signal STGLA. (D) shows the waveform of the control signal STGLB. In this example, at the timing t21, the optical pulse L0 rises, the control signal STGLA rises, and the control signal STGLB falls. Then, at the timing t23 whose phase is delayed by 180 degrees from the timing t21, the optical pulse L0 falls, the control signal STGLA falls, and the control signal STGLB rises. Similarly, at the timing t25 whose phase is delayed by 180 degrees from the timing t23, the optical pulse L0 rises, the control signal STGLA rises, and the control signal STGLB falls. Then, at the timing t26 whose phase is delayed by 180 degrees from the timing t25, the optical pulse L0 falls, the control signal STGLA falls, and the control signal STGLB rises.

The phase of the reflected light pulse L1 is shifted by a phase φ from the phase of the light pulse L0 (FIG. 8 (B)). This phase φ corresponds to the distance D from the photodetector 1 to the object 100. In this example, the reflected light pulse L1 rises at the timing t22 which is delayed by the time corresponding to the phase φ from the timing t21, and the reflected light pulse L1 falls at the timing t24 which is delayed by the time corresponding to the phase φ from the timing t23. .. The photodiode PD of pixel P1 generates an electric charge in the period of timings t22 to t24 based on the reflected light pulse L1.

The transistor TGA transfers the charge generated by the photodiode PD to the floating diffusion FDA during the period when the control signal STGLA is at a high level (timing t22 to t23), and the transistor TGB has a control signal STGLB at a high level. In a certain period (timing t23 to t25), the charge generated by the photodiode PD is transferred to the floating diffusion FDB. As a result, the charge QA is accumulated in the floating diffusion FDA during the timing t22 to t23, and the charge QB is accumulated in the floating diffusion FDB during the timing t23 to t24.

The difference between the amount of charge of the charge QA and the amount of charge of the charge QB (charge difference ΔQ) changes according to the phase φ. In other words, the charge difference ΔQ changes according to the distance D from the photodetector 1 to the object 100. Specifically, for example, the shorter the distance D, the larger the charge QA and the smaller the charge QB. Further, for example, as the distance D increases, the charge QA decreases and the charge QB increases.

As shown in FIGS. 7 and 8, the pixel P1 repeats the operation at the timings t21 to t25. As a result, the charge QA is repeatedly accumulated in the floating diffusion FDA, and the charge QB is repeatedly accumulated in the floating diffusion FDB. As a result, the voltage VFDA of the floating diffusion FDA and the voltage VFDB of the floating diffusion FDB gradually decrease (FIGS. 7 (C) and 7 (D)). The amount of voltage change in the voltage VFDA corresponds to the amount of charge of the charge QA, and the amount of voltage change in the voltage VFDB corresponds to the amount of charge of the charge QB. In this example, the degree of change in voltage VFDA is greater than the degree of change in voltage VFDB.

Then, at the timing t12, the light emitting unit 11 ends the light emitting operation OP1 (FIG. 7 (A)). Further, the drive unit 22 stops the generation of the control signals STGLA and STGLB at this timing t12, and lowers the voltage of the control signals STGLA and STGLB to a low level (FIGS. 7 (E) and 7 (F)). As a result, the transistors TGA and TGB are turned off. In this way, the exposure period T1 ends.

After that, during the period from timing t13 to t16, the photodetector 1 performs the read operation OP3. In this read operation OP3, the drive unit 22 sets the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, both the transistors SELA and SELB are turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs the pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs the pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output.

During the period from timing t13 to t14, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 7 (G). ), (H)). At the timing t14, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a low level to a high level (FIG. 7B). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned on, and the voltages VFDA and VFDB in the floating diffusion FDA and FDB are set to the power supply voltage VDD (FIGS. 7 (C) and 7 (D)). In this way, the floating diffusion FDA and FDB are reset. At the timing t15, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a high level to a low level (FIG. 7B). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. Then, during the period from timing t15 to t16, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 7). (G), (H)). That is, the AD conversion unit 25A performs AD conversion based on the pixel signal SIGA after the floating diffusion FDA is reset, and the AD conversion unit 25B performs AD conversion based on the pixel signal SIGB after the floating diffusion FDB is reset. Perform the conversion.

The AD conversion unit 25A generates a digital code CODE (digital code CODEA) by subtracting the digital value converted in the period from timing t17 to t18 from the digital value converted in the period from timing t13 to t14. Similarly, the AD conversion unit 25B generates a digital code CODE (digital code CODEB) by subtracting the digital value converted in the period from timing t17 to t18 from the digital value converted in the period from timing t13 to t14. .. The reading unit 24 sequentially transfers these digital codes CODEA and CODEB to the processing unit 26 as an image signal DATA0.

The pixel value calculation unit 28 of the processing unit 26 calculates the pixel value of the pixel P1 based on, for example, the difference between the two digital codes CODEA and CODEB related to the pixel P1. This pixel value indicates a value for a distance D to the object 100. The processing unit 26 generates a distance image PIC1 using a plurality of pixel values obtained from a plurality of pixels P in the pixel array 21, and outputs the distance image PIC1 as an image signal DATA.

(About imaging operation)
Next, the imaging operation for imaging the object 100 in the imaging operation mode MI will be described in detail.

9A and 9B show an example of the imaging operation of the light detection device 1, in which FIG. 9A shows the waveform of the control signal SRSLB, FIG. 9B shows the waveform of the control signal STGLB, and FIG. 9C shows the control signal SRSLA. (D) shows the waveform of the control signal STGLA, (E) shows the operation of the AD conversion unit 25 (AD conversion unit 25A) connected to the signal line SGLA related to the pixel P1, and (F) Indicates the operation of the AD conversion unit 25 (AD conversion unit 25B) connected to the signal line SGLB related to the pixel P1. 10A to 10E show the operating state of the main part of the pixel P1.

The photodetector 1 first performs the exposure operation OP2. In this exposure operation OP2, the drive unit 22 lowers the voltage of the control signal SSELL. As a result, in the pixel P1, both the transistors SELA and SELB are turned off, and the pixel P1 is electrically separated from the signal lines SGLA and SGLB. The floating diffusion FDA and FDB of the pixel P1 selectively store the electric charge generated by the photodiode PD. Specifically, the floating diffusion FDB accumulates the charge generated by the photodiode PD during the exposure period T2B, and the floating diffusion FDA accumulates the charge generated by the photodiode PD during the exposure period T2A. The exposure time ratio RT (time length of exposure period T2B / time length of exposure period T2A) is dynamically set to, for example, about 1 to 100. In other words, the "time length of the exposure period T2B: the time length of the exposure period T2A" is set to, for example, about "1: 1" to "100: 1". Then, the photodetector 1 performs the read operation OP3. In this read operation OP3, the drive unit 22 raises the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, both the transistors SELA and SELB are turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs the pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs the pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB. This operation will be described in detail below.

In the period before the timing t31, the drive unit 22 raises the voltage of the control signals SRSLA and SRSLB to a high level (FIGS. 9A and 9C). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned on, and the voltages VFDA and VFDB in the floating diffusion FDA and FDB are set to the power supply voltage VDD. In this way, the floating diffusion FDA and FDB are reset.

Next, at the timing t31, the drive unit 22 changes the voltage of the control signal STGLB from a low level to a high level (FIG. 9B). As a result, the transistor TGB is turned on in the pixel P1. During the period t31 to t32 (period T11), the photodiode PD and the floating diffusion FDB are electrically connected, as shown in FIG. 10A.

Next, at the timing t32, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a high level to a low level (FIGS. 9A and 9C). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. At this time, the electric charge QA0 is accumulated in the floating diffusion FDA, and the electric charge QB0 is accumulated in the floating diffusion FDB. In this way, the exposure period T2B starts. During the period from timing t32 to t33 (period T12), as shown in FIG. 10B, the charge generated by the photodiode PD is accumulated in the floating diffusion FDB as the charge QB1. In this way, the charge QB (= QB0 + QB1) is accumulated in the floating diffusion FDB.

Next, at the timing t33, the drive unit 22 changes the voltage of the control signal STGLB from a high level to a low level (FIG. 9B). As a result, in the pixel P1, the transistor TGB is turned off, and the photodiode PD and the floating diffusion FDB are electrically disconnected. In this way, the exposure period T2B ends.

Then, at this timing t33, the exposure period T2A starts. During the period from timing t33 to t35 (period T13), as shown in FIG. 10C, the electric charge generated by the photodiode PD is accumulated in the photodiode PD.

After that, during the period from timing t34 to t37, the photodetector 1 performs the read operation OP3. In this read operation OP3, the drive unit 22 sets the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, both the transistors SELA and SELB are turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs the pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs the pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output.

During this period from t34 to t35, as shown in FIG. 10C, the charge QA0 is accumulated in the floating diffusion FDA. During the period from timing t34 to t35, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 9 (E)). The digital value converted by the AD conversion unit 25A during this period is a value corresponding to the amount of electric charge QA0.

Next, at the timing t35, the drive unit 22 changes the voltage of the control signal STGLA from a low level to a high level (FIG. 9 (D)). As a result, the transistor TGA is turned on in the pixel P1. During the period t35-t36 (period T14), the photodiode PD and the floating diffusion FDA are electrically connected, as shown in FIG. 10D. Then, the electric charge generated by the photodiode PD is accumulated in the floating diffusion FDA as the electric charge QA1. In this way, the charge QA (= QA0 + QA1) is accumulated in the floating diffusion FDA.

Next, at the timing t36, the drive unit 22 changes the voltage of the control signal STGLA from a high level to a low level (FIG. 9D). As a result, in the pixel P1, the transistor TGA is turned off, and the photodiode PD and the floating diffusion FDA are electrically disconnected. In this way, the exposure period T2A ends.

During the period from timing t36 to t37 (period T15), as shown in FIG. 10E, the charge QA (= QA0 + QA1) is accumulated in the floating diffusion FDA. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 9 (E)). The digital value converted by the AD conversion unit 25A during this period T15 is a value corresponding to the amount of electric charge QA (= QA0 + QA1).

Further, during the period from timing t36 to t37 (period T15), as shown in FIG. 10E, the charge QB (= QB0 + QB1) is accumulated in the floating diffusion FDB. The AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 9 (F)). The digital value converted by the AD conversion unit 25B during this period is a value corresponding to the amount of electric charge QB (= QB0 + QB1).

Next, at the timing t37, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a low level to a high level (FIGS. 9A and 9C). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned on, and the voltages VFDA and VFDB in the floating diffusion FDA and FDB are set to the power supply voltage VDD. In this way, the floating diffusion FDA and FDB are reset.

The AD conversion unit 25A generates a digital code CODE (digital code CODEA) by subtracting the digital value converted at the timings t34 to t35 from the digital value converted during the period from the timing t36 to t37. This digital code CODEA is a value corresponding to the amount of electric charge QA1 (= QA-QA0) accumulated during the exposure period T2A. That is, the AD conversion unit 25A generates the digital code CODEA by using the principle of so-called Correlated double sampling (CDS). Since the AD conversion unit 25A performs such correlation double sampling, the noise component (charge QA0 in this example) can be removed, and as a result, the image quality of the captured image can be improved.

Further, the AD conversion unit 25B generates a digital code CODE (digital code CODEB) based on the digital values converted at the timings t36 to t37. This digital code CODEB is a value corresponding to the amount of electric charge QB (= QB0 + QB1) accumulated in the exposure period T2B.

The reading unit 24 sequentially transfers these digital codes CODEA and CODEB to the processing unit 26 as an image signal DATA0.

As will be described later, the pixel value calculation unit 28 of the processing unit 26 calculates the pixel value of the pixel P, for example, based on the digital code CODEA and the exposure time ratio RT related to each pixel P. The processing unit 26 generates an image PIC2 using a plurality of pixel values obtained from a plurality of pixels P in the pixel array 21, and outputs the image PIC2 as an image signal DATA.

FIG. 11 shows an operation example of the exposure control unit 27 and the pixel value calculation unit 28. In this example, the amount of charge that can be stored in the photodiode PD is "1,000e-" or less (less than 1,000 electrons), and the amount of charge that can be stored in each of the floating diffusion FDA and FDB is "100,000e". -"Or less (less than 100,000 electrons). In the following, the exposure amount of the taps A and B is shown by the amount of electric charge.

The exposure control unit 27 sets the exposure time ratio RT based on the digital code CODEB related to a certain pixel P (reference pixel PR) among the plurality of pixels P in the pixel array 21. The reference pixel PR is indicated by, for example, a control signal CTL supplied from the outside. The external device can determine the reference pixel PR according to the scene, for example. The pixel value calculation unit 28 calculates the pixel value of each pixel P, for example, based on the digital code CODEA and the exposure time ratio RT related to each pixel P. The photodetector 20 repeats this operation, for example, in frame period units. This operation will be described in detail below.

First, the photodetector 20 of the photodetector 1 performs an imaging operation (step S101). Specifically, the photodetector 20 performs the exposure operation OP2 and the read operation OP3 as shown in FIG. As a result, digital codes CODEA and CODEB for a plurality of pixels P of the pixel array 21 can be obtained.

Next, the exposure control unit 27 confirms whether or not the exposure amount at the tap B of the reference pixel PR is “1,000 e−” or less (step S102). Specifically, the exposure control unit 27 determines whether or not the amount of electric charge accumulated in the floating diffusion FDB in the reference pixel PR is "1,000 e-" or less based on the digital code CODEB related to the reference pixel PR. Confirm.

In step S102, when the exposure amount at tap B of the reference pixel PR is "1,000 e-" or less ("Y" in step S102), the exposure control unit 27 sets the exposure time ratio RT (exposure period T2B). It is confirmed whether or not the time length / time length of the exposure period T2A) is “1” (step S103). That is, the exposure control unit 27 confirms whether or not "the time length of the exposure period T2B: the time length of the exposure period T2A" is "1: 1".

If the exposure time ratio RT is “1” in step S103 (“Y” in step S103), the process proceeds to step S109.

If the exposure time ratio RT is not "1" in step S103 ("N" in step S103), the exposure control unit 27 sets the exposure time ratio RT to "1" (step S104). That is, in this case, since the exposure amount is the amount of electric charge that can be stored in the photodiode PD, the exposure control unit 27 sets the exposure time ratio RT to “1”. Then, the process proceeds to step S108.

In step S102, when the exposure amount at the tap B of the reference pixel PR is larger than “1,000e−” (“N” in step S102), the exposure control unit 27 determines that the exposure amount at the tap B of the reference pixel PR is increased. It is confirmed whether or not it is "100,000 e-" or less (step S105).

In step S105, when the exposure amount of the reference pixel PR at the tap B is "100,000 e-" or less ("Y" in step S105), the exposure control unit 27 determines the exposure amount of the reference pixel PR at the tap B. Is divided by the exposure time ratio RT to confirm whether or not the value is “1,000e−” or less (step S106).

In step S106, if the value obtained by dividing the exposure amount at tap B of the reference pixel PR by the exposure time ratio RT is “1,000 e−” or less (“Y” in step S106), the process proceeds to step S109.

In step S106, when the value obtained by dividing the exposure amount at tap B of the reference pixel PR by the exposure time ratio RT is more than "1,000 e-" ("N" in step S106), the exposure control unit 27 exposes. The time ratio RT is set to a value obtained by dividing the exposure amount at tap B of the reference pixel PR by "1,000 e-" (step S107).

Then, in step S108, the pixel value calculation unit 28 determines the pixel value of each of the plurality of pixels P including the reference pixel PR to a value corresponding to the exposure amount of the tap B of the pixel P. Specifically, the pixel value calculation unit 28 determines the pixel value of each pixel P to a value corresponding to the digital code CODEB related to the pixel P.

Further, in step S109, the pixel value calculation unit 28 responds to the pixel value of each of the plurality of pixels P including the reference pixel PR by multiplying the exposure amount of the tap A of the pixel P by the exposure time ratio RT. Determine the value.

In step S105, when the exposure amount at the tap B of the reference pixel PR is more than "100,000 e-" ("N" in step S105), the exposure control unit 27 determines the time length of the exposure period T2A and the exposure period T2B. The total exposure time of the time length is shortened (step S110). That is, in this case, since the exposure amount exceeds the charge amount that can be accumulated in the floating diffusion FDB and is saturated, the exposure control unit 27 shortens the total exposure time.

This is the end of this flow. The photodetector 20 repeats the operations of steps S101 to S110, for example, in frame period units.

Here, the reference pixel PR corresponds to a specific example of the "first pixel" in the present disclosure. Pixels P other than the reference pixel PR correspond to a specific example of the "second pixel" in the present disclosure. The AD conversion unit 25B corresponds to a specific example of the “first AD conversion unit” in the present disclosure. The AD conversion unit 25A corresponds to a specific example of the “second AD conversion unit” in the present disclosure. The digital code CODEB corresponds to a specific example of the "first digital code" in the present disclosure. The digital code CODEA corresponds to a specific example of the "second digital code" in the present disclosure.

As described above, the photodetector 1 can perform an imaging operation in addition to the distance measuring operation. As a result, the photodetector 1 can synthesize a distance image having a higher resolution by, for example, combining the distance image PIC1 and the captured image PIC2. In particular, in the photodetector 1, the photodetector 20 that performs the distance measuring operation by the indirect method is used to perform the imaging operation. In such an indirect method, high measurement accuracy can be realized, so that the photodetector 1 can further improve the accuracy of the distance image by further performing the imaging operation.

Further, in the photodetector 1, the time length of the exposure period T2A and the time length of the exposure period T2B are set based on the exposure amount of the tap B of the reference pixel PR. Specifically, for example, the exposure time ratio RT (the time length of the exposure period T2B / the time length of the exposure period T2A) is set based on the exposure amount of the tap B of the reference pixel PR. As a result, the photodetector 1 grasps the approximate exposure amount based on the exposure amount of the tap B of the reference pixel PR, and appropriately sets the exposure time ratio RT based on the information about the approximate exposure amount. can do. As described above, in the photodetector 1, the exposure time ratio RT can be appropriately set, so that the conversion efficiency can be freely set substantially continuously. As a result, the photodetector 1 can perform the imaging operation while ensuring the dynamic range.

Further, in the photodetector 1, the pixel value of the pixel P is calculated based on the exposure time ratio RT set in this way and the exposure amount of the tap A of the pixel P. In particular, in the photodetector 1, the exposure amount of the tap A of the pixel P is obtained by using the correlated double sampling, and the pixel value of the pixel P is calculated based on the obtained exposure amount and the exposure time ratio RT. I tried to calculate. As a result, the photodetector 1 can obtain the captured image PIC2 with less noise.

[effect]
As described above, in the present embodiment, the time length of the exposure period T2A and the time length of the exposure period T2B are set based on the exposure amount of the tap B of the reference pixel, so that the exposure time ratio is appropriately set. Since it can be set, the imaging operation can be performed while ensuring the dynamic range.

[Modification 1]
In the above embodiment, one exposure period T2B and one exposure period T2A are provided in the imaging operation, but the present invention is not limited to this. The photodetector 1A according to this modification will be described in detail below.

The photodetector 1A according to this modification includes a photodetector 20A, similarly to the photodetector 1 (FIG. 1) according to the embodiment. The light detection unit 20A has a processing unit 26A, similarly to the light detection unit 20 (FIG. 2) according to the above embodiment. The processing unit 26A has an exposure control unit 27A. The exposure control unit 27A is configured to set the time length of the exposure period T2A and the time length of the exposure period T2B in the imaging operation mode MI, similarly to the exposure control unit 27 according to the above embodiment. The exposure control unit 27A divides the exposure period T2B into two, and sets the exposure periods T2A and T2B in the order of the exposure period T2B, the exposure period T2A, and the exposure period T2B.

FIG. 12 shows an example of the imaging operation of the photodetector 1A. 13A to 13G show the operating state of the main part of the pixel P1.

In the period before the timing t41, the drive unit 22 raises the voltage of the control signals SRSLA and SRSLB to a high level (FIGS. 12A and 12C). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned on, and the voltages VFDA and VFDB in the floating diffusion FDA and FDB are set to the power supply voltage VDD.

Next, at the timing t41, the drive unit 22 changes the voltage of the control signal STGLB from a low level to a high level (FIG. 12 (B)). As a result, the transistor TGB is turned on in the pixel P1. During the period t41 to t42 (period T21), the photodiode PD and the floating diffusion FDB are electrically connected, as shown in FIG. 13A.

Next, at the timing t42, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a high level to a low level (FIGS. 23 (A) and 23 (C)). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. At this time, the electric charge QA0 is accumulated in the floating diffusion FDA, and the electric charge QB0 is accumulated in the floating diffusion FDB. In this way, the exposure period T2B starts. During the period from timing t42 to t43 (period T22), as shown in FIG. 13B, the charge generated by the photodiode PD is accumulated in the floating diffusion FDB as the charge QB1.

Next, at the timing t43, the drive unit 22 changes the voltage of the control signal STGLB from a high level to a low level (FIG. 12 (B)). As a result, in the pixel P1, the transistor TGB is turned off, and the photodiode PD and the floating diffusion FDB are electrically disconnected. In this way, the exposure period T2B ends once.

Then, at this timing t43, the exposure period T2A starts. During the period from timing t43 to t45 (period T23), as shown in FIG. 13C, the electric charge generated by the photodiode PD is accumulated in the photodiode PD.

During the period from timing t44 to t45, as shown in FIG. 13C, the charge QA0 is accumulated in the floating diffusion FDA. During the period from timing t44 to t45, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 12 (E)). The digital value converted by the AD conversion unit 25A during this period is a value corresponding to the amount of electric charge QA0.

Next, at the timing t45, the drive unit 22 changes the voltage of the control signal STGLA from a low level to a high level (FIG. 12 (D)). As a result, the transistor TGA is turned on in the pixel P1. During the period t45-t46 (period T24), the photodiode PD and the floating diffusion FDA are electrically connected, as shown in FIG. 13D. Then, the electric charge generated by the photodiode PD is accumulated in the floating diffusion FDA as the electric charge QA1.

Next, at the timing t46, the drive unit 22 changes the voltage of the control signal STGLA from a high level to a low level (FIG. 9D). As a result, in the pixel P1, the transistor TGA is turned off, and the photodiode PD and the floating diffusion FDA are electrically disconnected. In this way, the exposure period T2A ends. Then, the exposure period T2B starts again.

During the period from timing t46 to t47 (period T25), as shown in FIG. 13E, charge QA (= QA0 + QA1) is accumulated in the floating diffusion FDA. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 12 (E)). The digital value converted by the AD conversion unit 25A during this period T25 is a value corresponding to the amount of electric charge QA (= QA0 + QA1).

Next, at timing t47, the drive unit 22 changes the voltage of the control signal SRSLA from a low level to a high level (FIG. 12 (C)). As a result, in the pixel P1, the transistor RSTA is turned on, and the voltage VFDA in the floating diffusion FDA is set to the power supply voltage VDD. Further, at this timing t47, the drive unit 22 changes the voltage of the control signal STGLB from a low level to a high level (FIG. 12 (B)). As a result, the transistor TGB is turned on in the pixel P1. During the period t47-t48 (period T26), the photodiode PD and the floating diffusion FDB are electrically connected, as shown in FIG. 13F. Then, the electric charge generated by the photodiode PD is accumulated as the electric charge QB1 in the floating diffusion FDB.

Next, at the timing t48, the drive unit 22 changes the voltage of the control signal STGLB from a high level to a low level (FIG. 12 (B)). As a result, in the pixel P1, the transistor TGB is turned off, and the photodiode PD and the floating diffusion FDB are electrically disconnected. In this way, the exposure period T2B ends.

During the period from timing t48 to t49 (period T27), as shown in FIG. 13G, the charge QB (= QB0 + QB1) is accumulated in the floating diffusion FDB. The AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 12 (F)). The digital value converted by the AD conversion unit 25B during this period is a value corresponding to the amount of electric charge QB (= QB0 + QB1).

The AD conversion unit 25A generates a digital code CODE (digital code CODEA) by subtracting the digital value converted at the timings t44 to t45 from the digital value converted during the period from the timing t46 to t47. This digital code CODEA is a value corresponding to the amount of electric charge QA1 (= QA-QA0) accumulated during the exposure period T2A.

Further, the AD conversion unit 25B generates a digital code CODE (digital code CODEB) based on the digital values converted at the timings t48 to t49. This digital code CODEB is a value corresponding to the amount of electric charge QB (= QB0 + QB1) accumulated in the two exposure periods T2B.

The reading unit 24 sequentially transfers these digital codes CODEA and CODEB to the processing unit 26A as the image signal DATA0.

In this way, in the photodetector 1A, the exposure periods T2A and T2B are set in the order of the exposure period T2B, the exposure period T2A, and the exposure period T2B in the imaging operation. As a result, in the photodetector 1A, for example, when the illuminance changes monotonically during the exposure period, the exposure amount of the tap A (digital code CODEA) is set to the exposure amount corresponding to the center value of the change range of the illuminance. be able to.

[Modification 2]
In the above embodiment, in the imaging operation, when the exposure amount of the tap B of the reference pixel PR is "100,000 e-" or less, which is the amount of charge that can be accumulated by the floating diffusion FDA and FDB, the exposure time ratio RT Was set, but it is not limited to this. The photodetector 1B according to this modification will be described in detail below.

FIG. 14 shows an operation example of the exposure control unit 27B and the pixel value calculation unit 28 according to this modification. Steps S101 to S104 and S106 to S110 are the same as in the case of the above embodiment (FIG. 11).

In step S102, when the exposure amount at the tap B of the reference pixel PR is larger than “1,000e−” (“N” in step S102), the exposure control unit 27B determines that the exposure amount at the tap B of the reference pixel PR is increased. It is confirmed whether or not it is "10,000e-" or less (step S115).

In step S115, when the exposure amount at the tap B of the reference pixel PR is more than “10,000e−” (“N” in step S115), the exposure control unit 27B determines that the exposure amount at the tap B of the reference pixel PR is increased. It is confirmed whether or not it is "100,000 e-" or less (step S116). When the exposure amount at the tap B of the reference pixel PR is "100,000 e-" or less ("Y" in step S116), this flow ends. If the exposure amount at the tap B of the reference pixel PR is more than "100,000 e-" ("N" in step S116), the process proceeds to step S110.

As a result, the photodetector 1B sets the exposure time ratio RT when the exposure amount of the tap B of the reference pixel PR is “10,000e−” or less. Thereby, for example, when the exposure amount at the tap B exceeds "10,000e-" and the linearity deteriorates, the exposure time ratio RT can be set so that the exposure amount does not deteriorate the linearity. .. As a result, the photodetector 1B can improve the image quality of the captured image PIC2.

[Modification 3]
In the above embodiment, the floating diffusion FDA and FDB for accumulating the electric charge generated by the photodiode PD are provided, but the present invention is not limited to this, and in addition to this, a capacitive element may be further provided. This modification will be described below with some examples.

FIG. 15 shows an example of the pixel P in the pixel array 21C according to this modification. The pixel array 21C has a plurality of control lines FDGL. The control line FDGL is provided so as to extend in the horizontal direction (horizontal direction in FIG. 15), and a control signal SFDGL is applied to the control line FDGL by the drive unit 22C according to the present modification.

The pixel P has transistors FDGA and FDGB, and capacitive elements CAPA and CAPB. The transistors FDGA and FDGB are N-type MOS transistors in this example.

The gate of the transistor FDGA is connected to the control line FDGL, the drain is connected to the source of the capacitive element CAPA and the transistor RSTA, and the sources are connected to the floating diffusion FDA, the drain of the transistor TGA, and the gate of the transistor AMPA. One end of the capacitive element CAPA is connected to the drain of the transistor FDGA and the source of the transistor RSTA, and the other end is grounded.

The gate of the transistor FDGB is connected to the control line FDGL, the drain is connected to the source of the capacitive element CAPB and the transistor RSTB, and the source is connected to the floating diffusion FDB, the drain of the transistor TGB, and the gate of the transistor AMPB. One end of the capacitive element CAPB is connected to the drain of the transistor FDGB and the source of the transistor RSTB, and the other end is grounded.

Based on the control signal SFDGL, the transistors FDGA and FDGB are turned on. By turning on the transistors FDGA and FDGB, the floating diffusion FDA and the capacitive element CAPA are electrically connected, and the floating diffusion FDB and the capacitive element CAPB are electrically connected. As a result, in the pixel P, the capacitance value at the taps A and B can be increased, so that the amount of charge that can be stored at the taps A and B can be increased.

FIG. 16 shows an example of the pixel P in another pixel array 21D according to this modification. The pixel array 21D has a plurality of control lines FDGLA and a plurality of control lines FDGLB. The control line FDGLA is provided so as to extend in the horizontal direction (horizontal direction in FIG. 16), and a control signal SFDGLA is applied to the control line FDGLA by the drive unit 22D according to the present modification. The control line FDGLB is provided so as to extend in the horizontal direction, and a control signal SFDGLB is applied to the control line FDGLB by the drive unit 22D according to the present modification.

The pixel P has transistors FDGA and FDGB, and capacitive elements CAPA and CAPB. The gate of the transistor FDGA is connected to the control line FDGLA. The gate of the transistor FDGB is connected to the control line FDGLB.

Based on the control signal SFDGLA, the transistor FDGA is turned on. By turning on the transistor FDGA, the floating diffusion FDA and the capacitive element CAPA are electrically connected. As a result, in the pixel P, the amount of electric charge that can be stored in the tap A can be increased. Similarly, the transistor FDGB is turned on based on the control signal SFDGLB. By turning on the transistor FDGB, the floating diffusion FDB and the capacitive element CAPB are electrically connected. As a result, in the pixel P, the amount of electric charge that can be stored in the tap B can be increased.

In this way, in the pixel P, the taps A and B can be set individually. Thereby, for example, the setting can be made according to the roles of the taps A and B. Specifically, for example, if saturation occurs even if "time length of exposure period T2B: time length of exposure period T2A" is set to "100: 1", the transistor FDGA is turned off and the transistor FDGB is turned on. It can be turned on. This can increase the amount of charge that can be stored in the tap B. In other words, the conversion efficiency in the tap B can be lowered and the conversion efficiency in the tap A can be increased.

[Modification example 4]
In the above embodiment, in the imaging operation, all the pixels P in the pixel array 21 operate at the same exposure time ratio RT, but the present invention is not limited to this. Instead of this, for example, a plurality of pixels P in the pixel array may be divided into a plurality of groups so that the plurality of pixels P belonging to the same group operate at the same exposure time ratio RT. Hereinafter, the photodetector 1D provided with the two groups will be described in detail by taking as an example.

FIG. 17 shows a configuration example of the pixel array 21D and the drive unit 22D in the photodetector 1D according to this modification. The pixel array 21D has a plurality of pixels P. The plurality of pixels P are divided into two groups G1 and G2 in this example. In FIG. 17, the pixel P (pixel PG1) belonging to the group G1 is shown by a thick line, and the pixel P (pixel PG2) belonging to the group G2 is shown by a thin line.

The pixel array 21D includes a plurality of control lines TGLA1, a plurality of control lines TGLA2, a plurality of control lines TGLB1, a plurality of control lines TGLB2, a plurality of control lines RSLA1, a plurality of control lines RSLA2, and a plurality of controls. It has a line RSLB1 and a plurality of control lines RSLB2. The control line TGLA1 is provided so as to extend in the horizontal direction (horizontal direction in FIG. 17), and a control signal STGLA1 is applied to the control line TGLA1 by the drive unit 22D. The control line TGLA2 is provided so as to extend in the horizontal direction, and a control signal STGLA2 is applied to the control line TGLA2 by the drive unit 22D. The control line TGLB1 is provided so as to extend in the horizontal direction, and a control signal STGLB1 is applied to the control line TGLB1 by the drive unit 22D. The control line TGLB2 is provided so as to extend in the horizontal direction, and a control signal STGLB2 is applied to the control line TGLB2 by the drive unit 22D. The control line RSLA1 is provided so as to extend in the horizontal direction, and a control signal SRSLA1 is applied to the control line RSLA1 by the drive unit 22D. The control line RSLA2 is provided so as to extend in the horizontal direction, and a control signal SRSLA2 is applied to the control line RSLA2 by the drive unit 22D. The control line RSLB1 is provided so as to extend in the horizontal direction, and a control signal SRSLB1 is applied to the control line RSLB1 by the drive unit 22D. The control line RSLB2 is provided so as to extend in the horizontal direction, and a control signal SRSLB2 is applied to the control line RSLB2 by the drive unit 22D.

The pixel PG1 is connected to the control lines TGLA1, TGLB1, RSLA1, RSLB1. The pixel PG2 is connected to the control lines TGLA2, TGLB2, RSLA2, RSLB2.

The exposure control unit 27D in the light detection device 1D sets the exposure time ratio RT (exposure time ratio RT1) based on the exposure amount of the tap B of a certain reference pixel PG1R among the plurality of pixels PG1 in the imaging operation mode MI. At the same time, the exposure time ratio RT (exposure time ratio RT2) is set based on the exposure amount of the tap B of a certain reference pixel PG2R among the plurality of pixels PG2. Then, the plurality of pixels PG1 can operate at the exposure time ratio RT1, and the plurality of pixels PG2 can operate at the exposure time ratio RT2.

Here, the reference pixel PG1R corresponds to a specific example of the "first pixel" in the present disclosure. The reference pixel PG2R corresponds to a specific example of the "second pixel" in the present disclosure. The exposure time ratio RT1 corresponds to a specific example of the "first exposure time ratio" in the present disclosure. The exposure time ratio RT2 corresponds to a specific example of the "second exposure time ratio" in the present disclosure.

[Modification 5]
In the above embodiment, in the imaging operation, the exposure time ratio RT is set based on the exposure amount of the tap B of a certain reference pixel among the plurality of pixels P, and the plurality of pixels P operate at the exposure time ratio RT. I tried to do it, but it is not limited to this. Instead of this, for example, the exposure time ratio RT may be set based on the exposure amount of the tap B of each pixel P so that the pixel P operates at the exposure time ratio RT. The photodetector 1E according to this modification will be described in detail below.

FIG. 18 shows a configuration example of the photodetector 20E in the photodetector 1E. The photodetector 20E is formed on two semiconductor chips 30 and 40. A pixel array 21E is formed on the semiconductor chip 30. The pixel array 21E has a plurality of pixels P arranged in a matrix. A plurality of circuits C arranged in a matrix are formed on the semiconductor chip 40. The plurality of circuits C are provided corresponding to the plurality of pixels P, respectively. The semiconductor chips 30 and 40 are superposed on each other and electrically connected to each other. Specifically, the pixel P and the circuit C are electrically connected to each other by using, for example, a through electrode.

FIG. 19 shows a configuration example of the pixel P and the circuit C. The circuit C includes a drive unit 42, AD conversion units 45A and 45B, and a processing unit 46. The drive unit 42, the AD conversion units 45A and 45B, and the processing unit 46 correspond to the drive unit 22, the AD conversion units 25A and 25B, and the processing unit 26 according to the above embodiment, respectively.

The drive unit 42 is configured to drive the pixel P based on an instruction from the processing unit 46. Specifically, the drive unit 42 supplies the control signal STGLA to the transistor TGA of the pixel P, supplies the control signal STGLB to the transistor TGB of the pixel P, and supplies the control signal to the transistor RSTA of the pixel P. The SRSLA is supplied, the control signal SRSLB is supplied to the transistor RSTB of the pixel P, and the control signal SSELL is supplied to the transistors SELA and SELB of the pixel P. The drive unit 42 has a signal generation unit 43. The signal generation unit 43 is configured to generate control signals STGLA and STGLB.

The AD conversion unit 45A is configured to generate a digital code CODEA by performing AD conversion based on the voltage of the pixel signal SIGA supplied from the pixel P. The AD conversion unit 45B is configured to generate a digital code CODEB by performing AD conversion based on the voltage of the pixel signal SIGB supplied from the pixel P.

The processing unit 46 is configured to control the operations of the pixels P and the circuit C by supplying control signals to the drive unit 42 and the AD conversion units 45A and 45B based on the instruction from the control unit 13. The processing unit 46 includes an exposure control unit 47 and a pixel value calculation unit 48.

The exposure control unit 47 is configured to set the time length of the exposure period T2A and the time length of the exposure period T2B in the imaging operation mode MI. Specifically, the exposure control unit 47 sets, for example, the ratio of the time length of the exposure period T2A to the time length of the exposure period T2B (exposure time ratio RT). Then, the processing unit 46 supplies the drive unit 42 with information about the time lengths of the exposure periods T2A and T2B set by the exposure control unit 47.

The pixel value calculation unit 48 is configured to calculate the pixel value of the pixel P based on the digital codes CODEA and CODEB. In the distance measuring operation mode MD, the pixel value indicates a value for the distance D to the object 100. In the imaging operation mode MI, the pixel value indicates the amount of received light.

With this configuration, the photodetector 20E can set the exposure time ratio RT in units of pixels P.

Here, the pixel P corresponds to a specific example of the "first pixel" in the present disclosure. The exposure control unit 47 corresponds to a specific example of the "first exposure control unit" in the present disclosure. The drive unit 42 corresponds to a specific example of the "first drive unit" in the present disclosure. The pixel value calculation unit 48 corresponds to a specific example of the “first pixel value calculation unit” in the present disclosure.

[Other variants]
Moreover, you may combine two or more of these modified examples.

<2. Application example to moving body>
The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device mounted on a moving body of any kind such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, and a robot. You may.

FIG. 15 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving body control system to which the technique according to the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via the communication network 12001. In the example shown in FIG. 15, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside information detection unit 12030, an in-vehicle information detection unit 12040, and an integrated control unit 12050. Further, as a functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio image output unit 12052, and an in-vehicle network I / F (interface) 12053 are shown.

The drive system control unit 12010 controls the operation of the device related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 provides a driving force generator for generating the driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism for adjusting and a braking device for generating braking force of the vehicle.

The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, blinkers or fog lamps. In this case, the body system control unit 12020 may be input with radio waves transmitted from a portable device that substitutes for the key or signals of various switches. The body system control unit 12020 receives inputs of these radio waves or signals and controls a vehicle door lock device, a power window device, a lamp, and the like.

The vehicle exterior information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle outside information detection unit 12030 causes the image pickup unit 12031 to capture an image of the outside of the vehicle and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as a person, a vehicle, an obstacle, a sign, or characters on the road surface based on the received image.

The image pickup unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of the light received. The image pickup unit 12031 can output an electric signal as an image or can output it as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

The vehicle interior information detection unit 12040 detects information in the vehicle. For example, a driver state detection unit 12041 that detects the driver's state is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing.

The microcomputer 12051 calculates the control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit. A control command can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions including vehicle collision avoidance or impact mitigation, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane deviation warning, and the like. It is possible to perform cooperative control for the purpose of.

Further, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform coordinated control for the purpose of automatic driving that runs autonomously without depending on the operation.

Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the vehicle exterior information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the external information detection unit 12030, and performs coordinated control for the purpose of anti-glare such as switching the high beam to the low beam. It can be carried out.

The audio-image output unit 12052 transmits an output signal of at least one of audio and image to an output device capable of visually or audibly notifying the passenger of the vehicle or the outside of the vehicle. In the example of FIG. 15, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices. The display unit 12062 may include, for example, at least one of an onboard display and a heads-up display.

FIG. 16 is a diagram showing an example of the installation position of the imaging unit 12031.

In FIG. 16, the vehicle 12100 has image pickup units 12101, 12102, 12103, 12104, 12105 as the image pickup unit 12031.

The imaging units 12101, 12102, 12103, 12104, 12105 are provided at positions such as, for example, the front nose, side mirrors, rear bumpers, back doors, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The imaging units 12102 and 12103 provided in the side mirrors mainly acquire images of the side of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or the back door mainly acquires an image of the rear of the vehicle 12100. The images in front acquired by the imaging units 12101 and 12105 are mainly used for detecting the preceding vehicle, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

Note that FIG. 16 shows an example of the photographing range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and the imaging range 12114 indicates the imaging range of the imaging units 12102 and 12103. The imaging range of the imaging unit 12104 provided on the rear bumper or the back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 as viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image pickup units 12101 to 12104 may be a stereo camera composed of a plurality of image pickup elements, or may be an image pickup element having pixels for phase difference detection.

For example, the microcomputer 12051 has a distance to each three-dimensional object within the imaging range 12111 to 12114 based on the distance information obtained from the imaging units 12101 to 12104, and a temporal change of this distance (relative velocity with respect to the vehicle 12100). By obtaining, it is possible to extract as the preceding vehicle a three-dimensional object that is the closest three-dimensional object on the traveling path of the vehicle 12100 and that travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or more). it can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in front of the preceding vehicle in advance, and can perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform coordinated control for the purpose of automatic driving or the like in which the vehicle travels autonomously without depending on the operation of the driver.

For example, the microcomputer 12051 converts three-dimensional object data related to a three-dimensional object into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that can be seen by the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured image of the imaging units 12101 to 12104. Such pedestrian recognition includes, for example, a procedure for extracting feature points in an image captured by an imaging unit 12101 to 12104 as an infrared camera, and pattern matching processing for a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. It is done by the procedure to determine. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a square contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to superimpose and display. Further, the audio image output unit 12052 may control the display unit 12062 so as to display an icon or the like indicating a pedestrian at a desired position.

The example of the vehicle control system to which the technique according to the present disclosure can be applied has been described above. The technique according to the present disclosure can be applied to the imaging unit 12031 among the configurations described above. As a result, in the vehicle control system 12000, the measurement accuracy in distance measurement can be improved, so that the vehicle collision avoidance or collision mitigation function, the follow-up running function based on the inter-vehicle distance, the vehicle speed maintenance running function, the vehicle collision warning function, It is possible to improve the accuracy of the vehicle lane deviation warning function and the like.

Although the present technology has been described above with reference to embodiments and modifications, and specific application examples thereof, the present technology is not limited to these embodiments and the like, and various modifications are possible.

For example, in each of the above embodiments, the present technology is applied to a distance measuring device that measures a distance to an object 100, but is not limited thereto. Instead of this, for example, the present technology may be applied to a time measuring device that measures the flight time of light from the light emitting unit emitting the light pulse L0 to the light receiving unit detecting the reflected light pulse L1. ..

It should be noted that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can have the following configuration. According to this technology having the following configuration, it is possible to perform an imaging operation while ensuring a dynamic range.

(1) A first photoelectric conversion element capable of performing a photoelectric conversion operation for generating a first received light charge based on light, a first charge storage unit capable of accumulating the first received light charge, and a first charge accumulating unit. The charge storage unit 2 and the first switch capable of connecting the first photoelectric conversion element to the first charge storage unit when turned on, and the first photoelectric conversion when turned on. A first pixel having a second switch capable of connecting the element to the second charge storage unit, and
The first photoelectric conversion element is accumulated in the first charge storage unit based on the first charge amount of the first received charge accumulated in the first charge storage unit. The first time length of the first exposure period for generating the received light charge, and the second exposure for which the first photoelectric conversion element generates the first received light charge accumulated in the second charge storage unit. A first exposure control unit capable of performing a first setting process for setting a second time length of a period, and a first exposure control unit.
A photodetector including a first switch and a first drive unit capable of controlling the operation of the second switch based on the result of the first setting process.
(2) The second charge amount of the first received charge accumulated in the second charge storage unit is set to the first exposure time ratio of the first time length and the second time length. The photodetector according to (1) above, further comprising a first pixel value calculation unit capable of calculating a first pixel value based on the first pixel value.
(3) In the above (2), the first driving unit can control the operation of the first switch so as to keep the first switch on during the first exposure period. The photodetector described.
(4) Further equipped with a first AD conversion unit,
The first pixel can output a first voltage corresponding to the first charge amount in the first charge storage unit in the first period after the first exposure period. Has an output section
The first AD conversion unit can convert the first voltage into a first digital code in the first period.
The photodetector according to (3), wherein the first exposure control unit performs the first setting process based on the first digital code.
(5) The photodetector according to (3) or (4) above, wherein the first exposure period includes two sub-exposure periods sandwiching the second exposure period.
(6) The second exposure period includes a first sub-exposure period and a second sub-exposure period after the first sub-exposure period.
The first drive unit keeps the second switch off during the first sub-exposure period and keeps the second switch on during the second sub-exposure period. The photodetector according to any one of (2) to (5), wherein the operation of the second switch can be controlled.
(7) Further equipped with a second AD conversion unit,
The first pixel can output a second voltage according to the amount of electric charge in the second charge storage unit in the second period within the period of the first sub-exposure period, and the second pixel can output the second voltage. In the third period after the sub-exposure period of the above, the second output unit capable of outputting the third voltage corresponding to the second charge amount is provided.
The second AD conversion unit can convert the difference voltage between the second voltage in the second period and the third voltage in the third period into a second digital code.
The photodetector according to (6), wherein the first pixel value calculation unit can calculate the pixel value based on the second digital code and the first exposure time ratio.
(8) The first exposure control unit can set the first exposure time ratio based on the first charge amount according to any one of (2) to (7). Photodetector.
(9) The first exposure control unit makes the first time length and the second time length equal to each other when the first charge amount is smaller than the first threshold value. The photodetector according to (8) above, wherein the first exposure time ratio can be set.
(10) When the first charge amount is larger than the first threshold value and smaller than the second threshold value, the first exposure control unit determines the first charge amount according to the first charge amount. The light detection device according to (9), wherein the first exposure time ratio can be set so that the first time length is longer than the second time length.
(11) The first exposure control unit shortens the total time length of the first time length and the second time length when the first charge amount is larger than the third threshold value. The photodetector according to any one of (2) to (10), wherein the first time length and the second time length can be set accordingly.
(12) A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received charge based on light, a third charge accumulating portion capable of accumulating the second received charge, and a second. The charge storage unit 4 and the third switch capable of connecting the second photoelectric conversion element to the third charge storage unit when turned on, and the second photoelectric conversion when turned on. Further comprising a second pixel having a fourth switch capable of connecting the element to the fourth charge storage section.
In the first exposure control unit, the second photoelectric conversion element is further based on the third charge amount of the second received charge accumulated in the third charge storage unit. The third time length of the third exposure period for generating the second received charge accumulated in the charge storage unit, and the second photoelectric conversion element in which the second photoelectric conversion element is accumulated in the fourth charge storage unit. It is possible to perform a second setting process for setting the fourth time length of the fourth exposure period for generating the received charge of 2.
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the second setting process.
The first pixel value calculation unit further includes a fourth charge amount of the second received charge accumulated in the fourth charge storage unit, the third time length, and the fourth time length. The photodetector according to any one of (2) to (11) above, wherein the second pixel value can be calculated based on the second exposure time ratio of the above.
(13) A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received charge based on light, a third charge accumulating portion capable of accumulating the second received charge, and a second. The charge storage unit 4 and the third switch capable of connecting the second photoelectric conversion element to the third charge storage unit when turned on, and the second photoelectric conversion when turned on. A second pixel having a fourth switch capable of connecting the element to the fourth charge storage unit, and
The second photoelectric conversion element is accumulated in the third charge storage unit based on the third charge amount of the second received charge accumulated in the third charge storage unit. The third time length of the third exposure period for generating the received light charge, and the fourth exposure for which the second photoelectric conversion element generates the second received light charge accumulated in the fourth charge storage unit. A second exposure control unit capable of performing a second setting process for setting a fourth time length of the period, and a second exposure control unit.
A second drive unit capable of controlling the operation of the third switch and the fourth switch based on the result of the second setting process.
A second based on the fourth charge amount of the second received charge accumulated in the fourth charge storage unit and the second exposure time ratio of the third time length and the fourth time length. The photodetector according to any one of (2) to (11) above, further comprising a second pixel value calculation unit capable of calculating a pixel value of 2.
(14) A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received charge based on light, a third charge accumulating portion capable of accumulating the second received charge, and a second. The charge storage unit 4 and the third switch capable of connecting the second photoelectric conversion element to the third charge storage unit when turned on, and the second photoelectric conversion when turned on. Further comprising a second pixel having a fourth switch capable of connecting the element to the fourth charge storage section.
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the first setting process.
The first pixel value calculation unit further determines the second pixel value based on the fourth charge amount of the second received charge accumulated in the fourth charge storage unit and the exposure time ratio. The photodetector according to any one of (2) to (11) above.
(15) The first pixel is
A fifth switch capable of applying a predetermined voltage to the first charge storage unit by being turned on, and
It further has a sixth switch capable of applying the predetermined voltage to the second charge storage unit by being turned on.
The first drive unit is further described in any one of (2) to (14), wherein the operation of the fifth switch and the sixth switch can be controlled based on the first setting process. Photodetector.
(16) Further, an operation mode setting unit capable of setting the operation mode to one of a plurality of operation modes including the first operation mode is provided.
The first exposure control unit can perform the first setting process in the first operation mode.
The first driving unit can control the operation of the first switch and the second switch based on the result of the first setting process in the first operation mode.
The first pixel value calculation unit can calculate the pixel value based on the second charge amount and the exposure time ratio in the first operation mode. Any of the above (2) to (14). The photodetector described in Crab.
(17) Further equipped with a light emitting unit
The plurality of operation modes include a second operation mode.
In the second operation mode, the light emitting unit can emit an optical pulse by alternately repeating light emission and quenching.
In the second operation mode, the first drive unit can alternately turn on the first switch and the second switch so as to synchronize with the optical pulse.
The light according to (16), wherein the first pixel value calculation unit can calculate the pixel value based on the first charge amount and the second charge amount in the second operation mode. Detection device.

1 ... light detection device, 9 ... semiconductor substrate, 11 ... light emitting unit, 12 ... optical system, 12A, 12B ... lens, 13 ... control unit, 14 ... operation mode setting unit, 20, 20E ... light detection unit, 21,21D , 21E ... pixel array, 22, 22D ... drive unit, 23, 23D ... signal generation unit, 24 ... reading unit, 25, 25A, 25B ... AD conversion unit, 26 ... processing unit, 27 ... exposure control unit, 28 ... pixel Value calculation unit, 30, 40 ... semiconductor chip, 42 ... drive unit, 43 ... signal generation unit, 45A, 45B ... AD conversion unit, 46 ... processing unit, 47 ... exposure control unit, 48 ... pixel value calculation unit, 90 ... Substrate, 91 ... holder, 91A, 91B ... opening, 92 ... lens holder, 93 ... cover glass, 94 ... light diffuser, 95 ... infrared filter, AMPA, AMPB, FDGA, FDGB, RSTA, RSTB, SELA, SELB , TGA, TGB ... Transistor, FDGL, FDGLA, FDGLB, RSLA, RSLB, SELL, TGLA, TGLA1, TGLA2, TGLB, TGLB1, TGLB2 ... Control line, C ... Circuit, OP1 ... Light emission operation, BUS ... Bus wiring, CAPA, CAPB ... Capacitive element, DATA, DATA0 ... Image signal, FDA, FDB ... Floating diffusion, L0 ... Optical pulse, L1 ... Reflected light pulse, L2 ... Light, M ... Operation mode, MD ... Distance measurement operation mode MD, MI ... Imaging Operation mode, OP2 ... exposure operation, OP3 ... read operation, P, PG1, PG2 ... pixel, PR ... reference pixel, QA, QB ... charge, RT ... exposure time ratio, SGL, SGLA, SGLB ... signal line, SFDGL, SFDGLA , SFDGLB, SRSLA, SRSLB, SSELL, STGLA, STGLA1, STGLA2, STGLB, STGLB1, STGLB2 ... Control signal, T1, T2A, T2B ... Exposure period, VFDA, VFDB ... Voltage, φ ... Phase.

Claims (17)

A first photoelectric conversion element capable of performing a photoelectric conversion operation for generating a first received light charge based on light, a first charge storage unit capable of accumulating the first received light charge, and a second charge. The storage unit, the first switch capable of connecting the first photoelectric conversion element to the first charge storage unit when turned on, and the first photoelectric conversion element when turned on. A first pixel having a second switch that can be connected to a second charge storage unit, and
The first photoelectric conversion element is accumulated in the first charge storage unit based on the first charge amount of the first received charge accumulated in the first charge storage unit. The first time length of the first exposure period for generating the received light charge, and the second exposure for which the first photoelectric conversion element generates the first received light charge accumulated in the second charge storage unit. A first exposure control unit capable of performing a first setting process for setting a second time length of a period, and a first exposure control unit.
A photodetector including a first switch and a first drive unit capable of controlling the operation of the second switch based on the result of the first setting process. Based on the second charge amount of the first received charge accumulated in the second charge storage unit, and the first exposure time ratio of the first time length and the second time length. The photodetector according to claim 1, further comprising a first pixel value calculation unit capable of calculating the pixel value of 1. The photodetection according to claim 2, wherein the first driving unit can control the operation of the first switch so as to keep the first switch on during the first exposure period. apparatus. Further equipped with a first AD conversion unit,
The first pixel can output a first voltage corresponding to the first charge amount in the first charge storage unit in the first period after the first exposure period. Has an output section
The first AD conversion unit can convert the first voltage into a first digital code in the first period.
The photodetector according to claim 3, wherein the first exposure control unit performs the first setting process based on the first digital code. The photodetector according to claim 3, wherein the first exposure period includes two sub-exposure periods sandwiching the second exposure period. The second exposure period includes a first sub-exposure period and a second sub-exposure period after the first sub-exposure period.
The first drive unit keeps the second switch in the off state during the first sub-exposure period and keeps the second switch in the on state during the second sub-exposure period. The photodetector according to claim 2, wherein the operation of the second switch can be controlled. Further equipped with a second AD conversion unit,
The first pixel can output a second voltage according to the amount of electric charge in the second charge storage unit in the second period within the period of the first sub-exposure period, and the second pixel can output the second voltage. In the third period after the sub-exposure period of the above, the second output unit capable of outputting the third voltage corresponding to the second charge amount is provided.
The second AD conversion unit can convert the difference voltage between the second voltage in the second period and the third voltage in the third period into a second digital code.
The photodetector according to claim 6, wherein the first pixel value calculation unit can calculate the pixel value based on the second digital code and the first exposure time ratio. The photodetector according to claim 2, wherein the first exposure control unit can set the first exposure time ratio based on the first charge amount. The first exposure control unit makes the first time length and the second time length equal to each other when the first charge amount is smaller than the first threshold value. The light detection device according to claim 8, wherein the exposure time ratio can be set. When the first charge amount is larger than the first threshold value and smaller than the second threshold value, the first exposure control unit determines the first charge amount according to the first charge amount. The light detection device according to claim 9, wherein the first exposure time ratio can be set so that the time length is longer than the second time length. The first exposure control unit shortens the total time length of the first time length and the second time length when the first charge amount is larger than the third threshold value. The photodetector according to claim 2, wherein the first time length and the second time length can be set. A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received charge based on light, a third charge accumulating portion capable of accumulating the second received charge, and a fourth charge. The storage unit, a third switch capable of connecting the second photoelectric conversion element to the third charge storage unit when turned on, and the second photoelectric conversion element when turned on. Further comprising a second pixel with a fourth switch connectable to a fourth charge store
In the first exposure control unit, the second photoelectric conversion element is further based on the third charge amount of the second received charge accumulated in the third charge storage unit. The third time length of the third exposure period for generating the second received charge accumulated in the charge storage unit, and the second photoelectric conversion element in which the second photoelectric conversion element is accumulated in the fourth charge storage unit. It is possible to perform a second setting process for setting the fourth time length of the fourth exposure period for generating the received charge of 2.
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the second setting process.
The first pixel value calculation unit further includes a fourth charge amount of the second received charge accumulated in the fourth charge storage unit, the third time length, and the fourth time length. The photodetector according to claim 2, wherein the second pixel value can be calculated based on the second exposure time ratio of the above. A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received charge based on light, a third charge accumulating portion capable of accumulating the second received charge, and a fourth charge. The storage unit, a third switch capable of connecting the second photoelectric conversion element to the third charge storage unit when turned on, and the second photoelectric conversion element when turned on. A second pixel having a fourth switch that can be connected to a fourth charge storage unit, and
The second photoelectric conversion element is accumulated in the third charge storage unit based on the third charge amount of the second received charge accumulated in the third charge storage unit. The third time length of the third exposure period for generating the received light charge, and the fourth exposure for which the second photoelectric conversion element generates the second received light charge accumulated in the fourth charge storage unit. A second exposure control unit capable of performing a second setting process for setting a fourth time length of the period, and a second exposure control unit.
A second drive unit capable of controlling the operation of the third switch and the fourth switch based on the result of the second setting process.
A second based on the fourth charge amount of the second received charge accumulated in the fourth charge storage unit and the second exposure time ratio of the third time length and the fourth time length. The photodetector according to claim 2, further comprising a second pixel value calculation unit capable of calculating the pixel value of 2. A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received charge based on light, a third charge accumulating portion capable of accumulating the second received charge, and a fourth charge. The storage unit, a third switch capable of connecting the second photoelectric conversion element to the third charge storage unit when turned on, and the second photoelectric conversion element when turned on. Further comprising a second pixel with a fourth switch connectable to a fourth charge store
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the first setting process.
The first pixel value calculation unit further determines the second pixel value based on the fourth charge amount of the second received charge accumulated in the fourth charge storage unit and the exposure time ratio. The light detection device according to claim 2, wherein the above can be calculated. The first pixel is
A fifth switch capable of applying a predetermined voltage to the first charge storage unit by being turned on, and
It further has a sixth switch capable of applying the predetermined voltage to the second charge storage unit by being turned on.
The photodetector according to claim 2, wherein the first drive unit can further control the operation of the fifth switch and the sixth switch based on the first setting process. Further provided with an operation mode setting unit capable of setting the operation mode to one of a plurality of operation modes including the first operation mode.
The first exposure control unit can perform the first setting process in the first operation mode.
The first drive unit can control the operation of the first switch and the second switch based on the result of the first setting process in the first operation mode.
The photodetector according to claim 2, wherein the first pixel value calculation unit can calculate the pixel value based on the second charge amount and the exposure time ratio in the first operation mode. With a light emitting part
The plurality of operation modes include a second operation mode.
In the second operation mode, the light emitting unit can emit an optical pulse by alternately repeating light emission and quenching.
In the second operation mode, the first drive unit can alternately turn on the first switch and the second switch so as to synchronize with the optical pulse.
The light detection according to claim 16, wherein the first pixel value calculation unit can calculate the pixel value based on the first charge amount and the second charge amount in the second operation mode. apparatus.

Images