WO2020179288A1

WO2020179288A1 - Light detection device

Info

Publication number: WO2020179288A1
Application number: PCT/JP2020/002912
Authority: WO
Inventors: 前田　俊治
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2019-03-01
Filing date: 2020-01-28
Publication date: 2020-09-10
Also published as: JP2020141362A

Abstract

This light detection device comprises: a first pixel having a first photoelectric conversion element that generates a first light receiving charge, a first and a second charge accumulation unit that accumulate the first light receiving charges, a first switch that connects the first photoelectric conversion element to the firs charge accumulation unit, and a second switch that connects the first photoelectric conversion element to the second charge accumulation unit; a first exposure control unit that performs a first setting process for setting, on the basis of a first charge amount of first light receiving charges accumulated in the first charge accumulation unit, a first time length of a first exposure period in which the first photoelectric conversion element generates first light receiving charges to be accumulated in the first charge accumulation unit, and a second time length of a second exposure period in which the first photoelectric conversion element generates first light receiving charges to be accumulated in the second charge accumulation unit; and a first drive unit that controls the operation of the first and the second switches on the basis of the results of the first setting process.

Description

Photodetector

The present disclosure relates to a light detection device capable of detecting light.

When measuring the distance to the object to be measured, the TOF (Time Of Flight) method is often used. In this TOF method, light is emitted and the reflected light reflected by the measurement object is detected. Then, in the TOF method, the distance to the measurement object is measured by measuring the time difference between the timing at which light is emitted and the timing at which reflected light is detected (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2013-0764645

By the way, in the photodetector, there may be a case where it is desired to perform an imaging operation in addition to the distance measuring operation for measuring the distance to the object to be measured. In the image pickup operation, it is desirable to secure a dynamic range.

It is desirable to provide a photodetector that can perform imaging operations while ensuring a dynamic range.

The photodetector according to the embodiment of the present disclosure includes a first pixel, a first exposure controller, and a first driver. The first pixel includes a first photoelectric conversion element capable of performing a photoelectric conversion operation to generate a first received light charge based on light, and a first charge storage unit capable of storing the first received light charge. And a second charge storage portion, a first switch capable of connecting the first photoelectric conversion element to the first charge storage portion by being turned on, and a first photoelectric conversion element by being turned on And a second switch connectable to the second charge storage section. The first exposure control unit stores the first photoelectric conversion element in the first charge storage unit based on the first charge amount of the first received charge stored in the first charge storage unit. A first time length of a first exposure period for generating a first light-receiving charge, and a second exposure for generating a first light-receiving charge accumulated in a second charge storage section by the first photoelectric conversion element. It is possible to perform the first setting process for setting the second time length of the period. The first drive unit is configured to be able to control the operation of the first switch and the second switch based on the result of the first setting process.

In the photodetector according to the embodiment of the present disclosure, the first photoelectric conversion element generates the first received light charge, and the first switch is turned on, so that the first received charge becomes the first light receiving charge. The first light receiving charge is stored in the first charge storage unit and the second switch is turned on, so that the first received light charge is stored in the second charge storage unit. Based on the first charge amount of the first received charge accumulated in the first charge storage unit, the first time length of the first exposure period and the second time length of the second exposure period are set. The first setting process for setting is performed. The first exposure period is a period in which the first photoelectric conversion element generates the first received light charge accumulated in the first charge storage section, and the second exposure period is the period in which the first photoelectric conversion element is generated. This is a period in which the first received light charges accumulated in the second charge accumulation unit are generated. Then, the operation of the first switch and the second switch is controlled based on the result of the first setting process.

FIG. 3 is a block diagram illustrating a configuration example of a photodetector device according to an embodiment of the present disclosure. FIG. 3 is a block diagram showing a configuration example of a photodetector section shown in FIG. 1. FIG. 3 is a circuit diagram illustrating a configuration example of the pixel illustrated in FIG. 2. 4 is a cross-sectional view illustrating an example of a cross-sectional structure of a main part of the pixel illustrated in FIG. 3. FIG. 3 is a cross-sectional view illustrating a configuration example of the photodetector device illustrated in FIG. 1. FIG. 6 is a timing chart showing an example of a distance measuring operation in the photodetector shown in FIG. 1. FIG. 6 is a timing diagram illustrating an example of an image pickup operation in the photodetector shown in FIG. 1. FIG. 6 is a timing waveform chart showing an example of a distance measuring operation in the photodetector shown in FIG. 1. It is another timing waveform diagram which shows an example of the distance measuring operation in the photodetector shown in FIG. FIG. 6 is a timing waveform chart showing an example of an image pickup operation in the photodetector shown in FIG. 1. It is explanatory drawing which shows one operation state of the pixel shown in FIG. FIG. 9 is an explanatory diagram illustrating another operation state of the pixel illustrated in FIG. 2. FIG. 9 is an explanatory diagram illustrating another operation state of the pixel illustrated in FIG. 2. FIG. 9 is an explanatory diagram illustrating another operation state of the pixel illustrated in FIG. 2. FIG. 9 is an explanatory diagram illustrating another operation state of the pixel illustrated in FIG. 2. 6 is a flowchart showing an example of an image pickup operation in the photodetector shown in FIG. 1. FIG. 13 is a timing waveform chart illustrating an example of an image pickup operation in the photodetector according to the modification. It is explanatory drawing which shows one operation state of the pixel which concerns on a modification. It is explanatory drawing showing the other operation state of the pixel which concerns on a modification. It is explanatory drawing showing the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing which shows the other operation state of the pixel which concerns on a modification. It is explanatory drawing showing the other operation state of the pixel which concerns on a modification. 9 is a flowchart showing an example of an image pickup operation in a photodetector according to another modification. It is a circuit diagram which shows one composition example of the pixel which concerns on another modification. It is a circuit diagram which shows one composition example of the pixel which concerns on another modification. FIG. 11 is a block diagram illustrating a configuration example of a pixel array and a driving unit according to another modification. It is explanatory drawing showing the structural example of the photon detection part which concerns on another modification. FIG. 19 is a circuit diagram illustrating a configuration example of the pixel and the circuit illustrated in FIG. 18. It is a block diagram showing an example of a schematic structure of a vehicle control system. It is explanatory drawing which shows an example of the installation position of a vehicle exterior information detection part and an imaging part.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be given in the following order.
1. Embodiment 2. Application example to mobile

<1. Embodiment>
[Configuration example]
FIG. 1 shows a configuration example of a photodetector (photodetector 1) according to an embodiment. The photodetector 1 has two operation modes M (distance measuring operation mode MD and imaging operation mode MI). The distance measuring operation mode MD is an operation mode for performing a distance measuring operation for measuring the distance to the object 100 by an indirect method, and the imaging operation mode MI is a mode for performing an imaging operation for imaging the object 100. The photodetector 1 includes a light emitting unit 11, an optical system 12, a photodetector 20, and a control unit 13.

The light emitting unit 11 is configured to emit an optical pulse L0 toward the object 100 in the distance measuring operation mode MD. The light emitting unit 11 emits the light pulse L0 by performing a light emitting operation in which light emission and non-light emission are alternately repeated based on an instruction from the control unit 13. The light emitting unit 11 has, for example, a light source that emits infrared light. This light source is configured using, for example, a laser light source or an LED (Light Emitting Diode).

The optical system 12 is configured to include a lens that forms an image on the light receiving surface S of the light detection unit 20. In the distance measurement operation mode MD, a light pulse (reflected light pulse L1) emitted from the light emitting unit 11 and reflected by the object 100 enters the optical system 12. Further, in the image pickup operation mode MI, the light from the subject enters the optical system 12.

The light detection unit 20 is configured to generate an image PIC by detecting light based on an instruction from the control unit 13. Specifically, in the distance measurement operation mode MD, the light detection unit 20 receives the reflected light pulse L1 to generate the image PIC (distance image PIC1). Each of the plurality of pixel values included in the distance image PIC1 indicates a value for the distance D to the object 100. Further, the photodetector 20 generates an image PIC (image PIC2) by receiving light L2 in the image pickup operation mode MI. Each of the plurality of pixel values included in the captured image PIC2 indicates the amount of received light. Then, the photodetector 20 outputs the generated image PIC as the image signal DATA.

The control unit 13 supplies a control signal to the light emitting unit 11 and the photodetecting unit 20 to control the operations thereof, thereby controlling the operation of the photodetecting device 1. The control unit 13 has an operation mode setting unit 14. The operation mode setting unit 14 sets the operation mode M to one of the distance measurement operation mode MD and the imaging operation mode MI based on the instruction included in the control signal CTL supplied from the outside. The control unit 13 is configured to control the operations of the light emitting unit 11 and the light detection unit 20 according to the operation mode M set by the operation mode setting unit 14.

FIG. 2 shows an example of the configuration of the light detection unit 20. The light detection unit 20 includes a pixel array 21, a driving unit 22, a reading unit 24, and a processing unit 26.

The pixel array 21 has a plurality of pixels P arranged in a matrix. Each pixel P outputs pixel signals SIGA and SIGB.

FIG. 3 shows an example of a configuration of pixel P. FIG. 4 shows an example of a configuration of a main part of the pixel P. The pixel array 21 includes a plurality of control line TGLA, a plurality of control line TGLB, a plurality of control line RSLA, a plurality of control line RSLB, a plurality of control line SELL, a plurality of signal lines SGLA, and a plurality of signals. It has a line SGLB. The control line TGLA is provided so as to extend in the horizontal direction (horizontal direction in FIGS. 2 and 3), and the drive unit 22 applies the control signal STGLA to the control line TGLA. The control line TGLB is provided so as to extend in the horizontal direction, and the drive unit 22 applies the control signal STGLB to the control line TGLB. The control line RSLA is provided so as to extend in the horizontal direction, and the drive unit 22 applies a control signal SRSLA to the control line RSLA. The control line RSLB is provided so as to extend in the horizontal direction, and the control signal SRSLB is applied to the control line RSLB by the drive unit 22. The control line SELL is provided so as to extend in the horizontal direction, and a control signal SSELL is applied to the control line SELL by the drive unit 22. The signal line SGLA is provided so as to extend in the vertical direction (longitudinal direction in FIGS. 2 and 3) and transmits the pixel signal SIGA to the reading unit 24. The signal line SGLB is provided so as to extend in the vertical direction, and transmits the pixel signal SIGB to the reading unit 24.

The pixel P has a photodiode PD, transistors TGA and TGB, floating diffusions FDA and FDB, transistors RSTA and RSTB, transistors AMPA and AMPB, and transistors SELA and SELB. The transistors TGA, TGB, RSTA, RSTB, AMPA, AMPB, SELA, SELB are N-type MOS (Metal Oxide Semiconductor) transistors in this example.

The photodiode PD is a photoelectric conversion element that generates electric charges according to the amount of received light. The anode of the photodiode PD is grounded, and the cathode is connected to the sources of the transistors TGA and TGB. In this example, the amount of charge that can be stored in the photodiode PD is smaller than the amount of charge that can be stored in the floating diffusion FDA and the amount of charge that can be stored in the floating diffusion FDB.

The gate of the transistor TGA is connected to the control line TGLA, the source is connected to the cathode of the photodiode PD and the source of the transistor TGB, and the drain is connected to the floating diffusion FDA, the source of the transistor RSTA, and the gate of the transistor AMPA. The floating diffusion FDA is configured to accumulate the electric charge supplied from the photodiode PD via the transistor TGA. For example, as shown in FIG. 4, a diffusion layer formed on the surface of the semiconductor substrate 9 is used. It is composed. In FIG. 3, the floating diffusion FDA is shown using a capacitive element symbol. In this example, the amount of charge that can be stored in the floating diffusion FDA is larger than the amount of charge that can be stored in the photodiode PD. The gate of the transistor RSTA is connected to the control line RSLA, the drain is supplied with a power supply voltage VDD, and the source is connected to the floating diffusion FDA, the drain of the transistor TGA, and the gate of the transistor AMPA. The gate of the transistor AMPA is connected to the floating diffusion FDA, the drain of the transistor TGA, and the source of the transistor RSTA, the drain is supplied with the power supply voltage VDD, and the source is connected to the drain of the transistor SELA. The gate of the transistor SELA is connected to the control line SELL, the drain is connected to the source of the transistor AMPA, and the source is connected to the signal line SGLA. Hereinafter, the circuit including the photodiode PD, the floating diffusion FDA, and the transistors RSTA, AMPA, and SELA is also referred to as tap A.

The gate of the transistor TGB is connected to the control line TGLB, the source is connected to the cathode of the photodiode PD and the source of the transistor TGA, and the drain is connected to the floating diffusion FDB, the source of the transistor RSTB, and the gate of the transistor AMPB. The floating diffusion FDB is configured to accumulate the electric charge supplied from the photodiode PD via the transistor TGB. For example, as shown in FIG. 4, a diffusion layer formed on the surface of the semiconductor substrate 9 is used. It is composed. In FIG. 3, similarly to the floating diffusion FDA, the floating diffusion FDB is shown using the symbol of the capacitive element. In this example, the amount of charge that can be stored in the floating diffusion FDB is larger than the amount of charge that can be stored in the photodiode PD. The gate of the transistor RSTB is connected to the control line RSLB, the drain is supplied with the power supply voltage VDD, and the source is connected to the floating diffusion FDB, the drain of the transistor TGB, and the gate of the transistor AMPB. The gate of the transistor AMPB is connected to the floating diffusion FDB, the drain of the transistor TGB, and the source of the transistor RSTB, the power supply voltage VDD is supplied to the drain, and the source is connected to the drain of the transistor SELB. The gate of the transistor SELB is connected to the control line SELL, the drain is connected to the source of the transistor AMPB, and the source is connected to the signal line SGLB. Hereinafter, the circuit including the photodiode PD, the floating diffusion FDB, and the transistors RSTB, AMPB, and SELB is also referred to as tap B.

With this configuration, in the pixel P, the floating diffusion FDA is reset by turning on the transistor RSTA, and the floating diffusion FDB is reset by turning on the transistor RSTB. Then, by turning on one of the transistors TGA and TGB, the charges generated by the photodiode PD are selectively accumulated in the floating diffusion FDA and the floating diffusion FDB. When the transistors SELA and SELB are turned on, the pixel P outputs the pixel signal SIGA corresponding to the amount of charges accumulated in the floating diffusion FDA, and the amount of charges accumulated in the floating diffusion FDB is changed. The corresponding pixel signal SIGB is output.

The drive unit 22 (FIG. 2) is configured to drive the plurality of pixels P based on an instruction from the processing unit 26. Specifically, the drive unit 22 applies the control signal STGLA to the control line TGLA, the control signal STGLB to the control line TGLB, and the control signal SRSLA to the control line RSLA to perform control. The control signal SRSLB is applied to the line RSLB, and the control signal SSELL is applied to the control line SELL. The drive unit 22 has a signal generation unit 23. The signal generator 23 is configured to generate the control signals STGLA and STGLB. The drive unit 22 applies the single control signal STGLA generated by the signal generation unit 23 to the plurality of control lines TGLA, and applies the single control signal STGLB generated by the signal generation unit 23 to the plurality of control lines TGLB. It is designed to do.

The reading unit 24 performs AD (Analog to Digital) conversion based on the pixel signal SIG (pixel signal SIGA, SIGB) supplied from the pixel array 21 via the signal line SGL (signal line SGLA, SGLB). It is configured to generate the image signal DATA0.

The reading unit 24 has a plurality of AD conversion units 25 and a bus line BUS. The plurality of AD conversion units 25 are provided corresponding to the plurality of signal lines SGL (signal lines SGLA, SGLB), respectively. The AD conversion unit 25 is configured to generate a digital code CODE by performing AD conversion based on the voltage of the pixel signal SIG supplied via the corresponding signal line SGL. Then, the AD conversion unit 25 supplies the digital code CODE to the bus wiring BUS.

The bus wiring BUS has a plurality of wirings and is configured to transmit the digital code CODE output from the AD conversion unit 25. Using this bus wiring BUS, the reading unit 24 sequentially transfers a plurality of digital code CODEs supplied from the plurality of AD conversion units 25 to the processing unit 26 as an image signal DATA0.

The processing unit 26 is configured to control the operation of the light detection unit 20 by supplying a control signal to the driving unit 22 and the reading unit 24 based on an instruction from the control unit 13. The processing unit 26 has an exposure control unit 27 and a pixel value calculation unit 28.

In the image pickup operation mode MI, the exposure control unit 27 determines the time length of the period (exposure period T2A) in which the photodiode PD accumulates electric charges accumulated in the floating diffusion FDA, and the electric charges accumulated in the photodiode PD in the floating diffusion FDB. Is configured to set the time length of the period (exposure period T2B) for generating the. Specifically, the exposure control unit 27 sets, for example, the ratio of the time length of the exposure period T2A and the time length of the exposure period T2B (exposure time ratio RT). In this example, the exposure time ratio RT is obtained by dividing the time length of the exposure period T2B by the time length of the exposure period T2A. Then, the processing unit 26 supplies the driving unit 22 with information about the time lengths of the exposure periods T2A and T2B set by the exposure control unit 27.

The pixel value calculation unit 28 is configured to calculate the pixel value of each of the plurality of pixels P based on the digital code CODE relating to the plurality of pixels P in the pixel array 21. In the distance measurement operation mode MD, the pixel value indicates the value for the distance D to the object 100. The processing unit 26 generates the distance image PIC1 using the pixel values of the plurality of pixels P in the pixel array 21, and outputs the distance image PIC1 as the image signal DATA. Further, in the imaging operation mode MI, the pixel value indicates the amount of received light. The processing unit 26 is configured to generate a captured image PIC2 by using the pixel values of the plurality of pixels P in the pixel array 21, and output the captured image PIC2 as an image signal DATA.

FIG. 5 shows a configuration example of the photodetector 1. The light emitting unit 11, the light detection unit 20, and the holder 91 are arranged on the substrate of the substrate 90. The holder 91 is provided with

openings

91A and 91B. The light emitting unit 11 is arranged on the substrate of the substrate 90 at a position corresponding to the opening 91A, and the light detecting unit 20 is arranged at a position on the substrate of the substrate 90 corresponding to the opening 91B. The holder 91 holds the cover glass 93. The cover glass 93 is configured to protect the light emitting unit 11 and the light detection unit 20 from dust and the external atmosphere. The cover glass 93 is provided with a light diffusing film 94 and an infrared filter 95. The light diffusion film 94 is provided in a region of the cover glass 93 corresponding to the light emitting unit 11, and is configured to diffuse the light pulse L0. The infrared filter 95 is provided in a region of the cover glass 93 corresponding to the photodetector 20, and is configured to transmit infrared light. A lens holder 92 is provided in the opening 91B of the holder 91. The lens holder 92 holds the

lenses

12A and 12B. The

lenses

12A and 12B form an optical system 12.

Here, the photodiode PD corresponds to a specific but not limitative example of “first photoelectric conversion element” in one embodiment of the present disclosure. The floating diffusion FDB corresponds to a specific but not limitative example of “first charge storage section” in one embodiment of the disclosure. The floating diffusion FDA corresponds to a specific but not limitative example of “second charge storage section” in one embodiment of the present disclosure. The transistor TGB corresponds to a specific example of the "first switch" in the present disclosure. The transistor TGA corresponds to a specific example of the "second switch" in the present disclosure. The transistors AMPB and SELB correspond to a specific but not limitative example of “first output section” in one embodiment of the disclosure. The transistors AMPA and SELA correspond to a specific but not limitative example of “second output section” in one embodiment of the disclosure. The exposure control unit 27 corresponds to a specific but not limitative example of “first exposure control unit” in one embodiment of the present disclosure. The exposure period T2B corresponds to a specific but not limitative example of “first exposure period” in one embodiment of the present disclosure. The exposure period T2A corresponds to a specific but not limitative example of “second exposure period” in one embodiment of the present disclosure. The exposure time ratio RT corresponds to a specific but not limitative example of “first exposure time ratio” in one embodiment of the present disclosure. The drive unit 22 corresponds to a specific example of the “first drive unit” in the present disclosure. The pixel value calculation unit 28 corresponds to a specific but not limitative example of “first pixel value calculation unit” in one embodiment of the present disclosure.

[Operation and action]
Next, the operation and action of the photodetector 1 according to the present embodiment will be described.

(Overview of overall operation)
First, with reference to FIGS. 1 to 3, an outline of the overall operation of the photodetection device 1 will be described. When performing a distance measuring operation for measuring the distance to the object 100, the photodetector 1 operates in the distance measuring operation mode MD. In this distance measurement operation mode MD, the light detection device 1 emits the light pulse L0 by performing the light emission operation OP1 and receives the reflected light pulse L1 reflected by the object 100 by performing the exposure operation OP2. Then, the charge is selectively accumulated in the floating diffusion FDA and FDB in each of the plurality of pixels P. Then, the photodetector 1 performs AD conversion based on the pixel signal SIG supplied from the plurality of pixels P via the signal line SGL by performing the read operation OP3, and generates the image signal DATA0. Then, the photodetector 1 generates a distance image PIC1 based on the image signal DATA0, and outputs the distance image PIC1 as an image signal DATA.

Further, when performing an imaging operation for imaging the object 100, the photodetector 1 operates in the imaging operation mode MI. In this imaging operation mode MI, the photodetector 1 receives the light L2 by performing the exposure operation OP2, and selectively accumulates charges in the floating diffusion FDA and FDB in each of the plurality of pixels P. Then, the photodetection device 1 performs the read operation OP3 to perform AD conversion based on the pixel signal SIG supplied from the plurality of pixels P via the signal line SGL to generate the image signal DATA0. Then, the photodetection device 1 generates a captured image PIC2 based on the image signal DATA0 and outputs this captured image PIC2 as the image signal DATA.

(Detailed operation)
6A and 6B show an example of the operation in the photodetector 1, FIG. 6A shows the distance measuring operation in the distance measuring operation mode MD, and FIG. 6B shows the imaging operation in the imaging operation mode MI. In FIGS. 6A and 6B, the upper end indicates the uppermost portion of the pixel array 21, and the lower end indicates the lowermost portion of the pixel array 21.

In the distance measurement operation mode MD, as shown in FIG. 6A, the photodetector 1 performs the light emission operation OP1 and the exposure operation OP2 during the period from timing t1 to t2. Specifically, the control unit 13 controls the operation of the light emitting unit 11, and the light emitting unit 11 emits the light pulse L0 by performing the light emitting operation OP1 that alternately repeats light emission and non-light emission. Further, the control unit 13 controls the operation of the light detection unit 20, and the drive unit 22 of the light detection unit 20 drives the plurality of pixels P in the pixel array 21. The plurality of pixels P receive the reflected light pulse L1 reflected by the object 100. Then, the photodetection device 1 performs the read operation OP3 during the period from timing t2 to timing t3. Specifically, the drive unit 22 sequentially drives the plurality of pixels P in the pixel array 21 in pixel line units, and the plurality of pixels P outputs the pixel signal SIG and the signal line SGL (signal lines SGLA and SGLB). It is supplied to the reading unit 24 via. Then, the reading unit 24 generates the image signal DATA0 by performing AD conversion based on the pixel signal SIG. The photodetector 1 alternately repeats the light emitting operation OP1 and the exposing operation OP2 and the reading operation OP3. The processing unit 26 generates a distance image PIC1 in which each pixel value indicates a value for the distance D based on the image signal DATA0.

In the imaging operation mode MI, the photodetection device 1 performs the exposure operation OP2 in the period of timing t4 to t5, as shown in FIG. 6B. Specifically, the control unit 13 controls the operation of the light detection unit 20, and the drive unit 22 of the light detection unit 20 drives the plurality of pixels P in the pixel array 21. Then, the photodetection device 1 performs the read operation OP3 during the period from timing t5 to t6. Specifically, the drive unit 22 sequentially drives the plurality of pixels P in the pixel array 21 in pixel line units, and the plurality of pixels P outputs the pixel signal SIG and the signal line SGL (signal lines SGLA and SGLB). It is supplied to the reading unit 24 via. Then, the reading unit 24 generates the image signal DATA0 by performing AD conversion based on the pixel signal SIG. The photodetector 1 alternately repeats such an exposure operation OP2 and a read operation OP3. The processing unit 26 generates a captured image PIC2 in which each pixel value indicates the amount of received light, based on the image signal DATA0.

(About distance measurement operation)
Next, the distance measuring operation for measuring the distance to the object 100 in the distance measuring operation mode MD will be described in detail. In the following, a certain pixel P (pixel P1) among the plurality of pixels P will be focused on, and the operation of the focused pixel P1 will be described in detail by taking as an example.

7A and 7B show an example of the distance measuring operation of the photodetector 1, where FIG. 7A shows the waveform of the light pulse L0 emitted by the light emitting unit 11, and FIG. 7B shows the waveforms of the control signals SRSLA and SRSLB. (C) shows the waveform of the voltage VFDA in the floating diffusion FDA, (D) shows the waveform of the voltage VFDB in the floating diffusion FDB, (E) shows the waveform of the control signal STGLA, and (F) shows the control signal. The waveform of STGLB is shown, (G) shows the operation of the AD conversion unit 25 (AD conversion unit 25A) connected to the signal line SGLA related to the pixel P1, and (H) is connected to the signal line SGLB related to the pixel P1. The operation of the AD converter 25 (AD converter 25B) will be described.

First, the photodetector 1 performs light emission operation OP1 and exposure operation OP2. In this exposure operation OP2, the drive section 22 sets the voltage of the control signal SSELL to a low level. As a result, in the pixel P1, the transistors SELA and SELB are both turned off, and the pixel P1 is electrically disconnected from the signal lines SGLA and SGLB. The floating diffusions FDA and FDB of the pixel P1 selectively accumulate the charges generated by the photodiode PD. Then, the photodetector 1 performs the read operation OP3. In this read operation OP3, the drive section 22 sets the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, both the transistors SELA and SELB are turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs a pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and a pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB. The operation will be described in detail below.

In the period before the timing t11, the drive unit 22 sets the voltages of the control signals SRSLA and SRSLB to the high level (FIG. 7(B)). As a result, in the pixel P1, the transistors RSTA and RSTB are both turned on, and the voltages VFDA and VFDB in the floating diffusions FDA and FDB are set to the power supply voltage VDD (FIGS. 7C and 7D). In this way, the floating diffusions FDA and FDB are reset.

Then, at the timing t11, the drive unit 22 changes the voltage of the control signals SRSLA and SRSLB from the high level to the low level (FIG. 7(B)). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. Further, at this timing t11, the light emitting unit 11 starts the light emitting operation OP1 in which light emission and non-light emission are alternately repeated (FIG. 7 (A)). Then, the drive unit 22 starts generating the control signals STGLA and STGLB at this timing t11 (FIGS. 7E and 7F). The control signals STGLA and STGLB are signals that alternately repeat high level and low level. The phase of the control signal STGLB lags the phase of the control signal STGLA by 180 degrees. As shown in FIGS. 7A and 7E, the frequency of the light pulse L0 is the same as the frequency of the control signal STGLA, and the phase of the light pulse L0 is the same as the phase of the control signal STGLA. That is, the control unit 13 controls the operations of the light emitting unit 11 and the light detection unit 20 so that the light pulse L0 and the control signals STGLA and STGLB are synchronized.

In this way, the exposure period T1 starts at this timing t11. In the exposure period T1, in the pixel P1, the photodiode PD generates an electric charge based on the reflected light pulse L1 corresponding to the light pulse L0. The transistor TGA is turned on/off based on the control signal STGLA, and the transistor TGB is turned on/off based on the control signal STGLB. That is, one of the transistors TGA and TGB is turned on. As a result, the electric charge generated by the photodiode PD is selectively accumulated in the floating diffusion FDA and the floating diffusion FDB.

FIG. 8 shows an operation example of the pixel P1, where (A) shows the waveform of the light pulse L0, (B) shows the waveform of the reflected light pulse L1, and (C) shows the waveform of the control signal STGLA. And (D) shows the waveform of the control signal STGLB. In this example, at the timing t21, the light pulse L0 rises, the control signal STGLA rises, and the control signal STGLB falls. Then, at a timing t23 where the phase is delayed by 180 degrees from the timing t21, the optical pulse L0 falls, the control signal STGLA falls, and the control signal STGLB rises. Similarly, the optical pulse L0 rises, the control signal STGLA rises, and the control signal STGLB falls at a timing t25 when the phase is delayed by 180 degrees from the timing t23. Then, at timing t26 in which the phase is delayed by 180 degrees from timing t25, the optical pulse L0 falls, the control signal STGLA falls, and the control signal STGLB rises.

The phase of the reflected light pulse L1 is shifted from the phase of the light pulse L0 by the phase φ (FIG. 8(B)). This phase φ corresponds to the distance D from the light detection device 1 to the object 100. In this example, the reflected light pulse L1 rises at a timing t22 which is delayed by a time corresponding to the phase φ from the timing t21, and the reflected light pulse L1 falls at a timing t24 which is delayed by a time corresponding to the phase φ from the timing t23. .. The photodiode PD of the pixel P1 generates an electric charge in the period from timing t22 to t24 based on the reflected light pulse L1.

The transistor TGA transfers the charge generated by the photodiode PD to the floating diffusion FDA during the period when the control signal STGLA is at a high level (timing t22 to t23), and the transistor TGB has a control signal STGLB at a high level. In a certain period (timing t23 to t25), the charge generated by the photodiode PD is transferred to the floating diffusion FDB. As a result, the charge QA is accumulated in the floating diffusion FDA during the period from timing t22 to t23, and the charge QB is accumulated in the floating diffusion FDB during the period from timing t23 to t24.

The difference between the amount of charge of the charge QA and the amount of charge of the charge QB (charge difference ΔQ) changes according to the phase φ. In other words, the charge difference ΔQ changes according to the distance D from the photodetector 1 to the object 100. Specifically, for example, the shorter the distance D, the more the charge QA and the less the charge QB. Further, for example, as the distance D increases, the charge QA decreases and the charge QB increases.

As shown in FIGS. 7 and 8, the pixel P1 repeats the operation at timings t21 to t25. As a result, the charge QA is repeatedly accumulated in the floating diffusion FDA, and the charge QB is repeatedly accumulated in the floating diffusion FDB. As a result, the voltage VFDA of the floating diffusion FDA and the voltage VFDB of the floating diffusion FDB gradually decrease (FIGS. 7C and 7D). The amount of change in voltage VFDA corresponds to the amount of charge QA, and the amount of change in voltage VFDB corresponds to the amount of charge QB. In this example, the degree of change in voltage VFDA is greater than the degree of change in voltage VFDB.

Then, at the timing t12, the light emitting unit 11 finishes the light emitting operation OP1 (FIG. 7(A)). Further, the drive unit 22 stops the generation of the control signals STGLA and STGLB at this timing t12, and sets the voltage of the control signals STGLA and STGLB to the low level (FIGS. 7E and 7F). As a result, the transistors TGA and TGB are turned off. In this way, the exposure period T1 ends.

After that, in the period from timing t13 to timing t16, the photodetecting device 1 performs the reading operation OP3. In this read operation OP3, the drive section 22 sets the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, the transistors SELA and SELB are both turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs a pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs a pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output.

In the period from timing t13 to t14, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (see FIG. ), (H)). At timing t14, the drive section 22 changes the voltage of the control signals SRSLA and SRSLB from low level to high level (FIG. 7(B)). As a result, in the pixel P1, the transistors RSTA and RSTB are both turned on, and the voltages VFDA and VFDB in the floating diffusions FDA and FDB are set to the power supply voltage VDD (FIGS. 7C and 7D). In this way, the floating diffusions FDA and FDB are reset. At timing t15, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from the high level to the low level (FIG. 7(B)). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. Then, in the period from timing t15 to t16, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 7). (G), (H)). That is, the AD conversion unit 25A performs AD conversion based on the pixel signal SIGA after the floating diffusion FDA is reset, and the AD conversion unit 25B performs AD conversion based on the pixel signal SIGB after the floating diffusion FDB is reset. Perform the conversion.

The AD conversion unit 25A generates a digital code CODE (digital code CODEA) by subtracting the digital value converted in the period of timing t17 to t18 from the digital value converted in the period of timing t13 to t14. Similarly, the AD conversion unit 25B generates a digital code CODE (digital code CODEB) by subtracting the digital value converted in the period of timing t17 to t18 from the digital value converted in the period of timing t13 to t14. .. The reading unit 24 sequentially transfers these digital codes CODEA and CODEB to the processing unit 26 as an image signal DATA0.

The pixel value calculation unit 28 of the processing unit 26 calculates the pixel value of the pixel P1 based on, for example, the difference between the two digital codes CODEA and CODEB related to the pixel P1. This pixel value indicates a value for the distance D to the object 100. The processing unit 26 uses the plurality of pixel values obtained from the plurality of pixels P in the pixel array 21 to generate the distance image PIC1 and outputs the distance image PIC1 as the image signal DATA.

(About imaging operation)
Next, the imaging operation for imaging the target object 100 in the imaging operation mode MI will be described in detail.

9A and 9B show an example of the image pickup operation of the photodetector 1, where FIG. 9A shows the waveform of the control signal SRSLB, FIG. 9B shows the waveform of the control signal STGLB, and FIG. 9C shows the control signal SRSLA. , (D) shows the waveform of the control signal STGLA, (E) shows the operation of the AD conversion section 25 (AD conversion section 25A) connected to the signal line SGLA related to the pixel P1, and (F). Shows the operation of the AD conversion unit 25 (AD conversion unit 25B) connected to the signal line SGLB related to the pixel P1. 10A to 10E show the operating state of the main part of the pixel P1.

The photodetector 1 first performs the exposure operation OP2. In this exposure operation OP2, the drive unit 22 lowers the voltage of the control signal SSELL. As a result, in the pixel P1, the transistors SELA and SELB are both turned off, and the pixel P1 is electrically disconnected from the signal lines SGLA and SGLB. The floating diffusions FDA and FDB of the pixel P1 selectively accumulate the charges generated by the photodiode PD. Specifically, the floating diffusion FDB accumulates the charge generated by the photodiode PD during the exposure period T2B, and the floating diffusion FDA accumulates the charge generated by the photodiode PD during the exposure period T2A. The exposure time ratio RT (time length of exposure period T2B/time length of exposure period T2A) is dynamically set to, for example, about 1 to 100. In other words, “time length of exposure period T2B:time length of exposure period T2A” is set to, for example, about “1:1” to “100:1”. Then, the photodetector 1 performs the read operation OP3. In this read operation OP3, the drive section 22 sets the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, the transistors SELA and SELB are both turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs a pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs a pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA, and the AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB. The operation will be described in detail below.

In the period before the timing t31, the drive unit 22 sets the voltages of the control signals SRSLA and SRSLB to the high level (FIGS. 9A and 9C). As a result, in the pixel P1, the transistors RSTA and RSTB are both turned on, and the voltages VFDA and VFDB in the floating diffusions FDA and FDB are set to the power supply voltage VDD. In this way, the floating diffusion FDA and FDB are reset.

Next, at timing t31, the drive unit 22 changes the voltage of the control signal STGLB from low level to high level (FIG. 9(B)). As a result, in the pixel P1, the transistor TGB is turned on. In the period from timing t31 to t32 (period T11), the photodiode PD and the floating diffusion FDB are electrically connected as shown in FIG. 10A.

Next, at timing t32, the drive unit 22 changes the voltage of the control signals SRSLA and SRSLB from the high level to the low level (FIGS. 9A and 9C). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. At this time, the charge QA0 is accumulated in the floating diffusion FDA and the charge QB0 is accumulated in the floating diffusion FDB. In this way, the exposure period T2B starts. In the period from timing t32 to t33 (period T12), as shown in FIG. 10B, the charges generated by the photodiode PD are accumulated in the floating diffusion FDB as the charges QB1. In this way, the charge QB (=QB0+QB1) is accumulated in the floating diffusion FDB.

Next, at timing t33, the drive unit 22 changes the voltage of the control signal STGLB from the high level to the low level (FIG. 9(B)). As a result, in the pixel P1, the transistor TGB is turned off, and the photodiode PD and the floating diffusion FDB are electrically disconnected. In this way, the exposure period T2B ends.

Then, at this timing t33, the exposure period T2A starts. In the period from timing t33 to t35 (period T13), the charges generated by the photodiode PD are accumulated in the photodiode PD, as shown in FIG. 10C.

After that, in the period from timing t34 to timing t37, the photodetecting device 1 performs the reading operation OP3. In this read operation OP3, the drive section 22 sets the voltage of the control signal SSELL to a high level. As a result, in the pixel P1, the transistors SELA and SELB are both turned on, and the pixel P1 is electrically connected to the signal lines SGLA and SGLB. The pixel P1 outputs a pixel signal SIGA corresponding to the amount of electric charge accumulated in the floating diffusion FDA to the signal line SGLA, and outputs a pixel signal SIGB corresponding to the amount of electric charge accumulated in the floating diffusion FDB to the signal line SGLB. Output.

During this period from timing t34 to t35, the charge QA0 is accumulated in the floating diffusion FDA, as shown in FIG. 10C. During the period from timing t34 to timing t35, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 9(E)). The digital value converted by the AD conversion unit 25A during this period is a value corresponding to the amount of electric charge QA0.

Next, at timing t35, the drive unit 22 changes the voltage of the control signal STGLA from low level to high level (FIG. 9(D)). As a result, the transistor TGA is turned on in the pixel P1. In the period from timing t35 to t36 (period T14), the photodiode PD and the floating diffusion FDA are electrically connected as shown in FIG. 10D. Then, the charge generated by the photodiode PD is accumulated as the charge QA1 in the floating diffusion FDA. In this way, the charge QA (=QA0+QA1) is accumulated in the floating diffusion FDA.

Next, at timing t36, the drive unit 22 changes the voltage of the control signal STGLA from the high level to the low level (FIG. 9(D)). As a result, in the pixel P1, the transistor TGA is turned off and the photodiode PD and the floating diffusion FDA are electrically disconnected. In this way, the exposure period T2A ends.

During the period from timing t36 to t37 (period T15), as shown in FIG. 10E, the charge QA (= QA0 + QA1) is accumulated in the floating diffusion FDA. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 9 (E)). The digital value converted by the AD conversion unit 25A in the period T15 is a value corresponding to the charge amount of the charge QA (=QA0+QA1).

During the period from the timing t36 to t37 (period T15), the electric charge QB (=QB0+QB1) is accumulated in the floating diffusion FDB, as shown in FIG. 10E. The AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 9(F)). The digital value converted by the AD conversion unit 25B in this period is a value corresponding to the charge amount of the charge QB (=QB0+QB1).

Next, at the timing t37, the drive unit 22 changes the voltages of the control signals SRSLA and SRSLB from a low level to a high level (FIGS. 9A and 9C). As a result, in the pixel P1, the transistors RSTA and RSTB are both turned on, and the voltages VFDA and VFDB in the floating diffusions FDA and FDB are set to the power supply voltage VDD. In this way, the floating diffusions FDA and FDB are reset.

The AD conversion unit 25A generates a digital code CODE (digital code CODEA) by subtracting the digital value converted at the timing t34 to t35 from the digital value converted at the timing t36 to t37. The digital code CODEA has a value corresponding to the amount of charge QA1 (=QA-QA0) accumulated in the exposure period T2A. That is, the AD conversion unit 25A generates the digital code CODEA using the principle of so-called correlated double sampling (CDS). Since the AD converter 25A is configured to perform such correlated double sampling, the noise component (charge QA0 in this example) can be removed, and as a result, the image quality of the captured image can be improved.

The AD conversion unit 25B also generates a digital code CODE (digital code CODEB) based on the digital value converted at the timings t36 to t37. The digital code CODEB has a value corresponding to the charge amount of the charge QB (=QB0+QB1) accumulated in the exposure period T2B.

The reading unit 24 sequentially transfers these digital codes CODEA and CODEB to the processing unit 26 as an image signal DATA0.

As will be described later, the pixel value calculation unit 28 of the processing unit 26 calculates the pixel value of the pixel P based on, for example, the digital code CODEA relating to each pixel P and the exposure time ratio RT. The processing unit 26 generates a captured image PIC2 using the plurality of pixel values obtained from the plurality of pixels P in the pixel array 21, and outputs the captured image PIC2 as the image signal DATA.

FIG. 11 shows an operation example of the exposure controller 27 and the pixel value calculator 28. In this example, the amount of charge that can be stored in the photodiode PD is "1,000e-" or less (less than 1,000 electrons), and the amount of charge that can be stored in each of the floating diffusion FDA and FDB is "100,000e". -" or less (100,000 electrons or less). In the following, the exposure amount at the taps A and B is indicated by the charge amount.

The exposure control unit 27 sets the exposure time ratio RT based on the digital code CODEB relating to a certain pixel P (reference pixel PR) of the plurality of pixels P in the pixel array 21. The reference pixel PR is indicated by, for example, a control signal CTL supplied from the outside. The external device can determine the reference pixel PR according to the scene, for example. The pixel value calculation unit 28 calculates the pixel value of the pixel P based on, for example, the digital code CODEA relating to each pixel P and the exposure time ratio RT. The photodetector 20 repeats this operation, for example, in frame period units. The operation will be described in detail below.

First, the light detection unit 20 of the light detection device 1 performs an image pickup operation (step S101). Specifically, the photodetector 20 performs the exposure operation OP2 and the read operation OP3 as shown in FIG. As a result, digital codes CODEA and CODEB for the plurality of pixels P of the pixel array 21 are obtained.

Next, the exposure control unit 27 confirms whether the exposure amount at the tap B of the reference pixel PR is “1,000e−” or less (step S102). Specifically, the exposure control unit 27 determines whether or not the amount of electric charge accumulated in the floating diffusion FDB in the reference pixel PR is "1,000 e-" or less based on the digital code CODEB related to the reference pixel PR. Check.

In step S102, when the exposure amount at the tap B of the reference pixel PR is equal to or less than “1,000e−” (“Y” in step S102), the exposure control unit 27 sets the exposure time ratio RT (of the exposure period T2B). It is confirmed whether or not (time length/time length of exposure period T2A) is "1" (step S103). That is, the exposure control unit 27 confirms whether "time length of exposure period T2B:time length of exposure period T2A" is "1:1".

When the exposure time ratio RT is “1” in step S103 (“Y” in step S103), the process proceeds to step S109.

If the exposure time ratio RT is not "1" in step S103 ("N" in step S103), the exposure control unit 27 sets the exposure time ratio RT to "1" (step S104). That is, in this case, since the exposure amount is the amount of charge that can be accumulated in the photodiode PD, the exposure control unit 27 sets the exposure time ratio RT to “1”. Then, the process proceeds to step S108.

In step S102, when the exposure amount of the reference pixel PR at the tap B is larger than “1,000e−” (“N” in step S102), the exposure control unit 27 determines that the exposure amount of the reference pixel PR at the tap B is increased. It is confirmed whether or not the value is "100,000e-" or less (step S105).

In step S105, when the exposure amount of the reference pixel PR at the tap B is "100,000 e-" or less ("Y" in step S105), the exposure control unit 27 determines the exposure amount of the reference pixel PR at the tap B. It is confirmed whether or not the value obtained by dividing by the exposure time ratio RT is "1,000e-" or less (step S106).

In step S106, when the value obtained by dividing the exposure amount at tap B of the reference pixel PR by the exposure time ratio RT is “1,000e−” or less (“Y” in step S106), the process proceeds to step S109.

In step S106, when the value obtained by dividing the exposure amount at the tap B of the reference pixel PR by the exposure time ratio RT is larger than “1,000e−” (“N” in step S106), the exposure control unit 27 causes the exposure to be performed. The time ratio RT is set to a value obtained by dividing the exposure amount at tap B of the reference pixel PR by "1,000e-" (step S107).

Then, in step S108, the pixel value calculation unit 28 determines the pixel value of each of the plurality of pixels P including the reference pixel PR to a value according to the exposure amount of the tap B of the pixel P. Specifically, the pixel value calculation unit 28 determines the pixel value of each pixel P to a value according to the digital code CODEB relating to the pixel P.

Further, in step S109, the pixel value calculation unit 28 determines the pixel value of each of the plurality of pixels P including the reference pixel PR according to the value obtained by multiplying the exposure amount of the tap A of the pixel P by the exposure time ratio RT. Determine the value.

In step S105, when the exposure amount at the tap B of the reference pixel PR is larger than “100,000 e−” (“N” in step S105), the exposure control unit 27 determines the time length of the exposure period T2A and the exposure period T2B. The total exposure time of the time length is shortened (step S110). That is, in this case, the exposure amount exceeds the charge amount that can be accumulated in the floating diffusion FDB and is saturated, so the exposure control unit 27 shortens the total exposure time.

This is the end of this flow. The photodetection unit 20 repeats the operations of these steps S101 to S110, for example, in frame period units.

Here, the reference pixel PR corresponds to a specific example of the "first pixel" in the present disclosure. Pixels P other than the reference pixel PR correspond to a specific example of the "second pixel" in the present disclosure. The AD conversion unit 25B corresponds to a specific example of the “first AD conversion unit” in the present disclosure. The AD conversion unit 25A corresponds to a specific example of the “second AD conversion unit” in the present disclosure. The digital code CODEB corresponds to a specific example of the "first digital code" in the present disclosure. The digital code CODEA corresponds to a specific example of the "second digital code" in the present disclosure.

As described above, the photodetector 1 can perform an imaging operation in addition to the distance measuring operation. Thereby, in the photodetection device 1, for example, by combining the distance image PIC1 and the captured image PIC2, it is possible to synthesize a distance image having higher resolution. In particular, in the photodetector 1, the photodetector 20 that performs the distance measuring operation by the indirect method is used to perform the imaging operation. Since such an indirect method can realize high measurement accuracy, the photodetection apparatus 1 can further improve the accuracy of the distance image by performing an imaging operation.

Further, in the photodetector 1, the time length of the exposure period T2A and the time length of the exposure period T2B are set based on the exposure amount of the tap B of the reference pixel PR. Specifically, for example, the exposure time ratio RT (time length of exposure period T2B/time length of exposure period T2A) is set based on the exposure amount of the tap B of the reference pixel PR. Thereby, in the photodetector 1, the approximate exposure amount is grasped based on the exposure amount of the tap B of the reference pixel PR, and the exposure time ratio RT is appropriately set based on the information about the approximate exposure amount. can do. As described above, in the photodetector 1, the exposure time ratio RT can be appropriately set, so that the conversion efficiency can be freely set almost continuously. As a result, the photodetector 1 can perform the imaging operation while ensuring the dynamic range.

Further, in the photodetector 1, the pixel value of the pixel P is calculated based on the exposure time ratio RT thus set and the exposure amount of the tap A of the pixel P. Particularly, in the photodetection device 1, the exposure amount of the tap A of the pixel P is obtained by using the correlated double sampling, and the pixel value of the pixel P is obtained based on the obtained exposure amount and the exposure time ratio RT. It was calculated. As a result, the photodetector 1 can obtain the captured image PIC2 with less noise.

[effect]
As described above, in the present embodiment, the time length of the exposure period T2A and the time length of the exposure period T2B are set based on the exposure amount of the tap B of the reference pixel, so that the exposure time ratio is appropriately set. Since it can be set, the imaging operation can be performed while ensuring the dynamic range.

[Modification 1]
In the above embodiment, one exposure period T2B and one exposure period T2A are provided in the imaging operation, but the present invention is not limited to this. The photodetector 1A according to this modification will be described in detail below.

The photodetector 1A according to the present modification includes a photodetector 20A, similar to the photodetector 1 according to the above-described embodiment (FIG. 1). The light detection unit 20A has a processing unit 26A, similarly to the light detection unit 20 (FIG. 2) according to the above embodiment. The processing unit 26A has an exposure control unit 27A. The exposure control unit 27A is configured to set the time length of the exposure period T2A and the time length of the exposure period T2B in the imaging operation mode MI, similarly to the exposure control unit 27 according to the above-described embodiment. The exposure control unit 27A divides the exposure period T2B into two, and sets the exposure periods T2A and T2B in the order of the exposure period T2B, the exposure period T2A, and the exposure period T2B.

FIG. 12 shows an example of the image pickup operation of the photodetector 1A. 13A to 13G show the operation state of the main part of the pixel P1.

In the period before the timing t41, the drive unit 22 sets the voltages of the control signals SRSLA and SRSLB to the high level (FIGS. 12A and 12C). As a result, in the pixel P1, the transistors RSTA and RSTB are both turned on, and the voltages VFDA and VFDB in the floating diffusions FDA and FDB are set to the power supply voltage VDD.

Next, at timing t41, the drive unit 22 changes the voltage of the control signal STGLB from low level to high level (FIG. 12(B)). As a result, in the pixel P1, the transistor TGB is turned on. During the period from timing t41 to t42 (period T21), the photodiode PD and the floating diffusion FDB are electrically connected as shown in FIG. 13A.

Next, at timing t42, the drive unit 22 changes the voltage of the control signals SRSLA and SRSLB from the high level to the low level (FIGS. 23(A) and (C)). As a result, in the pixel P1, both the transistors RSTA and RSTB are turned off. At this time, the electric charge QA0 is accumulated in the floating diffusion FDA, and the electric charge QB0 is accumulated in the floating diffusion FDB. In this way, the exposure period T2B starts. In the period from timing t42 to t43 (period T22), as shown in FIG. 13B, the charges generated by the photodiode PD are accumulated in the floating diffusion FDB as the charges QB1.

Next, at timing t43, the drive section 22 changes the voltage of the control signal STGLB from the high level to the low level (FIG. 12(B)). As a result, in the pixel P1, the transistor TGB is turned off, and the photodiode PD and the floating diffusion FDB are electrically disconnected. In this way, the exposure period T2B ends once.

Then, at this timing t43, the exposure period T2A starts. During the period from timing t43 to t45 (period T23), as shown in FIG. 13C, the electric charge generated by the photodiode PD is accumulated in the photodiode PD.

During the period from timing t44 to t45, the charge QA0 is accumulated in the floating diffusion FDA, as shown in FIG. 13C. During the period from timing t44 to timing t45, the AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 12(E)). The digital value converted by the AD conversion unit 25A during this period is a value corresponding to the amount of electric charge QA0.

Next, at timing t45, the drive unit 22 changes the voltage of the control signal STGLA from low level to high level (FIG. 12(D)). As a result, the transistor TGA is turned on in the pixel P1. During the period from timing t45 to timing t46 (period T24), the photodiode PD and the floating diffusion FDA are electrically connected as shown in FIG. 13D. Then, the electric charge generated by the photodiode PD is accumulated in the floating diffusion FDA as the electric charge QA1.

Next, at timing t46, the drive unit 22 changes the voltage of the control signal STGLA from the high level to the low level (FIG. 9(D)). As a result, in the pixel P1, the transistor TGA is turned off, and the photodiode PD and the floating diffusion FDA are electrically disconnected. In this way, the exposure period T2A ends. Then, the exposure period T2B starts again.

During the period from timing t46 to t47 (period T25), as shown in FIG. 13E, the charge QA (= QA0 + QA1) is accumulated in the floating diffusion FDA. The AD conversion unit 25A performs AD conversion based on the voltage of the signal line SGLA (FIG. 12E). The digital value converted by the AD conversion unit 25A in the period T25 is a value corresponding to the charge amount of the charge QA (=QA0+QA1).

Next, at timing t47, the drive section 22 changes the voltage of the control signal SRSLA from low level to high level (FIG. 12(C)). As a result, in the pixel P1, the transistor RSTA is turned on, and the voltage VFDA in the floating diffusion FDA is set to the power supply voltage VDD. Further, at the timing t47, the drive section 22 changes the voltage of the control signal STGLB from the low level to the high level (FIG. 12(B)). As a result, the transistor TGB is turned on in the pixel P1. In the period from timing t47 to t48 (period T26), the photodiode PD and the floating diffusion FDB are electrically connected as shown in FIG. 13F. Then, the electric charge generated by the photodiode PD is accumulated as the electric charge QB1 in the floating diffusion FDB.

Next, at the timing t48, the drive unit 22 changes the voltage of the control signal STGLB from the high level to the low level (FIG. 12(B)). As a result, in the pixel P1, the transistor TGB is turned off, and the photodiode PD and the floating diffusion FDB are electrically disconnected. In this way, the exposure period T2B ends.

During the period from timing t48 to t49 (period T27), as shown in FIG. 13G, the charge QB (= QB0 + QB1) is accumulated in the floating diffusion FDB. The AD conversion unit 25B performs AD conversion based on the voltage of the signal line SGLB (FIG. 12 (F)). The digital value converted by the AD conversion unit 25B in this period is a value corresponding to the charge amount of the charge QB (=QB0+QB1).

The AD conversion unit 25A generates a digital code CODE (digital code CODEA) by subtracting the digital value converted at the timings t44 to t45 from the digital value converted during the period from the timing t46 to t47. The digital code CODEA has a value corresponding to the amount of charge QA1 (=QA-QA0) accumulated in the exposure period T2A.

The AD conversion unit 25B also generates a digital code CODE (digital code CODEB) based on the digital value converted at the timings t48 to t49. This digital code CODEB has a value corresponding to the amount of charge QB (=QB0+QB1) accumulated in the two exposure periods T2B.

The reading unit 24 sequentially transfers these digital codes CODEA and CODEB to the processing unit 26A as an image signal DATA0.

In this way, in the photodetector 1A, the exposure periods T2A and T2B are set in the order of the exposure period T2B, the exposure period T2A, and the exposure period T2B in the imaging operation. As a result, in the photodetector 1A, for example, when the illuminance changes monotonically during the exposure period, the exposure amount of the tap A (digital code CODEA) is set to the exposure amount corresponding to the center value of the change range of the illuminance. be able to.

[Modification 2]
In the above embodiment, in the imaging operation, when the exposure amount of the tap B of the reference pixel PR is "100,000 e-" or less, which is the amount of charge that can be accumulated in the floating diffusion FDA and FDB, the exposure time ratio RT However, the present invention is not limited to this. Hereinafter, the photodetector 1B according to the present modification will be described in detail.

FIG. 14 illustrates an operation example of the exposure control unit 27B and the pixel value calculation unit 28 according to this modification. Steps S101 to S104 and S106 to S110 are the same as those in the above embodiment (FIG. 11).

In step S102, when the exposure amount at the tap B of the reference pixel PR is larger than “1,000e−” (“N” in step S102), the exposure control unit 27B determines that the exposure amount at the tap B of the reference pixel PR is increased. It is confirmed whether or not it is "10,000e-" or less (step S115).

In step S115, when the exposure amount of the reference pixel PR at the tap B is larger than “10,000e−” (“N” in step S115), the exposure control unit 27B determines that the exposure amount of the reference pixel PR at the tap B is increased. It is confirmed whether or not it is "100,000e-" or less (step S116). If the exposure amount at tap B of the reference pixel PR is equal to or less than “100,000 e−” (“Y” in step S116), this flow ends. If the exposure amount at the tap B of the reference pixel PR is larger than “100,000 e−” (“N” in step S116), the process proceeds to step S110.

With this, in the photodetector 1B, the exposure time ratio RT is set when the exposure amount of the tap B of the reference pixel PR is “10,000e−” or less. Thereby, for example, when the linearity deteriorates when the exposure amount on the tap B exceeds "10,000e-", the exposure time ratio RT can be set so that the exposure amount does not deteriorate the linearity. .. As a result, in the photodetector 1B, the image quality of the captured image PIC2 can be improved.

[Modification 3]
Although the floating diffusions FDA and FDB for accumulating the charges generated by the photodiode PD are provided in the above-described embodiment, the present invention is not limited to this, and a capacitive element may be further provided in addition to this. This modification will be described below with some examples.

FIG. 15 shows an example of the pixel P in the pixel array 21C according to this modification. The pixel array 21C has a plurality of control lines FDGL. The control line FDGL is provided so as to extend in the horizontal direction (horizontal direction in FIG. 15), and a control signal SFDGL is applied to the control line FDGL by the drive unit 22C according to the present modification.

The pixel P has transistors FDGA and FDGB and capacitors CAPA and CAPB. The transistors FDGA and FDGB are N-type MOS transistors in this example.

The gate of the transistor FDGA is connected to the control line FDGL, the drain is connected to the source of the capacitive element CAPA and the transistor RSTA, and the source is connected to the floating diffusion FDA, the drain of the transistor TGA, and the gate of the transistor AMPA. One end of the capacitive element CAPA is connected to the drain of the transistor FDGA and the source of the transistor RSTA, and the other end is grounded.

The gate of the transistor FDGB is connected to the control line FDGL, the drain is connected to the capacitive element CAPB and the source of the transistor RSTB, and the source is connected to the floating diffusion FDB, the drain of the transistor TGB, and the gate of the transistor AMPB. One end of the capacitive element CAPB is connected to the drain of the transistor FDGB and the source of the transistor RSTB, and the other end is grounded.

The transistors FDGA and FDGB are turned on based on the control signal SFDGL. By turning on the transistors FDGA and FDGB, the floating diffusion FDA and the capacitive element CAPA are electrically connected, and the floating diffusion FDB and the capacitive element CAPB are electrically connected. As a result, in the pixel P, the capacitance values at the taps A and B can be increased, so that the amount of charge that can be accumulated at the taps A and B can be increased.

FIG. 16 shows an example of a pixel P in another pixel array 21D according to this modification. The pixel array 21D has a plurality of control lines FDGLA and a plurality of control lines FDGLB. The control line FDGLA is provided so as to extend in the horizontal direction (horizontal direction in FIG. 16), and the control signal SFDGLA is applied to the control line FDGLA by the drive unit 22D according to the present modification. The control line FDGLB is provided so as to extend in the horizontal direction, and a control signal SFDGLB is applied to the control line FDGLB by the drive unit 22D according to the present modification.

The pixel P has transistors FDGA and FDGB and capacitors CAPA and CAPB. The gate of the transistor FDGA is connected to the control line FDGLA. The gate of the transistor FDGB is connected to the control line FDGLB.

The transistor FDGA is turned on based on the control signal SFDGLA. By turning on the transistor FDGA, the floating diffusion FDA and the capacitive element CAPA are electrically connected. As a result, in the pixel P, the amount of charge that can be accumulated in the tap A can be increased. Similarly, the transistor FDGB is turned on based on the control signal SFDGLB. By turning on the transistor FDGB, the floating diffusion FDB and the capacitive element CAPB are electrically connected. As a result, in the pixel P, the amount of charge that can be accumulated in the tap B can be increased.

In this way, in the pixel P, taps A and B can be individually set. As a result, for example, the settings can be set according to the roles of the taps A and B. Specifically, for example, if saturation occurs even if the "time length of the exposure period T2B: the time length of the exposure period T2A" is set to "100: 1", the transistor FDGA is turned off and the transistor FDGB is turned on. The ON state can be achieved, which can increase the amount of charge that can be accumulated in the tap B. In other words, the conversion efficiency in the tap B can be lowered and the conversion efficiency in the tap A can be increased.

[Modification example 4]
In the above embodiment, in the imaging operation, all the pixels P in the pixel array 21 operate at the same exposure time ratio RT, but the present invention is not limited to this. Instead of this, for example, a plurality of pixels P in the pixel array may be divided into a plurality of groups so that the plurality of pixels P belonging to the same group operate at the same exposure time ratio RT. The photodetection device 1D having two groups will be described in detail below as an example.

FIG. 17 shows a configuration example of the pixel array 21D and the drive unit 22D in the photodetector 1D according to this modification. The pixel array 21D has a plurality of pixels P. The plurality of pixels P are divided into two groups G1 and G2 in this example. In FIG. 17, the pixels P (pixels PG1) belonging to the group G1 are shown by thick lines, and the pixels P (pixels PG2) belonging to the group G2 are shown by thin lines.

The pixel array 21D includes a plurality of control lines TGLA1, a plurality of control lines TGLA2, a plurality of control lines TGLB1, a plurality of control lines TGLB2, a plurality of control lines RSLA1, a plurality of control lines RSLA2, and a plurality of controls. It has a line RSLB1 and a plurality of control lines RSLB2. The control line TGLA1 is provided so as to extend in the horizontal direction (horizontal direction in FIG. 17), and a control signal STGLA1 is applied to the control line TGLA1 by the drive unit 22D. The control line TGLA2 is provided so as to extend in the horizontal direction, and a control signal STGLA2 is applied to the control line TGLA2 by the drive unit 22D. The control line TGLB1 is provided so as to extend in the horizontal direction, and a control signal STGLB1 is applied to the control line TGLB1 by the drive unit 22D. The control line TGLB2 is provided so as to extend in the horizontal direction, and a control signal STGLB2 is applied to the control line TGLB2 by the drive unit 22D. The control line RSLA1 is provided so as to extend in the horizontal direction, and a control signal SRSLA1 is applied to the control line RSLA1 by the drive unit 22D. The control line RSLA2 is provided so as to extend in the horizontal direction, and a control signal SRSLA2 is applied to the control line RSLA2 by the drive unit 22D. The control line RSLB1 is provided so as to extend in the horizontal direction, and a control signal SRSLB1 is applied to the control line RSLB1 by the drive unit 22D. The control line RSLB2 is provided so as to extend in the horizontal direction, and a control signal SRSLB2 is applied to the control line RSLB2 by the drive unit 22D.

Pixel PG1 is connected to control lines TGLA1, TGLB1, RSLA1, RSLB1. The pixel PG2 is connected to the control lines TGLA2, TGLB2, RSLA2, RSLB2.

The exposure control unit 27D in the light detection device 1D sets the exposure time ratio RT (exposure time ratio RT1) based on the exposure amount of the tap B of a certain reference pixel PG1R among the plurality of pixels PG1 in the imaging operation mode MI. At the same time, the exposure time ratio RT (exposure time ratio RT2) is set based on the exposure amount of the tap B of a certain reference pixel PG2R among the plurality of pixels PG2. Then, the plurality of pixels PG1 can operate at the exposure time ratio RT1, and the plurality of pixels PG2 can operate at the exposure time ratio RT2.

Here, the reference pixel PG1R corresponds to a specific but not limitative example of “first pixel” in one embodiment of the present disclosure. The reference pixel PG2R corresponds to a specific but not limitative example of “second pixel” in one embodiment of the disclosure. The exposure time ratio RT1 corresponds to a specific but not limitative example of “first exposure time ratio” in one embodiment of the present disclosure. The exposure time ratio RT2 corresponds to a specific but not limitative example of “second exposure time ratio” in one embodiment of the present disclosure.

[Modification 5]
In the above-described embodiment, in the imaging operation, the exposure time ratio RT is set based on the exposure amount of the tap B of a certain reference pixel among the plurality of pixels P, and the plurality of pixels P operate at the exposure time ratio RT. However, the present invention is not limited to this. Instead of this, for example, the exposure time ratio RT may be set based on the exposure amount of the tap B of each pixel P so that the pixel P operates at the exposure time ratio RT. The photodetector 1E according to this modification will be described in detail below.

FIG. 18 illustrates a configuration example of the photodetection unit 20E in the photodetection device 1E. The photodetection section 20E is formed on the two

semiconductor chips

30 and 40. A pixel array 21E is formed on the semiconductor chip 30. The pixel array 21E has a plurality of pixels P arranged in a matrix. A plurality of circuits C arranged in a matrix are formed on the semiconductor chip 40. The plurality of circuits C are provided corresponding to the plurality of pixels P, respectively. The semiconductor chips 30 and 40 are overlaid on each other and electrically connected to each other. Specifically, the pixel P and the circuit C are electrically connected to each other by using, for example, a through electrode.

FIG. 19 shows a configuration example of the pixel P and the circuit C. The circuit C has a drive unit 42,

AD conversion units

45A and 45B, and a processing unit 46. The drive unit 42, the

AD conversion units

45A and 45B, and the processing unit 46 correspond to the drive unit 22, the

AD conversion units

25A and 25B, and the processing unit 26 according to the above embodiments, respectively.

The drive unit 42 is configured to drive the pixel P based on an instruction from the processing unit 46. Specifically, the drive unit 42 supplies the control signal STGLA to the transistor TGA of the pixel P, supplies the control signal STGLB to the transistor TGB of the pixel P, and supplies the control signal to the transistor RSTA of the pixel P. The SRSLA is supplied, the control signal SRSLB is supplied to the transistor RSTB of the pixel P, and the control signal SSELL is supplied to the transistors SELA and SELB of the pixel P. The drive unit 42 has a signal generation unit 43. The signal generator 43 is configured to generate the control signals STGLA and STGLB.

The AD conversion unit 45A is configured to generate a digital code CODEA by performing AD conversion based on the voltage of the pixel signal SIGA supplied from the pixel P. The AD conversion unit 45B is configured to generate a digital code CODEB by performing AD conversion based on the voltage of the pixel signal SIGB supplied from the pixel P.

The processing unit 46 is configured to control the operations of the pixels P and the circuit C by supplying control signals to the drive unit 42 and the

AD conversion units

45A and 45B based on the instruction from the control unit 13. The processing unit 46 has an exposure control unit 47 and a pixel value calculation unit 48.

The exposure control unit 47 is configured to set the time length of the exposure period T2A and the time length of the exposure period T2B in the imaging operation mode MI. Specifically, the exposure control unit 47 sets, for example, the ratio of the time length of the exposure period T2A and the time length of the exposure period T2B (exposure time ratio RT). Then, the processing unit 46 is configured to supply the driving unit 42 with information regarding the time lengths of the exposure periods T2A and T2B set by the exposure control unit 47.

The pixel value calculation unit 48 is configured to calculate the pixel value of the pixel P based on the digital codes CODEA and CODEB. In the distance measurement operation mode MD, the pixel value indicates the value for the distance D to the object 100. In the imaging operation mode MI, the pixel value indicates the amount of received light.

With this configuration, the light detection unit 20E can set the exposure time ratio RT in units of pixels P.

Here, the pixel P corresponds to a specific but not limitative example of “first pixel” in one embodiment of the present disclosure. The exposure control unit 47 corresponds to a specific but not limitative example of “first exposure control unit” in one embodiment of the present disclosure. The drive unit 42 corresponds to a specific but not limitative example of “first drive unit” in one embodiment of the present disclosure. The pixel value calculation unit 48 corresponds to a specific but not limitative example of “first pixel value calculation unit” in one embodiment of the present disclosure.

[Other modifications]
Also, two or more of these modifications may be combined.

<2. Application to mobiles>
The technology according to the present disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure is realized as a device mounted on a moving body of any kind such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility, an airplane, a drone, a ship, and a robot. May be.

FIG. 15 is a block diagram showing a schematic configuration example of a vehicle control system that is an example of a mobile body control system to which the technology according to the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in FIG. 15, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside information detection unit 12030, an in-vehicle information detection unit 12040, and an integrated control unit 12050. Further, as the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio/video output unit 12052, and an in-vehicle network I/F (interface) 12053 are shown.

The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 provides a driving force generator for generating the driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a steering mechanism for adjustment and a control device such as a braking device that generates a braking force of the vehicle.

The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, blinkers or fog lamps. In this case, the body system control unit 12020 may be input with radio waves transmitted from a portable device that substitutes for the key or signals of various switches. The body system control unit 12020 receives inputs of these radio waves or signals and controls a vehicle door lock device, a power window device, a lamp, and the like.

The vehicle exterior information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, the imaging unit 12031 is connected to the vehicle outside information detection unit 12030. The vehicle outside information detection unit 12030 causes the image pickup unit 12031 to capture an image of the outside of the vehicle and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as people, vehicles, obstacles, signs, or characters on the road surface based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of the light received. The imaging unit 12031 can output the electric signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

The in-vehicle information detection unit 12040 detects in-vehicle information. To the in-vehicle information detection unit 12040, for example, a driver state detection unit 12041 that detects the state of the driver is connected. The driver state detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 determines the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated or it may be determined whether the driver is asleep.

The microcomputer 12051 calculates the control target value of the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and the drive system control unit. A control command can be output to 12010. For example, the microcomputer 12051 realizes a function of ADAS (Advanced Driver Assistance System) that includes collision avoidance or impact mitigation of a vehicle, follow-up traveling based on an inter-vehicle distance, vehicle speed maintenance traveling, a vehicle collision warning, or a vehicle lane departure warning. It is possible to perform cooperative control for the purpose.

Further, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, and the like based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform cooperative control for the purpose of autonomous driving or the like that autonomously travels without depending on operation.

Also, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the external information detection unit 12030, and performs cooperative control for the purpose of antiglare such as switching the high beam to the low beam. It can be carried out.

The voice image output unit 12052 transmits an output signal of at least one of a voice and an image to an output device capable of visually or audibly notifying information to an occupant of the vehicle or the outside of the vehicle. In the example of FIG. 15, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

FIG. 16 is a diagram showing an example of the installation position of the imaging unit 12031.

In FIG. 16, the vehicle 12100 has

imaging units

12101, 12102, 12103, 12104, 12105 as the imaging unit 12031.

The

imaging units

12101, 12102, 12103, 12104, 12105 are provided at positions such as the front nose, side mirrors, rear bumpers, back doors, and the upper part of the windshield in the vehicle interior of the vehicle 12100, for example. The image capturing unit 12101 provided on the front nose and the image capturing unit 12105 provided on the upper part of the windshield in the vehicle interior mainly acquire an image in front of the vehicle 12100. The

imaging units

12102 and 12103 included in the side mirrors mainly acquire images of the side of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or the back door mainly acquires an image of the rear of the vehicle 12100. The front images acquired by the

imaging units

12101 and 12105 are mainly used for detecting a preceding vehicle or a pedestrian, an obstacle, a traffic signal, a traffic sign, a lane, or the like.

Note that FIG. 16 shows an example of the shooting range of the imaging units 12101 to 12104. The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the

imaging units

12102 and 12103 provided on the side mirrors, and the imaging range 12114 indicates The imaging range of the imaging part 12104 provided in a rear bumper or a back door is shown. For example, by overlaying image data captured by the image capturing units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera including a plurality of image capturing elements, or may be an image capturing element having pixels for phase difference detection.

For example, the microcomputer 12051 has a distance to each three-dimensional object within the imaging range 12111 to 12114 based on the distance information obtained from the imaging units 12101 to 12104, and a temporal change of this distance (relative velocity with respect to the vehicle 12100). By obtaining, it is possible to extract as the preceding vehicle a three-dimensional object that is the closest three-dimensional object on the traveling path of the vehicle 12100 and that travels in substantially the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or more). it can. Further, the microcomputer 12051 can set an inter-vehicle distance to be secured in front of the preceding vehicle in advance, and can perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform cooperative control for the purpose of autonomous driving or the like that autonomously travels without depending on the operation of the driver.

For example, the microcomputer 12051 converts three-dimensional object data related to a three-dimensional object into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, electric poles, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 identifies an obstacle around the vehicle 12100 into an obstacle visible to the driver of the vehicle 12100 and an obstacle difficult to see. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, the microcomputer 12051 via the audio speaker 12061 or the display unit 12062. By outputting an alarm to the driver and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition includes, for example, a procedure for extracting feature points in an image captured by an imaging unit 12101 to 12104 as an infrared camera, and pattern matching processing for a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. Is performed by the procedure for determining. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a square contour line for emphasizing the recognized pedestrian. The display unit 12062 is controlled so as to superimpose and display. Further, the audio image output unit 12052 may control the display unit 12062 so as to display an icon or the like indicating a pedestrian at a desired position.

Above, an example of the vehicle control system to which the technology according to the present disclosure can be applied has been described. The technology according to the present disclosure can be applied to the imaging unit 12031 among the configurations described above. As a result, in the vehicle control system 12000, the measurement accuracy in distance measurement can be improved, so that the vehicle collision avoidance or collision mitigation function, the follow-up running function based on the inter-vehicle distance, the vehicle speed maintenance running function, the vehicle collision warning function, The accuracy of the lane departure warning function of the vehicle can be improved.

The present technology has been described above with reference to the embodiments and modified examples, and specific application examples thereof, but the present technology is not limited to these embodiments and the like, and various modifications are possible.

For example, in each of the above-described embodiments, the present technology is applied to a distance measuring device that measures the distance to the object 100, but the present invention is not limited to this. Instead of this, for example, the present technology may be applied to a time measuring device that measures the flight time of light from the light emitting unit emitting the light pulse L0 to the light receiving unit detecting the reflected light pulse L1. ..

Note that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can be configured as follows. According to the present technology having the following configuration, it is possible to perform an imaging operation while ensuring a dynamic range.

(1) A first photoelectric conversion element capable of performing a photoelectric conversion operation to generate a first received light charge based on light, a first charge storage section capable of storing the first received charge, and a first photoelectric conversion element No. 2 charge storage section, a first switch capable of connecting the first photoelectric conversion element to the first charge storage section by being turned on, and the first photoelectric conversion element being turned on. A first pixel having a second switch capable of connecting the element to the second charge storage unit, and
The first photoelectric conversion element is stored in the first charge storage unit based on a first charge amount of the first received charge stored in the first charge storage unit. A first time length of a first exposure period for generating a received light charge, and a second exposure for generating the first received charge accumulated in the second charge storage section by the first photoelectric conversion element. A first exposure control unit capable of performing a first setting process for setting a second time length of a period, and a first exposure control unit.
A photodetector including a first switch and a first drive unit capable of controlling the operation of the second switch based on the result of the first setting process.
(2) The second charge amount of the first received charge accumulated in the second charge accumulating portion and the first exposure time ratio of the first time length and the second time length. The photodetector according to (1) above, further comprising a first pixel value calculation unit capable of calculating a first pixel value based on the first pixel value.
(3) The first drive unit can control the operation of the first switch so as to maintain the first switch in an ON state during the first exposure period. The photodetector described.
(4) A first AD converter is further provided,
The first pixel is capable of outputting a first voltage according to the first charge amount in the first charge storage section in a first period after the first exposure period. Has an output section,
The first AD conversion unit can convert the first voltage into a first digital code in the first period.
The photodetector according to (3), wherein the first exposure control unit performs the first setting process based on the first digital code.
(5) The photodetector according to (3) or (4), wherein the first exposure period includes two sub-exposure periods sandwiching the second exposure period.
(6) The second exposure period includes a first sub-exposure period and a second sub-exposure period after the first sub-exposure period.
The first drive unit maintains the second switch in an off state during the first sub-exposure period and maintains the second switch in an on state during the second sub-exposure period, The photodetector according to any one of (2) to (5), wherein the operation of the second switch can be controlled.
(7) A second AD converter is further provided,
The first pixel is capable of outputting a second voltage according to the amount of charge in the second charge storage section during a second period within the period of the first sub-exposure period, and the second pixel In the third period after the sub-exposure period of the above, the second output unit capable of outputting the third voltage corresponding to the second charge amount is provided.
The second AD converter is capable of converting a difference voltage between the second voltage in the second period and the third voltage in the third period into a second digital code,
The photodetector according to (6), wherein the first pixel value calculation unit can calculate the pixel value based on the second digital code and the first exposure time ratio.
(8) The first exposure control unit can set the first exposure time ratio based on the first charge amount. Light detection device.
(9) The first exposure controller controls the first time length and the second time length to be equal to each other when the first charge amount is smaller than a first threshold value. The photodetector according to (8) above, wherein the first exposure time ratio can be set.
(10) When the first charge amount is larger than the first threshold value and smaller than a second threshold value, the first exposure control unit may change the first charge amount according to the first charge amount. The photodetector according to (9), wherein the first exposure time ratio can be set so that the first time length is longer than the second time length.
(11) The first exposure control unit shortens a total time length of the first time length and the second time length when the first charge amount is larger than a third threshold value. The photodetector according to any one of (2) to (10), wherein the first time length and the second time length can be set accordingly.
(12) A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received light charge based on light, a third charge storage section capable of storing the second received charge, and a third No. 4 charge storage unit, a third switch capable of connecting the second photoelectric conversion element to the third charge storage unit by being turned on, and the second photoelectric conversion unit being turned on. Further comprising a second pixel having a fourth switch capable of connecting the element to the fourth charge storage section.
In the first exposure control unit, the second photoelectric conversion element may further include the third photoelectric conversion element based on the third charge amount of the second received charge accumulated in the third charge accumulation unit. A third time length of a third exposure period for generating the second received electric charge accumulated in the charge accumulating portion, and the second photoelectric conversion element in which the second photoelectric conversion element is accumulated in the fourth charge accumulating portion. It is possible to perform a second setting process for setting the fourth time length of the fourth exposure period for generating the received charge of 2.
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the second setting process.
The first pixel value calculation unit further includes a fourth charge amount of the second received charge accumulated in the fourth charge accumulation unit, the third time length, and the fourth time length. The photodetector according to any one of (2) to (11) above, wherein the second pixel value can be calculated based on the second exposure time ratio of the above.
(13) A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received light charge based on light, a third charge storage unit capable of storing the second received charge, and a third No. 4 charge storage unit, a third switch capable of connecting the second photoelectric conversion element to the third charge storage unit by being turned on, and the second photoelectric conversion unit being turned on. A second pixel having a fourth switch capable of connecting the element to the fourth charge storage unit, and
The second photoelectric conversion element is accumulated in the third charge accumulating portion based on a third charge amount of the second received electric charge accumulated in the third charge accumulating portion. A third time length of a third exposure period for generating received light charges, and a fourth exposure for generating the second received light charges accumulated by the second photoelectric conversion element in the fourth charge accumulation unit. A second exposure control unit capable of performing a second setting process for setting a fourth time length of the period, and a second exposure control unit.
A second drive unit capable of controlling the operation of the third switch and the fourth switch based on the result of the second setting process.
Based on a fourth charge amount of the second received charge accumulated in the fourth charge accumulating portion, a third exposure time length, and a second exposure time ratio of the fourth exposure time length. The photodetector according to any one of (2) to (11) above, further comprising a second pixel value calculation unit capable of calculating a pixel value of 2.
(14) A second photoelectric conversion element capable of performing a photoelectric conversion operation for generating a second received light charge based on light, a third charge storage section capable of storing the second received charge, and a third No. 4 charge storage unit, a third switch capable of connecting the second photoelectric conversion element to the third charge storage unit by being turned on, and the second photoelectric conversion unit being turned on. Further comprising a second pixel having a fourth switch capable of connecting the element to the fourth charge storage section.
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the first setting process.
The first pixel value calculation unit further includes a second pixel value based on a fourth charge amount of the second received charges accumulated in the fourth charge accumulation unit and the exposure time ratio. The photodetector according to any one of (2) to (11) above.
(15) The first pixel is
A fifth switch capable of applying a predetermined voltage to the first charge storage unit by being turned on,
It further has a sixth switch capable of applying the predetermined voltage to the second charge storage unit by being turned on.
The said 1st drive part can control the operation|movement of the said 5th switch and the said 6th switch further based on the said 1st setting process. In any one of said (2) to (14). Photo detector.
(16) Further, an operation mode setting unit capable of setting the operation mode to one of a plurality of operation modes including the first operation mode is provided.
The first exposure control unit can perform the first setting process in the first operation mode.
The first drive unit is capable of controlling operations of the first switch and the second switch in the first operation mode based on a result of the first setting process,
The first pixel value calculation unit can calculate the pixel value based on the second charge amount and the exposure time ratio in the first operation mode. (2) to (14) The photodetector according to claim 1.
(17) A light emitting unit is further provided,
The plurality of operation modes includes a second operation mode,
The light emitting unit can emit an optical pulse by alternately repeating light emission and quenching in the second operation mode.
In the second operation mode, the first driving unit can alternately turn on the first switch and the second switch so as to be synchronized with the optical pulse,
The first pixel value calculation unit can calculate the pixel value based on the first charge amount and the second charge amount in the second operation mode. The light according to (16) above. Detection device.

This application claims priority based on Japanese Patent Application No. 2019-037454 filed on March 1, 2019 at the Japan Patent Office, and this application is made by referring to all the contents of this application. Be used for.

One of ordinary skill in the art can conceive of various modifications, combinations, sub-combinations, and changes, depending on design requirements and other factors, but they are included in the appended claims and their equivalents. Be understood to be

Claims

A first photoelectric conversion element capable of performing a photoelectric conversion operation for generating a first received light charge based on light, a first charge storage section capable of storing the first received charge, and a second charge An accumulator, a first switch capable of connecting the first photoelectric conversion element to the first charge accumulation section by being turned on, and a first switch converting the first photoelectric conversion element by being turned on. A first pixel having a second switch connectable to a second charge storage section;
The first photoelectric conversion element is stored in the first charge storage unit based on a first charge amount of the first received charge stored in the first charge storage unit. A first time length of a first exposure period for generating a received light charge, and a second exposure for generating the first received charge accumulated in the second charge storage section by the first photoelectric conversion element. A first exposure control unit capable of performing a first setting process for setting a second time length of a period, and a first exposure control unit.
A photodetector including a first switch and a first drive unit capable of controlling the operation of the second switch based on the result of the first setting process.
Based on the second charge amount of the first received charge accumulated in the second charge storage unit, and the first exposure time ratio of the first time length and the second time length. The photodetector according to claim 1, further comprising a first pixel value calculation unit capable of calculating the pixel value of 1.
The photodetection according to claim 2, wherein the first driving unit can control the operation of the first switch so as to keep the first switch on during the first exposure period. apparatus.
Further comprising a first AD converter,
The first pixel is capable of outputting a first voltage according to the first charge amount in the first charge storage section in a first period after the first exposure period. Has an output section,
The first AD conversion unit can convert the first voltage into a first digital code in the first period.
The photodetector according to claim 3, wherein the first exposure control unit performs the first setting process based on the first digital code.
The photodetector according to claim 3, wherein the first exposure period includes two sub-exposure periods that sandwich the second exposure period.
The second exposure period includes a first sub-exposure period and a second sub-exposure period after the first sub-exposure period.
The first drive unit maintains the second switch in an off state during the first sub-exposure period and maintains the second switch in an on state during the second sub-exposure period, The photodetector according to claim 2, wherein the operation of the second switch can be controlled.
Further comprising a second AD converter,
The first pixel can output a second voltage according to the amount of electric charge in the second charge storage unit in the second period within the period of the first sub-exposure period, and the second pixel can output the second voltage. In the third period after the sub-exposure period of the above, the second output unit capable of outputting the third voltage corresponding to the second charge amount is provided.
The second AD converter is capable of converting a difference voltage between the second voltage in the second period and the third voltage in the third period into a second digital code,
The photodetector according to claim 6, wherein the first pixel value calculation unit can calculate the pixel value based on the second digital code and the first exposure time ratio.
The photodetector according to claim 2, wherein the first exposure control unit can set the first exposure time ratio based on the first charge amount.
The first exposure control unit sets the first time length and the second time length to be equal to each other when the first charge amount is smaller than a first threshold value. The photodetector according to claim 8, wherein the exposure time ratio can be set.
When the first charge amount is larger than the first threshold value and smaller than the second threshold value, the first exposure control unit determines the first charge amount according to the first charge amount. The light detection device according to claim 9, wherein the first exposure time ratio can be set so that the time length is longer than the second time length.
The first exposure control unit shortens the total time length of the first time length and the second time length when the first charge amount is larger than a third threshold value, The photodetector according to claim 2, wherein the first time length and the second time length can be set.
A second photoelectric conversion element capable of performing a photoelectric conversion operation to generate a second received light charge based on light, a third charge storage section capable of storing the second received charge, and a fourth charge A storage section; a third switch that can connect the second photoelectric conversion element to the third charge storage section by being turned on; and a second switch that connects the second photoelectric conversion element by turning on. Further comprising a second pixel with a fourth switch connectable to a fourth charge store
In the first exposure control unit, the second photoelectric conversion element is further based on the third charge amount of the second received charge accumulated in the third charge storage unit. The third time length of the third exposure period for generating the second received charge accumulated in the charge storage unit, and the second photoelectric conversion element in which the second photoelectric conversion element is accumulated in the fourth charge storage unit. It is possible to perform a second setting process for setting a fourth time length of a fourth exposure period for generating two received electric charges,
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the second setting process.
The first pixel value calculation unit further includes a fourth charge amount of the second received charge accumulated in the fourth charge storage unit, the third time length, and the fourth time length. The photodetector according to claim 2, wherein the second pixel value can be calculated based on the second exposure time ratio of the above.
A second photoelectric conversion element capable of performing a photoelectric conversion operation to generate a second received light charge based on light, a third charge storage section capable of storing the second received charge, and a fourth charge A storage section; a third switch that can connect the second photoelectric conversion element to the third charge storage section by being turned on; and a second switch that connects the second photoelectric conversion element by turning on. A second pixel having a fourth switch that can be connected to a fourth charge storage unit, and
The second photoelectric conversion element is accumulated in the third charge storage unit based on the third charge amount of the second received charge accumulated in the third charge storage unit. The third time length of the third exposure period for generating the received light charge, and the fourth exposure for which the second photoelectric conversion element generates the second received light charge accumulated in the fourth charge storage unit. A second exposure control unit capable of performing a second setting process for setting a fourth time length of the period, and a second exposure control unit.
A second drive unit capable of controlling the operation of the third switch and the fourth switch based on the result of the second setting process.
A second based on the fourth charge amount of the second received charge accumulated in the fourth charge storage unit and the second exposure time ratio of the third time length and the fourth time length. The photodetector according to claim 2, further comprising a second pixel value calculation unit capable of calculating the pixel value of 2.
A second photoelectric conversion element capable of performing a photoelectric conversion operation to generate a second received light charge based on light, a third charge storage section capable of storing the second received charge, and a fourth charge A storage section; a third switch that can connect the second photoelectric conversion element to the third charge storage section by being turned on; and a second switch that connects the second photoelectric conversion element by turning on. Further comprising a second pixel with a fourth switch connectable to a fourth charge store
The first drive unit can further control the operation of the third switch and the fourth switch based on the result of the first setting process.
The first pixel value calculation unit further determines the second pixel value based on the fourth charge amount of the second received charge accumulated in the fourth charge storage unit and the exposure time ratio. The photodetector according to claim 2, which is capable of calculating
The first pixel is
A fifth switch capable of applying a predetermined voltage to the first charge storage unit by being turned on,
It further has a sixth switch capable of applying the predetermined voltage to the second charge storage unit by being turned on.
The photodetector according to claim 2, wherein the first drive unit is further capable of controlling the operations of the fifth switch and the sixth switch based on the first setting process.
Further provided with an operation mode setting unit capable of setting the operation mode to one of a plurality of operation modes including the first operation mode.
The first exposure control unit is capable of performing the first setting process in the first operation mode,
The first drive unit is capable of controlling operations of the first switch and the second switch in the first operation mode based on a result of the first setting process,
The photodetection device according to claim 2, wherein the first pixel value calculation unit can calculate the pixel value based on the second charge amount and the exposure time ratio in the first operation mode.
Further equipped with a light emitting unit,
The plurality of operation modes includes a second operation mode,
The light emitting unit can emit an optical pulse by alternately repeating light emission and quenching in the second operation mode.
In the second operation mode, the first driving unit can alternately turn on the first switch and the second switch so as to be synchronized with the optical pulse,
The light detection according to claim 16, wherein the first pixel value calculation unit can calculate the pixel value based on the first charge amount and the second charge amount in the second operation mode. apparatus.