JP2022185168A - Light receiving element and method for driving the same, and ranging system - Google Patents

Abstract

To improve ranging accuracy at low speed while maintaining transfer efficiency during high speed operation.SOLUTION: A light receiving element includes: a pixel having a photoelectric conversion unit and a transfer transistor for transferring electric charges generated in the photoelectric conversion unit; and a voltage control unit for switching and controlling a voltage VSS and a voltage VSS1 as an off control voltage for turning off the transfer transistor. The present disclosure is applicable to, for example, a light receiving element that irradiates an object with irradiation light and receives reflected light from the object to measure distance thereto.SELECTED DRAWING: Figure 9

Description

The present disclosure relates to a light-receiving element, its driving method, and a distance measurement system, and in particular, a light-receiving element and its driving method that can improve distance measurement accuracy during low-speed operation while maintaining transfer efficiency during high-speed operation. It also relates to a ranging system.

A light receiving element using an indirect ToF (Time of Flight) method is known. In an indirect ToF photodetector, signal charges generated by photoelectric conversion of reflected light are distributed to two charge storage units by, for example, two transfer gates, and the distance is calculated from the distribution ratio of the signal charges. .

In indirect ToF distance measurement, it is possible to operate by setting multiple frequencies (modulation frequencies) distributed to the two charge storage units, so the same device must be able to handle both high-speed and low-speed frequencies. must. When the speed is high, the amount of charge accumulated in the photodiode is small, but the transfer time is shortened, so it is necessary to make a design that is advantageous for transfer. On the other hand, when the speed is low, a large amount of charge is accumulated in the photodiode, so it is important to increase the charge accumulation amount. Since these two are in a trade-off relationship, it is difficult to achieve them simultaneously. Also, if the amount of charge storage is small, the device is vulnerable to receiving large amounts of light.

For example, there has been proposed a sensitivity modulation element that prevents electrons from flowing back to a photodiode by causing the charges transferred to the two charge storage portions to periodically overflow to the pixel drain electrode (for example, See Patent Document 1).

Japanese Patent Application Laid-Open No. 2003-247809

As described above, the indirect ToF type light receiving element needs to be operated with a high modulation frequency, and the amount of charge accumulated in the photodiode may be smaller than that of the conventional viewing sensor. Therefore, surplus charges larger than the saturated charge amount of the photodiode may overflow and be unintentionally transferred to the charge accumulation section on the opposite side of the transfer direction of the main body. In this case, the ranging accuracy is lowered.

The present disclosure has been made in view of such circumstances, and is intended to improve distance measurement accuracy during low-speed operation while maintaining transfer efficiency during high-speed operation.

The light receiving element of the first aspect of the present disclosure includes
a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit;
and a voltage control unit that switches and controls a plurality of voltage values as an off-control voltage for turning off the transfer transistor.

A method for driving a light receiving element according to the second aspect of the present disclosure includes:
A light-receiving element comprising a pixel having a photoelectric conversion unit and a transfer transistor for transferring charges generated by the photoelectric conversion unit,
A plurality of voltage values are switched and controlled as an off-control voltage for turning off the transfer transistor.

The ranging system of the third aspect of the present disclosure comprises:
a light emitting unit that emits irradiation light;
a light-receiving element that receives the reflected light returned by the irradiation light reflected by an object,
The light receiving element is
a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit;
and a voltage control unit that switches and controls a plurality of voltage values as an off-control voltage for turning off the transfer transistor.

In the first to third aspects of the present disclosure, in a light-receiving element including a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit, off control for turning off the transfer transistor As the voltage, a plurality of voltage values are switch-controlled.

The light-receiving element and the distance measuring system may be independent devices or may be modules incorporated in other devices.

1 is a block diagram showing a configuration example of a ranging system according to an embodiment of the present disclosure; FIG. 2 is a block diagram showing a configuration example of a light receiving element in FIG. 1; FIG. It is a figure which shows the circuit structural example of a pixel. It is a figure explaining light emission of irradiation light, and a drive of a pixel. It is a figure explaining the calculation method of the depth value by a 2-Phase method and a 4-Phase method. It is a figure explaining the calculation method of the depth value by a 2-Phase method and a 4-Phase method. It is a figure explaining the calculation method of the depth value by a 2-Phase method and a 4-Phase method. FIG. 4 is a diagram for explaining driving of transfer transistors during high-speed operation; FIG. 10 is a diagram illustrating driving of transfer transistors during low-speed operation compared with high-speed operation; FIG. 4 is a diagram summarizing applied voltage control when a transfer transistor is turned off; 4 is a block diagram showing a configuration example of an off-voltage control section; FIG. 4 is a flowchart for explaining off-voltage control processing; 1 is a block diagram showing a configuration example of a smartphone as an electronic device equipped with a ranging system of the present disclosure; FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; FIG. FIG. 4 is an explanatory diagram showing an example of installation positions of an outside information detection unit and an imaging unit;

Hereinafter, modes for implementing the technology of the present disclosure (hereinafter referred to as embodiments) will be described with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description. The explanation is given in the following order.
1. Configuration example of distance measuring system 2. Configuration example of light-receiving element 3. Configuration example of pixel circuit 4. 5. Description of pixel operation and pixel signal. Drive voltage control of transfer transistor6. Configuration example of off-voltage control unit 7. Modification 8. Configuration example of electronic device 9. Example of application to mobile objects

<1. Configuration example of distance measuring system>
FIG. 1 is a block diagram showing a configuration example of a ranging system according to an embodiment of the present disclosure.

The ranging system 1 of FIG. 1 includes a control device 11, a light receiving device 12, and an illumination device 13. The light receiving device 12 includes a light receiving element 31 and a lens 32 . The illumination device 13 has an LD 21 and a light emitting section 22 .

The control device 11 is a device that controls the operation of the distance measuring system 1 as a whole. The control device 11 specifies an operation mode and supplies a measurement request to the light receiving device 12 . Further, the control device 11 acquires from the light receiving device 12 the measurement data generated based on the result of the light received by the light receiving device 12 in response to the measurement request. The control device 11 may be provided outside the ranging system 1 instead of being part of the configuration of the ranging system 1, for example, as a control unit of a host device in which the ranging system 1 is incorporated.

Here, the operation modes that can be specified by the control device 11 to the light receiving device 12 include a ranging mode for measuring the distance to an object by the indirect ToF method, and a 2D image shooting mode for generating a two-dimensional luminance image. There is Indirect ToF distance measurement is a distance measurement that calculates the distance to an object by detecting the flight time from the timing at which irradiated light is emitted to the timing at which reflected light is received as a phase difference. The 2D image shooting mode is a mode for outputting a two-dimensional luminance image according to the amount of light received, like a normal image sensor.

When a measurement request is supplied from the control device 11 , the light receiving element 31 of the light receiving device 12 supplies a light emission pulse to the LD 21 and causes the light emitting section 22 to emit irradiation light. The light receiving element 31 receives reflected light from subjects such as the objects 41 and 42 through the lens 32 . The lens 32 forms an image on the light-receiving surface of the light-receiving element 31 using the reflected light reflected from the object as incident light. Note that the configuration of the lens 32 is arbitrary, and for example, the lens 32 can be configured with a plurality of lens groups.

The light emission pulse is a timing signal that serves as a reference for the light emitting operation of the light emitting unit 22 and the light receiving operation of the light receiving element 31. For example, it is a pulse signal in which ON (High) and OFF (Low) are repeated at the modulation frequency Fmod. .

When the operation mode is the distance measurement mode, the light receiving element 31 generates a depth image storing distance information to the subject based on the light receiving result of the reflected light, and outputs it to the control device 11 as measurement data. On the other hand, when the operation mode is the 2D image shooting mode, the light receiving device 12 generates a brightness image of the subject based on the result of receiving the reflected light, and outputs it to the control device 11 as measurement data. A specific configuration of the light receiving element 31 will be described in detail after FIG.

The LD 21 of the illumination device 13 is, for example, a laser driver that drives the light emitting section 22, drives the light emitting section 22 based on the light emission pulse from the light receiving element 31, and causes the light emitting section 22 to output irradiation light. The light emitting unit 22 is composed of, for example, a VCSEL LED (Vertical Cavity Surface Emitting LASER LED) or the like, and emits irradiation light by driving the LD 21 . As the irradiation light, for example, infrared light (IR light) having a wavelength in the range of approximately 850 nm to 940 nm is used.

<2. Configuration example of light receiving element>
FIG. 2 is a block diagram showing a configuration example of the light receiving element 31 of FIG.

The light receiving element 31 includes a control section 51 , a pixel array section 52 , a pulse generation circuit 53 , a tap driving section 54 , a vertical driving section 55 , a column processing section 56 , a horizontal driving section 57 , a signal processing section 58 and an output section 59 . composed of

The control unit 51 controls the operation of the light receiving element 31 as a whole. For example, the control unit 51 acquires the operation mode and measurement request supplied from the control device 11 via an input unit (not shown). Further, the control unit 51 instructs the pulse generation circuit 53 to generate a light emission pulse with a predetermined modulation frequency Fmod according to the operation mode. The control unit 51 supplies control signals for performing operations according to the operation mode to the vertical driving unit 55, the column processing unit 56, the horizontal driving unit 57, and the like. Further, the control unit 51 instructs the signal processing unit 58 to perform predetermined signal processing performed by the signal processing unit 58 according to the operation mode, such as depth image generation processing and luminance image generation processing.

A plurality of pixels 61 are two-dimensionally arranged in a matrix in the pixel array section 52 . Although the circuit configuration of the pixels 61 will be described later with reference to FIG. 3, each of the pixels 61 arranged two-dimensionally in the pixel array section 52 includes a photodiode 71 as a photoelectric conversion section that generates an electric charge corresponding to the amount of light received. and two taps (FD73) for accumulating charges generated by the photodiode 71 . These two taps are also called the first tap and the second tap.

The pulse generation circuit 53 generates a light emission pulse with a predetermined modulation frequency Fmod under the control of the control unit 51 and outputs it to the LD 21 of the illumination device 13 . The modulation frequency Fmod of the light emission pulse is set to, for example, 20 MHz during low-speed driving, and is set to 100 MHz during high-speed driving. Note that this modulation frequency is merely an example, and is not limited to this numerical value.

Further, the pulse generation circuit 53 generates drive signals GDA and GDB corresponding to the light emission pulse with the modulation frequency Fmod and supplies them to the tap drive section 54 . The drive signal GDA is a drive signal for transferring the charge generated by the photoelectric conversion unit of each pixel 61 to the first tap, and the drive signal GDB transfers the charge generated by the photoelectric conversion unit of each pixel 61 to the first tap. It is a drive signal for transferring to 2 taps. The drive signal GDA and the drive signal GDB are signals in which the phase of the other is inverted with respect to that of the other.

The tap drive section 54 distributes the two drive signals GDA and GDB supplied from the pulse generation circuit 53 to generate a drive signal GDA' and a drive signal GDB' for each pixel column. supplied to each pixel column. The tap drive section 54 supplies the drive signal GDA' and the drive signal GDB' for each pixel column of the pixel array section 52, thereby controlling charge distribution to the two taps of each pixel 61. FIG.

In the pixel array section 52, a pixel drive line 63 is wired in the horizontal direction for each pixel row, and two vertical signal lines 64A and 64B are wired in the vertical direction for each pixel column. The pixel drive line 63 transmits a drive signal for driving when reading out the detection signal VSL from each pixel 61 . In FIG. 2, the pixel drive line 63 is shown as one wiring, but it is not limited to one. One end of the pixel driving line 63 is connected to an output terminal corresponding to each pixel row of the vertical driving section 55 . The vertical signal line 64A is a signal line for transmitting the detection signal VSLA of the first tap to the column processing section 56, and the vertical signal line 64B is a signal line for transmitting the detection signal VSLB of the second tap to the column processing section 56. be.

The vertical driving section 55 is composed of a shift register, an address decoder, and the like, and drives each pixel 61 of the pixel array section 52 simultaneously or simultaneously for each pixel row through a pixel driving line 63 arranged in the horizontal direction. This is a pixel driving unit that drives in units of rows or the like. The detection signals VSLA and VSLB output from each pixel 61 in the pixel row selectively scanned by the vertical driving section 55 are supplied as pixel signals to the column processing section 56 through the vertical signal lines 64A or 64B.

The column processing unit 56 performs predetermined signal processing on pixel signals input from each pixel 61 of the selected row via the vertical signal line 64A or 64B for each pixel column of the pixel array unit 52, and performs signal processing. Subsequent pixel signals are temporarily held. For example, the column processing unit 56 executes AD (analog-to-digital) conversion processing of pixel signals as signal processing.

The horizontal driving section 57 is composed of a shift register, an address decoder, and the like, and selects unit circuits corresponding to the pixel columns of the column processing section 56 in order. By selective scanning by the horizontal driving unit 57 , the pixel signals processed by the column processing unit 56 are sequentially output to the signal processing unit 58 .

The signal processing unit 58 has a predetermined arithmetic processing function, performs predetermined arithmetic processing on the pixel signals output from the column processing unit 56 as necessary, and outputs the signals to the control device via the output unit 59. 11 (FIG. 1).

For example, when the operation mode is the distance measurement mode, the signal processing unit 58 determines the distance to the subject based on the pixel data (detection signals VSLA and VSLB) for each tap of each pixel 61 supplied from the column processing unit 56. A process of calculating distance information and generating a depth image in which the distance information is stored as the pixel value of each pixel is performed.

For example, when the operation mode is the 2D image shooting mode, the signal processing unit 58 performs processing for generating a brightness image of the subject based on pixel data for each tap of each pixel 61 supplied from the column processing unit 56. conduct.

The output unit 59 is configured by a predetermined communication interface such as MIPI (Mobile Industry Processor Interface), for example, and outputs the depth image and the luminance image, which are the arithmetic processing results of the signal processing unit 58, to the control device 11 as measurement data. Output.

The light-receiving element 31 configured as described above performs a light-receiving operation in an operation mode specified by the control device 11 . When the distance measurement mode is designated as the operation mode, the light receiving element 31 switches the modulation frequency Fmod of the light emission pulse to at least two frequencies of low speed drive and high speed drive, performs light reception operation, and generates a depth image. and output. In this embodiment, the first modulation frequency Fmod1 for high-speed driving is set to 100 MHz, for example, and the second modulation frequency Fmod2 for low-speed driving is set to 20 MHz, for example. Also, when the 2D image shooting mode is specified as the operation mode, the light receiving element 31 generates and outputs a luminance image of the subject. In the 2D image capturing mode, the illumination light may or may not be emitted at a predetermined modulation frequency Fmod as in the ranging mode. When the irradiation light is not emitted, the output of the light emission pulse to the illumination device 13 is stopped. In the present embodiment, it is assumed that irradiation light is emitted and received at the second modulation frequency Fmod2 driven at low speed. Note that the modulation frequency Fmod for emitting irradiation light in the 2D image capturing mode does not need to be the same as any modulation frequency Fmod for operating in the ranging mode.

<3. Configuration Example of Pixel Circuit>
FIG. 3 shows a circuit configuration example of the pixel 61. As shown in FIG.

A pixel 61 in FIG. 3 includes a photodiode 71 as a photoelectric conversion unit. The pixel 61 also has two transfer transistors 72, FD (floating diffusion regions) 73, FD gate transistors 74, amplification transistors 75, reset transistors 76, and selection transistors 77 corresponding to two taps. That is, the pixel 61 has transfer transistors 72A and 72B, FDs 73A and 73B, FD gate transistors 74A and 74B, amplification transistors 75A and 75B, reset transistors 76A and 76B, and selection transistors 77A and 77B. Elements marked with are the elements on the first tap side, and elements marked with B are elements on the second tap side. FDs 73A and 73B correspond to two taps for accumulating charges generated by the photodiode 71. FIG. Pixel 61 also includes a charge drain transistor 78 . Each transistor included in the pixel 61 is composed of an N-type MOSFET.

The photodiode 71 generates and accumulates electric charges according to the amount of reflected light received.

When the drive signal GDA' supplied to the gate electrode becomes active (High), the transfer transistor 72A becomes conductive in response to this, thereby transferring the charges accumulated in the photodiode 71 to the FD 73A. When the drive signal GDB' supplied to the gate electrode becomes active (High), the transfer transistor 72B becomes conductive in response to this, thereby transferring the charge accumulated in the photodiode 71 to the FD 73B.

The FD 73A is a first-tap charge storage unit that temporarily stores and holds the charge transferred from the photodiode 71. FIG. FD 73B is a second-tap charge storage unit that temporarily stores and holds the charge transferred from the photodiode 71 .

When the FD drive signal FDG supplied to the gate electrode becomes active, the FD gate transistor 74A becomes conductive in response to this. connect to When the FD drive signal FDG supplied to the gate electrode becomes active, the FD gate transistor 74B becomes conductive in response to this. connect to By dynamically controlling the on/off of the FD gate transistor 74 according to the amount of incident light, the storage capacitance can be changed. In FIG. 3, for the sake of simplification, there is only one FD drive signal line 81, and the FD drive signal FDG is shared by the FD gate transistors 74A and 74B. They are individually provided for the FD gate transistors 74A and 74B, and are controlled to be on or off so that they operate exclusively.

The amplification transistor 75A is connected to a constant current source (not shown) by connecting the source electrode to the vertical signal line 64A through the selection transistor 77A, thereby forming a source follower circuit. The amplification transistor 75B is connected to a constant current source (not shown) by connecting the source electrode to the vertical signal line 64B via the selection transistor 77B, thereby forming a source follower circuit.

The reset transistor 76A resets the potential of the FD 73A by becoming conductive in response to the active state of the reset drive signal RST supplied to the gate electrode. The reset transistor 76B becomes conductive in response to the reset drive signal RST supplied to its gate electrode becoming active, thereby resetting the potential of the FD 73B. When reset transistors 76A and 76B are activated, FD gate transistors 74A and 74B are simultaneously activated. In FIG. 3, for the sake of simplification, there is only one reset driving signal line 82, and the reset driving signal RST is shared by the reset transistors 76A and 76B. Each of the transistors 76A and 76B is individually provided, and ON or OFF is controlled so that each operates exclusively. A predetermined drain voltage RSTDRAIN is supplied to the drains of the reset transistors 76A and 76B.

The selection transistor 77A is connected between the amplification transistor 75A and the vertical signal line 64A, becomes conductive when the selection signal SEL supplied to the gate electrode becomes active, and detects the detection signal VSLA output from the amplification transistor 75A. , to the vertical signal line 64A. The selection transistor 77B is connected between the amplification transistor 75B and the vertical signal line 64B, becomes conductive when the selection signal SEL supplied to the gate electrode becomes active, and detects the detection signal VSLB output from the amplification transistor 75B. , to the vertical signal line 64B. In FIG. 3, for the sake of simplification, there is only one selection signal line 83, and the selection signal SEL is shared by the selection transistors 77A and 77B. 77B are individually provided, and ON or OFF is controlled so that each operates exclusively.

When the discharge drive signal OFG supplied to the gate electrode through the discharge drive signal line 84 becomes active, the charge discharge transistor 78 becomes conductive in response to this, thereby discharging the charges accumulated in the photodiode 71 . Discharge.

<4. Description of Pixel Operation and Pixel Signal>
The operation of the pixel 61 in FIG. 3 will be described.

First, a reset operation for resetting the charges of the pixels 61 is performed in all pixels before light reception is performed. That is, the FD gate transistors 74A and 74B and the reset transistors 76A and 76B are turned on to discharge the accumulated charge of the FDs 73A and 73B, and the charge discharge transistor 78 is turned on to discharge the accumulated charge of the photodiode 71. .

After the accumulated charges are discharged, all pixels start receiving light. Specifically, as shown in FIG. 4, the light emitting unit 22 outputs irradiation light that is modulated such that the irradiation is repeatedly turned on and off for an irradiation time T, and a delay time ΔT corresponding to the distance to the subject. The reflected light is received by the photodiode 71 with a delay of . One cycle (=2T) of the irradiation light is 1/Fmod. The drive signal GDA' controls on/off of the transfer transistor 72A, and the drive signal GDB' controls on/off of the transfer transistor 72B. The drive signal GDA' is, for example, a signal having the same phase as that of the irradiation light, and the drive signal GDB' has an inverted phase of the drive signal GDA'.

Therefore, in FIG. 3, the charge generated by the photodiode 71 receiving the reflected light is transferred to the FD 73A while the transfer transistor 72A is on according to the driving signal GDA', and the transfer transistor 72A is transferred according to the driving signal GDB'. While 72B is on, it is transferred to FD 73B. As a result, during a predetermined period in which the irradiation of the irradiation light of the irradiation time T is periodically performed, the charges transferred via the transfer transistor 72A are sequentially accumulated in the FD 73A, and the charges transferred via the transfer transistor 72B are stored in the FD 73A. It is accumulated sequentially in the FD73B. Additional capacitance is also stored when FD gate transistors 74A and 74B are on.

As described above, the pixel 61 distributes the charge generated by the reflected light received by the photodiode 71 to the first tap (FD73A) and the second tap (FD73B) according to the delay time ΔT. It outputs a detection signal VSLA and a second tap detection signal VSLB.

However, in the pixel array unit 52, the detection signal VSLA and the detection signal VSLA are affected differently for each pixel 61 due to the characteristic deviation (sensitivity difference) of each element of the pixel transistor such as the photodiode 71 and the transfer transistor 72 of each pixel 61. signal VSLB. Therefore, in the distance measurement system 1 of the indirect ToF method, the sensitivity difference between the taps of each pixel 61 is eliminated by acquiring the detection signal VSLA and the detection signal VSLB that received the reflected light with the same pixel 61 changing the phase. Then, a method to improve the SN ratio is adopted.

As methods for receiving reflected light with different phases and calculating depth values, for example, a 2-Phase detection method (2-Phase method) and a 4-Phase detection method (4-Phase method) will be described.

In the 4-Phase method, as shown in FIG. 5, the light-receiving element 31 receives the reflected light at light-receiving timings shifted by 0°, 90°, 180°, and 270° with respect to the irradiation timing of the irradiation light. receive light. More specifically, the light-receiving element 31 receives light with a phase of 0° with respect to the irradiation timing of irradiation light in a certain frame period, receives light with a phase of 90° in the next frame period, and receives light with a phase of 90° in the next frame period. In a frame period, the reflected light is received with a phase of 180°, and in the next frame period, the phase is changed with a phase of 270°, and the reflected light is received by changing the phase.

A phase of 0°, 90°, 180°, or 270° represents the phase at the first tap of the pixel 61 unless otherwise specified. Since the second tap has a phase opposite to that of the first tap, when the first tap has a phase of 0°, 90°, 180°, or 270°, the second tap has a phase of 180°, 270°, respectively. °, 0°, or 90° phase.

FIG. 6 is a diagram showing the exposure periods of the first tap of the pixel 61 at each phase of 0°, 90°, 180°, and 270° so that the phase difference can be easily understood.

As shown in FIG. 6, at the first tap, the detection signal VSLA obtained by receiving light in the same phase (phase 0°) as the irradiation light is the detection signal A ₀ , and the phase (phase 90°) shifted from the irradiation light by 90°. °) is the detection signal A ₉₀ , the detection signal VSLA obtained by light reception at a phase shifted by 180 degrees from the irradiation light (phase 180 °) is the detection signal A ₁₈₀ , and the irradiation light is 270 degrees A detection signal VSLA obtained by receiving light with a shifted phase (phase 270°) is called a detection signal A ₂₇₀ .

Although not shown, the detection signal VSLB obtained by receiving light at the same phase (phase 0°) as the irradiation light at the second tap is designated as the detection signal B ₀ , and the phase shifted by 90° from the irradiation light (phase 90°). ), the detection signal VSLB obtained by receiving light at a phase shifted by 180 degrees from the irradiation light (phase 180 degrees) is the detection signal B ₁₈₀ , and the detection signal _VSLB obtained by light reception at a phase shifted by 180 degrees from the irradiation light is detection signal B 180. A detection signal VSLB obtained by receiving light with a shifted phase (phase 270°) is called a detection signal B ₂₇₀ .

FIG. 7 is a diagram for explaining how to calculate the depth value and reliability by the 2-Phase method and the 4-Phase method.

式（１）のｃは光速であり、ΔTは遅延時間であり、fは光の変調周波数Fmodを表す。また、式（１）のφは、反射光の位相ずれ量[rad]を表し、次式（２）で表される。

In the indirect ToF method, the depth value d can be obtained by the following formula (1).

In Equation (1), c is the speed of light, ΔT is the delay time, and f is the modulation frequency Fmod of light. Also, φ in Equation (1) represents the amount of phase shift [rad] of the reflected light and is expressed by the following Equation (2).

In the 4-Phase method, I and Q in Equation (2) convert detection signals A ₀ to A ₂₇₀ and detection signals B ₀ to B ₂₇₀ obtained by setting the phases to 0°, 90°, 180°, and 270°. is calculated by the following formula (3). I and Q are signals obtained by converting the phase of the sine wave from polar coordinates to a rectangular coordinate system (IQ plane), assuming that the luminance change of the irradiation light is a sine wave.
I = c ₀ - c ₁₈₀ = (A ₀ - B ₀ ) - (A ₁₈₀ - B ₁₈₀ )
Q= _c90 - _c270 =( _A90 - _{B90)-(A270-B270 ) (} 3)

In the 4-Phase method, for example, like “A ₀ −A ₁₈₀ ” and “A ₉₀ −A ₂₇₀ ” in Equation (3), by taking the difference of detection signals of opposite phases in the same pixel, It is possible to remove the characteristic variation between the taps, that is, the sensitivity difference between the taps.

On the other hand, in the 2-Phase method, I and Q in Equation (2) can be calculated using detection signals with two phases of 0° and 90°. That is, I and Q of the formula (2) in the 2Phase method are given by the following formula (4).
I = c ₀ - c ₁₈₀ = (A ₀ - B ₀ )
Q= _c90 - _c270 =( _A90 - _B90 ) (4)

Although the 2-Phase method cannot eliminate characteristic variations between taps that exist in each pixel, it is possible to obtain the depth value d to the object using only the detection signals of two phases. It is possible to measure the distance at a rate. Characteristic variations between taps can be adjusted by correction parameters such as gain and offset, for example.

式（５）から分かるように、信頼度cnfは、画素６１で受光した反射光の大きさ、即ち輝度情報（輝度値）に相当する。 The reliability cnf can be obtained by the following equation (5) in both the 2Phase method and the 4Phase method.

As can be seen from Equation (5), the reliability cnf corresponds to the magnitude of reflected light received by the pixel 61, that is, luminance information (luminance value).

Hereinafter, a unit in which each pixel 61 of the pixel array section 52 outputs pixel data (detection signal) of one phase such as 0°, 90°, 180°, or 270° is referred to as one frame (period). In the 4-Phase method, one depth image can be generated with four frames corresponding to four phases using one modulation frequency Fmod. One depth image can be generated from two corresponding frames.

However, the indirect ToF method suffers from aliasing. That is, in indirect ToF distance measurement, since the phase difference is detected and converted into distance, the maximum measurement range is determined according to the modulation frequency Fmod of the light emitting unit 22, and when the maximum measurement distance is exceeded, the detected position The phase difference starts again from zero. As a result, for example, when the modulation frequency Fmod is 100 MHz, 1.5 m and 3 m cannot be distinguished. Therefore, it is necessary to perform distance measurement with a plurality of modulation frequencies Fmod such as 100 MHz and 20 MHz. In the 4-Phase method, one modulation frequency Fmod requires four frames of received light data. Therefore, when performing distance measurement with a plurality of modulation frequencies Fmod, at least eight frames are required to output one depth image. It is necessary to emit light and receive light.

When the distance measurement mode is specified as the operation mode, the light receiving element 31 receives irradiation light at a high-speed first modulation frequency Fmod1 and at a second modulation frequency Fmod2 lower than the first modulation frequency. generates one depth image and outputs it to the control device 11 as measurement data.

On the other hand, when the 2D image capturing mode is specified as the operation mode, the light receiving element 31 receives the irradiation light at the second modulation frequency Fmod2 which is on the low speed side, thereby obtaining the reliability cnf as the brightness information described above. A luminance image is generated by calculating and stored as pixel values, and is output to the control device 11 as measurement data.

In the 2D image capturing mode, the light receiving element 31 does not use the reliability cnf as the pixel value, but uses the AD conversion data of the detection signal VSLA of the first tap and the A luminance image may be generated by storing data obtained by summing the AD conversion data of the detection signal VSLB as pixel values, and may be output to the control device 11 as measurement data.

<5. Driving Voltage Control of Transfer Transistor>
When the light-receiving element 31 performs light-receiving operation in synchronization with the high-speed first modulation frequency Fmod1, the charge amount accumulated in the photodiode 71 of each pixel 61 may be small, but the transfer time is short. It is necessary to make a design favorable to transfer. On the other hand, when the light-receiving element 31 performs light-receiving operation in synchronization with the low-speed second modulation frequency Fmod2, the amount of charge accumulated in the photodiode 71 increases, so it is important to increase the amount of accumulated charge. becomes. Also, when the operation mode is the 2D image shooting mode, or when the subject is a high-brightness subject, it is desirable to increase the charge storage amount of the photodiode 71 .

The light receiving element 31 can perform control to change the charge accumulation amount of the photodiode 71 according to the operation mode and the magnitude of the amount of light received.

FIG. 8 is a diagram illustrating driving of the transfer transistor 72 when the pixel 61 operates at high speed, in other words, when the pixel 61 operates at the first modulation frequency Fmod1.

In FIG. 8, "PD" represents the photodiode 71, "TGA" represents the transfer transistor 72A on the first tap side, and "FDA" represents the FD 73A which is the first tap. "TGB" represents the transfer transistor 72B on the second tap side, and "FDB" represents the FD 73B which is the second tap.

The diagram on the left side of FIG. 8 shows a state in which both transfer transistors 72A and 72B are controlled to be off. A predetermined voltage VSS is supplied to the gate electrodes of the transfer transistors 72A and _72B as an off-voltage TGL for turning off the transfer transistor 72A. The charge amount that can be stored in the photodiode 71 in this state is PDQs.

The central diagram in FIG. 8 shows a state in which the transfer transistor 72A on the first tap side is controlled to be ON. When turning on the transfer transistor 72A, the tap drive unit 54 applies a predetermined voltage VDD (>VSS) higher than the voltage VSS as the ON voltage _TGH to the gate electrode of the transfer transistor 72A. Thereby, the charge accumulated in the photodiode 71 is transferred to the FD 73A.

On the other hand, the diagram on the right side of FIG. 8 shows a state in which the transfer transistor 72B on the second tap side is controlled to be ON. When turning on the transfer transistor 72B, the tap drive unit 54 applies a voltage VDD (>VSS) higher than the voltage VSS as an on-voltage _TGH to the gate electrode of the transfer transistor 72B. Thereby, the charge accumulated in the photodiode 71 is transferred to the FD 73B.

FIG. 9 is a diagram for explaining the driving of the transfer transistor 72 during low-speed operation compared with high-speed operation.

When the pixel 61 operates at low speed, in other words, when the pixel 61 operates at the second modulation frequency _Fmod2 , the off-voltage TGL for turning off the transfer transistor 72 is the voltage VSS1 lower than the voltage VSS during high-speed operation. (<VSS). As a result, as shown in FIG. 9, the barrier when the transfer transistors 72A and 72B are turned off is increased, so that the amount of charge PDQs that can be stored in the photodiode 71 can be made larger than during high-speed operation. Note that the on-voltage _TGH for turning on the transfer transistor 72 during low-speed operation is the same voltage VDD as during high-speed operation.

In FIGS. 8 and 9, the operation mode is the ranging mode, and the difference in applied voltage when the transfer transistor is turned off between high-speed operation and low-speed operation has been explained. is a high-brightness object, the applied voltage when the transfer transistor is turned off is changed to a voltage VSS1 lower than that during high-speed operation.

FIG. 10 is a table summarizing cases in which the voltage applied to the light-receiving element 31 when the transfer transistor is turned off is changed to a voltage VSS1 lower than that during high-speed operation.

The tap drive unit 54 of the light receiving element 31 controls to set the off voltage TGL of the transfer transistor 72 to the voltage VSS when the operation mode is the distance measurement mode and the light receiving operation is performed at the high-speed first modulation frequency _Fmod1 . do. On the other hand, when the operation mode is the ranging mode and the light receiving operation is performed at the low second modulation frequency _Fmod2 , the control unit 51 sets the off voltage TGL of the transfer transistor 72 to the voltage VSS1 lower than the voltage VSS. to control.

Further, when the operation mode is the 2D image capturing mode, the tap drive unit 54 controls to set the off voltage _TGL of the transfer transistor 72 to a voltage VSS1 lower than the voltage VSS regardless of the modulation frequency Fmod.

Furthermore, regardless of whether the operation mode is the distance measurement mode or the 2D image shooting mode, the tap drive unit 54 turns off the transfer transistor 72 when it is determined that the amount of light received by the pixel array unit 52 is large. The voltage _TGL is controlled to be set to a voltage VSS1 lower than the voltage VSS. Whether or not the amount of light received by the pixel array section 52 is large is determined, for example, as follows. For example, when the pixel data of the first frame is acquired, and the pixel data of a predetermined number or more or a predetermined ratio or more of the number of pixels from which the pixel data in the pixel array section 52 are acquired indicates a saturated value, It can be determined that the amount of received light is large. Alternatively, when saturation of pixel data occurs in an area of N×M pixels or more (N>1, M>1) in the pixel array section 52 from which pixel signals are acquired, it is determined that the amount of light received is large. You may

<6. Configuration Example of Off-Voltage Control Unit>
FIG. 11 is a block diagram showing a configuration example of an off-voltage control section that controls the off-voltage _TGL of the transfer transistor 72 according to the operation mode or the like.

The off-voltage control section 101 can be provided in the tap driving section 54 as shown in FIG. 11, for example.

The tap drive unit 54 distributes the two tap drive signals GDA and GDB supplied from the pulse generation circuit 53 to generate the drive signal GDA' and the drive signal GDB' for each pixel column, and the pixel array unit Each of the 52 pixel columns is supplied.

The off-voltage control unit 101 switches between the voltage VSS and the voltage _VSS1 as the low level of the drive signals GDA′ and GDB′, that is, the off-voltage TGL of the transfer transistor 72 . The High level of drive signals GDA' and GDB' is voltage VDD.

The off voltage control section 101 is composed of a circuit including an OR circuit 111 , an inverter 112 , and transistors 113 and 114 . Transistors 113 and 114 are N-type MOSFETs, and constitute a switch that switches between voltage VSS1 and voltage VSS as an off-control voltage for turning off transfer transistor 72 .

An operation mode determination signal, a frequency determination signal, and a high light amount determination signal are input to the OR circuit 111 . The OR circuit 111 performs a logical sum operation of the operation mode determination signal, the frequency determination signal, and the high light intensity determination signal, and outputs a high (1) or low (0) signal as the operation result to the gate electrode of the transistor 113. , to the inverter 112 .

The operation mode determination signal is a signal that becomes High (1) when the operation mode is the 2D image shooting mode and becomes Low (0) when the operation mode is the ranging mode, and is supplied from the control unit 51, for example.

The frequency determination signal is a signal that becomes High (1) when the modulation frequency Fmod is equal to or less than a predetermined threshold frequency Fmod_TH, and becomes Low (0) when it is greater than the threshold frequency Fmod_TH. The frequency determination signal can be generated, for example, by referring to a specific address signal that is always fixed to High or Low when the frequency divider in the pulse generation circuit 53 divides the frequency by X or more. The first modulation frequency Fmod1 in this embodiment is higher than the threshold frequency Fmod_TH (Fmod1>Fmod_TH), and the second modulation frequency Fmod2 is lower than the threshold frequency Fmod_TH (Fmod2=<Fmod_TH).

The large light amount determination signal is a signal that becomes High (1) when the amount of received light is large and becomes Low (0) when the amount of received light is not large. The signal processing unit 58 outputs the result of determining whether or not a predetermined number or more or a predetermined ratio or more of pixel data is saturated with respect to the number of pixels from which pixel data is acquired in the pixel array unit 52 as a high light amount determination signal. It is supplied to the off voltage control section 101 . Factors that cause the amount of received light to be large include a short distance to the subject, a large amount of emitted light, and a high reflectance of the subject.

The output of the OR circuit 111 is directly supplied to the gate electrode of the transistor 113, and is supplied to the gate electrode of the transistor 114 via the inverter 112. Therefore, when the transistor 113 is on, the transistor 114 is off, and the transistor 114 is off. When 113 is off, transistor 114 is on.

A voltage VSS1 is supplied to the source of the transistor 113 . A voltage VSS (>VSS1) is supplied to the source of the transistor 114 . Voltages VSS and VSS1 may be provided exclusively for off-voltage control of transfer transistor 72, or may be shared with other controls. For example, the voltage VSS can be shared with the ground voltage applied to the Pwell of the semiconductor substrate, and the voltage VSS1 can be shared with the off control voltage of other pixel transistors (eg, select transistor 77).

Therefore, when the operation mode is the 2D image capturing mode, when the modulation frequency Fmod is equal to or less than the predetermined threshold frequency Fmod_TH, or when the amount of received light is large, the driving signal GDA' and the Low level of GDB' is controlled to voltage VSS1. Conversely, when the operation mode is the ranging mode, when the modulation frequency Fmod is greater than the predetermined threshold frequency Fmod_TH, and when the amount of light received is not large, the drive signals GDA' and GDB' are at the Low level. is controlled to the voltage VSS.

The off-voltage control processing performed by the off-voltage control unit 101 will be described with reference to the flowchart of FIG. 12 . This processing is started, for example, when a measurement request is supplied from the control device 11 to the light receiving device 12 and the light receiving element 31 starts light receiving operation. Note that the initial value of the off-voltage _TGL of the transfer transistor 72 is set to the voltage VSS corresponding to high-speed driving.

First, in step S1, the off-voltage control unit 101 determines whether the operation mode is the 2D image shooting mode. If it is determined in step S1 that the operation mode is the 2D image shooting mode, the process proceeds to step S4, which will be described later.

On the other hand, if it is determined in step S1 that the operation mode is not the 2D image capturing mode but the operation mode is the ranging mode, the process proceeds to step S2, and the off-voltage control unit 101 sets the modulation frequency Fmod at a low speed. Yes, in other words, it is determined whether the modulation frequency Fmod is equal to or lower than a predetermined threshold frequency Fmod_TH. If it is determined in step S2 that the modulation frequency Fmod is low, the process proceeds to step S4, which will be described later.

On the other hand, if it is determined that the modulation frequency Fmod is not low (high), the process proceeds to step S3, and the off voltage control section 101 determines whether the amount of received light is large. Since the determination in step S3 requires acquisition of the depth value d for at least one frame, if it is determined in step S2 that the modulation frequency Fmod is high, the light receiving element 31 turns off the transfer transistor 72. The light receiving operation is performed with the voltage _TGL at the voltage VSS set as the initial value. Then, the off-voltage control unit 101 determines whether the amount of light received is large based on the light reception result of one frame. When a predetermined number or more or a predetermined ratio or more of pixel data is saturated with respect to the number of pixels from which pixel data is acquired in the pixel array section 52, it is determined that the amount of received light is large.

If it is determined in step S3 that the amount of received light is large, the process proceeds to step S4, and if it is determined that the amount of received light is not large, the process proceeds to step S5.

In step S4, the off-voltage control unit 101 controls the off-voltage _TGL of the transfer transistor 72 to be set to a voltage VSS1 lower than the voltage VSS.

In step S5, the off-voltage control unit 101 controls the off-voltage _TGL of the transfer transistor 72 to be set to the voltage VSS.

The off-voltage control process is completed as described above. The off-voltage control process of FIG. 12 can be repeatedly executed while the light receiving operation continues.

According to the off-voltage control processing by the off-voltage control unit 101, when the modulation frequency Fmod of the light emission pulse is higher than the predetermined threshold frequency Fmod_TH and the transfer transistor 72 is driven at high speed, the off-voltage TG _L of the transfer transistor 72 is set to the default (initial value) voltage VSS and controlled to shorten the transfer time.

On the other hand, when the modulation frequency Fmod of the light emission pulse is equal to or less than the predetermined threshold frequency Fmod_TH, when the operation mode is the 2D image capturing mode, or when the amount of received light is large, the off voltage TG of the transfer transistor 72 _L is set to a voltage VSS1 that is lower than voltage VSS. As a result, the charge storage amount of the photodiode 71 can be increased, and saturation can be prevented. In addition, it is possible to prevent the charge accumulated in the photodiode 71 from being transferred to the reverse tap (FD73) that is different from the transfer direction of the main body. Therefore, the light-receiving element 31 can prevent inflow of reverse taps at low speed while maintaining transfer efficiency at high speed, thereby improving distance measurement accuracy.

Note that if the charge storage amount of the photodiode 71 can be increased by setting the off-voltage _TGL of the transfer transistor 72 to the voltage VSS1, there is also an idea that it should always be set to the voltage VSS1. However, the power consumption increases as the drive voltage difference ΔTG=(ON voltage TG _H −OFF voltage TG _L ) increases. Therefore, if the OFF voltage TG _L is always set to the voltage VSS1, the power consumption increases. That is, the power consumption of the light-receiving element 31 can be reduced by setting the off-voltage _TGL at the time of high-speed driving to the voltage VSS as the basic control. In addition, since the operating power is proportional to the operating frequency, even if the operating voltage range is slightly widened, the power consumption of low-speed operation exceeds the power consumption of high-speed operation by suppressing the modulation frequency Fmod. no.

<7. Variation>
The following modifications can be applied to the light receiving element 31 .

Although the light-receiving element 31 described above has two transfer transistors 72 in each pixel 61, one or more transfer transistors 72 may be arranged in one pixel. For example, the same off-voltage control as described above can be applied to one transfer transistor 72 for one pixel or four transfer transistors 72 for one pixel, such as an image sensor for viewing. In the case of the light receiving element 31 having one transfer transistor 72 for one pixel, the detection signal VSLA of the first tap and the detection signal VSLB of the second tap are obtained in separate frames. In the case of the light receiving element 31 having four transfer transistors 72 in one pixel, it can be operated so as to obtain four phase detection signals VSL of 0°, 90°, 180° and 270° in one frame.

In the above-described embodiment, an example has been described in which two types of voltages VSS and VSS1 are provided as the OFF control voltage for the transfer transistor 72, and the voltage VSS and the voltage VSS1 are switched. However, the off control voltage for the transfer transistor 72 may have three or more voltage values. For example, there are three types of voltages VSS, VSS1, and VSS2 (VSS>VSS1>VSS2) as off-control voltages for the transfer transistor 72, and the modulation frequency Fmod is classified into three stages, and the amount of received light is classified into three stages. , and three types of OFF control voltages may be switched. Further, for example, the voltage VSS may be used for the distance measurement mode, and the voltages VSS1 and VSS2 may be used for the 2D image capturing mode, and the voltages VSS1 and VSS2 may be further switched according to the type of scene in the 2D image capturing mode.

In the above-described embodiment, the off-voltage control unit 101 is provided in the tap drive unit 54, but the off-voltage control unit 101 extends from the pulse generation circuit 53 that generates the drive signals GDA and GDB to the tap drive unit 54. may be placed anywhere along the path between

<8. Configuration example of electronic device>
The distance measuring system 1 described above can be installed in electronic devices such as smartphones, tablet terminals, mobile phones, personal computers, game machines, television receivers, wearable terminals, digital still cameras, and digital video cameras.

FIG. 13 is a block diagram showing a configuration example of a smartphone as an electronic device equipped with the ranging system of the present disclosure.

As shown in FIG. 13 , the smartphone 201 includes a ranging module 202 , an imaging device 203 , a display 204 , a speaker 205 , a microphone 206 , a communication module 207 , a sensor unit 208 , a touch panel 209 , and a control unit 210 via a bus 211 . connected and configured. The control unit 210 also has functions as an application processing unit 221 and an operating system processing unit 222 by executing programs by the CPU.

The light receiving device 12 and the illumination device 13 of the ranging system 1 of FIG. 1 are applied to the ranging module 202 . For example, the distance measurement module 202 is arranged on the front surface of the smartphone 201, and performs distance measurement for the user of the smartphone 201, thereby obtaining the depth value of the surface shape of the user's face, hands, fingers, etc. as a distance measurement result. can be output as The functions of the controller 11 of the ranging system 1 of FIG. 1 can be performed by the control unit 210 .

The imaging device 203 is arranged in front of the smartphone 201 and acquires an image of the user by imaging the user of the smartphone 201 as a subject. Note that the ranging module 202 and the imaging device 203 may be arranged on the back side of the smartphone 201, or may be arranged on both the front side and the back side.

The display 204 displays an operation screen for processing by the application processing unit 221 and the operation system processing unit 222, an image captured by the imaging device 203, and the like. The speaker 205 and the microphone 206 output the voice of the other party and pick up the voice of the user, for example, when making a call using the smartphone 201 .

A communication module 207 performs communication via a communication network. The sensor unit 208 senses speed, acceleration, proximity, and the like, and the touch panel 209 acquires a user's touch operation on the operation screen displayed on the display 204 .

The application processing unit 221 performs processing for providing various services using the smartphone 201 . For example, the application processing unit 221 can create a computer graphics face that virtually reproduces the user's facial expression based on the depth image supplied from the distance measurement module 202 and display the face on the display 204 . . Also, the application processing unit 221 can perform processing for creating three-dimensional shape data of an arbitrary three-dimensional object, for example, based on the depth image supplied from the distance measurement module 202 .

The operation system processing unit 222 performs processing for realizing basic functions and operations of the smartphone 201 . For example, the operation system processing unit 222 can authenticate the user's face and unlock the smartphone 201 based on the depth image supplied from the ranging module 202 . Further, the operation system processing unit 222 performs, for example, a process of recognizing a user's gesture based on the depth image supplied from the distance measurement module 202, and performs a process of inputting various operations according to the gesture. can be done.

In the smartphone 201 configured as described above, by incorporating the distance measurement system 1 described above, for example, distance measurement information can be calculated and a two-dimensional luminance image can be generated. Also, the accuracy of the ranging information can be improved.

<9. Example of application to moving objects>
The technology (the present technology) according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure can be realized as a device mounted on any type of moving body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility, airplanes, drones, ships, and robots. may

FIG. 14 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology according to the present disclosure can be applied.

Vehicle control system 12000 comprises a plurality of electronic control units connected via communication network 12001 . In the example shown in FIG. 14, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an exterior information detection unit 12030, an interior information detection unit 12040, and an integrated control unit 12050. Also, as the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio/image output unit 12052, and an in-vehicle network I/F (interface) 12053 are illustrated.

Drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the driving system control unit 12010 includes a driving force generator for generating driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, and a steering angle of the vehicle. It functions as a control device such as a steering mechanism to adjust and a brake device to generate braking force of the vehicle.

Body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as headlamps, back lamps, brake lamps, winkers or fog lamps. In this case, the body system control unit 12020 can receive radio waves transmitted from a portable device that substitutes for a key or signals from various switches. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, etc. of the vehicle.

External information detection unit 12030 detects information external to the vehicle in which vehicle control system 12000 is mounted. For example, the vehicle exterior information detection unit 12030 is connected with an imaging section 12031 . The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image of the exterior of the vehicle, and receives the captured image. The vehicle exterior information detection unit 12030 may perform object detection processing or distance detection processing such as people, vehicles, obstacles, signs, or characters on the road surface based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electric signal according to the amount of received light. The imaging unit 12031 can output the electric signal as an image, and can also output it as distance measurement information. Also, the light received by the imaging unit 12031 may be visible light or non-visible light such as infrared rays.

The vehicle interior information detection unit 12040 detects vehicle interior information. The in-vehicle information detection unit 12040 is connected to, for example, a driver state detection section 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 detects the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041. It may be calculated, or it may be determined whether the driver is dozing off.

The microcomputer 12051 calculates control target values for the driving force generator, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and controls the drive system control unit. A control command can be output to 12010 . For example, the microcomputer 12051 realizes functions of ADAS (Advanced Driver Assistance System) including collision avoidance or shock mitigation of vehicles, follow-up driving based on inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane deviation warning, etc. Cooperative control can be performed for the purpose of

In addition, the microcomputer 12051 controls the driving force generator, the steering mechanism, the braking device, etc. based on the information about the vehicle surroundings acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, so that the driver's Cooperative control can be performed for the purpose of autonomous driving, etc., in which vehicles autonomously travel without depending on operation.

Further, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the information detection unit 12030 outside the vehicle. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or the oncoming vehicle detected by the vehicle exterior information detection unit 12030, and performs cooperative control aimed at anti-glare such as switching from high beam to low beam. It can be carried out.

The audio/image output unit 12052 transmits at least one of audio and/or image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle. In the example of FIG. 14, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

FIG. 15 is a diagram showing an example of the installation position of the imaging unit 12031. As shown in FIG.

In FIG. 15 , vehicle 12100 has imaging units 12101 , 12102 , 12103 , 12104 , and 12105 as imaging unit 12031 .

The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose, side mirrors, rear bumper, back door, and windshield of the vehicle 12100, for example. An image pickup unit 12101 provided in the front nose and an image pickup unit 12105 provided above the windshield in the passenger compartment mainly acquire images in front of the vehicle 12100 . Imaging units 12102 and 12103 provided in the side mirrors mainly acquire side images of the vehicle 12100 . An imaging unit 12104 provided in the rear bumper or back door mainly acquires an image behind the vehicle 12100 . Forward images acquired by the imaging units 12101 and 12105 are mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

Note that FIG. 15 shows an example of the imaging range of the imaging units 12101 to 12104 . The imaging range 12111 indicates the imaging range of the imaging unit 12101 provided in the front nose, the imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided in the side mirrors, respectively, and the imaging range 12114 The imaging range of an imaging unit 12104 provided on the rear bumper or back door is shown. For example, by superimposing the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 determines the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and changes in this distance over time (relative velocity with respect to the vehicle 12100). , it is possible to extract, as the preceding vehicle, the closest three-dimensional object on the course of the vehicle 12100, which runs at a predetermined speed (for example, 0 km/h or more) in substantially the same direction as the vehicle 12100. can. Furthermore, the microcomputer 12051 can set the inter-vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including following stop control) and automatic acceleration control (including following start control). In this way, cooperative control can be performed for the purpose of automatic driving in which the vehicle runs autonomously without relying on the operation of the driver.

For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 converts three-dimensional object data related to three-dimensional objects to other three-dimensional objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, and utility poles. It can be classified and extracted and used for automatic avoidance of obstacles. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into those that are visible to the driver of the vehicle 12100 and those that are difficult to see. Then, the microcomputer 12051 judges the collision risk indicating the degree of danger of collision with each obstacle, and when the collision risk is equal to or higher than the set value and there is a possibility of collision, an audio speaker 12061 and a display unit 12062 are displayed. By outputting an alarm to the driver via the drive system control unit 12010 and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be performed.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not the pedestrian exists in the captured images of the imaging units 12101 to 12104 . Such recognition of a pedestrian is performed by, for example, a procedure for extracting feature points in images captured by the imaging units 12101 to 12104 as infrared cameras, and performing pattern matching processing on a series of feature points indicating the outline of an object to determine whether or not the pedestrian is a pedestrian. This is done by a procedure that determines
When the microcomputer 12051 determines that a pedestrian exists in the images captured by the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 outputs a rectangular outline for emphasis to the recognized pedestrian. is superimposed on the display unit 12062 . Also, the audio/image output unit 12052 may control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the vehicle exterior information detection unit 12030 and the vehicle interior information detection unit 12040 among the configurations described above. Specifically, by using the distance measurement information from the distance measurement system 1 as the vehicle exterior information detection unit 12030 and the vehicle interior information detection unit 12040, a process of recognizing the driver's gesture is performed, and various types (for example, , audio system, navigation system, air conditioning system) and more accurately detect the driver's condition. In addition, by using the distance measurement information from the distance measurement system 1, it is possible to measure the distance to an obstacle and use it for collision prevention judgment, or to recognize the unevenness of the road surface and reflect it in the control of the suspension. .

The embodiments of the present disclosure are not limited to the embodiments described above, and various modifications are possible without departing from the gist of the technology of the present disclosure.

Each of the techniques described in this specification can be implemented independently and singly unless inconsistent. Of course, it is also possible to use any number of the present techniques in combination. For example, part or all of the present technology described in any embodiment can be combined with part or all of the present technology described in other embodiments. Also, part or all of any of the techniques described above may be implemented in conjunction with other techniques not described above.

Further, for example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). Further, it is of course possible to add a configuration other than the above to the configuration of each device (or each processing unit). Furthermore, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit) as long as the configuration and operation of the system as a whole are substantially the same. .

Furthermore, in this specification, a system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device housing a plurality of modules in one housing, are both systems. .

Note that the effects described in this specification are merely examples and are not limited, and there may be effects other than those described in this specification.

In addition, the technique of this disclosure can take the following configurations.
(1)
a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit;
A light-receiving element comprising: a voltage control section that switches and controls a plurality of voltage values as an off-control voltage for turning off the transfer transistor.
(2)
The pixels are
two said transfer transistors, first and second transfer transistors;
a first charge storage unit that holds the charge transferred by the first transfer transistor;
a second charge storage unit that holds the charge transferred by the second transfer transistor;
The photoelectric conversion unit receives and photoelectrically converts reflected light emitted from a predetermined light source and reflected by an object.
(1) above, wherein the voltage control unit switches between a first voltage and a second voltage lower than the first voltage as the off-control voltages for the first and second transfer transistors. Light receiving element.
(3)
the first and second transfer transistors are alternately turned on at a predetermined modulation frequency;
(2) above, wherein the voltage control unit switches the off-control voltages of the first and second transfer transistors to the first voltage or the second voltage according to the predetermined modulation frequency light receiving element.
(4)
the predetermined modulation frequencies include a first modulation frequency and a second modulation frequency slower than the first modulation frequency;
The voltage control unit switches the off-control voltage to the first voltage when the modulation frequency is the first modulation frequency, and switches the off-control voltage to the second voltage when the modulation frequency is the second modulation frequency. The light-receiving element according to (3), wherein the light-receiving element is switched to a voltage.
(5)
The voltage control unit switches the OFF control voltage to the first voltage when the predetermined modulation frequency is greater than a predetermined threshold frequency, and switches the OFF control voltage to the first voltage when the predetermined modulation frequency is less than or equal to a predetermined threshold frequency. In addition, the off-control voltage is switched to the second voltage. The light receiving element according to (3).
(6)
The voltage control unit switches the off-control voltages of the first and second transfer transistors to the first voltage or the second voltage according to the operation mode of (2) to (5). The light receiving element according to any one of the above.
(7)
The voltage control unit switches the off-control voltage to the first voltage when the operation mode is a ranging mode for measuring a distance, and the operation mode is 2D image capturing for generating a two-dimensional luminance image. The light-receiving element according to (6), wherein the off-control voltage is switched to the second voltage when in the mode.
(8)
The voltage control unit switches the off-control voltages of the first and second transfer transistors to the first voltage or the second voltage according to the amount of received light. The light receiving element according to any one of the above.
(9)
The voltage control unit switches the off-control voltage to the first voltage when it is determined that the amount of received light is not large, and when it is determined that the amount of received light is large, The light receiving element according to (8), wherein the OFF control voltage is switched to the second voltage.
(10)
The above (9), wherein when it is determined that the amount of received light is large, a predetermined number or more or a predetermined ratio or more of pixel data with respect to the number of pixels from which pixel data is acquired is saturated. Light receiving element.
(11)
The voltage control unit according to any one of (1) to (10) above, including a transistor that switches between a first voltage and a second voltage lower than the first voltage as the off-control voltage. Light receiving element.
(12)
A light-receiving element comprising a pixel having a photoelectric conversion unit and a transfer transistor for transferring charges generated by the photoelectric conversion unit,
A method of driving a light-receiving element, comprising switching between a plurality of voltage values as an off-control voltage for turning off the transfer transistor.
(13)
a light emitting unit that emits irradiation light;
a light-receiving element that receives the reflected light returned by the irradiation light reflected by an object,
The light receiving element is
a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit;
A distance measuring system, comprising: a voltage control section that switches and controls a plurality of voltage values as an off-control voltage for turning off the transfer transistor.

1: Ranging System, 11: Control Device, 12: Light Receiving Device, 13: Lighting Device, 21: LD, 22: Light Emitting Section, 31: Light Receiving Element, 51: Control Section, 52: Pixel Array Section, 53: Pulse Generation Circuit 54: Tap drive section 58: Signal processing section 61: Pixel 64A, 64B: Vertical signal line 71: Photodiode 72A, 72B: Transfer transistor 74A, 74B: FD gate transistor 75A, 75B: Amplification transistors, 76A, 76B: reset transistors, 77A, 77B: selection transistors, 101: OFF voltage control unit, 111: OR circuit, 112: inverter, 113: transistor, 114: transistor, 201: smart phone, 202: ranging module

Claims (13)

a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit;
A light-receiving element comprising: a voltage control section that switches and controls a plurality of voltage values as an off-control voltage for turning off the transfer transistor. The pixels are
two said transfer transistors, first and second transfer transistors;
a first charge storage unit that holds the charge transferred by the first transfer transistor;
a second charge storage unit that holds the charge transferred by the second transfer transistor;
The photoelectric conversion unit receives and photoelectrically converts reflected light emitted from a predetermined light source and reflected by an object.
The light receiving according to claim 1, wherein the voltage control section switches between a first voltage and a second voltage lower than the first voltage as the off control voltages for the first and second transfer transistors. element. the first and second transfer transistors are alternately turned on at a predetermined modulation frequency;
3. The voltage control unit according to claim 2, wherein the off control voltage of the first and second transfer transistors is switched to the first voltage or the second voltage according to the predetermined modulation frequency. Light receiving element. the predetermined modulation frequencies include a first modulation frequency and a second modulation frequency slower than the first modulation frequency;
The voltage control unit switches the off-control voltage to the first voltage when the modulation frequency is the first modulation frequency, and switches the off-control voltage to the second voltage when the modulation frequency is the second modulation frequency. The light-receiving element according to claim 3, wherein the light-receiving element is switched to a voltage. The voltage control unit switches the OFF control voltage to the first voltage when the predetermined modulation frequency is greater than a predetermined threshold frequency, and switches the OFF control voltage to the first voltage when the predetermined modulation frequency is less than or equal to a predetermined threshold frequency. 4. The light receiving element according to claim 3, wherein the OFF control voltage is switched to the second voltage at the same time. 3. The light receiving element according to claim 2, wherein the voltage control section switches the off-control voltages of the first and second transfer transistors to the first voltage or the second voltage according to an operation mode. The voltage control unit switches the off-control voltage to the first voltage when the operation mode is a ranging mode for measuring a distance, and the operation mode is 2D image capturing for generating a two-dimensional luminance image. 7. The light receiving element according to claim 6, wherein the off control voltage is switched to the second voltage when in the mode. 3. The light receiving element according to claim 2, wherein the voltage control section switches the off control voltages of the first and second transfer transistors to the first voltage or the second voltage according to the amount of light received. The voltage control unit switches the off-control voltage to the first voltage when it is determined that the amount of received light is not large, and when it is determined that the amount of received light is large, The light receiving element according to claim 8, wherein the OFF control voltage is switched to the second voltage. 10. The light receiving according to claim 9, wherein when the amount of received light is determined to be a large amount of light, a predetermined number or more or a predetermined ratio or more of pixel data with respect to the number of pixels from which pixel data is obtained is saturated. element. The light receiving element according to claim 1, wherein the voltage control section has a transistor that switches between a first voltage and a second voltage lower than the first voltage as the off control voltage. A light-receiving element comprising a pixel having a photoelectric conversion unit and a transfer transistor for transferring charges generated by the photoelectric conversion unit,
A method of driving a light-receiving element, comprising switching a plurality of voltage values as an off-control voltage for turning off the transfer transistor. a light emitting unit that emits irradiation light;
a light-receiving element that receives the reflected light returned by the irradiation light reflected by an object,
The light receiving element is
a pixel having a photoelectric conversion unit and a transfer transistor that transfers charges generated by the photoelectric conversion unit;
A distance measuring system, comprising: a voltage control section that switches and controls a plurality of voltage values as an off-control voltage for turning off the transfer transistor.

Images