JP2022183614A - Distance image photographing device and distance image photographing method - Google Patents

Abstract

To provide a distance image photographing device and distance image photographing method capable of measuring a distance to an object at a short distance and determining presence/absence of an object at a long distance.SOLUTION: A distance image photographing device includes a light source, a light reception section, an electric charge discharge section, and a distance image processing section. The distance image processing section executes unit accumulation processing a plurality of times in one frame period, and determines a measurement distance to a subject with the use of electric charge amounts accumulated respectively in three or more electric charge accumulation sections. The unit accumulation processing comprises: a short-distance object measurement process for allowing a part of the electric charge accumulation section among the three or more electric charge accumulation sections to distribute and accumulate electric charges corresponding to reflection light being an optical pulse reflected against the subject; a discharge process for allowing the electric charge discharge sections to discharge the electric charges generated by photoelectric conversion elements; and a long-distance object measurement process for allowing the electric charge accumulation sections including the electric charge accumulation section different from the part of the electric charge accumulation sections to accumulate the electric charges corresponding to the reflection light. In the unit accumulation processing, the processes are sequentially executed.SELECTED DRAWING: Figure 1

Description

The present invention relates to a range image capturing device and a range image capturing method.

Conventionally, as a technique for measuring the distance to an object, there is a technique for measuring the flight time of light pulses. Such a technique is called Time of Flight (hereinafter referred to as ToF). ToF utilizes the fact that the speed of light is known, and irradiates an object with light pulses in the near-infrared region. Then, the time difference between the time when the light pulse is applied and the time when the reflected light of the applied light pulse is received by the object is measured. The distance to the object is calculated based on this time difference. A distance measuring sensor (ToF sensor) that detects light for measuring distance using a photodiode (photoelectric conversion element) has been put to practical use.

In recent years, distance measuring sensors have been put into practical use that can obtain depth information for each pixel in a two-dimensional image including the object, that is, three-dimensional information about the object, in addition to the distance to the object. Such a distance measuring sensor is also called a distance image pickup device. In the depth image pickup device, a plurality of pixels including photodiodes are arranged in a two-dimensional matrix on a silicon substrate, and light reflected by an object is received by the pixel surface. A depth image pickup device outputs a photoelectric conversion signal for one image based on the amount of light (charge) received by each pixel. distance information can be obtained. For example, Japanese Patent Application Laid-Open No. 2002-200003 discloses a technique in which three charge storage units are provided in one pixel and the distance is calculated based on the amount of charge stored in each charge storage unit.

Japanese Patent No. 4235729

In a ToF sensor, in order to improve the measurement accuracy when measuring the distance to an object at a short distance, it is conceivable to shorten the irradiation time of the light pulse and the accumulation time for accumulating charges in the charge accumulation unit. However, in this case, the measurable distance range is shortened. As a countermeasure, it is conceivable to increase the number of charge storage units in order to extend the measurable distance range while maintaining the relationship between the irradiation time and the storage time that improves the measurement accuracy. However, due to restrictions on the area occupied by one pixel, there is a limit to the number of charge storage units, and it has been difficult to increase the measurable distance range while increasing the measurement accuracy of an object at a short distance. In particular, when measuring the distance to an object around the moving body using a range imaging device mounted on a moving body that moves at a high speed, it is possible to measure not only the distance to an object at a short distance, but also It is necessary to control the moving body by detecting the presence or absence of an object at a long distance that is relatively approaching by moving the body and taking avoidance action. Therefore, there is a demand for a range imaging apparatus that can determine the presence or absence of an object at a long distance while measuring the distance to an object at a short distance with high accuracy.

SUMMARY OF THE INVENTION The present invention has been made based on the above problems, and provides a distance image capturing apparatus capable of measuring the distance to an object at a short distance and determining the presence or absence of an object at a long distance. It is an object of the present invention to provide an image capturing method.

The distance image pickup device of the present invention comprises a light source unit that irradiates a light pulse into a measurement space where an object exists, a photoelectric conversion element that generates electric charges according to the incident light, and three or more electric charges that accumulate the electric charges. a light-receiving unit comprising: a pixel having an accumulating portion; and a pixel driving circuit that distributes and accumulates the charge in each of the charge accumulating portions of the pixel at a predetermined timing synchronized with irradiation of the light pulse; a charge discharging unit for discharging charges generated by photoelectric conversion elements; and a distance image processing unit for determining a measured distance to the subject using the amount of charge accumulated in each of the charge accumulating units, The distance image processing unit performs unit accumulation processing a plurality of times in one frame period, determines the measured distance to the subject using the amount of charge accumulated in each of the three or more charge accumulation units, and In the unit accumulation processing, short-distance object measurement processing for distributing and accumulating electric charges according to the reflected light, which is the light pulse reflected by the subject, in some of the three or more electric charge accumulation units; discharge processing in which the charge discharge unit discharges the charge generated by the photoelectric conversion element; Distance object measurement processing is performed in order.

The distance image capturing method of the present invention includes a light source unit that irradiates a light pulse into a measurement space in which an object exists, a photoelectric conversion element that generates charges according to the incident light, and three or more charges that accumulate the charges. a light-receiving unit comprising: a pixel having an accumulating portion; and a pixel driving circuit that distributes and accumulates the charge in each of the charge accumulating portions of the pixel at a predetermined timing synchronized with irradiation of the light pulse; A distance image, comprising: a charge discharging unit for discharging charges generated by a photoelectric conversion element; and a distance image processing unit for determining a measured distance to the subject using the amount of charge accumulated in each of the charge accumulating units. In a range image capturing method using an image capturing device, the range image processing unit performs unit accumulation processing a plurality of times in one frame period, and uses the charge amount accumulated in each of the three or more charge accumulation units. to determine the measurement distance to the object; A short-distance object measurement process of distributing and accumulating charges, a discharge process of discharging the charges generated by the photoelectric conversion element by the charge discharging section, and a charge accumulating section including a charge accumulating section different from the part of the charge accumulating sections. , long-distance object measurement processing for accumulating charges corresponding to the reflected light is sequentially performed.

According to the present invention, it is possible to measure the distance to an object at a short distance and determine the presence or absence of an object at a long distance.

1 is a block diagram showing a schematic configuration of a distance imaging device 1 of an embodiment; FIG. 3 is a block diagram showing a schematic configuration of a distance image sensor 32 of the embodiment; FIG. 3 is a circuit diagram showing an example of the configuration of a pixel 321 of the embodiment; FIG. 4 is a timing chart showing an example of conventional driving timing; 4 is a timing chart showing an example of drive timings according to the first embodiment; 4 is a flow chart showing the flow of processing performed by a distance image processing unit 4 of the first embodiment; 9 is a timing chart showing an example of drive timings according to the second embodiment; 9 is a flow chart showing the flow of processing performed by a distance image processing unit 4 of the second embodiment; FIG. 11 is a timing chart showing an example of drive timings of modification 1 according to the second embodiment; FIG. 9 is a flow chart showing the flow of processing performed by a distance image processing unit 4 of Modification 1 according to the second embodiment. FIG. 11 is a timing chart showing an example of drive timings of modification 2 according to the second embodiment; FIG. 10 is a flow chart showing the flow of processing performed by a distance image processing unit 4 of modification 2 according to the second embodiment.

Hereinafter, the distance image capturing device of the embodiment will be described with reference to the drawings.

(Basic configuration of embodiment)
First, the basic configuration of the embodiment will be described. FIG. 1 is a block diagram showing a schematic configuration of a distance image pickup device according to an embodiment of the present invention. A distance image pickup device 1 configured as shown in FIG. FIG. 1 also shows an object OB, which is an object whose distance is to be measured in the distance image pickup device 1 .

Under the control of the distance image processing unit 4, the light source unit 2 irradiates the space of the image pickup object in which the object OB whose distance is to be measured in the distance image pickup device 1 exists with a light pulse PO. The light source unit 2 is, for example, a surface emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL: Vertical Cavity Surface Emitting Laser). The light source unit 2 includes a light source device 21 and a diffuser plate 22 .

The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) as a light pulse PO to irradiate the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting device. The light source device 21 emits pulsed laser light under the control of the timing control section 41 .

The diffuser plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 over a surface that irradiates the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO, and is irradiated onto the object OB.

The light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the object OB whose distance is to be measured in the distance image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving section 3 includes a lens 31 and a distance image sensor 32 .

The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32 . The lens 31 emits the incident reflected light RL to the distance image sensor 32 side, and causes pixels provided in the light receiving area of the distance image sensor 32 to receive the light (incident).

The distance image sensor 32 is an image pickup device used in the distance image pickup device 1 . The distance image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the distance image sensor 32 is provided with one photoelectric conversion element, a plurality of charge storage units corresponding to this one photoelectric conversion element, and a component for distributing the charge to each charge storage unit. . In other words, the pixel is an imaging device having a distribution structure in which charges are distributed and accumulated in a plurality of charge accumulation units.

The distance image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge storage units according to control from the timing control unit 41 . Also, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage section. A plurality of pixels are arranged in a two-dimensional matrix in the range image sensor 32, and pixel signals for one frame corresponding to each pixel are output.

The distance image processing unit 4 controls the distance image capturing device 1 and calculates the distance to the subject OB. The distance image processing section 4 includes a timing control section 41 , a distance calculation section 42 , a measurement control section 43 and a storage section 44 . Note that part of the functional units (timing control unit 41 , distance calculation unit 42 , measurement control unit 43 , and storage unit 44 ) of distance image processing unit 4 may be incorporated in distance image sensor 32 .

The timing control section 41 controls the timing of outputting various control signals required for measurement under the control of the measurement control section 43 . The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for allocating and accumulating the reflected light RL to a plurality of charge accumulation units, and controlling the number of times of distribution (number of times of accumulation) per frame. such as a signal to The number of distributions is the number of repetitions of the process of distributing charges to the charge storage section CS (see FIG. 3).

The distance calculator 42 calculates the ToF distance using the pixel signal output from the distance image sensor 32 and Equation (1), which will be described later. The distance calculator 42 corrects the calculated ToF distance using the correction information 440, and uses the corrected ToF distance as the distance (measured distance) to the subject OB. The correction information 440 and the method by which the distance calculator 42 corrects the ToF distance using the correction information 440 will be described later in detail.

The measurement control section 43 controls the timing control section 41 . For example, the measurement control unit 43 sets the number of allocation times for one frame, the accumulation time Ta, and the like, and controls the timing control unit 41 so that imaging is performed according to the set contents.

The storage unit 44 is a storage medium such as a HDD (Hard Disk Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read/write Memory), ROM (Read Only Memory), or any of these any combination of storage media.

With such a configuration, in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL that is reflected by the object OB from the light pulse PO in the near-infrared wavelength band that the light source unit 2 irradiates the object OB. A distance image processing unit 4 outputs distance information obtained by measuring the distance to the object OB.

Although FIG. 1 shows the distance image pickup device 1 having a configuration in which the distance image processing unit 4 is provided inside, the distance image processing unit 4 is a component provided outside the distance image pickup device 1. may

Next, the configuration of the distance image sensor 32 used as an imaging device in the distance image pickup device 1 will be described. FIG. 2 is a block diagram showing a schematic configuration of an imaging device (distance image sensor 32) used in the distance image pickup device 1 of the embodiment.

As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a sorting operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325 .

The light-receiving region 320 is a region in which a plurality of pixels 321 are arranged, and FIG. 2 shows an example in which the pixels are arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixel 321 accumulates charges corresponding to the amount of light received. A control circuit 322 controls the distance image sensor 32 in an integrated manner. The control circuit 322 controls the operation of the components of the range image sensor 32 according to instructions from the timing control section 41 of the range image processing section 4, for example. It should be noted that the components provided in the distance image sensor 32 may be controlled directly by the timing control section 41, in which case the control circuit 322 may be omitted.

The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 for each row according to control from the control circuit 322 . The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge accumulated in each charge accumulation portion CS of the pixel 321 . In this case, the vertical scanning circuit 323 distributes the charges converted by the photoelectric conversion elements to the respective charge accumulating portions of the pixels 321 . That is, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

The pixel signal processing circuit 325 performs predetermined signal processing (for example, noise suppression processing) on the voltage signals output to the corresponding vertical signal lines from the pixels 321 in each column under the control of the control circuit 322 . , A/D conversion processing, etc.).

The horizontal scanning circuit 324 is a circuit that sequentially outputs signals output from the pixel signal processing circuit 325 to horizontal signal lines under the control of the control circuit 322 . As a result, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to the distance image processing section 4 via the horizontal signal line.

In the following description, it is assumed that the pixel signal processing circuit 325 performs A/D conversion processing and the pixel signal is a digital signal.

Here, the configuration of the pixels 321 arranged in the light receiving area 320 provided in the distance image sensor 32 will be described. FIG. 3 is a circuit diagram showing an example of the configuration of the pixels 321 arranged within the light receiving area 320 of the distance image sensor 32 of the embodiment. FIG. 3 shows an example of the configuration of one pixel 321 among the plurality of pixels 321 arranged in the light receiving region 320. As shown in FIG. In the example of this figure, the pixel 321 is an example of a configuration including four pixel signal reading units.

As shown in FIG. 3, the pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and four pixel signal readout units RU (pixel signal readout units RU1 to RU4). Each pixel signal readout unit RU outputs a voltage signal from an output terminal O. FIG.

In the following description, each pixel signal readout unit RU is distinguished by adding a number "1", "2", "3" or "4" after the code of the four pixel signal readout units RU. do. Similarly, each constituent element provided in the four pixel signal readout units RU is distinguished by adding a numeral after each reference numeral.

Each pixel signal readout unit RU includes a charge distribution gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a select gate transistor SL. In each pixel signal readout unit RU, the floating diffusion FD and the charge storage capacitor C constitute a charge storage unit CS. Specifically, the pixel signal readout unit RU1 includes a charge distribution gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, a source follower gate transistor SF1, and a select gate transistor SL1. . In the pixel signal readout unit RU1, the charge storage unit CS1 is composed of the floating diffusion FD1 and the charge storage capacitor C1. The pixel signal readout units RU2 to RU4 also have the same configuration. The configuration of the charge distribution gate transistor G is not limited to the transfer method, and may be a photogate method for charge distribution.

The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate charges and accumulate the generated charges. The structure of the photoelectric conversion element PD may be arbitrary. The photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. It may be a PIN photodiode.

In the pixel 321, the charges generated by photoelectrically converting the light incident on the photoelectric conversion element PD are distributed to the four charge storage units CS, respectively, and voltage signals corresponding to the amounts of the distributed charges are output to the pixels. Output to the signal processing circuit 325 .

The configuration of the pixels arranged in the distance image sensor 32 is not limited to the configuration provided with four pixel signal readout units RU as shown in FIG. Any pixel of the configuration may be used. That is, the number of pixel signal readout units RU (charge storage units CS) provided for the pixels arranged in the distance image sensor 32 may be two, three, or five or more. There may be.

Also, in the pixel 321 having the configuration shown in FIG. 3, an example in which the charge storage portion CS is configured by the floating diffusion FD and the charge storage capacitor C is shown. However, the charge storage section CS may be composed of at least the floating diffusion FD, and the pixel 321 may be configured without the charge storage capacitor C. FIG.

Further, although the pixel 321 having the configuration shown in FIG. 3 has an example of the configuration including the drain gate transistor GD, the configuration is not limited to this. For example, when it is not necessary to discard the charge remaining in the photoelectric conversion element PD without being accumulated in the charge accumulation section CS, the configuration without the drain gate transistor GD may be employed.

(conventional drive timing)
Here, the conventional driving timing will be described with reference to FIG. FIG. 4 is a timing chart showing conventional drive timings (timings for driving pixels).

In FIG. 4, the time required for accumulating charges in each of the charge accumulating units CS in one sorting process is represented as "unit accumulation time UT". After repeating the sorting process (unit accumulation process) performed in the "unit accumulation time UT" for the number of times of accumulation corresponding to one frame, the process of reading out the amount of charge accumulated during that period is performed. The time during which the process of reading out the accumulated charge amount is expressed as a “readout period”.

In FIG. 4, "L" is the timing for irradiating the light pulse PO, "R" is the timing for receiving the reflected light RL, "G1" is the timing for driving the charge distribution gate transistor G1, and "G1" is the timing for driving the charge distribution gate transistor G2. "G2" for the timing to drive the charge distribution gate transistor G3, "G3" for the timing for driving the charge distribution gate transistor G3, "G4" for the timing for driving the charge distribution gate transistor G4, and "GD" for the timing of the drive signal RSTD. , respectively.

The vertical scanning circuit 323 accumulates charges in the charge accumulation units CS1 to CS4 at timing synchronized with the irradiation of the light pulse PO. In the example of FIG. 4, charges are accumulated in the charge storage section CS1 at the same timing as the timing of irradiating the light pulse PO, and after the charges are accumulated in the charge storage section CS1, charges are sequentially accumulated in the charge storage sections CS2 to CS4. Accumulate.

The example of FIG. 4 shows a timing chart when the reflected light RL is received by the distance image sensor 32 with a delay time Td after the time when the light pulse PO is emitted. Charges corresponding to the reflected light RL are distributed to and accumulated in the charge accumulation units CS1 and CS2 according to the delay time Td. At the timing when the charge accumulating units CS3 and CS4 accumulate charges, the reflected light RL is not received, and charges corresponding to external light components such as background light are accumulated in the charge accumulating units CS3 and CS4.

Specifically, first, the vertical scanning circuit 323 irradiates the light pulse PO. The vertical scanning circuit 323 turns off the drain gate transistor GD at the same timing as the irradiation timing, and turns on the charge distribution gate transistor G1 for the accumulation time Ta. After turning on the charge distribution gate transistor G1 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G1. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the charge distribution gate transistor G1 is controlled to be on is accumulated in the charge storage section CS1 via the charge distribution gate transistor G1.

Next, the vertical scanning circuit 323 turns on the charge distribution gate transistor G2 for the accumulation time Ta at the timing when the charge distribution gate transistor G2 is turned off. After turning on the charge distribution gate transistor G2 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G2. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the charge distribution gate transistor G2 is controlled to be on is accumulated in the charge storage section CS2 via the charge distribution gate transistor G2.

Next, the vertical scanning circuit 323 turns on the charge distribution gate transistor G3 for the accumulation time Ta at the timing when the charge distribution gate transistor G2 is turned off. After turning on the charge distribution gate transistor G3 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G3. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the charge distribution gate transistor G3 is controlled to be on is accumulated in the charge storage section CS3 via the charge distribution gate transistor G3.

Next, the vertical scanning circuit 323 turns on the charge distribution gate transistor G4 for the accumulation time Ta at the timing when the charge accumulation in the charge accumulation section CS3 is completed. After turning on the charge distribution gate transistor G4 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G4. The vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge distribution gate transistor G4 is turned off. By turning on the drain gate transistor GD, the charges photoelectrically converted by the photoelectric conversion element PD during this period are discarded via the drain gate transistor GD without being accumulated in the charge storage unit CS.

The vertical scanning circuit 323 repeats the above-described driving a predetermined number of times over one frame. After that, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in each charge accumulation section CS. Specifically, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in the charge accumulation unit CS1 via the pixel signal readout unit RU1 by turning on the selection gate transistor SL1 for a predetermined time. Output from O1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 to SL4 to output voltage signals corresponding to the amounts of charge accumulated in the charge accumulation units CS2 to CS4 from the output terminals O2 to O4. Let As a result, an electric signal corresponding to the amount of charge for one frame accumulated in each of the charge accumulation units CS is output to the distance calculation unit 42 .

In the above description, the case where the charge distribution gate transistor G1 is turned on at the timing when the light pulse PO is irradiated has been described as an example. However, it is not limited to this. The light pulse PO may be applied at the timing at which the charges according to at least the reflected light RL are distributed and accumulated in the charge accumulation units CS1 and CS2, CS2, or CS3.

In FIG. 4, from the relationship between the timing of irradiating the light pulse PO, the timing of receiving the reflected light RL, and the timing of accumulating charges in each of the charge accumulating portions CS, reflected light is applied to the charge accumulating portions CS1 and CS2. A charge amount corresponding to RL is distributed and accumulated. Further, the charge amount corresponding to the external light component is accumulated in the charge accumulation units CS1 to CS4.

The distribution (distribution ratio) of the amount of charge distributed to the charge storage units CS1 and CS2 is a ratio according to the delay time Td from the irradiation time of the light pulse PO to the reception time of the reflected light RL. Using this principle, the distance calculator 42 calculates the delay time Td by the following equation (1). R in the formula (1) is a charge ratio indicating a distribution ratio of the reflected light RL. Equation (1) assumes that the charge amount corresponding to the external light component accumulated in each of the charge accumulation units CS1 to CS3 is the same.

Td=To×R (1)
However, R=(Q2-Q3)/(Q1+Q2-2×Q3)
To is the time interval during which the optical pulse PO is irradiated
Q1 is the amount of charge accumulated in the charge accumulation section CS1
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Q3 is the charge amount accumulated in the charge accumulation section CS3

The distance calculation unit 42 multiplies the delay time Td obtained by the formula (1) by the speed of light (velocity) in the short-distance light-receiving pixel, thereby calculating the round-trip distance to the object OB. Then, the distance calculation unit 42 obtains the ToF distance by halving the round trip distance calculated above.

(Drive timing of the first embodiment)
Here, the driving timing of the first embodiment will be described with reference to FIG. FIG. 5 is a timing chart showing driving timings of the pixels 321 in the first embodiment.

FIG. 5 shows a timing chart of the sorting process (unit accumulation process) performed in the unit accumulation time UT of this embodiment. In FIG. 5, as in FIG. 4, the timing for driving the charge distribution gate transistor G1 is "G1", the timing for driving the charge distribution gate transistor G2 is "G2", and the timing for driving the charge distribution gate transistor G3 is "G3". ”, “G4” for the timing of driving the charge distribution gate transistor G4, and “GD” for the timing of the drive signal RSTD. Also, here, at the start of the unit accumulation time UT, each of the charge distribution gate transistors G1 to G4 is in an off state, and the drain gate transistor GD is in an on state.

In the present embodiment, short-distance object measurement processing, discharge processing, and long-distance object measurement processing are sequentially performed in the unit accumulation processing. Each process will be described below.

First, short-distance object measurement processing will be described. The short-distance object measurement process is a process of measuring the distance to an object at a relatively short distance, and is performed by driving the charge storage units CS1 to CS3 of the pixels 321 at conventional drive timings.

Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD at a timing corresponding to the irradiation of the light pulse PO, for example, at the same timing as the irradiation of the light pulse PO, to turn off the drain gate transistor GD and the charge distribution gate. The transistors G1 to G3 are sequentially turned on for the accumulation time Ta.

Next, the discharge process will be explained. The discharge process is a process for discharging charges generated by the photoelectric conversion element PD, and is performed by turning on the drain gate transistor GD.

Specifically, the vertical scanning circuit 323 turns on the drain gate transistor GD for a certain time (time corresponding to zone Z3 in the example of this figure) at the timing when the charge accumulation in the charge accumulation unit CS3 is completed. state.

Next, long-distance object measurement processing will be described. The long-distance object measurement process is a process of determining the presence or absence of an object existing at a long distance. The long-distance object measurement process is performed by turning on the charge distribution gate transistor G4 after the discharge process.

Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the charge distribution gate transistor G4 for a certain period of time (accumulation time Tb in the example of this figure). After that, the vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge accumulation unit CS4 is finished.

Also, in this figure, zones Z1 to Z4 indicate time intervals in which the reflected light RL reaches the depth image pickup device 1 according to the distance to the object OB. Here, the distance to the object OB is classified into first to fourth distances according to which of the zones Z1 to Z4 the reflected light RL is received.

The first distance is the distance corresponding to zone Z1. The reflected light RL reflected by the object OB existing at the first distance is received in the zone Z1. As the first distance, for example, a distance of approximately 0 (zero) [m] to 2.5 [m] is assumed.

The second distance is the distance corresponding to zone Z2. The second distance is longer than the first distance, and the reflected light RL reflected by the object OB existing at the second distance is received in the zone Z2. As the second distance, for example, a distance of about 2.5 [m] to 5.0 [m] is assumed.

The third distance is the distance corresponding to zone Z3. The third distance is longer than the second distance, and the reflected light RL reflected by the object OB existing at the third distance is received in the zone Z3. As the third distance, for example, a distance of approximately 5.0 [m] to 15.0 [m] is assumed.

The fourth distance is the distance corresponding to zone Z4. The fourth distance is a distance greater than the third distance, and the reflected light RL reflected by the object OB existing at the fourth distance is received in the zone Z4. As the fourth distance, for example, a distance of approximately 15.0 [m] to 25.0 [m] is assumed.

When the pixel 321 is driven as shown in the example of this figure, the charges corresponding to the reflected light received in the zone Z1 are distributed and accumulated in the charge storage section CS1 and the charge storage section CS2. In this case, the charges accumulated in the charge accumulation units CS3 and CS4 correspond to the external light component. When the charge accumulated in the charge storage section CS4 is used as an external light component, the charge storage section CS4 may be set to have a different accumulation time than the charge storage sections CS1 to CS3. Therefore, by multiplying Ta/Tb, it is necessary to derive an external light component corresponding to the accumulation time of the charge accumulation section CS4. There is a possibility that the noise component that is not related to the accumulation time may be increased or decreased (changed) when executing the calculation for multiplying Ta/Tb. Therefore, it is preferable to derive the charge corresponding to the external light component based on the charge accumulated in the charge storage section CS3, which does not require calculation when deriving the external light component.

Charge corresponding to the reflected light received in the zone Z2 is distributed to and accumulated in the charge accumulating portion CS2 and the charge accumulating portion CS3. In this case, the charges accumulated in the charge accumulation units CS1 and CS4 correspond to the external light component. When the charge accumulated in the charge storage section CS4 is used as an external light component, the charge storage section CS4 may be set to have a different accumulation time than the charge storage sections CS1 to CS3. Therefore, by multiplying Ta/Tb, it is necessary to derive an external light component corresponding to the accumulation time of the charge accumulation section CS4. There is a possibility that the noise component that is not related to the accumulation time may be increased or decreased (changed) when executing the calculation for multiplying Ta/Tb. Therefore, it is preferable to derive the charge corresponding to the external light component based on the charge accumulated in the charge storage section CS1, which does not require calculation when deriving the external light component.

The charge corresponding to the reflected light received in zone Z3 is discharged without being accumulated in any charge accumulation section CS. That is, the distance to the object OB that exists at the third distance corresponding to zone Z3 is not measured. The third distance is an example of the "measurement range out-of-range distance".

Charge corresponding to the reflected light received in zone Z4 is accumulated in charge accumulation unit CS4. In this case, the charges accumulated in the charge accumulation units CS1 to CS3 correspond to the external light component.

After repeating the allocation process shown in FIG. 5 for the number of times of accumulation corresponding to one frame, the process of reading out the amount of charge accumulated during that period is performed.

(Processing flow of the first embodiment)
Here, the flow of processing according to the first embodiment will be described with reference to FIG. FIG. 6 is a flow chart showing the flow of processing performed by the distance image processing section 4 of the first embodiment.

First, the distance image processing unit 4 carries out charge accumulation for one frame at the driving timing shown in FIG. 5 (step S10).

Next, the distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S11). The distance image processing unit 4 sets, for example, the smallest charge amount among the charge amounts accumulated in each of the charge accumulation units CS1 to CS3 as the charge amount Qg corresponding to the external light component.

Next, the distance image processing unit 4 calculates a corrected charge amount Q4# obtained by correcting the charge amount Q4 accumulated in the charge accumulation unit CS4 (step S12). The corrected charge amount Q4# is a value obtained by correcting the charge amount Q4 stored in the charge storage section CS4 to a charge amount corresponding to the storage time in the other charge storage sections CS (charge storage sections CS1 to CS3). The distance image processing unit 4 calculates the post-correction charge amount Q4# by the following equation (2).

Q4#=Ta/Tb×Q4 (2)
where Ta is the time interval during which charges are accumulated in the charge accumulation unit CS1
Tb is the time interval during which charges are accumulated in the charge accumulation unit CS4.
Q4 is the amount of charge accumulated in the charge accumulation section CS4

Next, the distance image processing unit 4 determines in which charge storage unit CS the amount of charge corresponding to the reflected light is stored (step S13). The distance image processing unit 4 compares the charge amount Qg calculated in step S11 with the charge amount accumulated in each of the charge accumulation units CS. In this case, the distance image processing section 4 compares the charge amount Qg with the charge amounts Q1 to Q3 accumulated in the charge accumulation sections CS1 to CS3, respectively. The distance image processing unit 4 compares the charge amount Qg with the corrected charge amount Q4# calculated in step S12 for the charge storage unit CS4. The distance image processing unit 4 determines the charge storage section CS in which the charge amount larger than the charge amount Qg is stored as the charge storage section CS in which the charge amount corresponding to the reflected light RL is stored.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS1 and CS2, the distance image processing unit 4 determines that there is an object at a distance (first distance) corresponding to the zone Z1, and ( 1) Using the formula, calculate the distance to the object (step S14).

The distance image processing unit 4 performs control when there is an object at a distance (first distance) corresponding to the zone Z1 (step S15). The control that the object is present at the distance (first distance) corresponding to the zone Z1 here is arbitrarily set according to the environment in which the range image pickup device 1 is mounted. For example, in an environment where the range image capturing apparatus 1 is mounted on a moving body and moving, the risk of collision increases if the moving body continues to move while an object is present at a short distance. Therefore, processing such as emergency stop of the moving body is performed.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS2 and CS3, the distance image processing unit 4 determines that there is an object at a distance (second distance) corresponding to the zone Z2, and ( 1) Using the formula, calculate the distance to the object (step S16). In this case, the charge ratio R in equation (1)=(Q3-Q1)/(Q2+Q3-2.times.Q1). When the distance is obtained from the charge ratio R, the distance to the object is determined by adding the base distance corresponding to the zone. For example, for zone Z2, the base distance is 2.5 [m]. In the case of zone Z2, the distance obtained by adding the base distance of 2.5 [m] to the distance calculated based on the charge ratio R is the distance to the object OB existing in zone Z2.

The distance image processing unit 4 performs control when there is an object at a distance (second distance) corresponding to the zone Z2 (step S17). The control that the object is present at the distance (second distance) corresponding to the zone Z2 here is arbitrarily set according to the environment in which the range imaging device 1 is installed. For example, a process such as an avoidance action to change the traveling direction of the moving object is performed.

When determining that the charge corresponding to the reflected light is accumulated in the charge accumulating section CS4, the distance image processing section 4 determines that there is an object at the distance (fourth distance) corresponding to the zone Z4 (step S18). .

The distance image processing unit 4 performs control when there is an object at a distance (fourth distance) corresponding to the zone Z4 (step S19). The control that the object is present at the distance (fourth distance) corresponding to the zone Z4 here is arbitrarily set according to the environment in which the range imaging device 1 is mounted. For example, processing such as reducing the moving speed of the moving object is performed.

In the above description, the case where the distance image processing unit 4 performs the processes shown in steps S15, S17, and S19 has been described as an example. However, it is not limited to this. The distance image processing unit 4 may perform at least the processing shown in steps S14, S15, or S18. For example, consider a case in which an external device is provided to control a moving object on which the distance image capturing device 1 is mounted. The external device is communicably connected to the distance image pickup device 1 . The distance imaging device 1 outputs information indicating the distance to the object calculated in step S14 or S15 to an external device. Alternatively, the distance imaging device 1 outputs information indicating the presence or absence of an object determined in step SS18 to an external device. The external device controls the moving object based on the information received from the range image capturing device 1 .

As described above, the range image pickup device 1 of the first embodiment includes the light source section 2, the light receiving section 3, the drain gate transistor GD, and the range image processing section 4. The light source unit 2 irradiates the measurement space in which the object OB exists with a light pulse PO. The light receiving unit 3 includes pixels 321 and a vertical scanning circuit 323 (an example of a driving circuit). The vertical scanning circuit 323 distributes and accumulates electric charges in each of the charge accumulation units CS in the pixels 321 at a predetermined timing synchronized with the irradiation of the light pulse PO. The drain gate transistor GD discharges the charges generated by the photoelectric conversion element PD. The distance image processing section 4 determines the measured distance to the subject OB using the amount of charge accumulated in each of the charge accumulation sections CS. The distance image processing unit 4 performs unit accumulation processing multiple times in one frame period, and determines the measurement distance to the object OB using the amount of charge accumulated in each of the charge accumulation units CS. In the unit accumulation process, a short-distance object measurement process, an ejection process, and a long-distance object measurement process are performed in order. The short-distance object measurement process is a process of distributing and accumulating charges according to the reflected light RL in the charge storage units CS1 to CS3 (part of the charge storage units CS). The discharging process is a process for discharging the charges generated by the photoelectric conversion element PD by the drain gate transistor GD. The long-distance object measurement process is a process of accumulating electric charges corresponding to the reflected light RL in the charge accumulator CS4 (the charge accumulator CS including the charge accumulator CS different from some of the charge accumulators CS).

As a result, the distance image capturing device 1 of the first embodiment can measure the distance to an object at a short distance in the short distance object measurement process. In addition, it is possible to determine the presence or absence of a long-distance object in the long-distance object measurement process. That is, it is possible to measure the distance to an object at a short distance and determine the presence or absence of an object at a long distance. In addition, in the distance image capturing apparatus 1 according to the first embodiment, after performing the short-distance object measurement process, before performing the long-distance object measurement process, the discharge process is performed. For this reason, it is possible to prevent charges from being accumulated in the charge accumulation section CS during this period, and to accumulate charges corresponding to reflected light reflected from an object at a long distance with high accuracy.

In the first embodiment, during the unit accumulation time UT, the drain gate transistor GD is turned off so that unnecessary charges are not accumulated at a timing other than the time at which charges are accumulated in the charge accumulation section CS. This avoids the continuous accumulation of charges during the time (zone Z3) in which the distance to the short-distance object is not measured and the presence or absence of the long-distance object is not determined.

(Drive timing of the second embodiment)
Here, the drive timing of the second embodiment will be described with reference to FIG. FIG. 7 is a timing chart showing driving timings of the pixels 321 in the second embodiment.

FIG. 7 shows a timing chart of the sorting process (unit accumulation process) performed in the unit accumulation time UT of this embodiment. In FIG. 7, as in FIG. 5, the timing for driving the charge distribution gate transistor G1 is "G1", the timing for driving the charge distribution gate transistor G2 is "G2", and the timing for driving the charge distribution gate transistor G3 is "G3". ”, “G4” for the timing of driving the charge distribution gate transistor G4, and “GD” for the timing of the drive signal RSTD. Also, here, it is assumed that each of the charge distribution gate transistors G1 to G4 is off and the drain gate transistor GD is on at the start of the unit accumulation time UT.

This embodiment differs from the first embodiment in that the distance to an object is measured in the long-distance object measurement process. Differences from the first embodiment will be described below.

Since the short-distance object measurement process and the ejection process are the same as those in the first embodiment, description thereof will be omitted.

A long-distance object measurement process according to the present embodiment will be described. The long-distance object measurement process is a process of determining the distance to an object existing at a long distance. The long-distance object measurement process is performed by sequentially turning on the charge distribution gate transistors G4 and G1 after the discharge process.

Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the charge distribution gate transistor G4 for the accumulation time Ta. After turning on the charge distribution gate transistor G4 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G4. The vertical scanning circuit 323 turns on the charge distribution gate transistor G1 for the accumulation time Ta at the timing when the charge accumulation in the charge accumulation unit CS4 is completed. After turning on the charge distribution gate transistor G1 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G1. The vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge accumulation unit CS1 is completed.

The fourth distance corresponding to zone Z4 in this embodiment is the entire range or a partial range of the fourth distance in the first embodiment. The fourth distance in this embodiment is determined according to the relationship between the accumulation time Tb in the first embodiment and the accumulation time Ta in the charge distribution gate transistors G4 and G1 in the long-distance object measurement process in this embodiment. It may be determined arbitrarily. For example, the fourth distance in this embodiment is assumed to be approximately 17.5 [m] to 20.0 [m].

(Processing flow of the second embodiment)
Here, the flow of processing according to the second embodiment will be described with reference to FIG. FIG. 8 is a flow chart showing the flow of processing performed by the distance image processing unit 4 of the second embodiment.

Since step S20 is the same as step S10 in FIG. 5, its description is omitted.

The distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S21). The distance image processing unit 4 sets, for example, the smallest charge amount among the charge amounts accumulated in each of the charge accumulation units CS2 to CS4 as the charge amount Qg corresponding to the external light component.

Next, the distance image processing unit 4 calculates a corrected charge amount Q1# obtained by correcting the charge amount Q1 accumulated in the charge accumulation unit CS1 (step S22). The corrected charge amount Q1# is a value obtained by correcting the charge amount Q1 stored in the charge storage section CS1 to a charge amount corresponding to the storage time in the other charge storage sections CS (charge storage sections CS2 to CS4). The distance image processing unit 4 calculates the post-correction charge amount Q1# by the following equation (3).

Q1#=Q1-Qg (3)
however,
Q1 is the amount of charge accumulated in the charge accumulating section CS1 Qg is the amount of charge corresponding to the external light component It is assumed that the time interval (Ta in FIG. 7) at which charges are accumulated in the charge accumulation unit CS1 in the distance object measurement process is the same value.

Next, the distance image processing section 4 determines in which charge accumulation section CS the charge amount corresponding to the reflected light is accumulated (step S23). The distance image processing unit 4 compares the charge amount Qg calculated in step S21 with the charge amount accumulated in each of the charge accumulation units. In this case, the distance image processing section 4 compares the charge amount Qg with the charge amounts Q2 to Q4 accumulated in the charge storage sections CS2 to CS4, respectively. The distance image processing unit 4 compares the charge amount Qg with the corrected charge amount Q1# calculated in step S22 for the charge accumulation unit CS1. The distance image processing unit 4 determines the charge storage section CS in which the charge amount larger than the charge amount Qg is stored as the charge storage section CS in which the charge amount corresponding to the reflected light is stored.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS1 and CS2, the distance image processing unit 4 determines that there is an object at a distance (first distance) corresponding to the zone Z1, and ( 1) Using the formula, calculate the distance to the object (step S24). In this case, the charge ratio R in equation (1)=(Q2-Q3)/(Q1#+Q2-2.times.Q3).

Steps S25 to S27 are the same as steps S15 to S17 in FIG. 5, so description thereof will be omitted.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS4 and CS1, the distance image processing unit 4 determines that there is an object at a distance (fourth distance) corresponding to the zone Z4, and ( 1) Using the formula, calculate the distance to the object (step S28). In this case, the charge ratio R = (Q1#-Q2) / (Q4 + Q1#-2 x Q2) in equation (1), or the charge ratio R = (Q1#-Q3) / (Q4 + Q1#-2 x Q3) be. When the distance is obtained from the charge ratio R, the distance to the object is determined by adding the base distance corresponding to the zone.

Since step S29 is the same as step S19 in FIG. 5, its description is omitted.

As described above, in the distance image capturing device 1 of the second embodiment, in the long-distance object measurement process, the charge storage units CS4 and CS1 (including the charge storage units CS different from some of the charge storage units CS) Charges corresponding to the reflected light RL are distributed and accumulated in the accumulation section CS).

Thereby, the distance image capturing device 1 of the second embodiment can measure the distance to the object in the long-distance object measurement process. Therefore, it is possible to perform processing according to the distance to an object at a long distance.

(Drive Timing of Modification 1 According to Second Embodiment)
Here, the drive timing of Modification 1 according to the second embodiment will be described with reference to FIG. FIG. 9 is a timing chart showing driving timings of the pixels 321 in Modification 1 according to the second embodiment.

FIG. 9 shows a timing chart of the sorting process (unit accumulation process) performed during the unit accumulation time UT of this modification. In FIG. 9, as in FIG. 5, the timing for driving the charge distribution gate transistor G1 is "G1", the timing for driving the charge distribution gate transistor G2 is "G2", and the timing for driving the charge distribution gate transistor G3 is "G3". ”, “G4” for the timing of driving the charge distribution gate transistor G4, and “GD” for the timing of the drive signal RSTD. Also, here, it is assumed that each of the charge distribution gate transistors G1 to G4 is off and the drain gate transistor GD is on at the start of the unit accumulation time UT.

This modification differs from the second embodiment in that the measurable range of the distance to the object is set larger in the long-distance object measurement process. Below, points different from the second embodiment will be described.

In this modification, as shown in this figure, zone Z5 is shown in addition to zones Z1 to Z4. The fourth distance corresponding to zone Z4 is, for example, about 17.5 [m] to 20.0 [m], as in the second embodiment.

In this modified example, the distance corresponding to zone Z5 is set as the fifth distance. The reflected light RL reflected by the object OB present at the fifth distance is received by the zone Z5. As the fifth distance, for example, a distance of approximately 20.0 [m] to 22.5 [m] is assumed.

Since the short-distance object measurement process and the ejection process are the same as those in the second embodiment, description thereof will be omitted.

A long-distance object measurement process of this modification will be described. The long-distance object measurement process is performed by sequentially turning on the charge distribution gate transistors G4, G1, and G2 after the discharge process.

Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD, and turns on the charge distribution gate transistors G4 and G1 for the accumulation time Ta as in the second embodiment. After turning on the charge distribution gate transistor G1 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G1. The vertical scanning circuit 323 turns on the charge distribution gate transistor G2 for the accumulation time Ta at the timing when the charge accumulation in the charge accumulation unit CS4 is completed. After turning on the charge distribution gate transistor G2 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G2. The vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge accumulation unit CS2 is completed.

(Processing flow of modification 1 according to the second embodiment)
Here, the flow of processing of modification 1 according to the second embodiment will be described with reference to FIG. 10 . FIG. 10 is a flow chart showing the flow of processing performed by the distance image processing unit 4 of Modification 1 according to the second embodiment.

Since step S30 is the same as step S10 in FIG. 5, its description is omitted.

The distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S31). The distance image processing unit 4, for example, sets the smallest charge amount among the charge amounts accumulated in each of the charge accumulation units CS3 and CS4 as the charge amount Qg corresponding to the external light component.

Next, the distance image processing unit 4 calculates post-correction charge amounts Q1# and Q2# (step S32). The corrected charge amount Q1# is a value obtained by correcting the charge amount Q1 stored in the charge storage section CS1 to a charge amount corresponding to the storage time in the other charge storage sections CS (the charge storage sections CS3 and CS4), (3) Calculated by the formula. The corrected charge amount Q2# is a value obtained by correcting the charge amount Q2 stored in the charge storage section CS2 to a charge amount corresponding to the storage time in the other charge storage sections CS (charge storage sections CS3 to CS4), It is calculated by the following formula (4).

Q2#=Q2-Qg (4)
however,
Q2 is the amount of charge accumulated in the charge accumulating section CS2 Qg is the amount of charge corresponding to the external light component It is assumed that the time interval (Ta in FIG. 9) at which charges are accumulated in the charge accumulation unit CS2 in the distance object measurement process is the same value.

Next, the distance image processing section 4 determines in which charge accumulation section CS the charge amount corresponding to the reflected light is accumulated (step S33). The distance image processing unit 4 compares the charge amount Qg calculated in step S31 with the charge amount accumulated in each of the charge accumulation units. In this case, the distance image processing section 4 compares the charge amount Qg with the charge amounts Q3 and Q4 accumulated in the charge accumulation sections CS3 and CS4, respectively. The distance image processing unit 4 compares the charge amount Qg with the corrected charge amounts Q1# and Q2# calculated in step S22 for the charge storage units CS1 and CS2. The distance image processing unit 4 determines the charge storage section CS in which the charge amount larger than the charge amount Qg is stored as the charge storage section CS in which the charge amount corresponding to the reflected light is stored.

When it is determined that charges corresponding to reflected light are accumulated in the charge accumulation units CS1 and CS2, the distance image processing unit 4 sets the sum (Q1#+Q2#) of the corrected charge amounts Q1# and Q2# as the threshold value. It is determined whether or not it is greater than Th (step S34).

The threshold Th is a value set according to the amount of charge corresponding to the boundary between the intensity of reflected light from an object at a short distance and the intensity of reflected light from an object at a long distance. That is, when the sum (Q1#+Q2#) is greater than the threshold Th, it can be determined that reflected light from an object at a short distance is distributed and accumulated in the charge accumulation units CS1 and CS2. On the other hand, when the sum (Q1#+Q2#) is less than the threshold Th, it can be determined that reflected light from a distant object is distributed and accumulated in the charge accumulation units CS1 and CS2.

If the sum (Q1#+Q2#) is greater than the threshold Th, the distance image processing unit 4 determines that the object is at the distance (first distance) corresponding to the zone Z1, and uses equation (1) to determine the distance to the object. is calculated (step S35). In this case, the charge ratio R in equation (1)=(Q2#-Q3)/(Q1#+Q2#-2.times.Q3). Since step S36 is the same as step S15 in FIG. 5, its description is omitted.

On the other hand, when the sum (Q1#+Q2#) is less than the threshold Th, the distance image processing unit 4 determines that there is an object at a distance (fifth distance) corresponding to zone Z5, and uses equation (1) , the distance to the object is calculated (step S37). In this case, the charge ratio R in equation (1)=(Q2#-Q3)/(Q1#+Q2#-2.times.Q3). Since step S38 is the same as step S19 in FIG. 5, its description is omitted.

The processing (step S39) when it is determined in step S33 that charges corresponding to the reflected light are accumulated in the charge accumulation units CS2 and CS3 is the same as step S16 in FIG. . Also, the post-processing (step S40) is the same as step S17 in FIG. 5, so the description thereof is omitted.

The process (step S41) when it is determined in step S33 that the charge corresponding to the reflected light is accumulated in the charge accumulation units CS4 and CS1 is the same as step S28 in FIG. 8, so the explanation thereof is omitted. . Also, the post-processing (step S42) is the same as step S29 in FIG. 8, so the description thereof will be omitted.

As described above, in the distance imaging device 1 of Modification 1 according to the second embodiment, in the long-distance object measurement process, the charge storage units CS4, CS1, and CS2 (some of the charge storage units CS are Charges according to the reflected light RL are sorted and accumulated in the charge storage sections CS including different charge storage sections CS.

As a result, the distance image capturing device 1 of Modification 1 according to the second embodiment measures the distance to an object with a three-tap configuration (charge storage units CS4, CS1, and CS2) in long-distance object measurement processing. can do. Therefore, it is possible to measure the distance to a distant object over a longer distance.

(Drive Timing of Modified Example 2 According to Second Embodiment)
Here, the drive timing of Modification 2 according to the second embodiment will be described with reference to FIG. 11 . FIG. 11 is a timing chart showing driving timings of the pixels 321 in Modification 2 according to the second embodiment.

FIG. 11 shows a timing chart of the sorting process (unit accumulation process) performed during the unit accumulation time UT of this modification. In FIG. 11, as in FIG. 5, the timing for driving the charge distribution gate transistor G1 is "G1", the timing for driving the charge distribution gate transistor G2 is "G2", and the timing for driving the charge distribution gate transistor G3 is "G3". ”, “G4” for the timing of driving the charge distribution gate transistor G4, and “GD” for the timing of the drive signal RSTD. Also, here, it is assumed that each of the charge distribution gate transistors G1 to G4 is off and the drain gate transistor GD is on at the start of the unit accumulation time UT.

This modification differs from the second embodiment in that the measurable range of the distance to the object is set larger in the long-distance object measurement process. Below, points different from the second embodiment will be described.

In this modification, as shown in this figure, zones Z1, Z3 to Z5 are shown without zone Z2. In this modification, the first distance corresponding to zone Z1 is, for example, a distance of approximately 0 (zero) [m] to 2.5 [m], as in the first embodiment.
In this modified example, the third distance corresponding to zone Z3 is, for example, a distance of approximately 2.5 [m] to 15.0 [m].
In this modified example, the fourth distance corresponding to zone Z4 is, for example, a distance of approximately 15.0 [m] to 17.5 [m].
In this modified example, the fifth distance corresponding to zone Z5 is, for example, a distance of approximately 17.5 [m] to 20.0 [m].

A short-distance object measurement process in this modified example will be described. In this modification, short-distance object measurement processing is performed by driving the charge storage units CS1 and CS2 of the pixel 321 at conventional driving timings.

Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD at a timing corresponding to the irradiation of the light pulse PO, for example, at the same timing as the irradiation of the light pulse PO, and turns off the charge distribution gate transistors G1 and G2. They are sequentially turned on for the accumulation time Ta.

Since the discharge process in this modified example is the same as the discharge process in the first embodiment, the description thereof will be omitted.

A long-distance object measurement process in this embodiment will be described. In this modification, after the discharging process, the charge distribution gate transistors G3, G4, and G1 are sequentially turned on.

Specifically, the vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the charge distribution gate transistor G3 for the accumulation time Ta. After turning on the charge distribution gate transistor G3 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G3. The vertical scanning circuit 323 turns on the charge distribution gate transistor G4 for the accumulation time Ta at the timing when the charge accumulation in the charge accumulation unit CS3 is completed. After turning on the charge distribution gate transistor G4 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G4. The vertical scanning circuit 323 turns on the charge distribution gate transistor G1 for the accumulation time Ta at the timing when the charge accumulation in the charge accumulation unit CS4 is completed. After turning on the charge distribution gate transistor G1 for the accumulation time Ta, the vertical scanning circuit 323 turns off the charge distribution gate transistor G1. The vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge accumulation unit CS1 is finished.

(Processing flow of modification 2 according to the second embodiment)
Here, the flow of processing of modification 2 according to the second embodiment will be described with reference to FIG. 12 . FIG. 12 is a flow chart showing the flow of processing performed by the distance image processing unit 4 of Modification 2 according to the second embodiment.

Since step S50 is the same as step S10 in FIG. 5, its description is omitted.

The distance image processing unit 4 calculates the charge amount Qg corresponding to the external light component (step S51). The distance image processing unit 4 sets, for example, the smallest charge amount among the charge amounts accumulated in each of the charge accumulation units CS2 to CS4 as the charge amount Qg corresponding to the external light component.

Next, the distance image processing unit 4 calculates a corrected charge amount Q1# obtained by correcting the charge amount Q1 accumulated in the charge accumulation unit CS1 (step S52). The corrected charge amount Q1# is a value obtained by correcting the charge amount Q1 stored in the charge storage section CS1 to a charge amount corresponding to the storage time in the other charge storage sections CS (charge storage sections CS2 to CS4). The distance image processing unit 4 calculates the post-correction charge amount Q1# by the following equation (5).

Q1#=Q1-Qg (5)
however,
Q1 is the amount of charge accumulated in the charge accumulating section CS1 Qg is the amount of charge corresponding to the external light component It is assumed that the time interval (Ta in FIG. 11) at which charges are accumulated in the charge accumulation unit CS1 in the distance object measurement process is the same value.

Next, the distance image processing unit 4 determines in which charge storage unit CS the amount of charge corresponding to the reflected light is stored (step S53). The distance image processing unit 4 compares the charge amount Qg calculated in step S51 with the charge amount accumulated in each of the charge accumulation units. In this case, the distance image processing section 4 compares the charge amount Qg with the charge amounts Q2 to Q4 accumulated in the charge storage sections CS2 to CS4, respectively. Distance image processing unit 4 compares charge amount Qg with corrected charge amount Q1# calculated in step S52 for charge storage unit CS1. The distance image processing unit 4 determines the charge storage section CS in which the charge amount larger than the charge amount Qg is stored as the charge storage section CS in which the charge amount corresponding to the reflected light is stored.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS1 and CS2, it is determined that there is an object at a distance (first distance) corresponding to the zone Z1, and using equation (1), A distance to the object is calculated (step S54). In this case, the charge ratio R in equation (1)=(Q2-Q3)/(Q1#+Q2-2.times.Q3). Since step S55 is the same as step S15 in FIG. 5, its description is omitted.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS3 and CS4, it is determined that there is an object at a distance (fourth distance) corresponding to the zone Z4, and using equation (1), A distance to the object is calculated (step S56). In this case, the charge ratio R in equation (1)=(Q4-Q2)/(Q3+Q4-2×Q2). Since step S57 is the same as step S19 in FIG. 5, its description is omitted.

When it is determined that charges corresponding to reflected light are accumulated in the charge storage units CS4 and CS1, it is determined that there is an object at a distance (fifth distance) corresponding to the zone Z5, and using equation (1), A distance to the object is calculated (step S58). In this case, the charge ratio R in equation (1)=(Q1#-Q2)/(Q4+Q1#-2.times.Q2). Since step S59 is the same as step S19 in FIG. 5, its description is omitted.

As described above, in the distance image capturing device 1 of the modification 2 according to the second embodiment, in the short-distance object measurement process, the charge storage units CS1 and CS2 (a part of the charge storage units CS) reflect light. Charges corresponding to RL are distributed and accumulated. In the long-distance object measurement process, charges corresponding to the reflected light RL are distributed and stored in the charge storage units CS3, CS4, and CS1 (the charge storage units CS including the charge storage units CS different from some of the charge storage units CS). Let

As a result, the distance image pickup device 1 of Modification 2 according to the second embodiment measures the distance to a very close object with a two-tap configuration (charge storage units CS1 and CS2) in short-distance object measurement processing. In the long-distance object measurement process, the distance to an object can be measured with a 3-tap configuration (charge storage units CS3, CS4 and CS1). Therefore, the distance to an object at a long distance can be measured up to a longer distance without executing the branching process shown in step S34 of FIG.

Note that in the above-described embodiment, the case where the allocation process performed in the unit accumulation time UT is repeatedly performed the number of times corresponding to one frame has been described as an example. However, it is not limited to this.

Since the sorting process of the embodiment includes a discharge process that takes a predetermined time, repeating the sorting process of the embodiment takes less time than repeating the conventional sorting process. It takes Therefore, there is a possibility that the maximum number of distribution processes that can be executed in one frame will be smaller than in the conventional case. As a countermeasure against this, the sorting process of the embodiment and the conventional sorting process may be mixed in one frame at a certain ratio.

Also, the time interval for performing the discharge process in zone Z3 may be arbitrarily set according to the environment in which the distance image capturing device 1 is applied. For example, regarding the relationship between the zone Z3 and the third distance, when the zone Z3 is 1 [ns], the third distance is approximately 15 [cm]. When the zone Z3 is 100 [ns], the third distance is approximately 15 [m]. When the zone Z3 is 200 [ns], the third distance is approximately 30 [m].

For example, consider a case in which the distance image pickup device 1 is mounted on a moving body and used for the purpose of avoiding collision with an object existing in the moving direction. In this case, for example, it is assumed that a moving object running at a speed of about 50 [km] per hour brakes 0.033 [seconds] after recognizing an object existing at a long distance. In this case, the idling distance is about 0.5 "m", and the braking distance is about 14.1 [m] depending on the road surface conditions. That is, if it is possible to recognize whether or not an object exists at a distance of about 15 [m], it is possible to avoid collision with that object. In this case, the upper limit of the third distance and the lower limit of the fourth distance are set to 15 [m]. Therefore, zone Z3 is set to about 100 [ns].

If this idea is applied to a moving body running at a speed of about 70 [km] per hour, the free running distance is about 0.6 "m", the braking distance is about 27.6 [m], and the braking distance is about 30 [m]. If it is possible to recognize whether or not an object exists at a distant distance, it is possible to avoid collision with that object. In this case, the upper limit of the third distance and the lower limit of the fourth distance are set to 30 [m]. Therefore, zone Z3 is set to about 200 [ns].

Further, in at least one embodiment described above, when the unit accumulation process is repeatedly performed on a frame-by-frame basis, based on the amount of charge accumulated in each of the charge accumulation units CS during the current one frame period, The details of the unit accumulation process in the period (the details of each of the short-distance object measurement process, the discharge process, and the long-distance object measurement process) may be determined. For example, the distance image processing unit 4 first determines whether an object exists at a long distance in the first frame according to the first embodiment. If the object exists in the long distance, the distance to the object in the long distance is measured in the next frame according to the second embodiment. If there is no object in the long distance, it is determined whether or not the object exists in the distance in the next frame as well according to the first embodiment. Accordingly, it is possible to change the processing contents for each frame depending on the situation, depending on whether an object exists at a long distance or not, and it is possible to respond according to the situation.

In at least one embodiment described above, the light source section 2 may include a plurality of light source devices having different properties. The plurality of light source devices here may use arbitrary light sources as long as at least the intensity of light emitted per unit area is different. As a light source, for example, an LED (Light Emitting Diode), laser, VCSEL, or the like can be applied.

The surface light source uniformly irradiates the space to be imaged with light pulses PO. When a space to be imaged is irradiated with a surface light source, the space can be uniformly irradiated, so there is an advantage that the resolution in the space can be improved. On the other hand, it is necessary to irradiate the light pulse PO uniformly over a wide area. Therefore, in order to comply with the so-called eye-safe standards stipulated in the safety standards for laser products (eg IEC 60825-1, JIS C6802, etc.), the intensity of the irradiated light must be reduced. There are disadvantages. This is because eye-safety is an index defined by the intensity of light irradiated per unit area.

For this reason, when the intensity of light emitted from a surface light source is suppressed to a low level, the intensity of reflected light reflected by an object existing at a long distance becomes weak. Therefore, even if the number of distribution processes performed in the unit accumulation time UT per frame is increased, the signal (charge corresponding to the reflected light) is small and likely to be buried in noise.

The Todd light source, for example, irradiates a light pulse PO having a predetermined dot pattern, and uniformly irradiates the space to be imaged with the light pulse PO. For example, light having a dot pattern can be irradiated by collimating light from a light source using a collimator lens or the like and passing the parallel light through a diffractive optical element (DOE). This is because one or both surfaces of the DOE are diffractive surfaces. Note that it is possible to omit the collimator lens by devising the light source and design (see, for example, International Publication No. 2020/066981).

In addition, although the case where the dot pattern is grid-like has been described above, the present invention is not limited to this. The dot pattern may be a regular pattern such as hexagonal, or may be a random or pseudo-random pattern.

When the dot light source is used to irradiate light, the space is irradiated non-uniformly, resulting in low spatial resolution. On the other hand, when using a dot light source, it is possible to increase the intensity of light per dot within the range of light intensity defined by eye-safe, compared to a surface light source. That is, when the light intensity per unit area is the same, the dot light source can make the local light intensity higher than the surface light source.

Therefore, when a dot light source is used, the signal (charge corresponding to the reflected light) due to the intensity of the reflected light reflected by an object at a long distance can be increased to the extent that it is not buried in noise. Then, it becomes possible to separate a reflected signal from a long distance (charge corresponding to reflected light reflected by an object existing at a long distance) from noise and detect it.

Also, the density and intensity of the dot pattern may be adjusted according to the range of measurable distance. For example, by increasing the density of the dot pattern to bring it closer to the surface light source, the spatial resolution is adjusted to be higher. Alternatively, considering eye-safe regulations, the density of the dot pattern may be set low so that the light intensity per unit area does not exceed the threshold.

Moreover, only dot light sources may be used, or a surface light source and a dot light source may be used in combination. When the surface light source and the dot light source are combined, the surface light source and the dot light source may be alternately irradiated, or the surface light source and the dot light source may be used together for irradiation at the same time.

When alternately irradiating the surface light source and the dot light source, the dot light source is radiated periodically, for example, every frame, every several frames, every unit accumulation time UT, or every unit accumulation time UT multiple times. A light pulse is irradiated using a surface light source, and at other irradiation timings, a light pulse is irradiated using a surface light source.

As described above, in the distance image pickup device 1 of the embodiment, the light source section 2 has a surface light source and a dot light source. The light intensity per unit area is the same between irradiation by the surface light source and irradiation by the dot light source. In the irradiation by the surface light source and the irradiation by the dot light source, the intensity of the light irradiated by the dot light source is higher than the intensity of the light irradiated by the surface light source, and the light intensity per unit area is the same. The distance image processing unit 4 performs measurement by switching between irradiation with a surface light source and irradiation with a dot light source. As a result, the distance image capturing device 1 of the embodiment can switch measurement depending on the purpose, such as whether to increase the resolution so as to measure the space uniformly or to increase the measurement accuracy of an object existing at a long distance. can.

Further, in the distance image capturing device 1 of the embodiment, the distance image processing unit 4 performs control such that the irradiation by the surface light source and the irradiation by the dot light source are periodically repeated. For example, the distance image processing unit 4 periodically changes the area light source and the dot light source in units of one frame, several frames, one unit accumulation time UT, or a plurality of unit accumulation times UT. and alternately. As a result, it is possible to improve the measurement accuracy of an object existing at a long distance while maintaining the resolution so that the space can be measured uniformly.

Note that the configuration described in at least one embodiment described above may be applied to other configurations. For example, the measurement for determining the presence or absence of a long-distance object according to the first embodiment and the measurement of the distance to the long-distance object according to the second embodiment may be combined to perform measurement. The dot light source according to the embodiment may be used when performing the measurement according to the first embodiment.

Further, in the above-described embodiment, in the equation (3), the time interval (Ta in FIG. 7) at which charges are accumulated in the charge accumulation unit CS1 in the short-distance object measurement process and the long-distance object measurement process is the same value. premised on. If both have different values, it is necessary to calculate the post-correction charge amount Q1# by distinguishing between short-distance object measurement processing and long-distance object measurement processing depending on whether the reflected light RL is received. There is

Whether the reflected light RL is received in the short-distance object measurement process or the reflected light RL is received in the long-distance object measurement process depends on the amount of charge Q2 and the amount of charge Q4 when driving is performed at the timing shown in FIG. It is possible to determine by comparison. Here, the charge amount Q2 is the charge amount accumulated in the charge accumulation section CS2. The charge amount Q4 is the charge amount accumulated in the charge accumulation section CS4. Specifically, when the charge amount Q2>the charge amount Q4, it can be determined that the reflected light RL has been received in the short-distance object measurement process. If the amount of charge Q2<the amount of charge Q4, it can be determined that the reflected light RL has been received in the long-distance object measurement process. In this case, it should be noted that the accumulation time Ta for accumulating charges in the charge accumulating unit CS2 in the short-distance object measurement process is the same as the accumulation time Ta for accumulating charges in the charge accumulating unit CS4 in the long-distance object measurement process. It is assumed. If they are not the same, the amount of charge Q4# estimated to be accumulated in the charge accumulating section CS4 when the accumulation times of the charge accumulating sections CS2 and CS4 are the same is calculated using the following equation (6). The amount of charge Q4# thus obtained is compared with the amount of charge Q2. The charge amount Q2 is the charge amount accumulated in the charge accumulation section CS2.

Q4#=Ta/Ta#×Q4 (6)
however,
Ta is the time interval during which charges are accumulated in the charge storage unit CS2 in the short distance object measurement process, Ta# is the time interval during which charges are accumulated in the charge storage unit CS4 in the long distance object measurement process, and Q4 is the time interval during which charges are accumulated in the charge storage unit CS4. The charge amount Q4# is the charge amount estimated to be accumulated in the charge accumulation unit CS4 when the accumulation time is Ta.

When the reflected light RL is received in the short-distance object measurement process, the post-correction charge amount Q1# is calculated by the following equation (7).

Q1#=(Q1-(Ta+Ta#)/Ta×Qg)+Qg (7)
however,
Ta is the time interval during which charges are accumulated in the charge storage unit CS1 in the short-distance object measurement process. Ta# is the time interval during which charges are accumulated in the charge storage unit CS1 in the long-distance object measurement process. The amount of charge Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated for the accumulation time Ta)

On the other hand, when the reflected light RL is received in the long-distance object measurement process, the post-correction charge amount Q1# is calculated by the following equation (8).

Q1#=(Q1-(Ta+Ta#)/Ta×Qg)×Ta/Ta#+Qg (8)
however,
Ta is the time interval during which charges are accumulated in the charge storage unit CS1 in the short-distance object measurement process. Ta# is the time interval during which charges are accumulated in the charge storage unit CS1 in the long-distance object measurement process. The amount of charge Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated for the accumulation time Ta)

Equation (4) requires further classification. Specifically, in the short-distance object measurement process, it is necessary to distinguish between cases in which the charge storage units CS1 and CS2 receive the reflected light RL, and cases in which the charge storage units CS2 and CS3 receive the reflected light RL. There is Further, in the long-distance object measurement process, it is necessary to distinguish between cases in which the charge storage units CS4 and CS1 receive the reflected light RL, and cases in which the charge storage units CS1 and CS2 receive the reflected light RL.

FIG. 9 shows the case where the charge storage units CS2 and CS3 receive the reflected light RL in the short-distance object measurement process and the case where the charge storage units CS4 and CS1 receive the reflected light RL in the long-distance object measurement process. It can be determined by comparing the amount of charge Q3 and the amount of charge Q4 when driven at the timing shown. Here, the charge amount Q3 is the charge amount accumulated in the charge accumulation section CS3. The charge amount Q4 is the charge amount accumulated in the charge accumulation section CS4. Specifically, when the charge amount Q3>the charge amount Q4, it can be determined that the reflected light RL has been received by the charge storage units CS2 and CS3 in the short-distance object measurement process. If the charge amount Q2<the charge amount Q4, it can be determined that the reflected light RL has been received by the charge storage units CS4 and CS1 in the long-distance object measurement process. In this case, it should be noted that the accumulation time Ta for accumulating charges in the charge accumulating unit CS3 in the short-distance object measurement process is the same as the accumulation time Ta for accumulating charges in the charge accumulating unit CS4 in the long-distance object measurement process. It is assumed. If they are not the same, the amount of charge Q4# estimated to be accumulated in the charge accumulating section CS4 when the accumulation times of the charge accumulating sections CS3 and CS4 are the same is calculated using the following equation (9). The charge amount Q4# thus obtained is compared with the charge amount Q3. The charge amount Q3 is the charge amount accumulated in the charge accumulation section CS3.

Q4#=Ta/Ta#×Q4 (9)
however,
Ta is the time interval during which charges are accumulated in the charge storage unit CS3 in the short distance object measurement process, Ta# is the time interval during which charges are accumulated in the charge storage unit CS4 in the long distance object measurement process, and Q4 is the time interval during which charges are accumulated in the charge storage unit CS4. The charge amount Q4# is the charge amount estimated to be accumulated in the charge accumulation unit CS4 when the accumulation time is Ta.

When the charge accumulation units CS2 and CS3 receive the reflected light RL in the short-distance object measurement process, the corrected charge amount Q2# is calculated by the following equation (10).

Q2#=(Q2-(Ta+Ta#)/Ta×Qg)+Qg (10)
however,
Ta is the time interval during which charges are accumulated in the charge storage unit CS2 in the short distance object measurement process Ta# is the time interval during which charges are accumulated in the charge storage unit CS2 in the long distance object measurement process Q2 is the time interval during which charges are accumulated in the charge storage unit CS2 The amount of charge Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated for the accumulation time Ta)

When the charge storage units CS4 and CS1 receive the reflected light RL in the long-distance object measurement process, the corrected charge amount Q1# is calculated by the following equation (11).

Q1#=(Q1-(Ta+Ta#)/Ta×Qg)×Ta/Ta#+Qg (11)
however,
Ta is the time interval during which charges are accumulated in the charge storage unit CS1 in the short-distance object measurement process. Ta# is the time interval during which charges are accumulated in the charge storage unit CS1 in the long-distance object measurement process. The amount of charge Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated for the accumulation time Ta)

The case where the reflected light RL is received by the charge storage units CS1 and CS2 in the short-distance object measurement process and the case where the charge storage units CS1 and CS2 receive the reflected light RL in the long-distance object measurement process are shown in FIG. It is considered that the amount of charge Q3 and the amount of charge Q4 are approximately the same when driven at the timing shown. That is, the amount of charge Q3=the amount of charge Q4, or the amount of charge Q3≈the amount of charge Q4. In this case, for example, provisional charge amounts Q1## and Q2## are once calculated by the following equation (12).

Q1##=(Q1-(Ta+Ta#)/Ta×Qg)
Q2##=(Q2-(Ta+Ta#)/Ta×Qg)
…(12)
however,
Ta is the time interval during which charges are accumulated in the charge storage units CS1 and CS2 in the short-distance object measurement process, Ta# is the time interval during which charges are accumulated in the charge storage units CS1 and CS2 in the long-distance object measurement process, and Q1 is the charge storage unit. The amount of charge accumulated in CS1 Q2 is the amount of charge accumulated in the charge storage section CS2 Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated over the accumulation time Ta)

In this case, it should be noted that the accumulation time Ta for accumulating charges in the charge accumulating unit CS3 in the short-distance object measurement process is the same as the accumulation time Ta for accumulating charges in the charge accumulating unit CS4 in the long-distance object measurement process. It is assumed. If they are not equal, the charge amount Q4# is calculated using the equation (9) as described above, and the calculated charge amount Q4# and the charge amount Q3 are approximately the same amount, and the charge amount Q3=the charge amount Q4. # or the amount of charge Q3≈the amount of charge Q4#. The charge amount Q3 is the charge amount accumulated in the charge accumulation section CS3. The amount of charge Q4# is the amount of charge estimated to be accumulated in the charge storage section CS4 when the storage times of the charge storage sections CS3 and CS4 are the same.

Next, the sum of the provisional charge amounts Q1## and Q2## (Q1##+Q2##) is compared with the threshold value Th. The threshold Th is a value set according to the amount of charge corresponding to the boundary between the intensity of reflected light from an object at a short distance and the intensity of reflected light from an object at a long distance. That is, when the sum (Q1##+Q2##) is greater than the threshold value Th, it can be determined that the reflected light RL has been received by the charge storage units CS1 and CS2 in the short-distance object measurement process. On the other hand, when the sum (Q1##+Q2##) is equal to or less than the threshold value Th, the reflected light RL is not received by the charge storage units CS1 and CS2 in the short-distance object measurement process, that is, the long-distance object measurement process , it can be determined that the reflected light RL is received by the charge storage units CS1 and CS2.

When the charge storage units CS1 and CS2 receive the reflected light RL in the short-distance object measurement process, the corrected charge amounts Q1# and Q2# are calculated by the following equation (13).

Q1#=(Q1-(Ta+Ta#)/Ta×Qg)+Qg
Q2#=(Q2-(Ta+Ta#)/Ta×Qg)+Qg
…(13)
however,
Ta is the time interval during which charges are accumulated in the charge storage units CS1 and CS2 in the short-distance object measurement process, Ta# is the time interval during which charges are accumulated in the charge storage units CS1 and CS2 in the long-distance object measurement process, and Q1 is the charge storage unit. The amount of charge accumulated in CS1 Q2 is the amount of charge accumulated in the charge storage section CS2 Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated over the accumulation time Ta)

When the charge storage units CS1 and CS2 receive the reflected light RL in the long-distance object measurement process, the corrected charge amounts Q1# and Q2# are calculated by the following equation (14).

Q1#=(Q1−(Ta+Ta#)/Ta×Qg)×Ta/Ta#+Qg
Q2#=(Q2−(Ta+Ta#)/Ta×Qg)×Ta/Ta#+Qg
…(14)
however,
Ta is the time interval during which charges are accumulated in the charge storage units CS1 and CS2 in the short-distance object measurement process, Ta# is the time interval during which charges are accumulated in the charge storage units CS1 and CS2 in the long-distance object measurement process, and Q1 is the charge storage unit. The amount of charge accumulated in CS1 Q2 is the amount of charge accumulated in the charge storage section CS2 Qg is the amount of charge corresponding to the external light component (the amount of charge when the charge is accumulated over the accumulation time Ta)

Using the charge amount calculated above, it is determined in which zone Z the object is located (steps S23, S33, S34, S53) and the distance to the object is calculated (steps S15, S17, S19, etc.). When it is determined that there is an object at a long distance, the distance is corrected using the charge ratio, the base distance, and the ratio between the accumulation times Ta and Ta#, and the corrected distance is taken as the distance to the object. Further, in the above example, the case where the amount of charge accumulated in the accumulation time Ta# is corrected in accordance with the accumulation time Ta has been illustrated, but the present invention is not limited to this. At least, the two charge amounts to be compared should be the charge amounts accumulated during the same accumulation time. The charge amount accumulated during the accumulation time Ta may be corrected in accordance with the accumulation time Ta#.

Thus, when the time intervals Ta and Ta# are not the same, it is necessary to correct the signal amount according to the distribution processing time (charge accumulation time). Correcting the signal amount may increase or decrease noise or increase the computational load. Therefore, it is desirable that the time intervals Ta and Ta# be the same in the short-distance object measurement process and the long-distance object measurement process.

If the time intervals Ta and Ta# are not the same, the distance image processing section 4 compares the charge amounts Q3 and Q4 accumulated in the charge accumulation sections CS3 and CS4 in step S33 of FIG. If the amount of charge Q3>the amount of charge Q4, the distance image processing section 4 proceeds to step S39. If the charge amount Q3<the charge amount Q4, the distance image processing unit 4 proceeds to step S41. If the charge amounts of both are substantially the same, and if the charge amount Q3=the charge amount Q4 or the charge amount Q3≈the charge amount Q4, the distance image processing section 4 proceeds to step S34. In step S34, the distance image processing unit 4 once calculates the provisional charge amounts Q1## and Q2## shown in equation (12), and determines whether the sum (Q1##+Q2##) is greater than the threshold value Th. determine whether If the sum (Q1##+Q2##) is greater than the threshold Th, the distance image processing section 4 proceeds to step S35. If the sum (Q1##+Q2##) is equal to or less than the threshold Th, the distance image processing section 4 proceeds to step S37.

All or part of the distance image capturing device 1 and the distance image processing unit 4 in the above-described embodiments may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically retains a program for a short period of time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include something that holds the program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing a part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be implemented using a programmable logic device such as FPGA.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design and the like are included within the scope of the gist of the present invention.

REFERENCE SIGNS LIST 1 distance image pickup device 2 light source unit 3 light receiving unit 32 distance image sensor 321 pixel 323 vertical scanning circuit 4 distance image processing unit 41 timing control unit 42 distance calculation unit 43 measurement control unit 44 Storage section CS... Charge storage section PO... Light pulse

Claims (10)

a light source unit that irradiates a light pulse into a measurement space in which an object exists;
a pixel comprising a photoelectric conversion element that generates charges according to incident light; and three or more charge storage units that store the charges; a light-receiving unit having a pixel drive circuit that distributes and accumulates the electric charge in each of the electric charge accumulation units;
a charge discharging unit for discharging the charge generated by the photoelectric conversion element;
a distance image processing unit that determines a measured distance to the subject using the amount of charge accumulated in each of the charge accumulation units;
with
The distance image processing unit performs unit accumulation processing a plurality of times in one frame period, and determines the measured distance to the subject using the amount of charge accumulated in each of the three or more charge accumulation units;
In the unit accumulation processing, a short-distance object measurement processing is performed for allocating and accumulating electric charges according to the reflected light, which is the light pulse reflected from the subject, in some of the three or more electric charge accumulation units. a discharge process in which the charge discharge unit discharges the charge generated by the photoelectric conversion element; and a charge storage unit including a charge storage unit different from the part of the charge storage unit stores the charge corresponding to the reflected light. long-distance object measurement processing is performed in order,
Range imaging device. The pixel is provided with a first charge storage portion, a second charge storage portion, a third charge storage portion, and a fourth charge storage portion, which are the four charge storage portions,
The distance image processing unit is
electric charges corresponding to the reflected light reflected by the subject at the first distance are accumulated in the first charge accumulation unit and the second charge accumulation unit in order;
charges corresponding to the reflected light reflected by the object at a second distance larger than the first distance are accumulated in the second charge storage unit and the third charge storage unit in order;
performing the discharge process at a time when the reflected light reflected by the subject located between the second distance and a third distance larger than the second distance by the distance outside the measurement range is received;
controlling so that the charge corresponding to the reflected light reflected by the subject at a fourth distance larger than the third distance is accumulated in the fourth charge accumulation unit;
The distance image pickup device according to claim 1. The pixel is provided with a first charge storage portion, a second charge storage portion, a third charge storage portion, and a fourth charge storage portion, which are the four charge storage portions,
The distance image processing unit is
electric charges corresponding to the reflected light reflected by the subject at the first distance are accumulated in the first charge accumulation unit and the second charge accumulation unit in order;
charges corresponding to the reflected light reflected by the object at a second distance larger than the first distance are accumulated in the second charge storage unit and the third charge storage unit in order;
performing the discharge process at a time when the reflected light reflected by the subject located between the second distance and a third distance larger than the second distance by the distance outside the measurement range is received;
Control is performed so that the electric charge corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance is distributed and accumulated in the fourth electric charge accumulation section and the first electric charge accumulation section in order. do,
The distance image pickup device according to claim 1. The pixel is provided with a first charge storage portion, a second charge storage portion, a third charge storage portion, and a fourth charge storage portion, which are the four charge storage portions,
The distance image processing unit is
electric charges corresponding to the reflected light reflected by the subject at the first distance are accumulated in the first charge accumulation unit and the second charge accumulation unit in order;
performing the discharge process at a time when the reflected light reflected by the subject, which is between the first distance and a third distance larger than the first distance by the distance outside the measurement range, is received;
electric charges corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance are distributed and accumulated in the third charge accumulation unit and the fourth charge accumulation unit in order;
Control is performed so that charges corresponding to the reflected light reflected by the subject at a fifth distance greater than the fourth distance are distributed and accumulated in the fourth charge storage unit and the first charge storage unit in order. do,
The distance image pickup device according to claim 1. The pixel is provided with a first charge storage portion, a second charge storage portion, a third charge storage portion, and a fourth charge storage portion, which are the four charge storage portions,
The distance image processing unit is
electric charges corresponding to the reflected light reflected by the subject at the first distance are accumulated in the first charge accumulation unit and the second charge accumulation unit in order;
charges corresponding to the reflected light reflected by the object at a second distance larger than the first distance are accumulated in the second charge storage unit and the third charge storage unit in order;
performing the discharge process at a time when the reflected light reflected by the subject located between the second distance and a third distance larger than the second distance by the distance outside the measurement range is received;
electric charges corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance are accumulated in the fourth charge accumulation unit and the first charge accumulation unit in order;
Control is performed so that charges corresponding to the reflected light reflected by the subject at a fifth distance larger than the fourth distance are distributed and accumulated in the first charge storage unit and the second charge storage unit in order. do,
The distance image pickup device according to claim 1. The pixel is provided with a first charge storage portion, a second charge storage portion, a third charge storage portion, and a fourth charge storage portion, which are the four charge storage portions,
The distance image processing unit is
electric charges corresponding to the reflected light reflected by the subject at the first distance are accumulated in the first charge accumulation unit and the second charge accumulation unit in order;
performing the discharge process at a time when the reflected light reflected by the subject, which is between the first distance and a third distance larger than the first distance by the distance outside the measurement range, is received;
electric charges corresponding to the reflected light reflected by the subject at a fourth distance greater than the third distance are distributed and accumulated in the third charge accumulation unit and the fourth charge accumulation unit in order;
electric charges corresponding to the reflected light reflected by the subject at a fifth distance that is greater than the fourth distance are accumulated in the fourth charge accumulation unit and the first charge accumulation unit in order;
Control is performed so that charges corresponding to the reflected light reflected by the subject at a sixth distance greater than the fifth distance are distributed and accumulated in the first charge storage unit and the second charge storage unit in order. do,
The distance image pickup device according to claim 1. The distance image processing unit performs the short-distance object measurement process, the discharge process, and the and determining processing details in each of the long-distance object measurement processing;
The distance image pickup device according to claim 1. The light source unit has a surface light source that uniformly irradiates the light pulse into the measurement space and a dot light source that non-uniformly irradiates the light pulse into the measurement space,
In the irradiation by the surface light source and the irradiation by the dot light source, the intensity of the light irradiated by the dot light source is greater than the intensity of the light irradiated by the surface light source, and the intensity of light per unit area is the same. ,
The distance image processing unit performs measurement by switching between irradiation by the surface light source and irradiation by the dot light source.
The distance image pickup device according to claim 1. The distance image processing unit controls such that the irradiation by the surface light source and the irradiation by the dot light source are periodically repeated.
The distance image pickup device according to claim 8. A light source unit for irradiating a measurement space in which an object exists with a light pulse, a photoelectric conversion element for generating charges according to the incident light, and pixels comprising three or more charge storage units for storing the charges; a pixel driving circuit for distributing and accumulating the electric charge in each of the electric charge accumulating portions of the pixel at a predetermined timing synchronized with the irradiation of the light pulse; A range image capturing method using a range image capturing device, comprising: a charge discharging unit for discharging charge; hand,
The distance image processing unit performs unit accumulation processing multiple times in one frame period, and determines the measured distance to the subject using the amount of charge accumulated in each of the three or more charge accumulation units;
In the unit accumulation processing, a short-distance object measurement processing is performed for allocating and accumulating electric charges according to the reflected light, which is the light pulse reflected from the subject, in some of the three or more electric charge accumulation units. a discharge process in which the charge discharge unit discharges the charge generated by the photoelectric conversion element; and a charge storage unit including a charge storage unit different from the part of the charge storage unit stores the charge corresponding to the reflected light. long-distance object measurement processing is performed in order,
Range image capturing method.

Images