JP2022112388A - Distance image pickup device, and distance image pickup method - Google Patents

Abstract

To provide a distance image pickup device that, even when a distance calculated on the basis of an amount of electric charges accumulated in charge accumulation units has an error, can correct the distance to be close to an actual distance.SOLUTION: A distance image pickup device comprises: a light source unit; a light receiving unit that has pixels each provided with a photoelectric converter and a plurality of charge accumulation units, and a pixel drive circuit distributing electric charges to the charge accumulation units to cause the charge accumulation units to accumulate the electric charges; a distance image processing unit that determines a measured distance to a subject on the basis of an amount of electric charges accumulated in the charge accumulation units; and a storage unit that, for every first variable based on the amount of electric charges accumulated per unit accumulation number, stores table information indicating a relationship between a second variable based on a charge ratio and a corresponding distance. The distance image processing unit calculates the first variable and the second variable on the basis of the amount of electric charges accumulated in the charge accumulation units, selects the table information corresponding to the calculated first variable, acquires the corresponding distance corresponding to the second variable by using the selected table information, and determines the measured distance by using the acquired corresponding distance.SELECTED DRAWING: Figure 1

Description

The present invention relates to a range image capturing device and a range image capturing method.

Conventionally, as a technique for measuring the distance to an object, there is a technique for measuring the flight time of light pulses. Such a technique is called Time of Flight (hereinafter referred to as TOF). TOF utilizes the fact that the speed of light is known, and irradiates an object with light pulses in the near-infrared region. Then, the time difference between the time when the light pulse is applied and the time when the reflected light of the applied light pulse is received by the object is measured. The distance to the object is calculated based on this time difference. 2. Description of the Related Art Ranging sensors that detect light for measuring distance using photodiodes (photoelectric conversion elements) have been put to practical use.

In recent years, distance measuring sensors have been put into practical use that can obtain depth information for each pixel in a two-dimensional image including the object, that is, three-dimensional information about the object, in addition to the distance to the object. Such a distance measuring sensor is also called a distance image pickup device. In the depth image pickup device, a plurality of pixels including photodiodes are arranged in a two-dimensional matrix on a silicon substrate, and light reflected by an object is received by the pixel surface. A depth image pickup device outputs a photoelectric conversion signal for one image based on the amount of light (charge) received by each pixel. distance information can be obtained. For example, Japanese Laid-Open Patent Publication No. 2004-100003 discloses a technique in which three charge storage units are provided in one pixel and the charges are distributed in order to calculate the distance.

Japanese Patent No. 4235729

In such a distance image pickup device, pixels receive reflected light reflected by an object, photoelectrically convert the light amount of the received reflected light into electric charge, accumulate the converted electric charge in the electric charge accumulation unit, and store the accumulated electric charge. Calculate distance information based on the quantity. However, the distance calculated from the accumulated charge amount may have an error with respect to the actual distance (actual distance).

The present invention has been made based on the above problem, and corrects the distance to be closer to the actual distance even if there is an error in the distance calculated based on the amount of charge accumulated in the charge accumulation unit. It is an object of the present invention to provide a distance image pickup device and a distance image pickup method capable of capturing a distance image.

The distance image pickup device of the present invention comprises a light source unit that irradiates a light pulse into a measurement space where an object exists, a photoelectric conversion element that generates electric charges according to the incident light, and three or more electric charges that accumulate the electric charges. a light-receiving unit comprising: a pixel having an accumulating portion; and a pixel driving circuit that distributes and accumulates the charge in each of the charge accumulating portions of the pixel at a predetermined timing synchronized with irradiation of the light pulse; a distance image processing unit that determines a measured distance to the subject based on the amount of charge accumulated in each charge accumulation unit; and a charge ratio for each first variable based on the amount of charge accumulated per unit accumulation count and a storage unit for storing table information indicating a relationship between a second variable based on the distance to the subject and a corresponding distance corresponding to the distance to the subject, wherein the first variable is stored in each of the charge storage units per unit accumulation count. The second variable is a variable corresponding to the sum of the amount of charge accumulated in the second variable, and the second variable is the amount of charge corresponding to the reflected light of the light pulse reflected from the subject among the plurality of charge accumulation units. is a charge ratio indicated by using a distance calculation charge amount obtained by subtracting an external light component from the charge amount accumulated in each of two or more distance calculation charge accumulation units, wherein the distance image processing unit calculating the first variable and the second variable based on the amount of charge accumulated in each storage unit, selecting the table information corresponding to the calculated first variable, and using the selected table information and obtaining the corresponding distance corresponding to the calculated second variable, and determining the measured distance using the obtained corresponding distance.

In the distance image pickup device of the present invention, the charge ratio is the sum of the distance calculation charge amount accumulated in each of the distance calculation charge accumulation units and the sum of the distance calculation charge amounts accumulated in each of the distance calculation charge accumulation units. Alternatively, it is the ratio of the charge amount for distance calculation by one or a combination of a plurality of them.

In the distance image pickup device of the present invention, the distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and the calculated first variable from the selected table information, a first distance associated with a charge ratio smaller than the second variable and a second distance associated with a charge ratio larger than the second variable and determining the measured distance by linearly interpolating the extracted first distance and the second distance.

In the distance image pickup device of the present invention, the pixel includes three or more of the charge storage units, and the distance image processing unit calculates the amount of charge accumulated in each of the three or more charge storage units , the smallest charge amount is set as the charge amount corresponding to the external light component.

In the distance image pickup device of the present invention, the pixel includes three or more charge storage units, and the distance image processing unit stores predetermined external light from among the three or more charge storage units. The amount of charge accumulated in the external light charge accumulating unit is controlled by controlling the timing of accumulating the charge in the external light accumulating charge accumulating unit so that the charge corresponding to the reflected light is not accumulated in the external light charge accumulating unit. is the charge amount corresponding to the external light component.

In the distance image capturing apparatus of the present invention, the storage unit stores first table information that is the table information corresponding to the case where the first variable is smaller than the threshold, and corresponding to the case that the first variable is larger than the threshold. second table information, which is the table information, and the threshold value is a value determined according to a distance deviation caused by charge transfer efficiency in a path from the photoelectric conversion element to the charge storage unit; The image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and if the calculated first variable is smaller than the threshold value, the first table Selecting information and selecting the second table information if the calculated first variable is greater than the threshold.

In the distance image pickup device of the present invention, the storage unit stores the table information for each combination of two distance calculation charge storage units, and the distance image processing unit stores three or more charge storage units. Of these, a combination in which the sum of the amount of charge accumulated in each of the two charge accumulation units whose charge accumulation timings are consecutive is greater than the amount of charge accumulated in each of the other two charge accumulation units, A combination of the distance-calculating charge storage units is determined, and the table information is selected according to the determined combination of the distance-calculating charge storage units.

In the range image pickup device of the present invention, the pixel is provided with three charge storage sections, namely, a first charge storage section, a second charge storage section, and a third charge storage section, and the range image processing section includes controlling the first charge storage unit, the second charge storage unit, and the third charge storage unit so that the charges are stored in this order at timing synchronized with the irradiation of the light pulse; A first calculated value that is a difference between the first charge amount and the third charge amount, using the first charge amount accumulated in the accumulation unit and the third charge amount accumulated in the third charge accumulation unit is calculated, and the first calculated value is set as the distance calculation charge amount of one of the two distance calculation charge storage units.

In the distance image pickup device of the present invention, the pixel is provided with a first charge storage unit, a second charge storage unit, a third charge storage unit, and a fourth charge storage unit, which are the four charge storage units, The distance image processing unit stores the charges in the order of the first charge storage unit, the second charge storage unit, the third charge storage unit, and the fourth charge storage unit at a timing synchronized with the irradiation of the light pulse. controlling the pixel drive circuit to accumulate the first charge using the first charge amount accumulated in the first charge accumulation unit and the third charge amount accumulated in the third charge accumulation unit; calculating a first calculated value that is a difference between the amount of charge and the amount of third charge, and calculating the amount of second charge accumulated in the second charge accumulation section and the amount of fourth charge accumulated in the fourth charge accumulation section; and calculating a second calculated value, which is the difference between the second charge amount and the fourth charge amount, and adding the absolute value of the first calculated value and the absolute value of the second calculated value Let the value be the sum of the distance calculation charge amounts in the two distance calculation charge storage units, the first calculated value be the distance calculation charge amount of one of the two distance calculation charge storage units, and The second calculated value is the other distance-calculating charge amount in the two distance-calculating charge storage units.

A distance image capturing method of the present invention includes a light source unit that irradiates a measurement space in which an object exists with a light pulse, a photoelectric conversion element that generates charges according to the incident light, and a plurality of charge storage units that store the charges. and a pixel driving circuit for distributing and accumulating the charge in each of the charge accumulating portions in the pixel at a predetermined timing synchronized with the irradiation of the light pulse, and the charge accumulating portion. and a first variable based on the charge amount accumulated per unit accumulation count, a first variable based on the charge ratio. A distance image capturing method using a distance image capturing device comprising: a storage unit storing table information indicating a relationship between two variables and a corresponding distance corresponding to the distance to the subject, wherein the first variable is unit accumulation The second variable is a variable corresponding to the sum of the amount of charge accumulated in each of the charge storage units per number of times, and the second variable is the reflection of the light pulse from the three or more charge storage units to the subject. A charge ratio indicated by using a distance calculation charge amount obtained by subtracting an external light component from the charge amount accumulated in each of the two distance calculation charge accumulation units in which the charge amount corresponding to light is distributed and accumulated, The distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and selects the table information corresponding to the calculated first variable. and obtaining the corresponding distance corresponding to the calculated second variable using the selected table information, and determining the measured distance using the obtained corresponding distance.

According to the present invention, even if there is an error in the distance calculated based on the amount of charge accumulated in the charge accumulation section, it is possible to correct the distance so that it approaches the actual distance.

1 is a block diagram showing a schematic configuration of a distance imaging device 1 of an embodiment; FIG. 3 is a block diagram showing a schematic configuration of a distance image sensor 32 of the embodiment; FIG. 3 is a circuit diagram showing an example of the configuration of a pixel 321 of the embodiment; FIG. 4 is a timing chart showing an example of timing for driving a pixel 321 of the embodiment; It is a figure which shows the example of a structure of the table information 440 of embodiment. It is a figure which shows the example of a structure of the table information 440 of embodiment. It is a figure which shows the example of a structure of the table information 440 of embodiment. It is a figure which shows the example of a structure of the table information 440 of embodiment. It is a figure explaining the table information 440 of embodiment. FIG. 10 is a diagram for explaining a process in which the distance image processing unit 4 of the embodiment performs linear interpolation using table information 440; FIG. 4 is a diagram for explaining processing for determining an external light component by the distance image processing unit 4 of the embodiment; FIG. 4 is a diagram for explaining processing for determining an external light component by the distance image processing unit 4 of the embodiment; 4 is a flow chart showing the flow of processing performed by the distance image processing unit 4 of the embodiment; It is a figure explaining the effect of embodiment. FIG. 13 is a diagram showing an example of the configuration of table information 440A in Modification 1 of the embodiment; FIG. 12 is a diagram showing an example of the configuration of table information 440B in Modification 2 of the embodiment; It is a figure explaining two time windows in the modification 3 of embodiment. FIG. 13 is a diagram showing an example of the configuration of table information 440C in Modification 3 of the embodiment; FIG. 13 is a diagram showing an example of the configuration of table information 440D in Modification 3 of the embodiment;

Hereinafter, the distance image capturing device of the embodiment will be described with reference to the drawings.

<Embodiment>
First, an embodiment will be described. FIG. 1 is a block diagram showing a schematic configuration of a distance image pickup device according to a first embodiment of the present invention. A distance image pickup device 1 configured as shown in FIG. FIG. 1 also shows an object OB, which is an object whose distance is to be measured in the distance image pickup device 1 .

Under the control of the distance image processing unit 4 , the light source unit 2 irradiates the space of the imaging target in which the object OB whose distance is to be measured in the distance imaging device 1 exists, with a light pulse PO. The light source unit 2 is, for example, a surface emitting semiconductor laser module such as a vertical cavity surface emitting laser (VCSEL: Vertical Cavity Surface Emitting Laser). The light source unit 2 includes a light source device 21 and a diffuser plate 22 .

The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band of 850 nm to 940 nm) as a light pulse PO to irradiate the subject OB. The light source device 21 is, for example, a semiconductor laser light emitting device. The light source device 21 emits pulsed laser light under the control of the timing control section 41 .

The diffuser plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 over a surface that irradiates the subject OB. The pulsed laser light diffused by the diffusion plate 22 is emitted as a light pulse PO, and is irradiated onto the object OB.

The light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the object OB whose distance is to be measured in the distance image pickup device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving section 3 includes a lens 31 and a distance image sensor 32 .

The lens 31 is an optical lens that guides the incident reflected light RL to the range image sensor 32 . The lens 31 emits the incident reflected light RL to the distance image sensor 32 side, and causes pixels provided in the light receiving area of the distance image sensor 32 to receive the light (incident).

The distance image sensor 32 is an image pickup device used in the distance image pickup device 1 . The distance image sensor 32 has a plurality of pixels in a two-dimensional light receiving area. Each pixel of the distance image sensor 32 is provided with one photoelectric conversion element, a plurality of charge storage units corresponding to this one photoelectric conversion element, and a component for distributing the charge to each charge storage unit. . In other words, the pixel is an imaging device having a distribution structure in which charges are distributed and accumulated in a plurality of charge accumulation units.

The distance image sensor 32 distributes the charges generated by the photoelectric conversion elements to the respective charge storage units according to control from the timing control unit 41 . Also, the distance image sensor 32 outputs a pixel signal corresponding to the amount of charge distributed to the charge storage section. A plurality of pixels are arranged in a two-dimensional matrix in the range image sensor 32, and pixel signals for one frame corresponding to each pixel are output.

The distance image processing unit 4 controls the distance image capturing device 1 and calculates the distance to the subject OB. The distance image processing section 4 includes a timing control section 41 , a distance calculation section 42 , a measurement control section 43 and a storage section 44 . Note that part of the functional units (timing control unit 41 , distance calculation unit 42 , measurement control unit 43 , and storage unit 44 ) of distance image processing unit 4 may be incorporated in distance image sensor 32 .

The timing control section 41 controls the timing of outputting various control signals required for measurement under the control of the measurement control section 43 . The various control signals here include, for example, a signal for controlling the irradiation of the light pulse PO, a signal for allocating and accumulating the reflected light RL to a plurality of charge accumulation units, and controlling the number of times of distribution (number of times of accumulation) per frame. such as a signal to The number of distributions is the number of repetitions of the process of distributing charges to the charge storage section CS (see FIG. 3). The exposure time is the product of the number of times of electric charge distribution and the time (accumulation time Ta, which will be described later) for accumulating electric charges in each electric charge accumulating section for each electric charge distributing process.

The distance calculator 42 calculates the distance to the subject OB based on the pixel signals output from the distance image sensor 32, and outputs the calculated distance information. A distance calculation unit 42 calculates the distance to the subject OB based on the charge amounts accumulated in the plurality of charge accumulation units.

In this embodiment, the distance calculator 42 determines the distance to the object OB using table information 440, which will be described later. The table information 440 will be explained later in detail. A method for determining the distance to the object OB by the distance calculation unit 42 using the table information 440 will be described in detail later.

The measurement control section 43 controls the timing control section 41 . For example, the measurement control unit 43 sets the number of allocation times for one frame, the accumulation time Ta, and the like, and controls the timing control unit 41 so that imaging is performed according to the set contents.

The storage unit 44 is a storage medium such as a HDD (Hard Disk Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read/write Memory), ROM (Read Only Memory), or any of these any combination of storage media. The storage unit 44 stores table information 440, for example. The table information 440 will be explained later in detail.

With such a configuration, in the distance image capturing device 1, the light receiving unit 3 receives the reflected light RL that is reflected by the object OB from the light pulse PO in the near-infrared wavelength band that the light source unit 2 irradiates the object OB. A distance image processing unit 4 outputs distance information obtained by measuring the distance to the subject OB.

Although FIG. 1 shows the distance image pickup device 1 having a configuration in which the distance image processing unit 4 is provided inside, the distance image processing unit 4 is a component provided outside the distance image pickup device 1. may

Next, the configuration of the distance image sensor 32 used as an imaging device in the distance image pickup device 1 will be described. FIG. 2 is a block diagram showing a schematic configuration of an imaging device (distance image sensor 32) used in the distance image pickup device 1 of the embodiment.

As shown in FIG. 2, the distance image sensor 32 includes, for example, a light receiving area 320 in which a plurality of pixels 321 are arranged, a control circuit 322, a vertical scanning circuit 323 having a sorting operation, a horizontal scanning circuit 324, and a pixel signal processing circuit 325 .

The light-receiving region 320 is a region in which a plurality of pixels 321 are arranged, and FIG. 2 shows an example in which the pixels are arranged in a two-dimensional matrix of 8 rows and 8 columns. The pixel 321 accumulates electric charge corresponding to the amount of light received. A control circuit 322 controls the distance image sensor 32 in an integrated manner. The control circuit 322 controls the operation of the components of the range image sensor 32 according to instructions from the timing control section 41 of the range image processing section 4, for example. It should be noted that the components provided in the distance image sensor 32 may be controlled directly by the timing control section 41, in which case the control circuit 322 may be omitted.

The vertical scanning circuit 323 is a circuit that controls the pixels 321 arranged in the light receiving region 320 for each row according to control from the control circuit 322 . The vertical scanning circuit 323 causes the pixel signal processing circuit 325 to output a voltage signal corresponding to the amount of charge accumulated in each charge accumulation portion CS of the pixel 321 . In this case, the vertical scanning circuit 323 distributes the charges converted by the photoelectric conversion elements to the respective charge accumulating portions of the pixels 321 . That is, the vertical scanning circuit 323 is an example of a "pixel driving circuit".

The pixel signal processing circuit 325 performs predetermined signal processing (for example, noise suppression processing) on the voltage signals output to the corresponding vertical signal lines from the pixels 321 in each column under the control of the control circuit 322 . , A/D conversion processing, etc.).

The horizontal scanning circuit 324 is a circuit that sequentially outputs signals output from the pixel signal processing circuit 325 to horizontal signal lines under the control of the control circuit 322 . As a result, pixel signals corresponding to the amount of charge accumulated for one frame are sequentially output to the distance image processing section 4 via the horizontal signal line.

In the following description, it is assumed that the pixel signal processing circuit 325 performs A/D conversion processing and the pixel signal is a digital signal.

Here, the configuration of the pixels 321 arranged in the light receiving area 320 provided in the distance image sensor 32 will be described. FIG. 3 is a circuit diagram showing an example of the configuration of the pixels 321 arranged within the light receiving area 320 of the distance image sensor 32 of the embodiment. FIG. 3 shows an example of the configuration of one pixel 321 among the plurality of pixels 321 arranged in the light receiving region 320. As shown in FIG. The pixel 321 is an example of a configuration including three pixel signal readout units.

As shown in FIG. 3, the pixel 321 includes one photoelectric conversion element PD, a drain gate transistor GD, and three pixel signal readout units RU for outputting voltage signals from corresponding output terminals O. As shown in FIG. Each pixel signal readout unit RU includes a readout gate transistor G, a floating diffusion FD, a charge storage capacitor C, a reset gate transistor RT, a source follower gate transistor SF, and a selection gate transistor SL. In each pixel signal readout unit RU, the floating diffusion FD and the charge storage capacitor C constitute a charge storage unit CS.

In FIG. 3, three pixel signal readout units RU are distinguished by adding numerals "1", "2" or "3" after the symbol "RU" of the three pixel signal readout units RU. do. Similarly, each component provided in the three pixel signal readout units RU is also indicated by a numeral representing each pixel signal readout unit RU after the symbol, so that each component corresponds to the pixel signal readout unit. RUs are distinguished.

In the pixel 321 shown in FIG. 3, the pixel signal readout unit RU1 that outputs a voltage signal from the output terminal O1 includes a readout gate transistor G1, a floating diffusion FD1, a charge storage capacitor C1, a reset gate transistor RT1, and a source follower. It includes a gate transistor SF1 and a selection gate transistor SL1. In the pixel signal readout unit RU1, the charge storage unit CS1 is composed of the floating diffusion FD1 and the charge storage capacitor C1. The pixel signal readout unit RU2 and the pixel signal readout unit RU3 also have the same configuration.

The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate charges and accumulate the generated charges. The structure of the photoelectric conversion element PD may be arbitrary. The photoelectric conversion element PD may be, for example, a PN photodiode having a structure in which a P-type semiconductor and an N-type semiconductor are joined together, or a structure in which an I-type semiconductor is sandwiched between a P-type semiconductor and an N-type semiconductor. It may be a PIN photodiode. Further, the photoelectric conversion element PD is not limited to a photodiode, and may be, for example, a photogate type photoelectric conversion element.

In the pixel 321, the charge generated by photoelectrically converting the light incident on the photoelectric conversion element PD is distributed to each of the three charge storage units CS, and each voltage signal corresponding to the charge amount of the distributed charge is output to the pixel. Output to the signal processing circuit 325 .

The configuration of the pixels arranged in the distance image sensor 32 is not limited to the configuration provided with three pixel signal readout units RU as shown in FIG. Any pixel of the configuration may be used. That is, the number of pixel signal readout units RU (charge storage units CS) provided in the pixels arranged in the distance image sensor 32 may be two, or may be four or more.

Also, in the pixel 321 having the configuration shown in FIG. 3, an example in which the charge storage portion CS is configured by the floating diffusion FD and the charge storage capacitor C is shown. However, the charge storage section CS may be composed of at least the floating diffusion FD, and the pixel 321 may be configured without the charge storage capacitor C. FIG.

Further, although the pixel 321 having the configuration shown in FIG. 3 has an example of the configuration including the drain gate transistor GD, the configuration is not limited to this. For example, when it is not necessary to discard the charge remaining in the photoelectric conversion element PD without being accumulated in the charge accumulation section CS, the configuration without the drain gate transistor GD may be employed.

Next, driving timing of the pixel 321 will be described with reference to FIG. FIG. 4 is a timing chart showing timings for driving the pixels 321 of the embodiment.

In FIG. 4, the timing of emitting the light pulse PO is "L", the timing of receiving the reflected light is "R", the timing of the driving signal TX1 is "G1", the timing of the driving signal TX2 is "G2", and the driving signal The timing of TX3 and the timing of the drive signal RSTD are indicated by item names of "G3" and "GD," respectively. The drive signal TX1 is a signal for driving the readout gate transistor G1. The same applies to the drive signals TX2 and TX3.

As shown in FIG. 4, it is assumed that the light pulse PO is emitted for the irradiation time To, and the reflected light RL is received by the distance image sensor 32 after the delay time Td. The vertical scanning circuit 323 accumulates charges in the order of the charge accumulation units CS1, CS2, and CS3 in synchronization with the irradiation of the light pulse PO. In FIG. 4, in one sorting process, the time from the irradiation of the light pulse PO until the charges are sequentially accumulated in the charge accumulation units CS is expressed as a "unit accumulation period". After repeating the allocation process performed in the "unit accumulation period" for the number of times of accumulation corresponding to one frame, the process of reading out the amount of charge accumulated during that period is performed. The time during which the process of reading out the accumulated charge amount is expressed as a “readout period”.

First, the case of receiving reflected light RL from an object at a short distance will be described with reference to FIG. The vertical scanning circuit 323 turns off the drain gate transistor GD and turns on the readout gate transistor G1 in synchronization with the timing of irradiating the light pulse PO. The vertical scanning circuit 323 turns off the readout gate transistor G1 after the accumulation time Ta has elapsed since turning on the readout gate transistor G1. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G1 is controlled to be on is accumulated in the charge storage unit CS1 via the readout gate transistor G1.

Next, the vertical scanning circuit 323 turns on the readout gate transistor G2 for the accumulation time Ta at the timing when the readout gate transistor G1 is turned off. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G2 is controlled to be on is accumulated in the charge storage unit CS2 via the readout gate transistor G2.

Next, the vertical scanning circuit 323 turns on the readout gate transistor G3 at the timing when the charge accumulation in the charge accumulation unit CS2 is finished, and turns off the readout gate transistor G3 after the accumulation time Ta has elapsed. do. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G3 is controlled to be on is accumulated in the charge storage unit CS3 via the readout gate transistor G3.

Next, the vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge accumulation section CS3 is completed to discharge the charge. As a result, charges photoelectrically converted by the photoelectric conversion element PD are discarded via the drain gate transistor GD.

The vertical scanning circuit 323 repeats the above-described driving a predetermined number of times over one frame. After that, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge distributed to each charge storage section CS. Specifically, the vertical scanning circuit 323 outputs a voltage signal corresponding to the amount of charge accumulated in the charge storage unit CS1 via the pixel signal readout unit RU1 by turning on the selection gate transistor SL1 for a predetermined time. Output from O1. Similarly, the vertical scanning circuit 323 sequentially turns on the selection gate transistors SL2 and SL3 to output voltage signals corresponding to the amounts of charge accumulated in the charge accumulation units CS2 and CS3 from the output terminals O2 and O3. Let Then, an electric signal corresponding to the amount of charge for one frame accumulated in each of the charge accumulation units CS is output to the distance calculation unit 42 via the pixel signal processing circuit 325 and the horizontal scanning circuit 324 .

In the above description, the case where the light source unit 2 irradiates the light pulse PO at the timing when the readout gate transistor G1 is turned on has been described as an example. However, it is not limited to this. The light source unit 2 may irradiate at a timing such that at least the reflected light RL from an object at a short distance is received across the charge storage units CS1 and CS2. For example, the light source unit 2 may emit light at a timing before the readout gate transistor G1 is turned on. Further, in the above description, the case where the irradiation time To for irradiating the light pulse PO is the same length as the accumulation time Ta has been described as an example. However, it is not limited to this. The irradiation time To and the accumulation time Ta may be different time intervals.

In the short-distance light-receiving pixel as shown in FIG. 4, the relationship between the timing of irradiating the light pulse PO and the timing of accumulating the charge in each of the charge accumulating portions CS causes the reflected light RL and the amount of charge corresponding to the external light component is distributed and held. Also, the charge storage section CS3 holds a charge amount corresponding to an external light component such as background light. In this case, the charge storage units CS1 and CS2 are an example of the "distance calculation charge storage unit". In this case, the charge amount corresponding to the reflected light RL accumulated in the charge accumulation units CS1 and CS2 is an example of the "distance calculation charge amount".

The distribution (distribution ratio) of the amount of charge distributed to the charge accumulating units CS1 and CS2 is a ratio corresponding to the delay time Td until the light pulse PO is reflected by the object OB and is incident on the range image pickup device 1 .

Using this principle, the distance calculation unit 42 calculates the delay time Td in the conventional short-distance light-receiving pixel using the following equation (1).

Td=To×(Q2-Q3)/(Q1+Q2-2×Q3) (1)

Here, To is the period during which the light pulse PO is irradiated, Q1 is the amount of charge accumulated in the charge storage section CS1, Q2 is the amount of charge accumulated in the charge storage section CS2, and Q3 is the amount of charge accumulated in the charge storage section CS3. shows the amount of charge. Note that the formula (1) assumes that the charge amount corresponding to the external light component among the charge amounts accumulated in the charge storage units CS1 and CS2 is the same as the charge amount accumulated in the charge storage unit CS3. and

The distance calculation unit 42 multiplies the delay time Td obtained by the formula (1) by the speed of light (velocity) in the short-distance light-receiving pixel, thereby calculating the round-trip distance to the object OB. Then, the distance calculation unit 42 obtains the distance to the object OB by halving the round trip distance calculated above.

Here, in the conventional distance image capturing device 1, the factors that cause an error in the distance (measured distance) calculated from the amount of accumulated charge will be described.

As a cause of the error, it is conceivable that a signal delay occurs in a circuit in the distance image pickup device 1, causing a delay in the rise and fall of the rectangle, resulting in rounding of the waveform. That is, the dullness of the light pulse and the gate pulse and the delay in charge transfer have been the factors that cause an error in the measured distance. As a countermeasure, for example, a method of correcting the error using a correspondence table (an example of "table information") in which the charge ratio R is associated with the distance D is adopted. The distance here is the distance to the subject OB.

The charge ratio R here is the ratio of the amount of charge accumulated in each of the two charge accumulation units CS (charge accumulation units CS1 and CS2 in FIG. 4) in which the reflected light RL is divided and accumulated. In this case, the charge ratio R is represented by, for example, the following equation (2) or (3).

R=Q1#/(Q1#+Q2#) (2)
R=Q2#/(Q1#+Q2#) (3)
However, Q1#=Q1-Qb
Q2#=Q2-Qb
Q1 is the amount of charge accumulated in the charge accumulation section CS1
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Qb is the charge amount corresponding to the external light component accumulated in the charge accumulation section CS.

By using the correspondence table in which the distance D is associated with the charge ratio R, even if an error occurs in the distance calculated by the formula (1), it is possible to correct the distance so as to reduce the error. Become. A method for calculating the charge amount Qb corresponding to the external light component will be described later in detail.

Another possible cause of errors is that the efficiency of transferring charges (transfer efficiency) varies depending on the absolute amount of charges generated in one photoelectric conversion. The transfer here means that charges are accumulated in each of the charge accumulation units CS from the photoelectric conversion element PD.

For example, when the intensity (amount of light) of light incident on the photoelectric conversion element PD is low, the transfer efficiency is lower than when the intensity of light incident on the photoelectric conversion element PD is high. This is thought to be because, for example, when a potential pocket (hole) is formed in the transfer path, electrons associated with the charge being transferred are used to fill the pocket.

For example, when the intensity of light incident on the photoelectric conversion element PD is high, the amount of charge generated in the photoelectric conversion element PD is large. Therefore, even if electrons are used to fill the pockets, the overall charge amount hardly changes, and the transfer efficiency does not decrease so much.

On the other hand, when the intensity of light incident on the photoelectric conversion element PD is low, the amount of charge generated in the photoelectric conversion element PD is small. Therefore, when electrons are used to fill the pockets, the total amount of charge is greatly reduced, resulting in a drop in transfer efficiency.

When a correspondence table in which the distance D is associated with the charge ratio R is used, it is impossible to consider whether the transfer efficiency is good or bad. This is because the charge ratio R is a value calculated based on the charge amount Q1 or Q2 accumulated per frame, and the charge amount accumulated in one accumulation cycle cannot be known. As a result, the measured distance is different from the actual distance, causing a distance deviation.

As a countermeasure against this, in this embodiment, the table information 440 is created for each unit accumulated charge amount Qint. The unit accumulated charge amount Qint is the sum of the amount of charge accumulated in each of the charge accumulation units CS per unit accumulation number (for example, one accumulation number). For example, the unit accumulated charge amount Qint is expressed by the following equation (4).

Qint=QSUM/int (4)
However, QSUM=Q1+Q2+Q3
Q1 is the amount of charge accumulated in the charge accumulation section CS1
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Q3 is the charge amount accumulated in the charge accumulation section CS3
int is the number of accumulations per frame

Alternatively, the table information 440 may be created for each unit accumulated electron number Nint instead of the unit accumulated charge amount Qint. The unit accumulated electron number Nint is the total number of electrons accumulated in each of the charge storage units CS per unit accumulation number (for example, one accumulation number). For example, the unit accumulated electron number Nint is represented by the following equation (5).

Nint=NSUM/int (5)
However, NSUM=Q1/CG1+Q2/CG2+Q3/CG3
NSUM is the sum of the number of electrons accumulated in the charge storage units CS1 to CS3 [e ⁻ ]
int is the number of accumulations per frame
Q1 is the charge amount [V] accumulated in the charge accumulation unit CS1
Q2 is the charge amount [V] accumulated in the charge accumulation section CS2
Q3 is the charge amount [V] accumulated in the charge accumulation section CS3.
CG1 is the conversion gain [V/e ⁻ ] of the charge amount accumulated in the charge accumulation section CS1.
CG2 is the conversion gain [V/e ⁻ ] of the charge amount accumulated in the charge accumulation section CS2.
CG3 is the conversion gain [V/e ⁻ ] of the charge amount accumulated in the charge accumulation section CS3.

The conversion gains CG1 to CG3 are coefficients for converting (converting) charges into the number of electrons, and are predetermined values, for example, in consideration of the layout. The conversion gains CG1 to CG3 may have different values, or all or part of the conversion gains CG1 to CG3 may have the same value.

In this way, with the configuration in which the table information 440 is created for each unit accumulated charge amount Qint (or unit accumulated electron number Nint), an appropriate Table information 440 can be selected.

For example, when the unit accumulated charge amount Qint is large and the transfer efficiency is not lowered, the normal table information 440 is used to correct the distance. On the other hand, when the unit accumulated charge amount Qint is small and the transfer efficiency is low, the distance is corrected using the table information 440 corresponding to the low transfer efficiency. This makes it possible to accurately correct the distance even when the charge transfer efficiency is different. Here, the unit accumulated charge amount Qint is an example of the "first variable". The charge ratio R is an example of a "second variable."

Here, the table information 440 will be explained using FIGS. 5A to 5D. 5A to 5D are diagrams showing examples of the configuration of the table information 440 of the embodiment. FIG. 5A shows table information 440 in which the distance is associated with the charge ratio R when the unit accumulated charge amount is Qint1. FIG. 5B shows table information 440 in which the distance is associated with the charge ratio R when the unit accumulated charge amount is Qint2. FIG. 5C shows table information 440 in which the distance is associated with the charge ratio R when the unit accumulated charge amount is Qint3. FIG. 5D shows table information 440 in which the distance is associated with the charge ratio R when the unit accumulated charge amount is Qint4.

In this example, even if the charge ratio is the same charge ratio R1, if the unit accumulated charge amount Qint1, the distance is D11. On the other hand, if the unit accumulated charge amount is Qint3, the distance is D31. In this way, even if the charge ratio R is the same value, the calculated distance can be set to a different value according to the amount of charge accumulated per accumulation process.

FIG. 6 is a diagram illustrating the table information 440 of the embodiment. FIG. 6 shows the relationship between the charge ratio R and the distance (denoted as actual distance D in FIG. 6) for a certain unit accumulated charge amount Qint. As shown in FIG. 6, the relationship between the charge ratio R and the distance is not always proportional and may be non-linear. In such a case, if the relationship between the charge ratio R and the distance is expressed as a function, it is considered that a complicated multi-dimensional relational expression is obtained. In such a case, a complicated multi-dimensional relational expression must be calculated each time the distance is corrected, resulting in a heavy processing load. By using a correspondence table as shown in the table information 440 of FIG. 5 for the relationship between the charge ratio R and the distance, even if the relationship between the charge ratio R and the distance D becomes non-linear, the processing load can be maintained without increasing the processing load. , it becomes possible to execute processing related to correction.

7A and 7B are diagrams for explaining the process of performing linear interpolation using the table information 440 by the distance image processing unit 4 of the embodiment. FIG. 7 shows part of the characteristics of FIG. 6 (range of charge ratios R1 to R2). In this example, the charge ratio calculated based on the amount of charge accumulated in the photoelectric conversion element PD is a charge ratio R corresponding to an intermediate value between the charge ratios R1 and R2. In this case, the distance calculator 42 may calculate the distance D corresponding to the charge ratio R by linearly interpolating the distance D11 corresponding to the charge ratio R1 and the distance D12 corresponding to the charge ratio R2. . By linear interpolation, it is possible to calculate the distance with higher accuracy.

Further, the interval of the charge ratio in the table information 440 may be set so that the accuracy of the interpolated distance is within a predetermined range when linear interpolation is performed. In this case, the charge ratio interval in the table information 440 is set within a linear range, for example.

Here, a method for calculating the charge amount Qb corresponding to the external light component will be described with reference to FIGS. 8 and 9. FIG. 8 and 9 are diagrams for explaining the process of determining the external light component by the distance image processing unit 4 of the embodiment.

(Method 1 for calculating the charge amount Qb corresponding to the external light component)
FIG. 8 shows a timing chart when receiving reflected light RL from an object at a longer distance than in FIG. 8, the timing at which the vertical scanning circuit 323 irradiates the light pulse PO, the timing at which the readout gate transistors G1 to G3 and the drain gate transistor GD are turned on, etc. are the same as in FIG. omitted.

As shown in FIG. 8, when the delay time Td is longer than in the timing chart of FIG. A charge amount Qb corresponding to the component is accumulated, and the charge amount Qb corresponding to the reflected light RL and the external light component are distributed and accumulated in the charge accumulation units CS2 and CS3. In this case, the charge storage units CS2 and CS3 are an example of the "distance calculation charge storage unit".

That is, when the delay time Td is not large (in the case of FIG. 4), the charge amount Qb corresponding to the external light component is accumulated in the charge storage section CS3, and when the delay time Td is large (in the case of FIG. 8), A charge amount Qb corresponding to the external light component is accumulated in the charge accumulation portion CS1. In either case of FIGS. 4 and 8, the charge amount Qb corresponding to the same amount of external light component is accumulated in each of the charge accumulation units CS1 to CS3. Therefore, the charge storage section CS in which the reflected light RL is distributed and accumulated accumulates a larger amount of charge than the charge storage section CS in which only other external light components are accumulated.

Using this property, the distance calculation unit 42 determines the smallest charge amount among the charge amounts accumulated in the charge accumulation units CS1 to CS3 as the charge amount Qb corresponding to the external light component.

Note that the charge ratio R here is the ratio of the amount of charge accumulated in each of the two charge accumulation units CS (charge accumulation units CS2 and CS3 in FIG. 8) in which the reflected light RL is distributed and accumulated. In this case, the charge ratio R is represented by, for example, the following equation (6) or (7).

R=Q2#/(Q2#+Q3#) (6)
R=Q3#/(Q2#+Q3#) (7)
However, Q2#=Q2-Qb
Q3#=Q3-Qb
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Q3 is the charge amount accumulated in the charge accumulation section CS3
Qb is the charge amount corresponding to the external light component accumulated in the charge accumulation section CS.

(Method 2 for calculating the charge amount Qb corresponding to the external light component)
The distance image pickup device 1 may control the timing so that only the charge amount Qb corresponding to the external light component is accumulated in the predetermined specific charge accumulation section CS. In this case, the distance calculation unit 42 can determine the charge amount accumulated in the specific charge accumulation unit CS as the charge amount Qb corresponding to the external light component regardless of the magnitude of the delay time Td.

FIG. 9 shows a timing chart when control is performed so that only the charge amount Qb corresponding to the external light component is accumulated in the charge accumulation section CS1. In FIG. 9, the timing for the vertical scanning circuit 323 to irradiate the light pulse PO, the timing for turning on the readout gate transistors G1 to G3 and the drain gate transistor GD, etc. are the same as in FIG. omitted.

As shown in the example of FIG. 9, by accumulating charges in the charge accumulating section CS1 before irradiating the light pulse PO, only the charge amount Qb corresponding to the external light component is accumulated in the charge accumulating section CS1. can be In this case, the distance calculation unit 42 determines the charge amount accumulated in the charge accumulation unit CS1 as the charge amount Qb corresponding to the external light component. In this case, the charge storage section CS1 is an example of the "predetermined external light storage charge storage section".

FIG. 10 is a flow chart showing the flow of processing performed by the distance image processing unit 4 of the embodiment. The distance calculation unit 42 acquires the charge amounts Q1 to Q3 accumulated in the charge accumulation units CS1 to CS3, respectively (step S10). The distance calculator 42 uses the acquired charge amounts Q1 to Q3 to calculate the charge amount Qb corresponding to the external light component (step S11). The distance calculation unit 42 may set the smallest charge amount among the charge amounts Q1 to Q3 as the charge amount Qb, or may be accumulated in a predetermined specific charge storage unit CS (for example, the charge storage unit CS1 in FIG. 9). The charge amount obtained may be used as the charge amount Qb.

The distance calculator 42 calculates the charge ratio R using the charge amounts Q1 to Q3 and the charge amount Qb (step S12). The distance calculator 42 calculates the charge ratio R by, for example, substituting the charge amounts Q1 to Q3 and the charge amount Qb into the equation (2) or (3). In this case, the distance calculation unit 42 selects either formula (2) or formula (3) so as to match the configuration of the charge ratio shown in the table information 440, and using the selected formula, A charge ratio R is calculated.

The distance calculator 42 calculates a unit accumulated charge amount Qint using the charge amounts Q1 to Q3 and the number of times of accumulation int (step S13). The accumulation count int is a predetermined value and is stored in the storage unit 44, for example. In this case, the distance calculation unit 42 reads out the number of times of accumulation int from the storage unit 44, and calculates the unit accumulated amount of charge Qint by substituting the amounts of charge Q1 to Q3 and the read out number of times of accumulation int into the equation (4). do.

Using the unit accumulated charge amount Qint calculated in step S13, the distance calculator 42 selects the table information 440 corresponding to the unit accumulated charge amount Qint (step S14). Using the selected table information 440, the distance calculator 42 selects the distance corresponding to the charge ratio R calculated in step S12. In this case, the distance calculation unit 42 may select two distances associated with the two charge ratios for linear interpolation, or may select the two distances that are most closely related to the charge ratio R calculated in step S12. One distance may be selected that corresponds to a close charge ratio.

If two distances are selected (step S16, YES), the distance calculator 42 determines the distance (measured distance) by linear interpolation (step S17). On the other hand, when one distance is selected (step S16, NO), the distance calculator 42 determines the selected distance as the corrected distance (measured distance) (step S18).

In the above description, the case where the pixel 321 of the distance image pickup device 1 includes the three charge storage units CS1 to CS3 has been described as an example. However, it is not limited to this. The pixel 321 of the distance image pickup device 1 may be configured to include four or more (for example, N, N≧4) charge accumulation units CS.

Further, in the flow shown in FIG. 10, it may be determined whether the calculation is valid or invalid. For example, the distance calculator 42 invalidates the calculation when the charge ratio R obtained in step S12 exceeds a predetermined upper limit threshold value (for example, 0.98). Further, the distance calculation unit 42 invalidates the calculation when the charge ratio R obtained in step S12 is below a predetermined lower limit threshold value (for example, 0.12).

When the pixel 321 of the distance image pickup device 1 includes N (N≧4) charge storage units CS, in step S10, the distance calculation unit 42 calculates the charge amount Q1 accumulated in each of the charge storage units CS1 to CSN. ~ Get QN. In step S11, the distance calculator 42 uses the acquired charge amounts Q1 to Q3 to calculate the charge amount Qb corresponding to the external light component. The method by which the distance calculation unit 42 calculates the amount of charge Qb is the same as in the case where the pixel 321 of the distance image pickup device 1 has three charge storage units CS1 to CS3.

In step S12, the distance calculation unit 42 selects two charge storage units CS (distance-measuring charge storage units) in which charges corresponding to the reflected light RL are distributed and stored from the charge storage units CS1 to CSN. The method by which the distance calculation unit 42 selects two charge storage units CS is, for example, based on the amount of charge stored in each charge storage unit CS out of a combination of two charge storage units CS in which charges are continuously stored. are the two charge storage units CS (distance-measuring charge storage units) in which the charges corresponding to the reflected light RL are distributed and accumulated. The distance calculation unit 42 uses the amount of charge accumulated in each of the two charge accumulation units CS in which the charges corresponding to the reflected light RL are distributed and accumulated, and the amount of charge Qb corresponding to the external light component, to calculate the charge ratio Calculate R. The processing of steps S13 to S18 is the same as in the case where the pixel 321 of the distance image pickup device 1 has three charge storage units CS1 to CS3.

Alternatively, the pixel 321 of the distance image pickup device 1 may be configured to include two charge storage units CS. In this case, the distance image pickup device 1 performs a process of accumulating only the charge corresponding to the external light component (referred to as the first process) and a process of accumulating the charge including the reflected light RL (second process) for each measurement. ) and two processes related to charge accumulation. For example, the range image capturing apparatus 1 performs the first process on the first frame and the second process on the next frame. When performing the first process, the distance image pickup device 1 accumulates charges in each of the charge accumulation units CS1 and CS2 without irradiating the light pulse PO. When performing the second process, the distance imaging device 1 irradiates a light pulse PO to accumulate electric charge in each of the electric charge accumulation units CS1 and CS2.

In this case, in step S10, the distance calculation unit 42 acquires the charge amounts Q1f and Q2f accumulated in the charge accumulation units CS1 and CS2 in the first process, respectively. Also, the distance calculation unit 42 acquires the charge amounts Q1s and Q2s accumulated in the charge accumulation units CS1 and CS2 in the second process. In step S11, the distance calculation unit 42 sets either the obtained charge amount Q1f or Q2f as the charge amount Qb corresponding to the external light component. In step S12, the distance calculator 42 calculates the charge ratio R using the acquired charge amounts Q1s, Q2s, and charge amount Qb. The method by which the distance calculation unit 42 calculates the charge ratio R is the same as in the case where the pixel 321 of the distance image pickup device 1 has three charge storage units CS1 to CS3.

As described above, the distance image capturing device 1 of the embodiment includes the light source unit 2, the light receiving unit 3, and the distance image processing unit 4. The light source unit 2 irradiates a light pulse PO into the measurement space in which the subject OB exists. The light receiving unit 3 includes pixels 321 and a vertical scanning circuit 323 (an example of a driving circuit). The pixel 321 includes a photoelectric conversion element PD and multiple charge storage units CS. The vertical scanning circuit 323 distributes and accumulates electric charges in each of the charge accumulation units CS in the pixels 321 at a predetermined timing synchronized with the irradiation of the light pulse PO. The distance image processing section 4 includes a storage section 44 . The storage unit 44 stores table information 440 . The table information 440 is information indicating the relationship between the charge ratio R (an example of the second variable) and the distance (corresponding distance) for each unit accumulated charge amount Qint (an example of the first variable). The unit accumulated charge amount Qint is a variable represented by equation (4) or (5). The unit accumulated charge amount Qint is a variable corresponding to the sum of the amount of charges accumulated in each of the charge accumulation units CS per unit number of times of accumulation. The charge ratio R is a variable represented by formula (2) or formula (3). The charge ratio R is the charge amount (for example, the charge amounts Q1 and Q2 in FIG. 4) accumulated in each of two or more distance calculation charge accumulation units (for example, the charge accumulation units CS1 and CS2 in FIG. 4). is a ratio indicated by using the amount of charge obtained by subtracting the external light component (the amount of charge Qb) from . The charge ratio is the ratio of the amount of charge for distance calculation by any one or a combination of two or more charge storage units for distance calculation. For example, the charge ratio is the ratio of either the charge amount (Q1# or Q2#) to the charge amount (Q1#+Q2#) corresponding to the reflected light RL, that is, the formula (2) or the formula (3) is a variable represented by The distance image processing section 4 calculates the unit accumulated charge amount Qint and the charge ratio R based on the charge amount accumulated in each of the charge accumulation sections CS. The distance image processing unit 4 selects the table information 440 corresponding to the calculated unit accumulated charge amount Qint. The distance calculator 42 uses the selected table information 440 to acquire the distance (corresponding distance) corresponding to the calculated charge ratio R. FIG. The distance calculator 42 uses the acquired corresponding distance to determine the corrected distance (measured distance).

As a result, the distance image capturing device 1 of the embodiment can acquire the distance corresponding to the charge ratio R using the table information 440, and based on the charge amounts Q1 to Q3 accumulated in the charge accumulation units CS1 to CS3. Even if there is an error in the distance calculated by , it can be corrected so as to approach the actual distance. Moreover, in this embodiment, the table information 440 is created for each unit accumulated charge amount Qint. Therefore, appropriate table information 440 can be selected according to the unit accumulated charge amount Qint. Therefore, even if the transfer efficiency varies greatly depending on the absolute value of the amount of charge accumulated per accumulation process, the distance can be accurately corrected.

Further, in the distance image capturing device 1 of the embodiment, the distance image processing unit 4 determines the distance after correction by linear interpolation. Using the table information 440, the distance image processing unit 4 uses the calculated first distance (for example, the distance D11) associated with the charge ratio (for example, the charge ratio R1) smaller than the calculated charge ratio R, and the distance from the charge ratio R A second distance (for example, distance D12) associated with a large charge ratio (for example, charge ratio R2) is extracted. The distance image processing unit 4 determines the distance obtained by linearly interpolating the extracted first distance (for example, distance D11) and second distance (for example, distance D12) as the corrected distance (measured distance). do. As a result, the distance imaging device 1 of the embodiment can correct the distance more accurately.

Also, the pixel 321 of the embodiment includes three or more charge storage units CS1 to CS3. The distance image processing unit 4 sets the smallest charge amount among the charge amounts accumulated in each of the charge accumulation units CS1 to CS3 as the charge amount Qb corresponding to the external light component. As a result, in the distance image pickup device 1 of the embodiment, the charge amount Qb corresponding to the external light component can be calculated by a simple process of comparing the charge amount accumulated in each of the charge accumulation units CS1 to CS3. can.

Also, the pixel 321 of the embodiment includes three or more charge storage units CS1 to CS3. The distance image processing unit 4 controls the accumulation timing so that only the amount of light corresponding to the external light component is accumulated in a specific charge accumulation unit CS among the charge accumulation units CS1 to CS3. The distance calculation unit 42 sets the charge amount accumulated in the specific charge accumulation unit CS as the charge amount Qb corresponding to the external light component. As a result, in the distance image pickup device 1 of the embodiment, the charge amount accumulated in the specific charge accumulation unit CS can be calculated as the charge amount Qb corresponding to the external light component, and the external light component can be easily handled. It is possible to determine the amount of charge Qb to be applied.

(Effect of Embodiment)
Here, the effect of the distance image capturing device 1 of the embodiment will be described with reference to FIG. 11 . FIG. 11 is a diagram explaining the effect of the embodiment. FIG. 11 shows the relationship between the actual distance (actual distance) and the measured distance. The horizontal axis of FIG. 11 indicates the actual distance, and the vertical axis indicates the measured distance. The distance here is the distance to the subject OB. In FIG. 11, the measured distance indicated by black circles when the table information 440 is not used (table information not used) is, for example, the distance calculated by substituting the charge amounts Q1 to Q3 into the equation (1). The measured distance when using the table information 440 indicated by black triangles (using table information) is the distance calculated using the table information 440 . As shown in this figure, the measured distance when using table information matches the actual distance. On the other hand, when the table information is not used, the measured distance does not match the actual distance and is a value including an error. That is, in the distance image pickup device 1 of the present embodiment, by determining the measurement distance using the table information 440, it is possible to calculate a value closer to the actual distance.

(Modification 1 of Embodiment)
Modification 1 of the embodiment will now be described. This modification differs from the above-described embodiment in that the storage unit 44 stores two pieces of table information 440 divided by the threshold value Th.

FIG. 12 is a diagram showing an example of the configuration of table information 440A in modification 1 of the embodiment. As shown in FIG. 12, the upper part of the table information 440A shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is less than the threshold value Th. The lower part of the table information 440A shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is equal to or greater than the threshold value Th. As shown in the example of this figure, the table information 440 (440A) may be a two-dimensional correspondence table in which the unit accumulated charge amount Qint and the charge ratio R are variables.

As already explained, the transfer efficiency depends on the absolute amount of charges transferred in one accumulation process. For this reason, it is considered that the transfer efficiency decreases when the charge amount less than a certain threshold Th is transferred, and the transfer efficiency does not decrease when the charge amount equal to or higher than the threshold is transferred. In this modified example, two pieces of table information 440 (upper and lower rows in the table information 440A) divided by such a threshold value Th are created in advance and stored in the storage unit 44 . The distance calculation unit 42 compares the calculated unit accumulated charge amount Qint with the threshold value Th, and selects either the upper stage or the lower stage of the two table information 440A according to the comparison result. Specifically, when the unit accumulated charge amount Qint is less than the threshold value Th, the distance calculation unit 42 selects the upper correspondence table of the table information 440A. When the unit accumulated charge amount Qint is equal to or greater than the threshold value Th, the distance calculator 42 selects the lower correspondence table of the table information 440A. The distance calculator 42 determines the distance using the selected correspondence table.

As described above, in the range image capturing apparatus 1 according to the modification 1 of the embodiment, the storage unit 44 stores information corresponding to the upper stage of the table information 440A (an example of the "first table information") and table information 440A. Two pieces of information are stored, one being information corresponding to the lower stage (an example of "second table information"). Information corresponding to the upper row of the table information 440A is information corresponding to the case where the unit accumulated charge amount Qint is smaller than the threshold value Th. Information corresponding to the lower part of the table information 440A is information corresponding to the case where the unit accumulated charge amount Qint is greater than the threshold value Th. The threshold Th is a value determined according to the charge transfer efficiency in the path from the photoelectric conversion element PD to the charge storage section CS. The distance image processing unit 4 calculates the unit accumulated charge amount Qint and the charge ratio R based on the charge amount accumulated in each charge accumulation unit CS. The distance image processing unit 4 selects information corresponding to the upper stage of the table information 440A when the calculated unit accumulated charge amount Qint is smaller than the threshold value Th. The distance image processing unit 4 selects the information corresponding to the lower part of the table information 440A when the calculated unit accumulated charge amount Qint is larger than the threshold value Th.

As a result, in the distance image pickup device 1 according to Modification 1 of the embodiment, one of the two table information 440 corresponding to the charge transfer efficiency can be selected, and the table to be stored in the storage unit 44 can be selected. Even if the number of pieces of information 440 is small, it is possible to determine the distance with high accuracy.

(Modification 2 of Embodiment)
Modification 2 of the embodiment will now be described. This modification differs from the above-described embodiment in that the storage unit 44 stores table information 440 corresponding to the range of the unit accumulated charge amount Qint.

FIG. 13 is a diagram showing an example of the configuration of table information 440B in Modification 2 of the embodiment. The upper portion of the table information 440B shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is equal to or greater than the threshold Th1 and less than the threshold Th2. The lower part of the table information 440B shows the relationship between the charge ratio R and the distance D when the unit accumulated charge amount Qint is equal to or greater than the threshold Th2 and less than the threshold Th3.

In this modification, the range of the unit accumulated charge amount Qint in the table information 440B corresponds to, for example, a range in which the tendency of transfer efficiency is similar. The distance calculation unit 42 determines which range of the unit accumulated charge amount Qint in the table information 440B the calculated unit accumulated charge amount Qint corresponds to, and selects one of the table information 440B according to the determination result. The distance calculator 42 determines the distance using the selected correspondence table. As a result, the table information 440 can be selected according to the tendency of the charge transfer efficiency, and the distance can be determined with high accuracy.

(Modification 3 of Embodiment)
Here, Modification 3 of the embodiment will be described. This modification differs from the above-described embodiment in that the storage unit 44 stores table information 440 for each of a plurality of time windows. The time window here corresponds to a combination of charge storage units for distance measurement.

For example, in FIG. 4, the combination of the charge storage units for distance measurement is the charge storage units CS1 and CS2, and this combination corresponds to the first time window. The distance D is determined according to the charge ratio R of the charges distributed and accumulated in the first time window.

Also, in FIG. 8, the combination of the charge storage units for distance measurement is the charge storage units CS2 and CS3. This combination corresponds to the second time window. The distance D is determined according to the charge ratio R of the charges distributed and accumulated in the second time window.

FIG. 14 is a diagram illustrating characteristics of two time windows in Modification 3 of the embodiment. The characteristic here is a characteristic indicating the correspondence relationship between the actual distance and the measured distance. The horizontal axis of FIG. 14 indicates the actual distance, and the vertical axis indicates the measured distance. A characteristic L0 indicates an ideal relationship between the actual distance and the measured distance. A characteristic L1 indicates the relationship between the actual distance and the measured distance in the first time window. A characteristic L2 indicates the relationship between the actual distance and the measured distance in the second time window. As shown in the example of this figure, the correspondence relationship between the actual distance and the measured distance is often different between one time window and another time window. Therefore, when correcting the distance in another time window using the table information 440 that allows accurate correction in a certain time window, accurate correction is not always possible.

As a countermeasure against this, in this modified example, table information 440 for each time window is created in advance in the storage unit 44 and stored in the storage unit 44 .

15 and 16 are diagrams showing examples of the configuration of table information in Modification 3 of the embodiment. FIG. 15 shows table information 440C used for a combination of charge storage units for distance measurement (for example, charge storage units CS1 and CS2) corresponding to a certain time window. FIG. 16 shows table information 440D used for a combination of charge storage units for distance measurement (for example, charge storage units CS2 and CS3) corresponding to another time window. As shown in FIGS. 15 and 16, in the table information 440C (400D), even if the unit accumulated charge amount Qint and the charge ratio R are the same value, if the time windows are different, the distance D is different. are mapped. By creating the table information 440 for each time window in this way, it is possible to perform correction according to each time window.

As described above, in the distance image capturing device 1 according to the modification 3 of the embodiment, the storage unit 44 stores the table information 440C (400D) for each time window (combination of two charge storage units for distance calculation). Remember. The distance image processing unit 4 determines a combination of the two charge storage units for distance calculation based on the amount of charge stored in each of the charge storage units CS. The distance image processing unit 4 calculates the sum of the amount of charge accumulated in each of the two charge accumulation units CS (for example, the charge accumulation units CS1 and CS2) of the charge accumulation units CS1 to CS3 whose charge accumulation timings are consecutive. A combination in which (Q1+Q2) is larger than the amount of charge accumulated in each of the other two charge accumulation units CS is determined as a combination of charge accumulation units for distance calculation. The distance image processing unit 4 selects the table information 440C (400D) in accordance with the determined combination of distance calculation charge storage units.

As a result, the distance image capturing device 1 according to the third modification of the embodiment can select the table information 440 according to the time window, and the correspondence relationship between the actual distance and the measured distance tends to differ for each time window. Even in this case, the table information 440 suitable for each time window can be selected, and the distance can be determined with high accuracy.

(Modification 4 of Embodiment)
Modification 4 of the embodiment will now be described. In this modification, the charge ratio R is calculated without specifying the charge storage portion CS in which only the external light is stored or the two charge storage portions CS (distance calculation charge storage portions) in which the reflected light RL is divided and stored. is different from the above-described embodiment in that it calculates .

In this modification, the distance calculator 42 uses the method described in Patent Document WO2019/031510. Patent document WO2019/031510 describes a technique of selecting an arithmetic expression to be used for distance calculation depending on whether an index value exceeds a predetermined threshold. The index value here is the "distance data validity determination signal" in Patent Document WO2019/031510. Further, the arithmetic expression here is the "distance reference value" in Patent Document WO2019/031510, and corresponds to the "charge ratio R" of the present embodiment. Specific methods for calculating the charge ratio R will be described below separately for the case where the pixel 321 includes three charge storage units CS and the case where the pixel 321 includes four charge storage units CS.

(When the pixel 321 has three charge storage units CS)
In this modification, the distance calculator 42 calculates the charge ratio R using the following equation (8) or (9). Here, according to the timing charts shown in FIGS. 4 and 8, charges are accumulated in the order of the charge accumulation units CS1, CS2, and CS3. That is, the distance calculation unit 42 controls the charge accumulation units CS1, CS2, and CS3 so that charges are accumulated in this order at the timing synchronized with the irradiation of the light pulse PO. In this case, the charge storage section CS1 is an example of the "first charge storage section". The charge storage section CS2 is an example of a "second charge storage section". The charge storage section CS3 is an example of a "third charge storage section". Also, the charge amount accumulated in the charge accumulation unit CS1 is an example of the "first charge amount". The charge amount accumulated in the charge accumulation unit CS2 is an example of the "second charge amount". The charge amount accumulated in the charge accumulation unit CS3 is an example of the "third charge amount".

R=1-(Q1-Q3)/SA (8)
R=(Q1-Q3)/SA (9)
however,
SA=|Q1−Q3|+Q2−0.5×SB
SB=|Q1+Q3|-|Q1-Q3|
Q1 is the amount of charge accumulated in the charge accumulation section CS1
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Q3 is the charge amount accumulated in the charge accumulation section CS3

The storage unit 44 stores table information 440 in which the distance is associated with the charge ratio R for each unit accumulated charge amount Qint. The distance calculator 42 calculates the unit accumulated charge amount Qint, and uses the table information 440 corresponding to the calculated unit accumulated charge amount Qint to set the distance associated with the charge ratio R as the measured distance.

(When the pixel 321 has four charge storage units CS)
First, the drive timing of the pixel 321 when the pixel 321 has four charge storage units CS will be described. In this case, for example, the item of the readout gate transistor G4 is added to FIGS. 4 and 8, and charges are accumulated in the order of the charge accumulation units CS1, CS2, CS3, and CS4. In this case, the charge storage section CS1 is an example of the "first charge storage section". The charge storage section CS2 is an example of a "second charge storage section". The charge storage section CS3 is an example of a "third charge storage section". The charge storage section CS4 is an example of a "fourth charge storage section". Also, the charge amount accumulated in the charge accumulation unit CS1 is an example of the "first charge amount". The charge amount accumulated in the charge accumulation unit CS2 is an example of the "second charge amount". The charge amount accumulated in the charge accumulation unit CS3 is an example of the "third charge amount". The charge amount accumulated in the charge accumulation unit CS4 is an example of the "fourth charge amount".

Specifically, at the timings shown in FIGS. 4 and 8, irradiation of the light pulse PO, control to turn off the drain gate transistor GD, and control to turn on the readout gate transistors G1 to G3 are performed. Next, the vertical scanning circuit 323 turns on the readout gate transistor G4 at the timing when the charge accumulation in the charge accumulation unit CS3 is completed, and turns off the readout gate transistor G4 after the accumulation time Ta has elapsed. do. As a result, the charge photoelectrically converted by the photoelectric conversion element PD while the readout gate transistor G4 is controlled to be on is accumulated in the charge storage unit CS4 via the readout gate transistor G4. Next, the vertical scanning circuit 323 turns on the drain gate transistor GD at the timing when the charge accumulation in the charge accumulation unit CS4 is completed to discharge the charge. As a result, charges photoelectrically converted by the photoelectric conversion element PD are discarded via the drain gate transistor GD.

Based on the amount of charge accumulated in each of the charge accumulation units CS controlled at the timing as described above, the distance calculation unit 42 uses the following equation (10) or (11) to calculate the charge ratio Calculate the XR.

XR=1-(Q1-Q3)/SA (10)
XR=(Q1-Q3)/SA (11)
however,
SA=|Q1−Q3|+|Q2−Q4|
Q1 is the amount of charge accumulated in the charge accumulation section CS1
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Q3 is the charge amount accumulated in the charge accumulation section CS3
Q4 is the amount of charge accumulated in the charge accumulation section CS4

Further, the distance calculator 42 calculates the charge ratio YR using the following equation (12) or (13).

YR=2-(Q2-Q4)/SA (12)
YR=1+(Q2-Q4)/SA (13)
however,
SA=|Q1−Q3|+|Q2−Q4|
Q1 is the amount of charge accumulated in the charge accumulation section CS1
Q2 is the amount of charge accumulated in the charge accumulation section CS2
Q3 is the charge amount accumulated in the charge accumulation section CS3
Q4 is the amount of charge accumulated in the charge accumulation section CS4

The distance calculator 42 compares the charge ratio XR with the threshold value ThR. The threshold ThR is set in the vicinity of the value of the charge ratio XR corresponding to the vicinity of switching of the time windows. The distance calculator 42 selects the charge ratio XR as the charge ratio R when the charge ratio XR is equal to or less than the threshold ThR. On the other hand, the distance calculator 42 selects the charge ratio YR as the charge ratio R when the charge ratio XR exceeds the threshold ThR.

The storage unit 44 stores table information 440 in which the distance is associated with the charge ratio R for each unit accumulated charge amount Qint. The distance calculator 42 calculates the unit accumulated charge amount Qint, and uses the table information 440 corresponding to the calculated unit accumulated charge amount Qint to set the distance associated with the charge ratio R as the measured distance.

As described above, in the distance image capturing device 1 according to Modification 4 of the embodiment, the pixel 321 includes three charge storage units CS1 to CS3. The distance image processing unit 4 controls the charge storage units CS1, CS2, and CS3 so that charges are accumulated in this order at timing synchronized with the irradiation of the light pulse PO. The distance calculation unit 42 converts (Q1-Q3) into the reflected light accumulated in one of the two distance calculation charge accumulation units CS as shown in the formula (8) or (9). It is assumed that the amount of charge corresponding to RL (the amount of charge for distance calculation). Q1 is the charge amount accumulated in the charge accumulation section CS1. Q3 is the amount of charge accumulated in the charge accumulation section CS3.

As a result, in the distance image pickup device 1 according to Modification 4 of the embodiment, the distance calculation accumulated in either one of the two distance calculation charge accumulation units is not specified without specifying the two distance calculation charge accumulation units. The amount of charge used can be calculated. For this reason, as indicated by SA in equation (8) or (9), the sum of the distance calculation charge amounts accumulated in the two distance calculation charge accumulation units is calculated to obtain the charge ratio R. It is possible to calculate Therefore, in the distance image capturing device 1 according to the modification 4 of the embodiment, the equation (2) or (3) is applied, or the equation (6) or (7) is applied depending on the length of the delay time Td. is applied, the charge ratio R can be easily calculated without specifying the charge storage portion CS in which only the external light is accumulated and calculating the charge amount Qb corresponding to the external light component. can be calculated. Furthermore, since the same arithmetic expression (equation (8) or (9)) can be applied to the boundaries of the two time windows, it is possible to improve the discontinuity at the boundaries of the time windows.

In addition, in the distance image pickup device 1 according to Modification 4 of the embodiment, the pixel 321 includes four charge storage units CS1 to CS4. The distance image processing unit 4 controls the charge storage units CS1, CS2, CS3, and CS4 so that charges are accumulated in this order at a timing synchronized with the irradiation of the light pulse PO. The distance calculation unit 42 sets (Q1-Q3) to the signal amount calculated from the charge amount accumulated in one of the two distance calculation charge accumulation units CS. The distance calculation unit 42 sets (Q2-Q4) to the signal amount calculated from the charge amount accumulated in the other charge accumulation unit CS of the two distance calculation charge accumulation units. The distance calculation unit 42 sets |Q1−Q3|+|Q2−Q4| to be the sum of the signal amounts calculated from the charge amounts accumulated in the respective charge accumulation units CS of the two distance calculation charge accumulation units. .

As a result, in the distance image pickup device 1 according to Modification 4 of the embodiment, the sum of the distance calculation charge amount accumulated in each of the two distance calculation charge accumulation units, or the two distance calculation charge accumulation units The amount of charge for distance calculation accumulated in one of them and the amount of charge for distance calculation accumulated in the other can be calculated. Therefore, the charge ratio R can be calculated without specifying the two distance calculation charge storage units.

Furthermore, in this case, the charge ratio XR in the equation (10) and the charge ratio YR in the equation (12) have the same value at the boundaries of the time windows. Therefore, it is possible to improve the discontinuity at the boundaries of the time windows.

In at least one embodiment described above, the charge ratio R of either one of the two distance-calculation charge storage units to the sum of the distance-calculation charge amount accumulated in each of the two distance-calculation charge storage units is the ratio of charge amounts for distance calculation. However, it is not limited to this. The charge ratio R may be a ratio indicated by using the distance calculation charge amount accumulated in each of at least two distance calculation charge accumulation units. For example, the charge ratio R may be the ratio of the distance calculation charge amount of one of the two distance calculation charge storage units to the distance calculation charge amount of the other.

Further, in FIG. 9, by accumulating charges in the charge accumulating section CS1 before irradiating the light pulse PO, control is performed so that only the charge amount Qb corresponding to the external light component is accumulated in the charge accumulating section CS1. was explained as an example. However, it is not limited to this. After irradiating the light pulse PO and receiving the reflected light RL, the electric charge may be accumulated in the electric charge accumulating section CS3. In this case, only the charge amount Qb corresponding to the external light component is accumulated in the charge accumulation section CS3. In this case, the charge storage section CS3 is an example of the "predetermined external light storage charge storage section".
Further, when the charge storage section CS for storing the charge amount Qb corresponding to the external light component is fixed, as shown in FIG. 8, the charge storage sections CS1 to CS3 are continuously turned on. good too.
Further, when the charge storage units CS1 to CS3 are continuously turned on, the driving may be controlled so as to insert a gap at the timing of switching each on state. The GAP here is driving for suppressing overlap in which the charge storage units CS are turned on at the same time, and controls all of the charge storage units CS to be turned off. For example, when the irradiation time To for irradiating the light pulse PO is 10 [clk], the accumulation time Ta for accumulating charges in the charge accumulation unit CS is 9 [clk], and the GAP is 1 [clk]. Then, control is performed to provide a gap at the timing of switching the charge storage section CS from the off state to the on state or at the timing of switching the charge storage section CS from the on state to the off state.

All or part of the distance image capturing device 1 and the distance image processing unit 4 in the above-described embodiments may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically retains a program for a short period of time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include something that holds the program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing a part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be implemented using a programmable logic device such as FPGA.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design and the like are included within the scope of the gist of the present invention.

REFERENCE SIGNS LIST 1 distance image pickup device 2 light source unit 3 light receiving unit 32 distance image sensor 321 pixel 323 vertical scanning circuit 4 distance image processing unit 41 timing control unit 42 distance calculation unit 43 measurement control unit 44 Memory unit 440 Table information CS Charge accumulation unit PO Optical pulse

Claims (10)

a light source unit that irradiates a light pulse into a measurement space in which an object exists;
A pixel including a photoelectric conversion element that generates a charge according to incident light and a plurality of charge storage units that store the charge, and the charge is stored in the pixel at a predetermined timing synchronized with the irradiation of the light pulse. a light-receiving unit having a pixel drive circuit that distributes and accumulates the electric charge in each of the units;
a distance image processing unit that determines a measured distance to the subject based on the amount of charge accumulated in each of the charge accumulation units;
a storage unit for storing table information indicating a relationship between a second variable based on the charge ratio and a corresponding distance corresponding to the distance to the object for each first variable based on the amount of charge accumulated per unit accumulation;
with
The first variable is a variable corresponding to the sum of the amount of charge accumulated in each of the charge accumulation units per unit accumulation count,
The second variable is each of two or more distance calculation charge storage units in which the amount of charge corresponding to the reflected light of the light pulse reflected by the object is distributed and accumulated among the three or more charge storage units. is the charge ratio indicated using the charge amount for distance calculation obtained by subtracting the external light component from the charge amount accumulated in
The distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and selects the table information corresponding to the calculated first variable. , using the selected table information to obtain the corresponding distance corresponding to the calculated second variable, and using the obtained corresponding distance to determine the measured distance;
Range imaging device. The charge ratio is defined by any one or a combination of two or more distance-calculating charge accumulating units with respect to the sum of the distance-calculating charge amounts accumulated in each of the distance-calculating charge accumulating units. which is the ratio of the charge amount for distance calculation,
The distance image pickup device according to claim 1. The distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and selects the table information corresponding to the calculated first variable. a first distance associated with a charge ratio smaller than the second variable and a second distance associated with a charge ratio larger than the second variable are extracted from the selected table information; determining the measured distance by linearly interpolating the first distance and the second distance;
The distance image pickup device according to claim 1. the pixel comprises three or more of the charge storage units;
The distance image processing unit sets the smallest charge amount among the charge amounts accumulated in each of the three or more charge accumulation units as the charge amount corresponding to the external light component,
The range imaging device according to any one of claims 1 to 3. the pixel comprises three or more of the charge storage units;
The distance image processing unit controls the external light charge accumulation unit so that the charge corresponding to the reflected light is not accumulated in a predetermined external light charge accumulation unit among the three or more charge accumulation units. controlling the timing of accumulating electric charge in the external light accumulating electric charge accumulating unit, and setting the amount of electric charge accumulated in the electric charge accumulating unit for accumulating external light as the electric charge amount corresponding to the external light component;
The range imaging device according to any one of claims 1 to 3. The storage unit stores first table information, which is the table information corresponding to the case where the first variable is smaller than the threshold, and second table information, which is the table information corresponding to the case where the first variable is larger than the threshold. and
The threshold value is a value determined according to the charge transfer efficiency in a path from the photoelectric conversion element to the charge storage unit,
The distance image processing unit calculates the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units, and calculates the first variable when the calculated first variable is smaller than the threshold value. selecting one table information and selecting the second table information when the calculated first variable is greater than the threshold;
The distance image pickup device according to claim 1. the storage unit stores the table information for each combination of the two distance calculation charge storage units;
In the distance image processing section, the sum of the amount of charge accumulated in each of two of the three or more charge accumulation sections whose charge accumulation timings are consecutive is equal to the sum of the amounts of charges accumulated in the other two charge accumulation sections. determining a combination larger than the amount of charge accumulated in each of the storage units as the combination of the distance-calculating charge storage units, and selecting the table information according to the determined combination of the distance-calculating charge storage units;
The distance image pickup device according to claim 1. the pixel is provided with a first charge storage unit, a second charge storage unit, and a third charge storage unit, which are the three charge storage units;
The distance image processing unit is
controlling so that the charge is accumulated in the order of the first charge storage unit, the second charge storage unit, and the third charge storage unit at a timing synchronized with the irradiation of the light pulse;
A difference between the first charge amount and the third charge amount using the first charge amount accumulated in the first charge accumulation unit and the third charge amount accumulated in the third charge accumulation unit calculating a first calculated value, and using the first calculated value as the distance calculation charge amount of one of the two distance calculation charge storage units;
The distance image pickup device according to claim 1. The pixel is provided with a first charge storage portion, a second charge storage portion, a third charge storage portion, and a fourth charge storage portion, which are the four charge storage portions,
The distance image processing unit is
The pixels are arranged such that the charges are accumulated in the order of the first charge accumulation unit, the second charge accumulation unit, the third charge accumulation unit, and the fourth charge accumulation unit at timing synchronized with the irradiation of the light pulse. control the drive circuit,
A difference between the first charge amount and the third charge amount using the first charge amount accumulated in the first charge accumulation unit and the third charge amount accumulated in the third charge accumulation unit Calculate a first calculated value,
A difference between the second charge amount and the fourth charge amount using the second charge amount accumulated in the second charge accumulation unit and the fourth charge amount accumulated in the fourth charge accumulation unit Calculate a second calculated value,
The addition value obtained by adding the absolute value of the first calculated value and the absolute value of the second calculated value is defined as the sum of the distance calculation charge amounts in the two distance calculation charge storage units, and the first calculated value is One of the distance calculation charge amounts in the two distance calculation charge storage units is taken as the other distance calculation charge amount in the two distance calculation charge storage units, and the second calculated value is taken as the other distance calculation charge amount in the two distance calculation charge storage units.
The distance image pickup device according to claim 1. A light source unit that irradiates a measurement space in which an object exists with a light pulse, a pixel that includes a photoelectric conversion element that generates charges according to the incident light, and a plurality of charge storage units that store the charges, and the light pulse. a light-receiving portion having a pixel drive circuit that distributes and accumulates the charge in each of the charge accumulation portions in the pixel at a predetermined timing synchronized with the irradiation of the pixel; a distance image processing unit that determines the measured distance to the subject based on the distance to the subject; A distance image capturing method using a distance image capturing device comprising a storage unit that stores table information indicating a relationship with a corresponding corresponding distance,
the first variable is a variable corresponding to the sum of the amount of charge accumulated in each of the charge accumulation units per unit accumulation count;
The second variable is each of two or more distance calculation charge storage units in which the amount of charge corresponding to the reflected light of the light pulse reflected by the object is distributed and accumulated among the three or more charge storage units. is the charge ratio indicated using the charge amount for distance calculation obtained by subtracting the external light component from the charge amount accumulated in
The distance image processing unit is
calculating the first variable and the second variable based on the amount of charge accumulated in each of the charge accumulation units;
selecting the table information corresponding to the calculated first variable;
Acquiring the corresponding distance corresponding to the calculated second variable using the selected table information,
determining the measured distance using the obtained corresponding distance;
Range image capturing method.

Images