JP7014324B1 - Distance image imaging device and distance image imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To remove a fluctuation corresponding to a charge amount of noise charge when correcting a variation of a charge amount read from each of the charge storage units by a transfer transistor, improve the accuracy of distance calculation, and capture an accurate distance image. An image imaging device is provided. According to the present invention, a light source unit that irradiates an optical pulse, a photoelectric conversion element that generates a charge according to incident light, a pixel circuit including a charge storage unit that stores an electric charge, and a pixel circuit that stores the electric charge are synchronized with the optical pulse. A light receiving unit having a pixel drive circuit that performs on / off processing of the transfer transistor to distribute and store charges, and a background light charge amount generated by background light and distributed from the photoelectric conversion element by the transfer transistor and stored in the charge storage unit. , The noise charge amount estimation unit that estimates the noise charge amount using the ratio to the noise charge amount that is generated by the background light and flows into the charge storage unit and is stored, and the accumulated charge stored in each of the charge storage units. It is provided with a distance calculation unit that calculates the distance to the subject by subtracting the amount of noise charge from the amount. [Selection diagram] Fig. 1

Description

The present invention relates to a distance image imaging device and a distance image imaging method.

Conventionally, time-of-flight (Time of Fright, hereinafter referred to as "ToF") method of distance image imaging that measures the distance to the subject based on the flight time of light by utilizing the known speed of light. There is an apparatus (see, for example, Patent Document 1).
The ToF distance image imaging device includes an imaging unit including a light source unit that irradiates light and a pixel array in which a plurality of pixel circuits for detecting light for measuring distance are arranged in a two-dimensional matrix (array shape). I have. Each of the pixel circuits has a photoelectric conversion element (for example, a photodiode) that generates an electric charge corresponding to the intensity of light as a component.
With this configuration, the ToF distance image imaging device can acquire (image) information on the distance between itself and the subject and an image of the subject in the measurement space (three-dimensional space).

JP-A-2015-29054

Since it is used for mobile terminals such as smartphones and tablet terminals, it is required to reduce the size and height (thinness) of the housing (imaging device) of the distance image imaging device.
Further, in order to improve the definition of the image captured by the distance image imaging device, it is required to increase the number of pixels in the imaging unit.
Therefore, the incident angle of the incident light incident on the pixel circuit arranged at the end of the pixel array is larger than the incident angle of the incident light incident on the pixel circuit arranged near the center. ..

As the incident angle of the incident light incident on the pixel circuit increases, not only the photoelectric conversion element in the pixel circuit but also the circuit around the photoelectric conversion element, for example, the charge storage portion for accumulating charges, is irradiated.
Then, not only in the photoelectric conversion element but also in the peripheral circuit (including the charge storage part), charges corresponding to the light irradiated by the photoelectric effect (hereinafter referred to as noise charges) are generated, and these are generated in the charge storage part having a high potential. Inflow.

Further, since the characteristics of the transfer transistor that distributes the charge from the photoelectric conversion element to each of the charge storage portions are different, it is necessary to correct the characteristics of the transfer transistor.
This correction is performed by multiplying the sensitivity correction coefficient set for each transfer transistor by the amount of charge read from the charge storage unit.
Here, since the noise charge is generated in the semiconductor substrate and flows into each of the charge storage units without passing through the transfer transistor, it is estimated that the amount of charge stored in each of the charge storage units is the same. To.

However, by multiplying the sensitivity read out from the charge storage unit by the sensitivity correction coefficient set for each charge storage unit, the charge amount due to the noise charge in each of the charge amounts is also corrected.
That is, since it is not affected by the variation in the characteristics of the transfer transistor, the amount of charge due to the noise charge, which is the same in each charge storage unit, is excessively corrected by the sensitivity correction coefficient.
As a result, the accuracy of the distance calculation is lowered because the amount of charge due to the noise charge varies in the amount of charge read from each charge storage unit.

The present invention has been made in view of such a situation, and when correcting the variation due to the transfer transistor in the amount of charge read from each of the charge storage units, the noise charge by multiplying the amount of charge by the sensitivity correction coefficient is used. It is an object of the present invention to provide a distance image imaging device and a distance image imaging method for capturing fluctuations due to the amount of electric charge, improving the accuracy of distance calculation, and capturing a more accurate distance image.

In order to solve the above-mentioned problems, the distance image imaging apparatus of the present invention includes a light source unit that irradiates the measurement space with an optical pulse, a photoelectric conversion element that generates an electric charge according to the light incident from the measurement space, and a frame. A plurality of pixel circuits including a plurality of charge storage units that store the charge in a cycle, and each of the charge storage units are subjected to on / off processing of each transfer transistor at a predetermined storage timing synchronized with the irradiation of the optical pulse. A light receiving unit having a pixel drive circuit that distributes and stores charges, and an amount of background light charges that are generated by background light in the measurement space and distributed from the photoelectric conversion element by the transfer transistor and stored in the charge storage unit. And adjusted from the background light charge amount using the noise charge inflow ratio, which indicates the ratio to the noise charge amount generated by the background light and flowing into and accumulating in the charge storage unit without passing through the transfer transistor. The adjusted noise charge amount is subtracted from each of the noise charge amount estimation unit for estimating the noise charge amount and the accumulated charge amount stored in each of the charge storage units, and the corrected charge amount of the subtraction result is obtained. It is characterized by having a distance calculation unit that calculates the distance to the subject by multiplying by an adjustment coefficient that corrects the variation due to each on characteristic of the transfer transistor and using the adjustment correction charge amount that is the result of the multiplication. And.

In the distance image imaging apparatus of the present invention, the pixel drive circuit transfers from the photoelectric conversion element to each of the charge storage units before the execution of the frame cycle for measuring the distance without irradiating the optical pulse. The first ratio acquisition frame, which performs on / off processing of each of the transistors to distribute and store the charges to acquire the reference accumulated charge amount, and the transfer transistor without irradiating the optical pulse and performing the on / off processing of each transfer transistor, said. The second ratio acquisition frame for acquiring the reference noise charge amount flowing into and accumulating in each of the charge storage units was executed, and the noise charge amount estimation unit subtracted the reference noise charge amount from the reference stored charge amount. It is characterized in that the noise charge inflow ratio is obtained from the reference background light charge amount of the result and the reference noise charge amount.

In the distance image imaging device of the present invention, the noise charge amount estimation unit is the total of the number of distributions, which is the number of distributions in the first ratio acquisition frame, multiplied by the first on-time when the transfer transistor is turned on. The unit reference background light charge amount is obtained by dividing the reference background light charge amount by the transfer time, and the unit reference background light charge amount is multiplied by the first storage period for accumulating the charge in the first ratio acquisition frame. The reference noise charge amount is divided by the result, and the division result is used as the noise charge inflow ratio.

In the distance image imaging apparatus of the present invention, the noise charge amount estimation unit is adjusted by multiplying the reference background light charge amount by an adjustment coefficient for correcting variations due to the on characteristics of each of the transfer transistors. The reference background light charge amount is used for calculating the noise charge inflow ratio.

In the distance image imaging apparatus of the present invention, the noise charge amount estimation unit obtains the adjusted noise charge amount by subtracting the background light charge amount from the minimum charge amount which is the minimum charge amount among the accumulated charge amounts. The first-order equation of 1 and the multiplication result of the second accumulation period in the frame period and the noise charge inflow ratio with respect to the background light charge amount are multiplied by the second on-time and the second vibration of the transfer transistor in the frame period. To solve the binary linear equation consisting of the second linear equation for obtaining the adjusted noise charge amount by dividing by the multiplication result with the number of minutes, and to obtain the adjusted noise charge amount and the background light charge amount which are unknown. It is a feature.

The range image imaging apparatus of the present invention has the minimum charge amount, which is the minimum charge amount among the accumulated charge amounts, the noise charge inflow ratio, the second storage period in the frame period, and the transfer transistor in the frame period. The adjusted noise charge amount is used by using a lookup table in which the multiplication result of the second on-time of the above and the second distribution number of the transfer transistor in the frame period is used as an input variable and the adjusted noise charge amount is output. Is characterized by finding.

In the range image imaging apparatus of the present invention, the noise charge amount estimation unit adjusts the minimum charge amount, which is the minimum charge amount among the accumulated charge amounts, to correct variations due to the on characteristics of the transfer transistors. It is characterized in that the adjusted minimum charge amount is used for calculating the adjusted noise charge amount by multiplying by a coefficient.

In the range image imaging apparatus of the present invention, the noise charge amount estimation unit adjusts the minimum charge amount, which is the minimum charge amount among the accumulated charge amounts, to correct variations due to the on characteristics of the transfer transistors. It is characterized in that the adjusted minimum charge amount is used for calculating the adjusted noise charge amount by multiplying by a coefficient.

The distance image imaging method of the present invention includes each of a plurality of pixel circuits including a photoelectric conversion element, a plurality of charge storage units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a noise charge amount estimation unit. It is a distance image imaging method that controls a distance image imaging device, and the pixel drive circuit transfers the electric charge generated by the photoelectric conversion element from the measurement space according to the incident light in a predetermined accumulation cycle synchronized with the irradiation of the optical pulse. In the frame period, each of the N (N ≧ 3) charge storage units is subjected to on / off processing of each of the transfer transistors for transferring the charge from the photoelectric conversion element to the charge storage unit, and the charge is distributed and stored. The noise charge amount estimation unit is generated by the background light amount generated by the background light in the measurement space, distributed from the photoelectric conversion element by the transfer transistor, and accumulated in the charge storage unit. The process of estimating the adjusted noise charge amount from the background light charge amount using the noise charge inflow ratio, which indicates the ratio to the noise charge amount that flows into and is accumulated in the charge storage unit without passing through the transfer transistor. , The distance calculation unit subtracts the adjusted noise charge amount from each of the accumulated charge amounts stored in each of the charge storage units, and turns on each of the transfer transistors for each of the correction charge amounts of the subtraction result. It is characterized by including a process of multiplying an adjustment coefficient for correcting variation due to characteristics and calculating the distance to the subject using the adjustment correction charge amount resulting from the multiplication .

As described above, according to the present invention, when correcting the variation due to the transfer transistor in the amount of charge read from each of the charge storage units, the amount of charge of the noise charge by multiplying the amount of charge by the sensitivity correction coefficient It is possible to provide a distance image imaging device and a distance image imaging method for removing fluctuations, improving the accuracy of distance calculation, and capturing a more accurate distance image.

It is a block diagram which showed the schematic structure of the distance image image pickup apparatus of one Embodiment of this invention. It is a figure which shows each of the accumulated charge amount Q accumulated in each charge storage part CS in the charge accumulation period in a frame period. It is a figure which shows each of the accumulated charge amount Q after adjustment by the adjustment noise charge amount QFP. It is a circuit diagram which showed an example of the structure of the pixel circuit 321 arranged in the distance image sensor 32 in the distance image image pickup apparatus of 1st Embodiment of this invention. It is a figure which shows the timing chart which transfers the charge generated by the photoelectric conversion element PD to each of charge storage part CS. It is a conceptual diagram explaining the time relationship that each of a control charge and a noise charge is accumulated in the charge storage part CS. It is a figure explaining the acquisition of the noise charge inflow ratio α in the noise charge amount acquisition mode. It is a flowchart which shows the operation of the process of the calculation of the noise charge inflow ratio α in the noise charge amount acquisition mode by this embodiment. It is a flowchart which shows the operation of the distance image acquisition processing in the distance measurement charge amount acquisition mode by this embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a distance image imaging device according to an embodiment of the present invention. The distance image imaging device 1 having the configuration shown in FIG. 1 includes a light source unit 2, a light receiving unit 3, and a distance image processing unit 4. Note that FIG. 1 also shows a subject S, which is an object for measuring a distance in the distance image imaging device 1. The distance image image sensor is, for example, a distance image sensor 32 (described later) in the light receiving unit 3.

According to the control from the distance image processing unit 4, the light source unit 2 irradiates the space of the photographing target in which the subject S whose distance is to be measured in the distance image imaging device 1 exists with the optical pulse PO. The light source unit 2 is a surface-emitting semiconductor laser module such as, for example, a vertical cavity surface emitting laser (VCSEL). The light source unit 2 includes a light source device 21 and a diffuser plate 22.

The light source device 21 is a light source that emits laser light in a near-infrared wavelength band (for example, a wavelength band having a wavelength of 850 nm to 940 nm) that serves as an optical pulse PO to irradiate the subject S. The light source device 21 is, for example, a semiconductor laser light emitting device. The light source device 21 emits a pulsed laser beam according to the control from the timing control unit 41.
The diffuser plate 22 is an optical component that diffuses the laser light in the near-infrared wavelength band emitted by the light source device 21 over the size of the surface that irradiates the subject S. The pulsed laser beam diffused by the diffuser plate 22 is emitted as an optical pulse PO and irradiates the subject S.

The light receiving unit 3 receives the reflected light RL of the light pulse PO reflected by the subject S whose distance is to be measured in the distance image imaging device 1, and outputs a pixel signal corresponding to the received reflected light RL. The light receiving unit 3 includes a lens 31 and a distance image sensor 32.
The lens 31 is an optical lens that guides the incident reflected light RL to the distance image sensor 32. The lens 31 emits the incident reflected light RL to the distance image sensor 32 side, and receives (incidents) the incident reflected light RL into the pixel circuit provided in the light receiving region of the distance image sensor 32.

The distance image sensor 32 is an image pickup device used in the distance image image pickup device 1. The distance image sensor 32 includes a plurality of pixel circuits 321 and a pixel drive circuit 322 that controls each of the pixel circuits 321 in a two-dimensional light receiving region.
The pixel circuit 321 includes one photoelectric conversion element (for example, a photoelectric conversion element PD described later) and a plurality of charge storage units (for example, charge storage units CS1 to CS4 described later) corresponding to the one photoelectric conversion element. , Each charge storage unit is provided with a component for distributing the electric charge.

The distance image sensor 32 distributes the charges generated by the photoelectric conversion element to the respective charge storage units according to the control from the timing control unit 41. Further, the distance image sensor 32 outputs a pixel signal according to the amount of charge distributed to the charge storage unit. A plurality of pixel circuits are arranged in a two-dimensional matrix in the distance image sensor 32, and a pixel signal for one frame corresponding to each pixel circuit is output.

The distance image processing unit 4 controls the distance image imaging device 1 and calculates the distance to the subject S.
The distance image processing unit 4 includes a timing control unit 41, a noise charge amount estimation unit 42, a distance calculation unit 43, and a measurement control unit 44.
The timing control unit 41 controls the timing of outputting various control signals required for distance measurement according to the control of the measurement control unit 44. The various control signals here are, for example, a signal for controlling the irradiation of the optical pulse PO, a signal for distributing the reflected light RL to a plurality of charge storage units, a signal for controlling the number of distributions per frame, and the like. The number of distributions is the number of times the process of distributing charges to the charge storage unit CS (see FIG. 5) is repeated.

The noise charge amount estimation unit 42 is generated by the background light in the measurement space, is distributed from the photoelectric conversion element PD by the transfer transistor G, and is stored in the charge storage unit CS (background light charge amount QB described later). And the noise charge inflow ratio α, which indicates the ratio to the amount of noise charge generated by the background light and flowing into and accumulating in the charge storage unit CS without passing through the transfer transistor G, is stored in the charge storage unit CS. As the accumulated noise charge amount for correcting the accumulated charge amount Q, the adjusted noise charge amount (adjusted noise charge amount QFP described later) is estimated from the background light charge amount. The stored charge amount Q stored in the charge storage unit CS includes the background light charge amount and the noise charge amount.

The distance calculation unit 43 subtracts the adjusted noise charge amount from the pixel signal (voltage value corresponding to the stored charge amount Q described later) output from the distance image sensor 32 under the control of the measurement control unit 44. Based on the quantity BC, the distance information obtained by calculating the distance to the subject S is output. The distance calculation unit 43 calculates the delay time Td from irradiating the optical pulse PO to receiving the reflected light RL based on the amount of electric charge accumulated in the plurality of charge storage units CS. The distance calculation unit 43 calculates the distance from the distance image imaging device 1 to the subject S according to the calculated delay time Td.

The measurement control unit 44 executes a ratio acquisition frame, which is a frame for acquiring the noise charge inflow ratio α used for estimating the amount of noise charge, in each mode of the frame repeated in the frame period. Select the ratio acquisition mode and the range-finding charge amount acquisition mode, which is a normal frame for measuring distance.
Then, the measurement control unit 44 controls the timing in the timing control unit 41 and the calculation in the distance calculation unit 43 in accordance with each mode of the noise charge amount acquisition mode and the distance measurement charge amount acquisition mode (). Will be described in detail later).

Here, the noise charge is a charge generated by incident light (background light and reflected light) from the measurement space and accumulated in the charge storage unit CS in the region of the semiconductor substrate other than the photoelectric conversion element PD.
This noise charge is a charge other than the charge distributed and accumulated by the on / off processing of the transfer transistor G, that is, the charge that flows into and is accumulated in the charge storage unit CS without passing through the transfer transistor G.

That is, the distance image imaging device according to the present embodiment calculates the distance between the subject and the distance image sensor 32 based on the charge accumulated in the charge storage unit CS. Therefore, in the calculation of distance measurement, noise charge due to background light or the like is accumulated in the charge storage unit CS until the time of reading in each frame, regardless of the charge read from the photoelectric conversion element at a predetermined timing. The calculated distance is less accurate.

Further, the characteristics (on-resistance, switching characteristics, etc.) of the transfer transistor G that distributes charges from the photoelectric conversion element PD to each of the charge storage units CS are different.
Therefore, even when the photoelectric conversion element PD generates a constant charge by the incident light of a constant intensity, a different amount of charge is distributed to each charge storage unit CS due to the variation in the characteristics of the transfer transistor G.

In order to suppress the variation due to the transfer transistor G, the correction coefficient β is set for each charge storage unit CS. This correction coefficient β is set for each transfer transistor G in each pixel circuit 321 and cancels the variation between the transfer transistors G.
Here, a correction coefficient β1 is set for the transfer transistor G1, a correction coefficient β2 is set for the transfer transistor G2, a correction coefficient β3 is set for the transfer transistor G3, and a correction coefficient β4 is set for the transfer transistor G4.

For example, when the photoelectric conversion element PD generates a constant charge by incident light of a constant intensity and distributes the charge to the CS of the charge storage unit by each of the transfer transistors G, the charge is read from each of the charge storage unit CS. By multiplying the accumulated charge amount by the correction coefficient β, each accumulated charge amount Q is adjusted to the same charge amount.
Therefore, the distance calculation unit 43 can calculate the distance between the subject S and the distance image imaging device 1 with high accuracy by multiplying the accumulated charge amount Q read from the charge storage unit CS by the correction coefficient β.

On the other hand, as described in the problem, each of the noise charge amount GF flowing into and being accumulated in each charge storage unit CS is the same in each of the charge storage unit CS in the pixel circuit 321.
The accumulated charge amount Q read from the charge storage unit CS is the charge amount QC (hereinafter, control charge amount QC) due to the charge distributed by the transfer transistor G from the photoelectric conversion element PD (hereinafter, control charge amount QC). Also includes a noise charge amount QF in which the noise charge that has flowed in without passing through the transfer transistor G is accumulated.

Therefore, when the distance calculation unit 43 multiplies the stored charge amount Q read from the charge storage unit CS by the correction coefficient β, not only the control charge amount QC but also the noise charge amount GF included in the stored charge amount Q is corrected. ..
On the contrary, due to the correction of the correction coefficient β, the noise charge amount QF included in the accumulated charge amount Q varies according to the variation of each characteristic of the transfer transistor G, and the subject S and the distance image imaging device 1 Distance cannot be calculated with high accuracy.

FIG. 2 is a diagram showing each of the accumulated charge amounts Q accumulated in each of the charge storage units CS during the charge accumulation period in the frame period. FIG. 2 shows a case where the reflected light RL from the subject S with respect to the light pulse PO is accumulated in each of the charge storage units CS2 and CS3.
FIG. 2A shows each of the accumulated charge amounts Q1, Q2, Q3, and Q4 in each of the charge storage units CS1 to CS4 (FIG.).

Here, the stored charge amount Q1 includes a background light charge amount QB1 and a noise charge amount QF1.
Further, the stored charge amount Q2 includes a background light charge amount QB2, a noise charge amount QF2, and a reflected light charge amount QL2.

The stored charge amount Q3 includes a background light charge amount QB3, a noise charge amount QF3, and a reflected light charge amount QL3.
The stored charge amount Q4 includes a background light charge amount QB4 and a noise charge amount QF4.
Noise charge amount Each of QF1, QF2, QF3 and QF4 has the same charge amount as described above.

Further, since the background light intensity is the same for each of the background light charges QB1 to QB4 due to the charge generated by the photoelectric conversion element PD due to the background light, the charge amount actually generated by the photoelectric conversion element PD is also the same. Therefore, each is equal.
However, the characteristics of the transfer transistors G1, G2, G3, and G4 that distribute the charge from the photoelectric conversion element PD are different from each of the charge storage units CS1, CS2, CS3, and CS4.

Therefore, as shown in FIG. 2A, the background light charge amounts QB1, QB2, QB3, and QB4 stored in each of the charge storage units CS1, CS2, CS3, and CS4 are different.
On the other hand, the noise charge amounts QF1, QF2, QF3, and QF4 stored in each of the charge storage units CS1, CS2, CS3, and CS4 are the same.

FIG. 2B shows the result of multiplying each of the stored charge amounts Q1, Q2, Q3 and Q4 by the correction coefficient β corresponding to each of the transfer transistors G1, G2, G3 and G4.
As a result of the correction by multiplying each of the correction coefficients ββ1, β2, β3 and β4, it is found that each of the background light charge amounts QB1, QB2, QB3 and QB4 is the same.
On the other hand, as a result of the correction by multiplying each of the correction coefficients β1, β2, β3 and β4, it can be seen that the noise charge amounts QF1, QF2, QF3 and QF4, which are the same, are different from each other.

FIG. 3 is a diagram showing each of the accumulated charge amounts Q after being adjusted by the adjusted noise charge amount QFP.
FIG. 3A shows after subtracting the adjusted noise charge amount QFP (corresponding to the noise charge amount QF) from each of the stored charge amounts Q1, Q2, Q3 and Q4 shown in FIG. 2A. .. Therefore, each of the accumulated charge amounts Q1, Q2, Q3, and Q4 does not include each of the noise charge amounts QF1, QF2, QF3, and QF4.

FIG. 3B shows the transfer transistor G1 by multiplying each of the stored charge amounts Q1, Q2, Q3 and Q4 by the correction coefficients β1, β2, β3 and β4, as in FIG. 2B. , Q'2, Q'2, Q'3 and Q'4 are shown after the characteristic variations of G2, G3 and G4 are corrected.
The variation of the background light charge amounts QB1, QB2, QB3, and QB4 in each of the stored charge amounts Q1, Q2, Q3, and Q4 in FIG. It has been resolved in '2, Q'3, and Q'4.
Further, in the accumulated charge amounts Q'1, Q'2, Q'3, and Q'4, each of the noise charge amounts QF1, QF2, QF3, and QF4 is removed to obtain the control charge amount QC, so that the subject S. The distance between the image and the distance image imaging device 1 can be obtained with high accuracy.

That is, when the correction coefficient β set corresponding to each of the charge storage unit CS is multiplied by each of the stored charge amount Q to perform the correction for suppressing the variation in the characteristics of the transfer transistor G in advance. It is necessary to subtract the noise charge amount QC from the stored charge amount Q so that the stored charge amount Q is a component of only the control charge amount QC.
Therefore, in the present embodiment, the distance calculation unit 43 subtracts the adjusted noise charge amount QFP (corresponding to the noise charge amount QF) estimated by the noise charge amount estimation unit 42 from the accumulated charge amount Q, and corrects the subtraction result. The distance between the subject S and the distance image imaging device 1 is calculated by multiplying the coefficient β and using the corrected accumulated charge amount Q'(that is, the control charge amount QC) (details will be described later).

With such a configuration, in the distance image imaging apparatus 1, the light receiving unit 3 receives the reflected light RL reflected by the subject S by the light pulse PO in the near infrared wavelength band irradiated by the light source unit 2 on the subject S. The distance image processing unit 4 outputs distance information obtained by measuring the distance between the subject S and the distance image imaging device 1.
Note that FIG. 1 shows a distance image image pickup device 1 having a distance image processing unit 4 inside, but the distance image processing unit 4 is a component provided outside the distance image image pickup device 1. You may.

Here, the configuration of the pixel circuit 321 in the distance image sensor 32 will be described. FIG. 4 is a circuit diagram showing an example of the configuration of the pixel circuit 321 arranged in the distance image sensor 32 in the distance image imaging apparatus according to the first embodiment of the present invention. The pixel circuit 321 of FIG. 4 is a configuration example including four pixel signal reading units RU1 to RU4.

The pixel circuit 321 includes one photoelectric conversion element PD, charge discharge transistors GD (GD1 and GD2 described later), and four pixel signal reading units RU (RU1 to RU4) that output voltage signals from the corresponding output terminals O. To prepare for. Each of the pixel signal reading units RU includes a transfer transistor G, a floating diffusion FD, a charge storage capacity C, a reset transistor RT, a source follower transistor SF, and a selection transistor SL. The floating diffusion FD and the charge storage capacity C constitute the charge storage unit CS.

In the pixel circuit 321 shown in FIG. 4, the pixel signal reading unit RU1 that outputs a voltage signal from the output terminal O1 includes a transfer transistor G1 (transfer MOS transistor), a floating diffusion FD1, a charge storage capacity C1, and a reset transistor RT1. And a source follower transistor SF1 and a selection transistor SL1. In the pixel signal reading unit RU1, the charge storage unit CS1 is composed of the floating diffusion FD1 and the charge storage capacity C1. The pixel signal reading units RU2, RU3, and RU4 have the same configuration.

The photoelectric conversion element PD is an embedded photodiode that photoelectrically converts incident light to generate an electric charge corresponding to the incident light (incident light) and stores the generated electric charge. In the present embodiment, the incident light is incident from the space to be measured.
In the pixel circuit 321, the charge generated by the photoelectric conversion element PD by photoelectric conversion of the incident light is distributed to each of the four charge storage units CS (CS1 to CS4), and each of them corresponds to the charge amount of the distributed charge. The voltage signal is output to the distance image processing unit 4.

Further, the configuration of the pixel circuit arranged in the distance image sensor 32 is not limited to the configuration including the four pixel signal reading units RU (RU1 to RU4) as shown in FIG. 4, and the pixel signal is not limited to the configuration. A pixel circuit having a configuration in which the reading unit RU includes a plurality of pixel signal reading units RU having one or more may be used.

In driving the pixel circuit 321 of the distance image imaging device 1, the light pulse PO is irradiated with the irradiation time To, and the reflected light RL is received by the distance image sensor 32 with a delay time Td delay. The pixel drive circuit 322, under the control of the timing control unit 41, charges the electric charge generated in the photoelectric conversion element PD to the transfer transistors G1, G2, G3, and G4 in synchronization with the irradiation of the optical pulse PO, and stores the charge signal. TX1 to TX4 are supplied and transferred at each timing, and the charge storage units CS1, CS2, CS3, and CS4 are stored in this order.

Then, the pixel drive circuit 322 controls each of the reset transistor RT and the selection transistor SL by the drive signals RST and SEL, respectively, and converts the charge accumulated in the charge storage unit CS into an electric signal by the source follower transistor SF. , The generated electric signal is output to the distance calculation unit 43 via the terminal O.
Further, the pixel drive circuit 322 causes the electric charge generated in the photoelectric conversion element PD to flow to the power supply VDD and discharge (eliminate the electric charge) by the drive signal RSTD under the control of the timing control unit 41.

FIG. 5 is a diagram showing a timing chart for transferring the charges generated by the photoelectric conversion element PD to each of the charge storage units CS.
In the timing chart of FIG. 5, the vertical axis shows the pulse level and the horizontal axis shows the time. The relative relationship between the optical pulse PO and the reflected light RL on the time axis, the timing of each of the storage drive signals TX1 to TX4 supplied to each of the transfer transistors G1 to G4, and the timing of the drive signal RSTD supplied to the charge discharge transistor GD. Shows.

The timing control unit 41 causes the light source unit 2 to irradiate the measurement space with the optical pulse PO. As a result, the light pulse PO is reflected by the subject and is received by the light receiving unit 3 as reflected light RL. Then, the photoelectric conversion element PD generates electric charges corresponding to each of the background light and the reflected light RL. The pixel drive circuit 322 controls the on / off of each of the transfer transistors G1 to G4 in order to transfer the charge generated by the photoelectric conversion element PD to each of the charge storage units CS1 to CS4.

That is, the pixel drive circuit 322 supplies each of the storage drive signals TX1 to TX4 to each of the transfer transistors G1 to G4 as an "H" level signal having a predetermined time width (the same width as the irradiation time To).

The pixel drive circuit 322 turns on, for example, the transfer transistor G1 provided on the transfer path for transferring charges from the photoelectric conversion element PD to the charge storage unit CS1. As a result, the charge photoelectrically converted by the photoelectric conversion element PD is accumulated in the charge storage unit CS1 via the transfer transistor G1. After that, the pixel drive circuit 322 turns off the transfer transistor G1. As a result, the transfer of electric charge to the electric charge storage unit CS1 is stopped. In this way, the pixel drive circuit 322 stores the electric charge in the electric charge accumulating unit CS1. The same applies to the other charge storage units CS2, CS3 and CS4.

At this time, in the charge storage period (the period in which the charge is stored in each of the charge storage units CS in the frame) in which the charge is distributed to the charge storage unit CS, each of the storage drive signals TX1, TX2, TX3, and TX4 is a transfer transistor. The accumulation cycle supplied to each of G1, G2, G3, and G4 is repeated.
Then, the electric charge corresponding to the incident light is transferred from the photoelectric conversion element PD to each of the charge storage units CS1, CS2, CS3, and CS4 via each of the transfer transistors G1, G2, G3, and G4. Multiple accumulation cycles are repeated during the charge accumulation period.
As a result, charges are accumulated in each of the charge storage units CS1, CS2, CS3, and CS4 for each storage cycle of the charge storage units CS1, CS2, CS3, and CS4 during the charge storage period.

Further, the pixel drive circuit 322 transfers the charge from the photoelectric conversion element PD after the transfer (transfer) of the charge to the charge storage unit CS4 is completed when the storage cycle of each of the charge storage units CS1, CS2, CS3 and CS4 is repeated. An "H" level drive signal RSTD is supplied to the charge discharge transistor GD provided on the discharge path for discharge to turn it on.

As a result, the charge discharge transistor GD discards the charge generated in the photoelectric conversion element PD after the accumulation cycle of the immediately preceding charge storage unit CS4 before the storage cycle for the charge storage unit CS1 is started (that is, photoelectric conversion). Reset the element PD). That is, one or a plurality (one or more) charge discharge transistors GD are provided, and the charge generated by the incident light is distributed to each of the charge storage units CS1, CS2, CS3 and CS4 from the photoelectric conversion element PD. Charges are discharged from the photoelectric conversion element PD at a time other than the storage period.

Then, the pixel drive circuit 322 sequentially performs A / D conversion processing and the like in units of rows (horizontal arrangement) of the pixel circuit 321 for voltage signals from each of all the pixel circuits 321 arranged in the light receiving unit 3. Signal processing is performed.
After that, the pixel drive circuit 322 outputs the voltage signal after the signal processing to the distance calculation unit 43 in the order of the columns arranged in the light receiving unit 3.

As described above, the accumulation of electric charge in the charge storage unit CS by the pixel drive circuit 322 and the discarding of the electric charge converted by the photoelectric conversion element PD are repeatedly performed over one frame. As a result, charges corresponding to the amount of light received by the distance image imaging device 1 in a predetermined time interval are accumulated in each of the charge storage units CS. The pixel drive circuit 322 outputs an electric signal corresponding to the amount of charge for one frame stored in each of the charge storage units CS to the distance calculation unit 43.

Due to the relationship between the timing of irradiating the optical pulse PO and the timing of accumulating charges in each of the charge storage units CS (CS1 to CS4), the charge storage unit CS1 has the background light before irradiating the optical pulse PO. The amount of charge corresponding to the external light component is retained. Further, the amount of charge corresponding to the reflected light RL and the external light component is distributed and held in the charge storage units CS2, CS3 and CS4. The distribution (distribution ratio) of the amount of charge distributed to the charge storage units CS2 and CS3 or the charge storage units CS3 and CS4 is the delay time until the optical pulse PO is reflected by the subject S and is incident on the distance image imaging device 1. The ratio corresponds to Td (FIG. 5).

Returning to FIG. 1, the distance calculation unit 43 uses this principle to calculate the delay time Td by the following equation (1) or (2).
Td = To × (Q'3-Q'1) / (Q'2 + Q'3-2 × Q'1)… (1)
Td = To + To × (Q'4-Q'1) / (Q'3 + Q'4-2 × Q'1)… (2)
Here, To is the period during which the optical pulse PO is irradiated. Q'1 is the stored charge amount obtained by multiplying the result of subtracting the adjusted noise charge amount QFP from the charge amount Q1 stored in the charge storage unit CS1 by the correction coefficient β1. Further, Q'2 is a stored charge amount obtained by multiplying the result of subtracting the adjusted noise charge amount QFP from the charge amount Q2 stored in the charge storage unit CS2 by the correction coefficient β2.

Q'3 is the stored charge amount obtained by multiplying the result of subtracting the adjusted noise charge amount QFP from the charge amount Q3 stored in the charge storage unit CS3 by the correction coefficient β3. Q'4 is the stored charge amount obtained by multiplying the result of subtracting the adjusted noise charge amount QFP from the charge amount Q4 stored in the charge storage unit CS4 by the correction coefficient β4.
For example, when Q'4 = Q'1, the distance calculation unit 43 calculates the delay time Td by the equation (1), while when Q'2 = Q'1, the distance calculation unit 43 calculates the delay time Td by the equation (2). Calculate the time Td.

In the equation (1), the charges generated by the reflected light are accumulated in the charge storage units CS2 and CS3, but are not accumulated in the charge storage units CS4. On the other hand, in the equation (2), the charges generated by the reflected light are accumulated in the charge storage units CS3 and CS4, but are not accumulated in the charge storage units CS2.
In addition, in the formula (1) or the formula (2), among the charge amounts stored in the charge storage units CS2, CS3 and CS4, the component corresponding to the external light component is the charge amount stored in the charge storage unit CS1. It is assumed that the amount is the same.

The distance calculation unit 43 calculates the round-trip distance to the subject S by multiplying the delay time obtained by the equation (1) or the equation (2) by the speed of light (speed).
Then, the distance calculation unit 43 halves the round-trip distance calculated above (delay time Td × c (speed of light) / 2), so that the distance image sensor 32 (that is, the distance image imaging device 1) is used. The distance from the subject S to the subject S is obtained.

FIG. 6 is a conceptual diagram illustrating the time relationship in which each of the control charge and the noise charge is accumulated in the charge storage unit CS.
As shown in FIG. 6, the frame period is composed of a storage period and a read period.
As a result, each of the accumulated charge amounts Q is accumulated in each of the charge storage units CS during the accumulation period for each frame.
Then, during the readout period, the accumulated accumulated charge amount is output to the distance image processing unit 4 from each of the charge storage units CS of the pixel circuit 321, and the distance image processing unit 4 is the subject S in each pixel. And the distance between the pixel and the pixel are calculated.

Further, the electric charge generated by the photoelectric conversion element PD from the photoelectric conversion element PD via the transfer transistor G is used as a control charge (charge generated by the reflected light RL and charge generated by the background light) with respect to the charge storage unit CS. The period to be distributed is the pulse width of the storage drive signal TX that turns on the transfer transistor G.
Then, the distribution from the photoelectric conversion element PD to the charge storage unit CS is performed a predetermined number of times (number of distributions) during the storage period in the frame.

On the other hand, the noise charge is generated in the semiconductor substrate by the incident light and flows into the charge storage unit CS without passing through the transfer transistor G. Therefore, the accumulated charge amount Q is generated in each of the charge storage units CS in each of the pixel circuits 321. It continuously flows in and accumulates until it is read out as, that is, during the accumulation period.
Further, since the period in which the reflected light RL is not included as the incident light is longer and the intensity of the background light incident on each of the photoelectric conversion element PD and the semiconductor substrate is the same, the reflected light charge amount QL and noise. There is a correlation with the amount of charge QF.

Therefore, in the present embodiment, the noise charge inflow ratio α used for estimating the adjusted noise charge amount QFP is obtained in advance from the reflected light charge amount QL and the noise charge amount QF.
By multiplying the pulse width of the storage drive signal TX when transferred by the transfer transistor G by the number of distributions, the time width TALL for supplying the charge generated by the photoelectric conversion element PD to each charge storage unit CS is obtained. Be done. This time width TALL corresponds to the time for the photoelectric conversion element PD to generate a charge to be supplied to each of the charge storage units CS.

Therefore, by dividing the background light charge amount QB by the time width TALL, the charge amount generated in the unit time by the photoelectric conversion element PD can be obtained.
Then, the noise charge inflow ratio is divided by dividing the amount of noise charge that has flowed into and accumulated in the charge storage unit CS by the amount of charge generated in the unit time by the photoelectric conversion element PD multiplied by the time of the storage period. α is required.

FIG. 7 is a diagram illustrating acquisition of the noise charge inflow ratio α in the noise charge amount acquisition mode. In the noise charge amount acquisition mode, the first ratio acquisition frame and the second ratio acquisition frame are executed.
That is, the measurement control unit 44 shifts the operating state from the distance measurement charge amount acquisition mode to the noise charge amount acquisition mode, for example, when the distance image imaging device 1 is activated or when the execution is arbitrarily instructed. Then, the measurement control unit 44 controls the timing in the timing control unit 41 and the operation in the distance calculation unit 43.

FIG. 7A shows the operation of the first ratio acquisition frame, in which only the background light is used as the incident light without irradiating the optical pulse PO, and the accumulated charge amount used for calculating the noise charge inflow ratio α is used. The reference stored charge amount QR is acquired.
Further, in the case of FIG. 7A, the transfer transistors G1 and G2 are each subjected to the storage drive signals TX1, TX2, TX3 and TX4 having a predetermined number of distributions of S times and a pulse width of TS during the storage period in the frame cycle. , G3 and G4 are turned on and off, and the charge is distributed from the photoelectric conversion element PD to the charge storage units CS1, CS2, CS3 and CS4, and the reference stored charge amounts QR1, QR2, QR3 and QR4 are stored.

Then, during the read-out period of the first ratio acquisition frame, the noise charge amount estimation unit 42 refers to and stores each of the charge storage units CS1, CS2, CS3, and CS4 for each pixel circuit 321 under the control of the pixel drive circuit 322. Each of the charge amounts QR1, QR2, QR3, and QR4 is acquired.
Each of the acquired reference stored charge amounts QR1, QR2, QR3 and QR4 includes reference background light charge amounts QBR1, QBR2, QBR3, QBR4 and reference noise charge amounts QFR1, QFR2, QFR3 and QFR4, respectively. There is.

FIG. 7B shows the operation of the second ratio acquisition frame, in which only the background light is used as the incident light without irradiating the optical pulse PO, and the accumulated charge amount used for calculating the noise charge inflow ratio α is used. The reference stored charge amount QR is acquired.
Further, in the case of FIG. 7B, during the storage period in the frame period, each of the storage drive signals TX1, TX2, TX3 and TX4 is not supplied, and each of the transfer transistors G1, G2, G3 and G4 remains in the off state. Then, the noise charge is flowed into each of the charge storage units CS1, CS2, CS3, and CS4, and the noise charge is accumulated as the reference noise charge amounts QFR1, QFR2, QFR3, and QFR4.

Then, the noise charge amount estimation unit 42 receives reference noise from each of the charge storage units CS1, CS2, CS3, and CS4 for each pixel circuit 321 under the control of the pixel drive circuit 322 during the read period of the second ratio acquisition frame. Each of the charge amounts QFR1, QFR2, QFR3, and QFR4 is acquired. Here, each of the reference noise charge amounts QFR1, QFR2, QFR3, and QFR4 is the same because of the amount of charge that flows into and is stored in each of the charge storage units CS1, CS2, CS3, and CS4.
The noise charge amount estimation unit 42 subtracts each of the reference noise charge amounts QFR1, QFR2, QFR3, and QFR4 from each of the reference stored charge amounts QR1, QR2, QR3, and QR4 acquired in the first ratio acquisition frame, and refers to them. The amount of background light charge QBR1, QBR2, QBR3, and QBR4 are obtained.

The noise charge amount estimation unit 42 extracts the minimum charge amount from each of the reference background light charge amounts QBR1, QBR2, QBR3 and QBR4 as the minimum reference background light charge amount.
Then, the noise charge amount estimation unit 42 multiplies the minimum reference background light charge amount by the corresponding correction coefficient β to obtain the adjusted reference background light charge amount.
The noise charge amount estimation unit 42 multiplies the pulse width (first on-time TP1) of the storage drive signal TX and the number of distributions (first distribution number, M1), and calculates the total transfer time as the multiplication result.

The noise charge amount estimation unit 42 divides the adjusted reference background light charge amount by this total transfer time, and obtains a unit reference background light charge amount corresponding to the charge amount generated by the photoelectric conversion element PD background light per unit time. Ask.
Then, the noise charge amount estimation unit 42 uses any one of the reference noise charge amounts QFR1, QFR2, QFR3, and QFR4 as the reference noise charge amount QFR, divides it by the time of the storage period, and flows into the charge storage unit CS per unit time. The unit time inflow charge amount to be accumulated is calculated.
As a result, the noise charge amount estimation unit 42 divides the unit time inflow charge amount by the unit reference background light charge amount to obtain the noise charge inflow ratio α.

That is, since the intensity of the background light is the same, the amount of charge generated per unit time generated by the photoelectric conversion element PD and the amount of charge generated by the reflected light and flowing into the charge storage unit CS per unit time. Is correlated.
That is, when measuring the distance, when the charge amount generated per unit time by the background light is obtained by the photoelectric conversion element PD, the noise charge amount estimation unit 42 determines the charge amount and the noise charge inflow ratio α. Therefore, the amount of electric charge of the noise charge flowing into the charge storage unit CS per unit time can be estimated.

Further, when the noise charge amount acquisition mode in FIG. 7 described above is completed, the measurement control unit 44 shifts the operating state from the noise charge amount acquisition mode to the distance measurement charge amount acquisition mode. Then, the measurement control unit 44 controls the timing in the timing control unit 41 and the operation in the distance calculation unit 43.
Then, the noise charge amount estimation unit 42 reads each of the stored charge amounts Q1, Q2, Q3, and Q4 from each of the charge storage units CS1, CS2, CS3, and CS4 during the read period of the frame period.

The noise charge amount estimation unit 42 extracts the minimum charge amount as the minimum charge amount (the charge component generated only by the background light without the charge component due to the reflected light RL) in the stored charge amounts Q1, Q2, Q3 and Q4. do.
This minimum charge amount QP is the background light charge amount QB of the control charge distributed from the photoelectric conversion element PD by the transfer transistor G and the noise charge amount QF (adjusted noise charge amount) due to the noise charge flowing in without passing through the transfer transistor G. QFP) is the amount of charge included.
Therefore, the minimum charge amount QP is expressed by the following equation (3).
QP = QB + QC ... (3)

Further, the adjusted noise charge amount QFP includes the second accumulation period T2 in the frame period in the distance measurement charge amount acquisition mode and the pulse width (second on time) TP2 (irradiation time To in the present embodiment) of the accumulation drive signal TX. , The number of distributions M2, the background light charge amount QB, and the noise charge inflow ratio α are expressed as the following equation (4).
QFP = (QB / (TP2 × M2)) × α × T2… (4)
The noise charge amount estimation unit 42 obtains the unknown background light charge amount QB and the adjusted noise charge amount QFP in the simultaneous linear equations according to each of the above equations (3) and (4).
Then, the noise charge amount estimation unit 42 outputs the calculated adjusted noise charge amount QFP to the distance calculation unit 43.

The distance calculation unit 43 subtracts the adjusted noise charge amount QFP from each of the stored charge amounts Q1, Q2, Q3 and Q4, respectively, and corrects the stored charge amounts CQ1 (= Q1-QF), CQ2 (= Q2-QF). CQ3 (= Q3-QF) and CQ4 (= Q4-QF) are obtained respectively.
The distance calculation unit 43 multiplies each of the modified stored charge amounts CQ1, CQ2, CQ3 and CQ4 by the correction coefficients β1, β2, β3 and β4, respectively, and the stored charge amounts Q'1, Q'2 and Q. Calculate '3 and Q'4.
Then, the distance calculation unit 43 calculates the delay time Td from the equations (1) and (2) already described, and calculates the distance between the distance image imaging device 1 and the subject S by the delay time Td.

FIG. 8 is a flowchart showing the operation of the process of calculating the noise charge inflow ratio α in the noise charge amount acquisition mode according to the present embodiment.
Step S101:
As shown in FIG. 7A, the measurement control unit 44 controls the timing control unit 41 in order to acquire the reference accumulated charge amount QR without irradiating the optical pulse PO, and the first ratio acquisition frame. To execute.

Then, the noise charge amount estimation unit 42 transfers the charge generated by the background light in the photoelectric conversion element PD to each of the charge storage units CS1, CS2, CS3 and CS4 during the storage period T1 to the transfer transistors G1, G2, G3. , G4 to distribute and accumulate.
The noise charge amount estimation unit 42 distributes each of the charge storage units CS1, CS2, CS3 and CS4 by the transfer transistors G1, G2, G3 and G4.

The noise charge amount estimation unit 42 acquires each of the reference stored charge amounts QR1, QR2, QR3, and QR4 that have flowed in and accumulated in the storage period T1 in the second ratio acquisition frame during the read-out period.
Here, the storage drive signals TX1, TX2, TX3, and TX4 for each of the transfer transistors G1, G2, G3, and G4 have the same pulse width TP1. The number of distributions is M1.

Step S102:
After the first ratio acquisition frame is completed, the measurement control unit 44 does not irradiate the optical pulse PO and does not drive the transfer transistors G1, G2, G3, and G4 as shown in FIG. 7B. In order to acquire each of the charge amounts QFR1, QFR2, QFR3, and QFR4, the timing control unit 41 is controlled to execute the second ratio acquisition frame.
The noise charge amount estimation unit 42 receives each of the reference noise charge amounts QFR1, QFR2, QFR3, and QFR4 that have flowed into and accumulated in the charge storage units CS1, CS2, CS3, and CS4 during the storage period T1 in the second ratio acquisition frame. , Obtained during the read period.

Step S103:
The noise charge amount estimation unit 42 has a reference noise charge amount QFR1, QFR2, QFR3, QFR4 acquired in the second ratio acquisition frame from each of the reference stored charge amounts QR1, QR2, QR3 and QR4 acquired in the first ratio acquisition frame. Subtract each of.
Then, the noise charge amount estimation unit 42 distributes the electric charge from the photoelectric conversion element PD to the transfer transistors G1, G2, G3, and G4 during the storage period T1 and stores the reference background light charge in the charge storage units CS1, CS2, CS3, and CS4. Each of the quantities QBR1, QBR2, QBR3 and QBR4 is determined.

Step S104:
The noise charge amount estimation unit 42 extracts the minimum charge amount as the minimum reference background light charge amount from each of the reference background light charge amounts QBR1, QBR2, QBR3 and QBR4.
Then, the noise charge amount estimation unit 42 multiplies the minimum reference background light charge amount by the corresponding correction coefficient β to obtain the adjusted reference background light charge amount.
At this time, for example, when the minimum reference background light charge amount is the reference background light charge amount QBR1, the noise charge amount estimation unit 42 distributes the charge to the charge storage unit CS1 using the transfer transistor G1, so that the correction coefficient β1 is referred to. Multiply the light charge amount QBR1 and use the multiplication result as the adjustment reference background light charge amount.

The noise charge amount estimation unit 42 multiplies the pulse width TP1 of the storage drive signal TX by the number of distributions M1 by the transfer transistor G to calculate TP1 × M1 as the total transfer time.
Then, the noise charge amount estimation unit 42 obtains the unit reference background light charge amount by dividing the adjustment reference background light charge amount by the total transfer time TP1 × M1.
The noise charge amount estimation unit 42 multiplies each of the unit reference background light charge amount and the accumulation period T1, and divides the reference noise charge amount QF from the multiplication result to obtain the noise charge inflow ratio α.

FIG. 9 is a flowchart showing the operation of the distance image acquisition process in the distance measurement charge amount acquisition mode according to the present embodiment.
Step S201:
As shown in FIG. 5, the measurement control unit 44 irradiates the optical pulse PO (pulse width To) at a predetermined timing in the second storage period T2 in the frame period, and the charge storage units CS1, CS2, CS3 and CS4 In each case, in order to store each of the stored charge amounts Q1, Q2, Q3, and Q4, the timing control unit 41 is controlled to execute a predetermined frame period.

Step S202:
Then, the pixel drive circuit 322 receives the reflected light RL of the light pulse PO irradiated at a predetermined timing in the second storage period T2 in the frame period, and the charge storage units CS1, CS2, CS3, and CS4 receive the reflected light RL. , Each of the accumulated charge amounts Q1, Q2, Q3, and Q4 is accumulated from the photoelectric conversion element PD by the transfer transistors G1, G2, G3, and G4.
Here, the storage drive signals TX1, TX2, TX3, and TX4 for each of the transfer transistors G1, G2, G3, and G4 have the same pulse width TP2. The number of distributions is M2.

Step S203:
The pixel drive circuit 322 processes the stored charge amounts Q1, Q2, Q3, and Q4 of the charge storage units CS1, CS2, CS3, and CS4 in order from each of the pixel circuits 321 during the read period in the frame cycle. Output to unit 4.
The noise charge amount estimation unit 42 sets the minimum charge amount from each of the stored charge amounts Q1, Q2, Q3, and Q4 by the stored charge amount that does not include the charge of the reflected light RL and contains only the charge of the background light. Extract a certain minimum amount of charge.

Step S204:
As shown in the above equation (3), the noise charge amount estimation unit 42 has the background light charge amount QB, which is the control charge of the extracted minimum charge amount QP, and the noise charge amount QF (that is, the noise charge amount QF) that flows in without control. Adjusted noise charge amount QFP) and included. The background light charge amount QB and the noise charge amount QF are unknown.
Here, each of the background light charge amounts QB1, QB2, QB3, and QB4 distributes the charges generated by the photoelectric conversion element PD by the background light by the storage drive signals TX1, TX2, TX3, and TX4 having the same pulse width TP2. Therefore, the amount of charge is the same.

Further, each of the noise charge amounts QF1, QF2, QF3 and QF4 is noise that flows into each of the charge storage units CS1, CS2, CS3 and CS4 in the same storage period T2 without being controlled by the pixel drive circuit 322. It is the same because it is the amount of electric charge.
The noise charge amount estimation unit 42 makes known the minimum charge amount QP, the pulse width TP2, the accumulation period T2, the number of distributions M2, and the noise charge inflow ratio α, and sets the background light charge amount QB and the adjusted noise charge. Equations (3) and (4) are solved with the quantity QFP as an unknown number, and each of the adjusted noise charge amount QFP and the background light charge amount QB is calculated.

Step S205:
The noise charge amount estimation unit 42 subtracts the adjusted noise charge amount QFP from each of the stored charge amounts Q1, Q2, Q3, and Q4 to obtain the modified stored charge amounts CQ1, CQ2, CQ3, and CQ4, respectively.
Then, the noise charge amount estimation unit 42 multiplies each of the modified stored charge amounts CQ1, CQ2, CQ3 and CQ4 by the correction coefficients β1, β2, β3 and β4, respectively, and the stored charge amount Q'1 and Find Q'2, Q'3, and Q'4.

Step S206:
Then, the distance calculation unit 43 calculates the delay time Td from the described equation (1) or equation (2) using each of the accumulated charge amounts Q'1, Q'2, Q'3 and Q'4. do.
The distance calculation unit 43 calculates the round-trip distance between the distance image imaging device 1 and the subject S by multiplying the calculated delay time Td by the speed of light (speed).
Then, the distance calculation unit 43 halves the round-trip distance calculated above (delay time Td × c (speed of light) / 2), so that the distance image sensor 32 (that is, the distance image imaging device 1) is used. The distance from the subject S to the subject S is obtained.

Step S207:
The distance calculation unit 43 determines whether or not the calculation of the distance in all the pixel circuits 321 (that is, the pixels) in the distance image sensor 32 is completed.
At this time, the distance calculation unit 43 advances the process to step S208 when the calculation of the distances in all the pixel circuits 321 is completed.
On the other hand, if the distance calculation unit 43 has not completed the calculation of the distances in all the pixel circuits 321, the distance calculation unit 43 proceeds to the process in step S203.

Step S208:
The distance calculation unit 43 determines the distance between the pixel and the subject S in each pixel according to the accumulated charge amounts Q1, Q2, Q3, and Q4 supplied from all the pixel circuits 321 in the distance image sensor 32. Generates a distance image, which is an image expressed as.
Then, the distance calculation unit 43 outputs the created distance image to a predetermined external device or the like, or writes and stores the created distance image in a storage unit that stores the distance image (not shown).

Step S209:
The measurement control unit 44 determines whether or not a control signal for ending the measurement to end the imaging of the distance image has been supplied.
At this time, the measurement control unit 44 ends the acquisition of the distance image (distance measurement processing) when the control signal for the end of the measurement is supplied.
On the other hand, the measurement control unit 44 advances the process to step S201 when the control signal for the end of measurement is not supplied.

As described above, according to the present embodiment, the noise charge included in the stored charge amount Q (that is, before multiplying by the correction coefficient β for correcting the sensitivity to the stored charge amount read from each of the charge storage units CS). , Adjusted noise charge amount QFP) is subtracted, so when correcting the variation in transfer efficiency due to the transfer transistor G, it is possible to suppress the variation by the amount of noise charge included in the stored charge amount, and it is possible to calculate the distance. The accuracy can be improved, and a distance image having more accurate range-finding information can be captured as compared with the conventional case.

Further, in the present embodiment, in the noise charge amount acquisition mode, the unit time inflow charge amount of the noise charge flowing into the charge storage unit CS per unit time is measured by the photoelectric conversion element PD per unit time (this). In the embodiment, the noise charge inflow ratio α is calculated by dividing by the unit reference background light charge amount, which is the amount of charge generated by the reflected light).
Therefore, in the present embodiment, by using the noise charge inflow ratio α in the noise charge acquisition mode different from the frame period in the noise charge acquisition mode, the accumulation period in the noise charge acquisition mode and the accumulation drive signal TX The adjusted noise charge amount QFP (that is, the noise charge amount QF) can be easily obtained by a simultaneous equation using the pulse width TP2, the number of distributions M2 by the transfer transistor G, and the actually measured accumulated charge amount Q.

1 ... Distance image image sensor 2 ... Light source unit 3 ... Light receiving unit 31 ... Lens 32 ... Distance image sensor (distance image image sensor)
321 ... Pixel circuit 322 ... Pixel drive circuit 4 ... Distance image processing unit 41 ... Timing control unit 42 ... Noise charge amount estimation unit 43 ... Distance calculation unit 44 ... Measurement control unit CS1, CS2, CS3, CS4 ... Charge storage unit FD1, FD2, FD3, FD4 ... Floating diffusion G1, G2, G3, G4 ... Transfer transistor GD ... Charge discharge transistor ML ... Microlens PD ... Photoelectric conversion element PO ... Optical pulse RT1, RT2, RT3, RT4 ... Reset transistor S ... Subject SF1 , SF2, SF3, SF4 ... Source follower transistor SL1, SL2, SL3, SL4 ... Selective transistor

Claims (8)

A light source that irradiates the measurement space with an optical pulse,
A photoelectric conversion element that generates electric charges according to the light incident from the measurement space, a plurality of pixel circuits including a plurality of electric charge storage units that accumulate the electric charges in the frame period, and a predetermined value synchronized with the irradiation of the optical pulse. A light receiving unit having a pixel drive circuit that performs on / off processing of each transfer transistor on each of the charge storage units at the storage timing to distribute and store the charges.
The amount of background light charge generated by the background light in the measurement space and distributed from the photoelectric conversion element by the transfer transistor and accumulated in the charge storage unit, and the amount of background light generated by the background light and not via the transfer transistor. A noise charge amount estimation unit that estimates an adjusted noise charge amount from the background light charge amount using a noise charge inflow ratio that indicates a ratio to the noise charge amount that flows into and is accumulated in the charge storage unit.
The adjusted noise charge amount is subtracted from each of the accumulated charge amounts stored in each of the charge storage units, and the variation due to the on-characteristics of the transfer transistor is corrected for each of the corrected charge amounts of the subtraction result. A distance image imaging device including a distance calculation unit that multiplies an adjustment coefficient and calculates the distance to the subject using the adjustment correction charge amount that is the result of the multiplication . Before the execution of the frame cycle for measuring the distance, the pixel drive circuit performs on / off processing of each transfer transistor from the photoelectric conversion element to each of the charge storage units without irradiating the optical pulse, and the charge is charged. The first ratio acquisition frame for acquiring the reference accumulated charge amount by distributing and accumulating the charges, and flowing into each of the charge accumulating portions without irradiating the optical pulse and performing on / off processing of each of the transfer transistors. Execute with the second ratio acquisition frame to acquire the accumulated reference noise charge amount,
The feature is that the noise charge amount estimation unit obtains the noise charge inflow ratio from the reference background light charge amount as a result of subtracting the reference noise charge amount from the reference stored charge amount and the reference noise charge amount. The distance image imaging device according to claim 1. The noise charge amount estimation unit
The unit reference background light is obtained by dividing the reference background light charge amount by the total transfer time obtained by multiplying the first distribution number, which is the number of distributions in the first ratio acquisition frame, and the first on time when the transfer transistor is turned on. The amount of charge is obtained, the reference noise charge amount is divided by the multiplication result obtained by multiplying the unit reference background light charge amount and the first accumulation period for accumulating the charge in the first ratio acquisition frame, and the division result is the noise charge. The distance image imaging apparatus according to claim 2, wherein the inflow ratio is used. The noise charge amount estimation unit
The reference background light charge amount is multiplied by an adjustment coefficient for correcting the variation due to the on characteristics of the transfer transistor, and the adjusted reference background light charge amount is used for calculating the noise charge inflow ratio. 3. The range image imaging device according to claim 3. The noise charge amount estimation unit
The first linear equation for obtaining the adjusted noise charge amount by subtracting the background light charge amount from the minimum charge amount, which is the minimum charge amount among the accumulated charge amounts, and the frame with respect to the background light charge amount. The adjusted noise charge amount is obtained by dividing the multiplication result of the second accumulation period in the period and the noise charge inflow ratio by the multiplication result of the second on time of the transfer transistor and the second distribution number in the frame period. The invention according to any one of claims 1 to 4, wherein the binary linear equation consisting of the second linear equation is solved to obtain the unknown adjusted noise charge amount and background light charge amount. Range image imager. The noise charge amount estimation unit
The minimum charge amount, which is the minimum charge amount among the accumulated charge amounts, the noise charge inflow ratio, the second storage period in the frame period, the second on-time of the transfer transistor in the frame period, and the frame. The claim is characterized in that the adjusted noise charge amount is obtained by using a lookup table in which the multiplication result with the second distribution number of the transfer transistor in the period is used as an input variable and the adjusted noise charge amount is output. The distance image imaging device according to any one of claims 1 to 4. The noise charge amount estimation unit
The minimum charge amount, which is the minimum charge amount among the accumulated charge amounts, is multiplied by an adjustment coefficient that corrects the variation due to the on-characteristics of the transfer transistor, and the adjusted minimum charge amount is calculated as the adjustment noise. The distance image imaging apparatus according to claim 5 or 6, characterized in that it is used for calculating the amount of electric charge. A distance image that controls a distance image image pickup device including each of a plurality of pixel circuits including a photoelectric conversion element, a plurality of charge storage units, and a transfer transistor, a pixel drive circuit, a distance calculation unit, and a noise charge amount estimation unit. It is an imaging method,
The pixel drive circuit stores N (N ≧ 3) charges generated by the photoelectric conversion element from the measurement space according to the incident light in a predetermined storage cycle synchronized with the irradiation of the light pulse in the frame cycle. A process of performing on / off processing of each of the transfer transistors for transferring the charge from the photoelectric conversion element to the charge storage unit in each of the units, and distributing and accumulating the charges.
The noise charge amount estimation unit is generated by the background light and the background light charge amount generated by the background light in the measurement space, distributed from the photoelectric conversion element by the transfer transistor and accumulated in the charge storage unit. The process of estimating the adjusted noise charge amount from the background light charge amount using the noise charge inflow ratio, which indicates the ratio to the noise charge amount that flows into and is accumulated in the charge storage unit without passing through the transfer transistor.
The distance calculation unit subtracts the adjusted noise charge amount from each of the accumulated charge amounts stored in each of the charge storage units, and for each of the corrected charge amounts of the subtraction result, each on characteristic of the transfer transistor. A distance image imaging method including a process of multiplying an adjustment coefficient for correcting the variation due to the above and calculating the distance to the subject using the adjustment correction charge amount resulting from the multiplication .

Images