JPWO2014132767A1 - Distance image sensor - Google Patents

Abstract

The present invention is a distance image sensor capable of measuring a distance even when the amount of received light is sequentially acquired for each light receiving element. In each of the light receiving elements, the clock signal generation unit is configured to receive the received light amount (A (T0) to A ( A clock signal (CLK) that changes in synchronization with the start and end of the light receiving period (T0 to T3) for T3)) is generated. The received light amount acquisition unit sequentially acquires the received light amount based on the clock signal (CLK) and acquires the frame data by repeating the step of acquiring the frame data a plurality of times, thereby acquiring the plurality of frame data. The amount of light received by each element corresponds to the amount of light received (A (T0) to A (T3)). The distance calculation unit calculates the phase difference based on the plurality of frame data, and calculates the distance from the irradiation unit to the measurement target.

Description

  The present invention relates to a distance image sensor.

  Non-Patent Document 1 describes an image sensor for acquiring a distance image. The distance image here is an image showing the depth (distance) of the measurement target. In Non-Patent Document 1, irradiation light is irradiated onto a measurement object, and reflected light reflected by the measurement object is received by a CCD (Charge-Coupled-Device) image sensor. Then, the distance from the light source to the measurement object is measured based on the phase difference (or time difference, hereinafter the same) between the irradiation light and the reflected light. A distance measurement method based on such a phase difference is called a TOF (Time-of-flight) method.

  In Non-Patent Document 1, a light receiving area is changed by forming a plurality of Poly-Si gates in a CCD image sensor and sequentially switching voltages applied to the gates. As a result, a received light signal necessary for distance measurement is obtained.

  Patent Documents 1 to 4 and Non-Patent Documents 2 to 4 are posted as techniques related to the present invention.

JP2012-29130A JP 2012-37526 A JP 2011-128024 A JP 2012-93131 A

Yusuke Hashimoto, 4 others, “Ambient Light Erasing Range Image Sensor”, Panasonic Electric Works Technical Report, Vol.57, No.2, p.4-9 Yusuke Hashimoto, 5 others, “High-sensitivity range image sensor by area demodulation method”, Panasonic Electric Works Technical Report, Vol. 59, No. 3, p. 45-50 H. Kim, 2 others, “A NOVEL WITH A SINGLE CARRIER MODULATION PHOTODETECTOR”, APCOT (Asia-Pacific Conference on Transducers and Micro-Nano Technology) 2008, Date: 22-25, June 2008 Ryohei Miyagawa, 1 other, “Smart Image Sensor that Performs Time-Series Operations by Electric Charge”, Television Society Technical Report, November 1995, Vol. 19, No. 65, pp. 37-41

  However, Non-Patent Document 1 requires a special structure in which a Poly-Si gate is formed and the voltage applied to the gate is sequentially switched, resulting in an increase in manufacturing cost.

  In Non-Patent Document 1, a CCD image sensor is employed. In general, a CCD image sensor employs a global shutter system that simultaneously obtains light reception signals of all light receiving elements.

  On the other hand, the technique of Non-Patent Document 1 can be applied as it is to a rolling shutter system in which the light reception signals of the light receiving elements are sequentially acquired, such as a CMOS (Complementary-Metal-Oxide-Semiconductor) image sensor. difficult.

  Accordingly, an object of the present invention is to provide a distance image sensor capable of measuring a distance even in a system in which the amount of received light is sequentially acquired for each light receiving element.

  The first aspect of the distance image sensor according to the present invention is configured to receive the reflected light from the measurement target and the irradiation unit that irradiates the measurement target with the irradiation light while modulating the intensity of the irradiation light at a certain modulation period. A plurality of light receiving elements and each of the light receiving elements in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light. A clock generation unit that generates a changing clock signal, and a step of acquiring frame data by sequentially acquiring the amount of light received by the light receiving element based on the clock signal to acquire a plurality of frame data by repeating a plurality of times. A plurality of received light amounts at the same light receiving element in the plurality of frame data, each of which corresponds to a received light amount in the plurality of light receiving periods; It calculates the phase difference on the basis of the frame data, and a distance calculation unit that calculates a distance from the irradiation portion to the measurement target.

  A second aspect of the distance image sensor according to the present invention is the distance image sensor according to the first aspect, wherein the plurality of light reception periods are four light reception periods, and the start periods of the four light reception periods are respectively It shifts by a quarter of the modulation period.

  A third aspect of the distance image sensor according to the present invention is the distance image sensor according to the first or second aspect, wherein the modulation period is an even multiple of the period of the clock signal, and the plurality of frame data Is the first to fourth frame data, and the amount of received light of each of the light receiving elements is acquired in response to the rising edge of either the even or odd number of the clock signal in the first frame data. In the second frame data, the second frame data is acquired with the other rising edge of the clock signal as an opportunity, and in the third frame data, either the even-numbered or odd-numbered falling edge of the clock signal is used as an opportunity. Acquired in the fourth frame data, triggered by the other falling edge of the clock signal.

  A fourth aspect of the distance image sensor according to the present invention is the distance image sensor according to the third aspect, wherein the received light amount acquisition unit determines the received light amount of the light receiving element at each rising edge of the clock signal. Obtaining the first and second frame data sequentially in a predetermined order, obtaining the received light amount of the light receiving element sequentially in the order at each falling edge of the clock signal, The third and fourth frame data are acquired.

  A fifth aspect of the distance image sensor according to the present invention is the distance image sensor according to the first or second aspect, wherein the received light amount of the light receiving element in one frame data among the plurality of frame data. All correspond to the amount of light received in one light receiving period among the plurality of light receiving periods.

  A sixth aspect of the distance image sensor according to the present invention is the distance image sensor according to the fifth aspect, wherein the modulation period is an even multiple of the period of the clock signal, and the plurality of frame data are the first frame data. To the fourth frame data, and the received light amount acquisition unit sequentially acquires the received light amount of the light receiving element in response to each of odd-numbered rising edges of the clock signal, and obtains the first frame data. And sequentially acquiring the amount of received light of the light receiving element in response to each of the even-numbered rising edges of the clock signal to acquire the second frame data, and the odd-numbered falling edges of the clock signal. In response to each, the received light amount of the light receiving element is sequentially acquired, the third frame data is acquired, and each even-numbered falling edge of the clock signal is triggered. The amount of light received by the light receiving element are sequentially acquired and to acquire the fourth frame data.

  A seventh aspect of the distance image sensor according to the present invention is the distance image sensor according to the first aspect, wherein the irradiation unit modulates the intensity of the irradiation light in a pulse shape, and the plurality of light receiving periods are A first period from the time when the intensity of the irradiation light rises to a time when the intensity falls, and a second period from the time when the intensity of the irradiation light rises to the time when the intensity of the irradiation light falls. It is.

  An eighth aspect of the distance image sensor according to the present invention is the distance image sensor according to the seventh aspect, wherein the plurality of frame data are first and second frame data, and the received light amount acquisition unit is Then, based on the clock signal, the light receiving amount of the light receiving element in the first period is sequentially obtained to obtain the first frame data, and on the basis of the clock signal, the second light receiving element is obtained. The second frame data is acquired by sequentially acquiring the received light amount in the period.

  A ninth aspect of the distance image sensor according to the present invention is the distance image sensor according to the fifth, sixth, or eighth aspect, wherein a motion vector detection unit that detects a motion vector for the measurement target; A motion compensation unit that associates the light receiving elements with each other in the plurality of frame data based on a motion vector is further provided.

  A distance image sensor according to a tenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the ninth aspect, wherein the irradiation unit outputs a second irradiation light of a DC component, and the motion vector detection unit is The motion vector is detected by using at least two motion vector frame data obtained by sequentially acquiring the amount of light received by the light receiving element in the state in which the second irradiation light is output.

  An eleventh aspect of the distance image sensor according to the present invention is the distance image sensor according to the ninth aspect, wherein the distance image sensor is provided between the light receiving elements and receives a light corresponding to a color. The light receiving element further includes a light receiving element, the light receiving amount acquiring unit sequentially acquires the light receiving amount of the light receiving element and the light receiving amount of the camera light receiving element, acquires the plurality of frame data, and the motion vector detecting unit Detects the motion vector based on the amount of light received by the camera light receiving element in the plurality of frame data.

  A twelfth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to eleventh aspects, wherein the distance image sensor is provided between the light receiving elements and corresponds to a color. And a correction unit that performs correction to subtract a value obtained by multiplying the received light amount of the light receiving element by a coefficient from the received light amount of the camera light receiving element.

  A thirteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the twelfth aspect, wherein the light receiving element used for the correction is the correction target among the light receiving elements. It is provided at a position closest to the camera light receiving element.

  A fourteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the twelfth or thirteenth aspect, wherein the irradiation light is infrared light, and each of the camera light receiving elements is red, The light corresponding to either blue or green is received, and the correction unit performs the correction only on the amount of light received by the camera light receiving element corresponding to the red.

  A fifteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to fourteenth aspects, wherein the irradiation unit is acquired in a state where the irradiation light is not irradiated. A calibration unit that outputs the amount of light received by the light receiving element as ambient light, subtracts the ambient light from the amount of light received by the light receiving element, and outputs the amount of light received from the calibration unit from analog data to digital data. And a conversion unit for converting.

  A sixteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to fifteenth aspects, wherein each of the light receiving elements has a current corresponding to the intensity of received light. And a plurality of capacitors through which the current flows, and a switch that changes a connection relationship between the photodetector and the plurality of capacitors, and the received light amount acquisition unit is connected to each of the light receiving elements. The voltage charged in the capacitor is acquired as the amount of received light.

  A seventeenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the sixteenth aspect, wherein the switch is controlled so that the smaller the irradiation light is, the more the condenser detects the light detector. It further includes a switch control unit that reduces the combined capacitance of the connected capacitors.

An eighteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the third to sixth aspects, wherein the modulation period is twice the period of the clock signal, The distance calculation unit includes the received light amounts A (T0), A (T2), A (T3), A (T1), the irradiation light, and the reflection in the first to fourth frame data of the light receiving element. Based on the relational expression with respect to the phase difference φ with respect to light, φ = tan ⁻¹ [{A (T3) −A (T1)} / {A (T0) −A (T2)}] Calculate the distance to the measurement object.

A nineteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the third to sixth aspects, wherein the modulation cycle is 2N of the clock signal cycle (N is 2 or more). The distance calculation unit calculates the sum A (T0) of the received light amounts in the first frame data of the N light receiving elements adjacent to each other and the N light receiving elements. The sum A (T2) of the received light amounts in the second frame data, the sum A (T3) of the received light amounts in the third frame data of the N light receiving elements, and the N light receiving elements. The relational expression between the received light amount sum A (T1) in the fourth frame data and the phase difference φ between the irradiation light and the reflected light, φ = tan ⁻¹ [{A (T3) −A (T1 )} / {A (T0) -A (T2)}] Based on, to calculate the distance from the irradiation portion to the measurement target.

  According to the first and second aspects of the distance image sensor according to the present invention, even in the method of acquiring the frame data by sequentially acquiring the received light amount based on the clock signal (for example, the global shutter method), the distance An image can be obtained.

  Regarding the third aspect of the distance image sensor according to the present invention, a case where the modulation period is twice the clock period will be described first. At this time, the clock cycle starting from the even-numbered rise corresponds to, for example, the first half of the modulation cycle (period T0), and the clock cycle starting from the odd-numbered rise is the half of the second half of the modulation cycle. This corresponds to (period T2).

  In addition, the received light amount of one light receiving element in the first and second frame data is acquired with the even-numbered and odd-numbered rises as triggers, respectively. Therefore, if the amount of received light in the first frame data of one light receiving element corresponds to the amount of received light A (T0) in period T0, the amount of received light in the second frame data of one light receiving element is the amount of received light A in period T2. This corresponds to (T2).

  Further, for example, if the period between the rising edge of the clock signal and the next falling edge corresponds to one quarter of the modulation period, the clock period starting from the falling edge of the clock signal is the period T0, T2. Is shifted by one quarter of the modulation period.

  The modulation period is twice that of the clock signal. Therefore, the clock cycle starting from the even-numbered fall corresponds to a half cycle (period T3) shifted by a quarter of the modulation period with respect to one of the periods T0 and T2 (for example, period T0). The clock cycle starting from the falling edge corresponds to a half cycle (period T1) shifted by a quarter of the modulation period with respect to the other of the periods T0 and T2 (for example, the period T2).

  In addition, in the third and fourth frame data, the received light amount of one light receiving element is acquired with the even-numbered and odd-numbered falling as a trigger, respectively. Therefore, the amount of received light in the third frame data of one light receiving element corresponds to, for example, the amount of received light A (T3) in the period T3, and the amount of received light in the fourth frame data of one light receiving element is, for example, the amount of received light in the period T1. This corresponds to A (T1).

  Therefore, the phase difference φ between the irradiated light and the reflected light can be calculated using these received light amounts using, for example, the following equation.

φ = tan ⁻¹ [{A (T3) −A (T1)} / {A (T0) −A (T2)}]

  In other words, it is possible to acquire a received light amount suitable for distance measurement.

  When the modulation period is 2N (N is an integer of 2 or more) times the period of the clock signal, if N light receiving elements are regarded as one virtual light receiving element, the virtual light receiving element in the first to fourth frame data Light receiving amounts A (T0), A (T2), A (T3), and A (T1) can be obtained. This point will be described in detail in the seventh embodiment.

  According to the fourth aspect of the distance image sensor according to the present invention, the amount of received light is acquired every time the clock signal rises. For example, compared to the case where the amount of received light is acquired by skipping one rising edge of the clock signal, The first and second frame data can be acquired in a short period. Similarly, the third and fourth frame data can be acquired in a relatively short period.

  According to the fifth aspect of the distance image sensor of the present invention, any light reception amount of the light receiving element in one frame data corresponds to the light reception amount in one light reception period. Therefore, it is possible to correspond to the amount of received light in a plurality of light receiving periods for each frame data. Therefore, a received light amount suitable for motion compensation can be obtained.

  According to the sixth aspect of the distance image sensor of the present invention, for example, the total amount of light received by the light receiving elements in the first frame data corresponds to the amount of received light A (T0), and the light receiving elements in the second frame data. All the received light amounts correspond to the received light amount A (T2), all the received light amounts of the light receiving elements in the third frame data correspond to the received light amount A (T3), and all the received light elements in the fourth frame data. The amount of received light corresponds to the amount of received light A (T1).

  As described above, it is possible to correspond to the received light amounts A (T0) to A (T3) for each frame data. Therefore, as described below, it is possible to obtain an amount of received light suitable for motion compensation. That is, the received light amounts A (T0) and A (T2) can be acquired regardless of which light receiving element in the second frame data corresponds to one light receiving element in the first frame data. Similarly, the received light amounts A (T1) and A (T3) can be acquired.

  According to the seventh aspect of the distance image sensor of the present invention, for each light receiving element, the amount of light received in the first period and the amount of light received in the second period can be obtained in the first and second frame data. .

  Therefore, the time difference tt between the irradiation light and the reflected light can be calculated for each light receiving element by using these received light amounts, for example, using the following equation. Hereinafter, the period length of the first period, and the amount of light received in the first period and the second period are denoted by ΔT ′, A (T0 ′), and A (T1 ′), respectively.

  tt = ΔT ′ · A (T1 ′) / {A (T0 ′) + A (T1 ′)}

  As described above, according to the seventh aspect, it is possible to acquire a received light amount suitable for distance measurement.

  According to the eighth aspect of the distance image sensor of the present invention, it is possible to obtain a received light amount suitable for motion compensation.

  According to the ninth aspect of the distance image sensor of the present invention, motion compensation can be performed.

  According to the tenth aspect of the distance image sensor of the present invention, since the motion vector is detected using the second irradiation light of the DC component, it is compared with the case where the motion vector is detected using the irradiation light of the modulation component. Detection accuracy is high.

  According to the eleventh aspect of the distance image sensor of the present invention, there is a received light amount corresponding to a color in a plurality of frame data, which contributes to creation of image data of a natural image. In addition, it is not necessary to acquire motion vector frame data used only for detecting a motion vector. Therefore, the required data amount can be reduced as compared with the case of further acquiring motion vector frame data.

  According to the twelfth aspect of the distance image sensor of the present invention, the manufacturing cost can be reduced as compared with the case where a filter for removing light having the same wavelength as the irradiation light is provided in the light receiving element for a camera.

  According to the thirteenth aspect of the distance image sensor of the present invention, the correction accuracy can be improved.

  According to the fourteenth aspect of the distance image sensor of the present invention, only the light receiving element for a camera corresponding to red that is most likely to receive light having the same wavelength as the irradiated light is corrected. Accordingly, it is possible to obtain image data closer to the actual color as compared with the case where only the other light receiving elements for cameras are corrected, while reducing the arithmetic processing.

  According to the fifteenth aspect of the distance image sensor of the present invention, it is possible to improve the S / N ratio of the received light amount input to the conversion unit.

  According to the sixteenth aspect of the distance image sensor of the present invention, the combined electrostatic capacitance of the capacitor connected to the photodetector can be changed. Therefore, the sensitivity in the amount of received light can be adjusted.

  According to the seventeenth aspect of the distance image sensor of the present invention, the sensitivity of the light receiving element can be improved when the amount of irradiation light is small. Therefore, the amount of received light can be appropriately acquired even if the amount of irradiation light is small.

  According to the eighteenth aspect of the distance image sensor of the present invention, the distance can be measured.

  According to the nineteenth aspect of the distance image sensor of the present invention, since the cycle of the clock signal can be shortened regardless of the performance of the irradiation apparatus, the period for acquiring the first to fourth frame data can be shortened. it can. In addition, the sum of the amounts of light received by the N light receiving elements is used. Thereby, N light receiving elements can be grasped as one virtual light receiving element, and a distance can be calculated for each virtual light receiving element.

  The objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

It is a figure which shows an example of a notional structure of a distance image sensor. It is a schematic diagram which shows an example of the arrangement | sequence of a light receiving element. It is a schematic diagram which shows an example of irradiated light and reflected light. It is a schematic diagram which shows an example of a clock signal and irradiation light. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of 1st frame data. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of 2nd frame data. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of 3rd frame data. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of 4th frame data. It is a schematic diagram which shows an example of a clock signal, irradiation light, and the reflected light received by one light receiving element. It is a schematic diagram which shows an example of a clock signal, irradiation light, and the reflected light received by one light receiving element. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of a clock signal, irradiation light, and the reflected light received by one light receiving element. It is a schematic diagram which shows an example of irradiated light and reflected light. It is a schematic diagram which shows an example of a clock signal and irradiation light. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of 1st frame data. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of 2nd frame data. It is a schematic diagram which shows an example of the arrangement | sequence of a light receiving element. It is a schematic diagram which shows an example of a measuring object. It is a schematic diagram which shows an example of a measuring object. It is a figure which shows an example of a notional structure of a distance image sensor. It is a figure which shows an example of a notional structure of a motion compensation part. It is a schematic diagram which shows an example of a measuring object. It is a schematic diagram which shows an example of a measuring object. It is a schematic diagram which shows an example of several frame data and a motion vector. It is a figure which shows an example of a notional structure of a distance image sensor. It is a figure which shows an example of a notional structure of a distance image sensor. It is a front view which shows an example of a conceptual external appearance of a distance image sensor. It is a side view which shows an example of the conceptual external appearance of a distance image sensor. It is a figure which shows an example of a notional structure of the light receiving element Pk. It is a figure which shows an example of a notional structure of a distance image sensor. It is a schematic diagram which shows an example of a clock signal and irradiation light. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount. It is a schematic diagram which shows an example of the acquisition timing of a clock signal and light reception amount.

First embodiment.
FIG. 1 is a block diagram illustrating an example of a conceptual configuration of the distance image sensor according to the first embodiment. The distance image sensor includes a light receiving device 1, a control arithmetic device 2, and an irradiation device 3.

  The irradiation device 3 irradiates the measurement target with the irradiation light L1 while adjusting (modulating) the intensity of the irradiation light L1. The irradiation device 3 is, for example, a light emitting diode (LED) that outputs near infrared light. The intensity is controlled by the control arithmetic unit 2 as will be described later.

  The light receiving device 1 includes a light receiving element group 11 and a received light amount acquisition unit 10. The light receiving element group 11 includes a plurality of light receiving elements P1, P2, P3,... As illustrated in FIG. Each of the light receiving elements P1, P2, P3,... Receives the light reflected by the measurement object as reflected light L2. Such a light receiving element group 11 is used in a CMOS sensor (for example, Non-Patent Document 3), and more specifically, for example, employed in SMPD (Single-Carrier-Modulation-Photo-Detector). In particular, SMPD is suitable because it has characteristics such as high sensitivity, high speed operation, wide dynamic range, wide spectral characteristics, manufacturability, and low power consumption.

  The light receiving elements P1, P2, P3,... Are two-dimensionally arranged, and in the example of FIG. Hereinafter, the light receiving elements P1, P2, P3,...

  For example, the light receiving element Pk includes a photodetector 11a and an amplifier circuit 11b. The photodetector 11a receives the reflected light L2 from the measurement object, converts it into a charge, and stores it in a charge storage section (not shown). The amplifier circuit 11b converts and amplifies the accumulated charge into a voltage. As a result, the amount of light received by the light receiving element Pk is output as a voltage.

  The received light amount acquisition unit 10 sequentially acquires the received light amount of each light receiving element Pk. For example, the light receiving elements in the first row are sequentially selected from the first column to the last column, the amount of received light is acquired, and then the light receiving elements in the second row from the first column to the last column. The received light amount is acquired by selecting sequentially. Thereafter, similarly, the light receiving elements from the third row to the last row are sequentially selected, and the amount of received light is acquired. Such an acquisition order is applied to a so-called rolling shutter type imaging apparatus.

  In order to execute such sequential acquisition, the received light amount acquisition unit 10 includes, for example, a timing control unit 12, a row decoder 13, a column decoder 14, a correlated double sample unit 15, an amplification unit 16, and an AD conversion unit 17 ( FIG. 1). The timing control unit 12 receives the clock signal CLK from the control arithmetic device 2. Based on the clock signal CLK, the timing controller 12 notifies the selection timing to the row decoder 13 and the column decoder 14 in order to sequentially select the light receiving elements Pk. The row decoder 13 appropriately selects the light receiving elements Pk for each row based on the selection timing, and the column decoder 14 appropriately selects the light receiving elements Pk for each column based on the selection timing.

  The received light amount of the selected light receiving element Pk is output to the correlated double sample unit 15. As is known, the correlated double sample unit 15 suppresses the noise of the received light amount and outputs the received light amount to the amplifying unit 16. The amplifying unit 16 amplifies the received amount of received light and outputs it to the AD converting unit 17. The AD conversion unit 17 converts the received light reception amount from analog data to digital data and outputs it to the control arithmetic unit 2.

  The control arithmetic device 2 has a recording unit 23, and the amount of light received by each light receiving element Pk is recorded in the recording unit 23 as frame data.

  The control arithmetic device 2 controls the modulation of the irradiation light L1 of the irradiation device 3 and the acquisition timing of the received light amount of the light receiving element Pk in the light receiving device 1. As a result, as will be described in detail below, a light reception amount suitable for calculating the distance from the irradiation device 3 to the measurement target is acquired for each light receiving element Pk. In the following, a method for calculating the distance from the light receiving device 1 to the measurement target will be described first, and then the timing for obtaining the modulation of the irradiation light L1 and the amount of light received by the light receiving element Pk will be described.

<Distance calculation>
The irradiation light L1 output from the irradiation device 3 is reflected by the measurement object, and is received by the light receiving device 1 as reflected light L2. Therefore, the longer the distance from the irradiation device 3 to the measurement target, the longer the period from when the irradiation light L1 is irradiated until it is received as the reflected light L2. In other words, the longer the distance, the larger the phase difference φ between the phase of the irradiation light L1 and the phase of the reflected light L2. In other words, the distance from the irradiation device 3 to the measurement target can be calculated by calculating the phase difference φ. Such a distance calculation method based on the phase difference φ is called a so-called TOF method.

  In order to calculate the phase difference φ in each light receiving element Pk, as is well known, the amount of light received in a plurality of light receiving periods is required in each light receiving element Pk. The plurality of light receiving periods are determined according to the modulation period. In the following, Non-Patent Document 1 will be described as an example of a method for calculating a phase difference so as to give a specific example.

  In Non-Patent Document 1, the intensity of the irradiation light L1 is periodically modulated with respect to time. For example, as shown in FIG. 3, the intensity of the irradiation light L1 is modulated in a sine wave shape. In the illustration of FIG. 3, the minimum value of the irradiation light L1 takes a value larger than zero. However, the present invention is not limited to this, and the minimum value may be substantially zero.

  Hereinafter, the period of the irradiation light L1 is referred to as a modulation period T. In the illustration of FIG. 3, the start time of the modulation period T is the time when the intensity of the irradiation light L1 takes the maximum value.

  Such irradiation light L1 is irradiated onto the measurement target, reflected by the measurement target, and incident on the light receiving element Pk as reflected light L2. Therefore, as shown in FIG. 3, the phase of the reflected light L2 received by the light receiving element Pk is delayed with respect to the irradiation light L1. As described in Non-Patent Document 1, this phase difference φ can be calculated by the following relational expression.

φ = tan ⁻¹ [{A (T3) -A (T1)} / {A (T0) -A (T2)}] (1)

  Here, T0 and T2 are half periods with respect to the modulation period T. In the example of FIG. 3, the start time of the period T0 is the time when the intensity of the irradiation light L1 takes the maximum value, and the start time of the period T2 is the irradiation light L1. Is the time when takes the minimum value. T3 and T1 are periods shifted by a quarter (= T / 4) of the modulation period T with respect to the periods T0 and T2, respectively. A (t) is an integral value of the intensity of the reflected light L2 in the period t (that is, the amount of light received in the period t).

  Therefore, if the amount of received light necessary for calculating the phase difference for each light receiving element Pk (here, A (T0) to A (T3)) can be obtained, the phase difference φ can be calculated for each light receiving element Pk. As a result, the distance from the irradiation device 3 to the measurement target can be calculated for each light receiving element Pk. Therefore, in order to obtain the amount of received light necessary for calculating the phase difference for each light receiving element Pk, the control calculation device 2 controls the modulation of the irradiation device 3 and the acquisition timing of the received light amount in the light receiving device 1 in association with each other. This will be described in detail below.

<Modulation of irradiation device and acquisition of received light amount>
Referring to FIG. 1, the control arithmetic device 2 includes a clock generation unit 21 and an irradiation light control unit 22. Here, the control arithmetic unit 2 includes a microcomputer and a storage device. The microcomputer executes each processing step (in other words, a procedure) described in the program. The storage device is, for example, a ROM (Read-Only-Memory), a RAM (Random-Access-Memory), a rewritable nonvolatile memory (EPROM (Erasable-Programmable-ROM), etc.), and various storage devices such as a hard disk device. One or more can be configured. The storage device stores various information, data, and the like, stores a program executed by the microcomputer, and provides a work area for executing the program. It can be understood that the microcomputer functions as various means corresponding to each processing step described in the program, or can realize that various functions corresponding to each processing step are realized. The control arithmetic device 2 is not limited to this, and various procedures executed by the control arithmetic device 2 or various means or various functions realized may be realized in hardware or in part by hardware.

  The clock generation unit 21 generates a clock signal CLK. The clock signal CLK changes in synchronization with each of the start and end of a plurality of light receiving periods (for example, periods T0 to T3) for obtaining a plurality of light receiving amounts necessary for calculating the phase difference. The clock signal CLK is output to the light receiving device 1 and the irradiation light control unit 22. Here, as an example, the active period and the inactive period of the clock signal CLK are equal to each other. In other words, the duty of the clock signal CLK is 50%.

  The irradiation light control unit 22 controls the irradiation device 3 so as to modulate the intensity of the irradiation light L1 with a modulation period T that is twice the period of the clock signal CLK, for example. As a result, the clock signal CLK changes in synchronization with the start and end of the light receiving period (periods T0 to T3) for the amount of light received necessary for distance calculation. Further, for example, the irradiation light control unit 22 generates a sine wave voltage based on the clock signal CLK, and outputs it to the irradiation device 3. The irradiation device 3 emits irradiation light L1 having an intensity corresponding to the magnitude of the voltage. As a result, the irradiation light L1 has a sinusoidal shape with a modulation period T as shown in FIG.

  The received light amount acquisition unit 10 repeatedly acquires the received light amount of the light receiving element Pk in a predetermined order based on the clock signal CLK and repeats the step of generating frame data, thereby acquiring a plurality of frame data. The amount of light received by the same light receiving element Pk in the plurality of frame data is made to correspond to the amount of light received in a plurality of light receiving periods necessary for calculating the phase difference. Here, as an example, the first to fourth frame data are acquired. This will be described in detail below.

  As illustrated in FIG. 5, for example, the received light amount acquisition unit 10 sequentially acquires the received light amounts of the light receiving elements Pk sequentially in a predetermined order with each rising edge of the clock signal CLK. However, the amount of light received here is the amount of light received in one cycle of the clock signal CLK (that is, a half cycle of the modulation cycle T). More specifically, the light receiving element Pk starts accumulating charges according to the amount of received light at the rise of the clock signal CLK, and the total amount of charges accumulated until the next rise is acquired as the amount of received light. Such a light receiving time is the same as an exposure time in a so-called imaging apparatus, and is controlled by a known technique. For example, a charge storage unit (not shown) into which a current from the photodetector 11a flows and a switch for controlling conduction between the photodetector 11a and the charge storage unit may be provided to control an on period (light reception time) of the switch.

  In the example of FIG. 5, first, the amount of light received by the light receiving element P1 in the first row and first column is acquired in response to the rising edge UE1 of the clock signal. Then, the amount of light received by the light receiving element P2 in the first row and the second column is acquired at the next rising UE2 after the rising UE1. Thereafter, similarly, the amount of light received by the light receiving element Pk is sequentially acquired in response to the rising edge of the clock signal CLK, and the amount of light received by the light receiving element in the last row and last column is acquired, whereby the frame data FD1 shown in FIG. Is acquired. T0 and T2 added in FIGS. 5 and 6 will be described later.

  Subsequently, as shown in FIG. 7, with each rising edge of the clock signal CLK, the received light amounts of the light receiving elements Pk are sequentially acquired in the same order as the frame data FD1 to acquire the frame data FD2 (FIG. 7). (See also 8). However, if the rising UE1 is defined as the first rising, in the frame data FD2, as shown in FIG. 7, the received light amount of the first light receiving element P1 is acquired with the even-numbered rising UE2n as a trigger. This technical significance will be described in detail later.

  In the example of FIG. 2, the number of light receiving elements Pk is (2n−2). Therefore, the rising UE2n-2 that triggers the acquisition of the amount of light received by the last light receiving element P2n-2 in the frame data FD1 is the even-numbered rising. Therefore, the next rising edge UE2n-1 is an odd-numbered rising edge. Therefore, in this example, the amount of light received by the light receiving element P1 is acquired with the next rising UE2n as a trigger, as shown in FIG. 7, without being triggered by the rising UE2n-1.

  On the other hand, if the number of light receiving elements Pk is an odd number, the rise that triggers the acquisition of the received light amount of the last light receiving element in the frame data FD1 is the odd number rise. Therefore, the light receiving amount of the first light receiving element P1 in the frame data FD2 may be acquired with the rising edge of the clock signal CLK immediately after that (even-numbered rising edge) as a trigger.

  Subsequently, as shown in FIG. 9, the received light amounts of the light receiving elements Pk are sequentially acquired in the same order as the frame data FD1, triggered by each falling edge of the clock signal CLK. In the example of FIG. 9, the received light amount of the first light receiving element P <b> 1 is acquired with the falling DE <b> 1 as a trigger. This falling DE1 may be a falling immediately after the acquisition of the frame data FD2. The amount of light received here is also the amount of light received in one cycle of the clock signal CLK. More specifically, the light receiving element Pk starts accumulating charges according to the amount of received light when the clock signal CLK falls. Then, the total amount of charges accumulated until the next fall is acquired as the received light amount. Such a light receiving time is also the same as an exposure time in a so-called imaging device, and is controlled by a known technique.

  Then, the amount of light received by the light receiving element P2 in the first row and the second column is acquired in response to the next falling DE2 after the falling DE1. Thereafter, in the same manner, each light receiving amount of the light receiving element Pk is sequentially acquired with each falling edge of the clock signal CLK, and the light receiving amount of the light receiving element in the last row and the last column is acquired, whereby frame data FD3 is acquired (see also FIG. 10).

  Subsequently, as shown in FIG. 11, each received light amount of the light receiving element Pk is sequentially acquired in the same order as the frame data FD1 in response to each falling edge of the clock signal CLK, thereby acquiring the frame data FD4. (See also FIG. 12). However, if the falling edge DE1 is defined as the first falling edge, in the frame data FD4, as shown in FIG. 11, the received light amount of the first light receiving element P1 is acquired with the even-numbered falling edge DE2n as a trigger. This technical significance will also be described in detail later.

  In the illustration of FIG. 2, the number of light receiving elements Pk is 2n−2. Therefore, the falling DE2n-2 that triggers the acquisition of the amount of light received by the last light receiving element P2n-2 in the frame data FD3 is the even-numbered falling. Therefore, the next falling edge DE2n-1 is an odd-numbered falling edge. Therefore, in this example, the amount of light received by the light receiving element P1 is acquired with the next falling DE2n as a trigger, as shown in FIG. 11, without being triggered by the falling DE2n-1.

  On the other hand, if the number of light receiving elements Pk is an odd number, the falling that triggers the acquisition of the light receiving amount of the last light receiving element in the frame data FD3 is the odd numbered falling. Therefore, the amount of light received by the first light receiving element P1 in the frame data FD4 may be acquired in response to the fall of the clock signal CLK immediately after that (even-numbered fall).

  Next, the amount of light received by such a procedure will be considered by focusing on one of the light receiving elements Pk. First, the light receiving element P1 will be described, and the other light receiving elements will be described later. FIG. 13 shows a schematic example of the clock signal CLK, the irradiation light L1, and the reflected light L2 received by the light receiving element P1.

  In the illustration of FIG. 13, the intensity of the irradiation light L1 takes the maximum value at the time of the rising edge UE1 of the clock signal CLK. Moreover, since the modulation period T is twice the clock period, one period starting from the rising UE1 corresponds to the period T0 (that is, the first half period of the modulation period T). Therefore, the amount of light received by the light receiving element P1 acquired with the rising UE1 as a trigger can be regarded as an integrated value (light reception amount A (T0)) of the reflected light L2 in the period T0. That is, the received light amount in the frame data FD1 of the light receiving element P1 can be regarded as the received light amount A (T0). In order to show this, in FIGS. 5 and 6, the symbol T0 is appended in parentheses corresponding to the light receiving element P1.

  Referring to FIG. 13, since one cycle starting from the first rising UE1 corresponds to the period T0, the clock cycle starting from the even-numbered rising UE2n is the period T2 (that is, the second half of the modulation period T). Half a cycle). Therefore, the amount of light received by the light receiving element P1 acquired with the rising UE2n as a trigger can be regarded as an integrated value (light reception amount A (T2)) of the reflected light L2 in the period T2. That is, the amount of received light in the frame data FD2 of the light receiving element P1 can be regarded as the amount of received light A (T2). In order to show this, in FIGS. 7 and 8, the symbol T2 is appended in parentheses corresponding to the light receiving element P1.

  The active period and the inactive period of the clock signal CLK are equal to each other. Therefore, both the active period and the inactive period are a quarter of the modulation period T. Therefore, one cycle starting from the falling DE1 of the clock signal CLK corresponds to one of the periods T1 and T3 (period T3 in the example of FIG. 13). Therefore, in the illustration of FIG. 13, the amount of light received by the light receiving element P1 acquired with the falling DE1 as an opportunity can be regarded as the integrated value of the reflected light L2 (the amount of received light A (T3)) in the period T3. That is, the amount of received light in the frame data FD3 of the light receiving element P1 can be regarded as the amount of received light A (T3). In order to show this, in FIGS. 9 and 10, the symbol T3 is appended in parentheses corresponding to the light receiving element P1.

  Also, since one cycle starting from the first falling DE1 corresponds to one of the periods T1 and T3, the clock cycle starting from the even-numbered falling DE2n is the other of the periods T1 and T3 (in the example of FIG. 13). Corresponds to the period T1). Therefore, in the illustration of FIG. 13, the amount of light received by the light receiving element P1 acquired with the falling DE2n as an opportunity can be regarded as the integrated value of the reflected light L2 (light reception amount A (T1)) in the period T1. That is, the received light amount in the frame data FD4 of the light receiving element P1 can be regarded as the received light amount A (T1). In order to show this, in FIGS. 11 and 12, the symbol T1 is attached in parentheses corresponding to the light receiving element P1.

  As described above, the received light amounts A (T0) to A (T3) can be acquired based on the received light amounts of the frame data FD1 to FD4 of the light receiving element P1. Therefore, the phase difference φ can be calculated in the light receiving element P1 based on the formula (1), and the distance of the measurement target (however, the portion received by the light receiving element P1) can be calculated based on the phase difference φ.

  Also in other light receiving elements Pk, the received light amounts A (T0) to A (T3) can be acquired. This is because the light receiving elements Pk are acquired in the same order in each of the frame data FD1 to FD4. 5 to 12, the period corresponding to the light receiving element Pk is indicated by the parenthesized reference numerals. For example, focusing on the light receiving element P2, the received light amount A (T2) is acquired in the frame data FD1, the received light amount A (T0) is acquired in the frame data FD2, and the received light amount A (T1) is acquired in the frame data FD3. The received light amount A (T3) is acquired in the frame data FD4.

  Therefore, it is possible to obtain the received light amounts A (T0) to A (T3) suitable for distance measurement also in the other light receiving elements Pk.

  In the illustration of FIG. 1, the control arithmetic device 2 includes a distance calculation unit 24. The distance calculation unit 24 uses the frame data FD1 to FD4 recorded in the recording unit 23 to calculate the phase difference φ from the equation (1) for each light receiving element Pk, and calculates the distance based on the phase difference φ. Thereby, image data indicating the distance can be obtained. For example, the image data is output to a display unit (not shown) and displayed on the display unit.

  In the above-described example, the received light amount is acquired at each rising edge and falling edge of the clock signal CLK. However, this is not necessarily the case. For example, when the cycle of the clock signal CLK is ¼ of the modulation cycle T, the received light amount may be acquired as follows.

  That is, as illustrated in FIG. 14, with each odd-numbered rising edge of the clock signal CLK as an opportunity, the received light amount in two cycles of the clock signal CLK is sequentially acquired in a predetermined order for each light receiving element Pk. Data FD1 is generated. Here, the rise that triggers the acquisition of the amount of light received by the first light receiving element P1 in the frame data FD1 is defined as the first rise.

  Then, after the generation of the frame data FD1, the received light amount of the first light receiving element P1 is acquired with the (4m-1) (m is a natural number) rise UE4m-1 of the clock signal CLK as an opportunity, and then the odd number The amount of light received by the light receiving element Pk is sequentially acquired in a predetermined order with each rising edge as a trigger to generate frame data FD2.

  Subsequently, after the generation of the frame data FD2, the received light amount of the first light receiving element P1 is obtained with the (4m'-2) (m 'is a natural number) rise U4m'-2 of the clock signal CLK as a trigger. In response to the even-numbered rise, the received light amount of the light receiving element Pk is sequentially acquired in a predetermined order, and the frame data FD3 is acquired.

  Subsequently, after the acquisition of the frame data FD3, the amount of light received by the first light receiving element P1 is acquired with the 4m ″ (m ″ is a natural number) th rising UE4m ″ of the clock signal CLK, and then the light receiving element. The amount of received light Pk is sequentially acquired in a predetermined order to acquire frame data FD4.

  The two cycles starting from the rising UE1 correspond to the light receiving period T0, the two cycles starting from the rising UE4m-1 correspond to the light receiving period T2, and the two cycles starting from the rising UE4m′-2 are the light receiving period T3. And two periods starting from the rising UE 4m ″ correspond to the light receiving period T1. This can be understood from the following explanation. That is, the two periods starting from the first to fourth rising UE1 to UE4 correspond to the light receiving periods T0, T3, T2, and T1, respectively, as can be understood from FIG. Considering periodically, two periods starting from the first, fifth,..., 4M-3 (M is a natural number) start correspond to the light receiving period T0, and the second, sixth,. The 2 cycles starting from the 4M-2th rise correspond to the light receiving period T3, and the 2nd cycle starting from the 3rd, 7th,..., 4M-1 start corresponds to the light receiving period T2. Two cycles starting from the 4Mth rise correspond to the light receiving period T1.

  Therefore, even with such an acquisition method, the received light amount of the first light receiving element P1 in the frame data FD1 to FD4 corresponds to the received light amount A (T0), A (T2), A (T3), and A (T1), respectively.

  Since the received light amounts are acquired in the same order in the frame data FD1 to FD4, the received light amounts A (T0) to A (T3) can be obtained from the frame data FD1 to FD4 for the other light receiving elements Pk.

  In this case, the falling edge of the clock signal CLK is not used. Therefore, the duty of the clock signal CLK is not limited to 50% and is arbitrary.

  In the above example, for example, two types of received light amounts A (T0) and A (T2) are acquired in the frame data FD1, and two types of received light amounts are acquired in each of the other frame data FD2 to FD4. However, in each of the plurality of frame data, one kind of received light amount necessary for calculating the phase difference may be acquired. However, the types in each frame data are different from each other. For example, the received light amount A (T0) is acquired in all the light receiving elements Pk of the frame data FD1, the received light amount A (T2) is acquired in all the light receiving elements Pk of the frame data FD2, and all the light receiving elements Pk of the frame data FD3. The light reception amount A (T3) may be acquired in step S1, and the light reception amount A (T1) may be acquired in all the light receiving elements Pk of the frame data FD4. Hereinafter, an example thereof will be described in detail with reference to FIG.

  That is, the received light amount acquisition unit 10 sequentially acquires the received light amounts of the light receiving elements Pk in response to the odd-numbered rising edges UE1, UE3,. Data FD1 is acquired. Next, the received light amount acquisition unit 10 sequentially acquires each received light amount of the light receiving element Pk with each of the even-numbered rising edges UE2m, UE2m + 2,..., UE4m-2 of the clock signal as frame data. Obtain FD2. Next, the received light amount acquisition unit 10 sequentially acquires the received light amounts of the light receiving elements Pk in response to the odd-numbered falling edges DE1, DE3,..., DE2m-1 of the clock signal CLK. , Frame data FD3 is acquired, and then each received light amount of the light receiving element Pk is sequentially acquired with each of the even-numbered falling edges DE2m, DE2m + 2,. Frame data FD4 is acquired.

  Accordingly, in the illustration of FIG. 15, the received light amount of any light receiving element Pk in the frame data FD1 corresponds to the received light amount A (T0), and the received light amount of any light receiving element Pk in the frame data FD2 is the received light amount A (T2). In the frame data FD3, the received light amount of an arbitrary light receiving element Pk corresponds to the received light amount A (T3), and in the frame data FD4, the received light amount of the arbitrary light receiving element Pk corresponds to the received light amount A (T1). . Therefore, the received light amounts A (T0) to A (T3) can be obtained for each light receiving element Pk also by such an acquisition method.

  In short, the amount of light received by the light receiving element Pk may be acquired so as to satisfy the following. First, the amount of light received by one light receiving element Pk is acquired in the frame data FD1 when one of the even-numbered and odd-numbered rising edges of the clock signal CLK is triggered, and in the frame data FD2, the other of the clock signal CLK is obtained. Acquired at the start of As a result, the received light amount of one light receiving element Pk in the frame data FD1 corresponds to one of the received light amounts A (T0) and A (T2), and the received light amount of one light receiving element Pk in the frame data FD2 is the received light amount. This corresponds to the other of A (T0) and A (T2).

  Second, one light receiving element Pk is acquired in response to one of the even-numbered and odd-numbered falling edges of the clock signal CLK in the frame data FD3, and the other falling edge of the clock signal CLK in the frame data FD4. Acquired as an opportunity. As a result, the received light amount of one light receiving element Pk in the frame data FD3 corresponds to one of the received light amounts A (T3) and A (T1), and the received light amount of one light receiving element Pk in the frame data FD4 is the received light amount. It corresponds to the other of A (T3) and A (T1).

  Therefore, the received light amounts A (T0) to A (T3) for one light receiving element Pk can be obtained from the frame data FD1 to FD4. The same applies to the other light receiving elements Pk.

  In addition, according to the acquisition method demonstrated with reference to FIGS. 5-12, light reception amount is acquired for every rising of a clock signal, and for every falling. Therefore, each frame data can be obtained earlier than the acquisition method of FIG.

  On the other hand, according to the acquisition method described with reference to FIG. 15, it is possible to obtain a received light amount suitable for motion compensation. This point will be described in detail in the third embodiment.

  In the above example, the calculation method of Non-Patent Document 1 is illustrated. Below, the calculation method of patent document 4 is also illustrated. In Patent Document 4, this phase difference φ can be calculated by the following relational expression.

φ = tan ⁻¹ [{A (t0) -A (t2)} / {A (t1) -A (t3)}] (2)

  Here, as shown in FIG. 16, the start of the periods t0 to t3 is shifted by a quarter of the modulation period T as in the periods T0 to T3. However, the periods t0 to t3 are different from the periods T0 to T3, and the modulation period T Is a quarter cycle. In the illustration of FIG. 16, the start time of the period t0 is a time when the intensity of the irradiation light L1 takes an average value.

  When formula (2) is employed, if the received light amounts A (t0) to A (t3) in the periods t0 to t3 can be obtained for each light receiving element Pk, the distance can be calculated for each light receiving element Pk.

  The clock signal CLK changes in synchronization with each of the start and end of the light receiving period (periods t0 to t3) for the amount of light received necessary for calculating the phase difference. For example, as shown in FIG. 16, the modulation period T is set to four times the period of the clock signal CLK. In FIG. 16, the clock signal CLK rises at each of the boundaries between periods t0 to t3. FIG. 16 also shows the reflected light L2 received by the light receiving element P1.

  The received light amount acquisition unit 10 sequentially acquires the received light amount of the light receiving element Pk based on the clock signal CLK and acquires the frame data FD1 to FD4, thereby obtaining the received light amount of the light receiving element Pk of the frame data FD1 to FD4. The amount of received light is set to correspond to A (t0) to A (t3).

  For example, every time the clock signal CLK rises, the amount of light received by the light receiving element Pk (here, the amount of light received for one period of the clock signal CLK) is sequentially obtained in a predetermined order to obtain the frame data FD1. Here, the rising edge of the clock signal CLK that triggers the acquisition of the amount of light received by the first light receiving element P1 in the frame data FD1 is defined as the first rising edge UE1.

  After acquiring the frame data FD1, the received light amount of the light receiving element Pk is sequentially acquired in the same order every time the clock signal CLK rises to acquire the frame data FD2. However, the amount of light received by the first light receiving element P1 in the frame data FD2 is acquired in response to the (4m-1) th rising UE4m-1 of the clock signal CLK.

  Next, every time the clock signal CLK rises, the light receiving elements Pk are sequentially acquired in the same order to acquire the frame data FD3, and then the frame data FD4 is acquired in the same manner. However, the received light amount of the first light receiving element P1 in the frame data FD3 and FD4 is acquired with the (4m′−2) th rising UE4m′−2 and the 4m ″ th rising UE4m ″ as a trigger, respectively.

  Here, one cycle starting from the first rising UE1 corresponds to the light receiving period t0, one cycle starting from the rising UE4m-1 corresponds to the light receiving period t1, and 1 starting from the rising UE4m′-2. The period corresponds to the light receiving period t3, and one period starting from the rising UE4m ″ corresponds to the light receiving period t3. Therefore, according to this acquisition method, the received light amount of the light receiving element P1 in the frame data FD1 to FD4 corresponds to the received light amounts A (t0) to A (t3), respectively.

  Since the received light amounts are acquired in the same order in the frame data FD1 to FD4, the received light amounts A (T0) to A (T3) can be obtained from the frame data FD1 to FD4 for the other light receiving elements Pk.

  Although the amount of received light is acquired in response to the rising edge of the clock signal CLK, it may be triggered in response to the falling edge, and each of the rising edge and falling edge of the clock signal CLK as shown in FIGS. The amount of received light may be acquired. However, in this case, the amount of light received in the half cycle of the clock signal CLK is acquired.

  In the above example, the frame data FD1 to FD4 are acquired in this order. However, the acquisition order of the frame data FD1 to FD4 is arbitrary. For example, after acquiring the frame data FD1, the frame data FD4 may be acquired, and then the frame data FD2 and FD3 may be acquired in this order.

  The frame data FD1 to FD4 are acquired in different time zones. Therefore, there is a time difference between the received light amounts A (T0), A (T2), A (T3), and A (T1) in each of the light receiving elements Pk (see also FIG. 13 for example). It is desirable that this time difference is short. This is because, for example, if the measurement object is moving, the light receiving element P1 receives reflected light L2 reflected by different measurement objects in the frame data FD1 to FD4. Therefore, as the time difference is smaller, the light receiving element P1 can receive the reflected light L2 from the same measurement object.

  This time difference is the time required to acquire each frame data FD1 to FD4 (the product of the number of light receiving elements Pk and the clock cycle), in other words, the number of frame data acquired per second (frame rate). Dependent. In order to reduce the time difference, it is desirable that the frame rate is large. For example, the frame rate is desirably larger than 30 frames / second.

  In the above-described example, the calculation method of Non-Patent Document 1 and the calculation method of Patent Document 4 are shown. Here, another calculation method is also described.

  Here, the intensity of the irradiation light L1 is modulated in a rectangular wave shape (pulse shape) as shown in FIG. The reflected light L2 is delayed from the irradiation light L1 as shown in FIG. More specifically, the reflected light L2 is delayed by a time (hereinafter referred to as a time difference) tt corresponding to the distance to the measurement target. Therefore, by calculating this time difference tt, the distance to the measurement object can be calculated. Therefore, it is intended to calculate this time difference tt.

  Here, in order to calculate the time difference tt, the received light amounts A (T0 ') and A (T1') in the periods T0 'and T1' are used. The period T0 'is a period from the time when the irradiation light L1 rises to the time when it falls, and the period T1' is a period following the period T0 ', for example, having the same period length ΔT' as the period T0 '.

  As can be understood from FIG. 17, the received light amount A (T1 ') takes a value corresponding to the time difference tt, and takes a larger value as the time difference tt is longer. The sum of the received light amounts A (T0 ') and A (T1') takes a value corresponding to the modulation period T (= sum of periods T0 'and T1'). If geometrical consideration is given in FIG. 17, the ratio between the time difference tt and the period length ΔT ′ is the difference between the received light amount A (T1 ′) and the sum of the received light amounts A (T0 ′) and A (T1 ′). Will be equal to the ratio. Therefore, the time difference tt can be calculated using the following equation.

  tt = ΔT ′ · A (T1 ′) / {A (T0 ′) + A (T1 ′)} (3)

  The time difference tt is the time required for the irradiation light L1 to reach the light receiving device 1 after being reflected from the irradiation device 3 by the measurement target. Therefore, for example, assuming that the distance from the measurement target to the light receiving device 1 and the distance from the measurement target to the irradiation device 3 are equal to each other, the distance R is calculated by the following equation using the speed of light c.

  R = c · tt / 2 (4)

  The distance to the measurement object can be calculated as described above. Here, in order to employ this calculation method for each light receiving element, it is intended to obtain the received light amounts A (T0 ′) and A (T1 ′) for each light receiving element Pk. For example, the intensity of the irradiation light L1 is modulated in a pulsed manner and periodically as shown in FIG. The modulation period of the irradiation light L1 is the sum of the periods T0 'and T1'.

  The clock cycle of the clock signal CLK is set to be the same as the half value of the modulation cycle, for example. More specifically, for example, the clock signal CLK rises at each of the start of the period T0 'and the start of the period T1'.

  As illustrated in FIG. 19, for example, the received light amount acquisition unit 10 sequentially acquires the received light amounts of the light receiving elements Pk sequentially in a predetermined order with each rising edge of the clock signal CLK. However, the amount of light received here is the amount of light received in one cycle of the clock signal CLK. Such a light receiving time is the same as an exposure time in a so-called imaging apparatus, and is controlled by a known technique.

  In the example of FIG. 19, first, the amount of light received by the light receiving element P <b> 1 in the first row and first column is acquired in response to the rising edge UE <b> 1 ′ of the clock signal. Then, the amount of light received by the light receiving element P2 in the first row and the second column is acquired with the next rising UE2 'after the rising UE1'. Thereafter, similarly, when the rising edge of the clock signal CLK is triggered, the light receiving amount of the light receiving element Pk is sequentially acquired, and the light receiving amount of the light receiving element in the last row and the last column is acquired, whereby the frame data FD1 shown in FIG. 'Is acquired.

  Subsequently, as shown in FIG. 21, the light reception amounts of the light receiving elements Pk are sequentially acquired in the same order as the frame data FD1 ′ with each rising edge of the clock signal CLK as a trigger to acquire the frame data FD2 ′. (See also FIG. 22). However, if the rising UE1 'is defined as the first rising, as shown in FIG. 21, the received light amount of the first light receiving element P1 is acquired in the frame data FD2' triggered by the even-numbered rising UE2n '.

  In the example of FIG. 2, the number of light receiving elements Pk is (2n−2). Therefore, the rising UE2n-2 ′ that is an opportunity to acquire the amount of light received by the last light receiving element P2n-2 in the frame data FD1 ′ is the even-numbered rising. Therefore, this next rising edge UE2n-1 'is an odd-numbered rising edge. Therefore, in this example, the amount of light received by the light receiving element P1 is acquired with the next rising UE2n 'as a trigger, as shown in FIG. 21, without being triggered by the rising UE2n-1'.

  On the other hand, if the number of light receiving elements Pk is an odd number, the rise that triggers the acquisition of the amount of light received by the last light receiving element in the frame data FD1 'is the odd number rise. Therefore, the received light amount of the first light receiving element P1 in the frame data FD2 'may be acquired with the rising edge (even-numbered rising edge) of the clock signal CLK immediately after that.

  Here, one cycle starting from the first rising UE1 'corresponds to the light receiving period T0', and one cycle starting from the rising UE2n 'corresponds to the light receiving period T1'. Therefore, according to this acquisition method, the received light amount of the light receiving element P1 in the frame data FD1 'and FD2' corresponds to the received light amounts A (T0 ') and A (T1'), respectively.

  Since the received light amounts are acquired in the same order in the frame data FD1 ′ and FD2 ′, the received light amounts A (T0 ′) and A (T1 ′) are obtained from the frame data FD1 ′ and FD2 ′ for the other light receiving elements Pk. be able to. Accordingly, it is possible to obtain the received light amounts A (T0 ′) and A (T1 ′) suitable for distance measurement also in the other light receiving elements Pk.

  As shown in FIGS. 20 and 22, the received light amounts A (T0 ') and A (T1') are mixed in the frame data FD1 'and FD2'. However, this is not a requirement. Each of the frame data FD1 'and FD2' may include one type of received light amount. That is, the frame data FD1 ′ includes only one of the received light amounts A (T0 ′) and A (T1 ′), and the frame data FD2 ′ includes only the other of the received light amounts A (T0 ′) and A (T1 ′). As described above, the amount of light received by each light receiving element Pk may be acquired.

  For example, in FIG. 19, by sequentially acquiring the light reception amount of the light receiving element Pk due to the rise of the odd-numbered clock signal CLK, the light reception amount A (T0) for all the light receiving elements Pk in the frame data FD1 ′. ') Can get. Similarly, in FIG. 21, the light reception amount of the light receiving element Pk is sequentially obtained due to the rising of the even-numbered clock signal CLK, so that the light reception amount A ( T1 ′) can be obtained. As described in the third embodiment, this acquisition method is a method suitable for motion compensation.

Second embodiment.
An example of a conceptual configuration of the distance image sensor according to the second embodiment is the same as that in FIG. However, the light receiving element group 11 further includes a plurality of light receiving elements for cameras (hereinafter also referred to as light receiving elements PP). The light receiving elements PP are arranged between the light receiving elements Pk and receive light corresponding to colors. Examples of the light receiving element PP include a light receiving element PR that receives light corresponding to red, a light receiving element PB that receives light corresponding to blue, and a light receiving element PG that receives light corresponding to green.

  Selection of light received by each light receiving element Pk, PP is realized by a filter provided in each light receiving element Pk, PP. That is, the light receiving elements PR, PB, and PG are provided with filters that allow light having wavelengths corresponding to red, blue, and green to pass through. Note that a filter that allows light having a wavelength corresponding to the irradiation light L1 (for example, infrared light) to pass therethrough may be provided in the light receiving element Pk. Here, the filter is provided.

  In the second embodiment, since the camera light receiving element PP corresponding to the color is provided, the light receiving element group 11 can also take an image or an image (hereinafter referred to as a natural image).

  These light receiving elements Pk and camera light receiving elements PP are two-dimensionally arranged. For example, as shown in FIG. 23, one pixel (pixels PX1 and PX2 in FIG. 23) is formed by four light receiving elements arranged in a matrix, and a plurality of the pixels are arranged in a matrix. In the illustration of FIG. 23, two types of pixels PX1 and PX2 are alternately arranged in each row and each column. The pixel PX1 is formed by the light receiving elements Pk, PR, PG, and PB, and the pixel PX2 is formed by the light receiving elements PR, PG, and PB. That is, the pixel PX2 has the same configuration as the conventional pixel, and the pixel PX1 has a new configuration including the light receiving element Pk.

  For example, the pixel PX1 is obtained by replacing one light receiving element PG with the light receiving element Pk with respect to the pixel PX2 including the light receiving elements PR and PB and the two light receiving elements PG. As a result, it is possible to acquire the reflected light L2 for distance measurement while acquiring the three lights corresponding to the colors in each pixel PX1.

  In the example of FIG. 23, both the pixel PX1 not including the light receiving element Pk and the pixel PX2 including the light receiving element Pk are provided. Thereby, the resolution of the natural image can be improved as compared with the structure in which only the pixel PX1 is provided.

  In view of the fact that the blue visibility is lower than that of other colors, the pixel PX1 may be employed as the pixel PX1 in which the light receiving element PB is replaced with the light receiving element Pk. In this case, it is essential that the pixel PX2 is provided to receive blue light.

  Next, an example of timing for acquiring the light reception amounts of the light receiving elements PP and Pk will be described. The received light amount acquisition unit 10 acquires only the received light amount of the light receiving element PP without acquiring the received light amount of the light receiving element Pk, acquires the image data of the natural image, and does not acquire the received light amount of the light receiving element PP. Although the distance image data may be acquired by acquiring only the received light amount of the light receiving element Pk, the first to fourth frame data are acquired at the same time here. To get.

  For example, the amount of light received by the light receiving elements PP and Pk is acquired in the same manner as in the first embodiment, regardless of the distinction between the light received. Although any of the methods described in the first embodiment may be employed, the light receiving elements in the first row are moved from the first column to the last column, for example, triggered by each rising edge of the clock signal CLK. The light receiving amount is acquired by selecting sequentially, and then the light receiving elements in the second row are sequentially selected in the same manner to acquire the received light amount. Thereafter, similarly, the light receiving elements are sequentially selected to obtain the received light amount, and the frame data FD1 is obtained.

  However, unlike the light receiving element Pk, the light receiving period in which the light receiving element PP receives light is not limited to one cycle of the clock signal CLK. For example, each of the light receiving elements PP may initialize the received light amount immediately after the received light amount is acquired, and may continue to receive the reflected light L2 until the received light amount is acquired in the next frame data.

  The frame data FD2 is also acquired in the same manner as in the first embodiment. In response to the even-numbered rise, the amount of light received by the first light receiving element in the frame data FD2 is acquired. Thereafter, the amount of light received by the light receiving element is sequentially obtained every time the clock signal CLK rises in the same order as the frame data FD1.

  Since the acquisition of the frame data FD3 and FD4 is obvious from the first embodiment and the above description, repeated description is avoided.

  Also in the second embodiment, the amount of received light in the frame data FD1 to FD4 of the light receiving element Pk corresponds to the amount of received light A (T0) to A (T3). This is because, in each of the frame data FD1 to FD4, the conditions that trigger the acquisition of the light reception amount of the first light receiving element are set in the same manner as in the first embodiment, and the light reception amounts are acquired in the same order. . A specific example will be described below.

  For example, in FIG. 23, attention is paid to the light receiving element P1 in the second row and the first column, and for the sake of simplicity, it is assumed that the light receiving elements up to the tenth column are arranged in the light receiving element group 11. Here, the amount of light received by the first light receiving element PR (light receiving element in the first row and first column) is acquired in response to the first rising edge UE1 of the clock signal CLK. Then, the amount of light received by the light receiving element P1 in the frame data FD1 is acquired at the 11th rise. Since the odd-numbered rise is a trigger, this received light amount corresponds to the received light amount A (T0).

  The amount of light received by the first light receiving element PR in the frame data FD2 is acquired in response to the even-numbered rise. In addition, the received light amount of the light receiving element is acquired in the same order as the frame data FD1 in the frame data FD2. Therefore, the amount of light received by the eleventh light receiving element P1 is acquired in response to the even-numbered rise. Since the even-numbered rise is a trigger, this received light amount corresponds to the received light amount A (T2).

  The fact that the amount of light received by the light receiving element P1 in the frame data FD3 and FD4 corresponds to the amount of received light A (T3) and A (T1) can be explained in the same way, and thus repeated explanation is avoided. The same applies to the other light receiving elements Pk, so that repeated description is avoided.

  Therefore, similarly to the first embodiment, the second embodiment can obtain the received light amounts A (T0) to A (T3) necessary for the calculation of the distance measurement.

  However, the distance calculation unit 24 needs to read the amount of light received by the light receiving element Pk from the recording unit 23 in distinction from the light receiving element PP. This can be performed, for example, as follows. That is, the arrangement of the light receiving elements PP and Pk in the light receiving element group 11 is recorded in the recording unit 23 in advance. Then, the distance calculation unit 24 reads the arrangement from the recording unit 23 and refers to the arrangement, so that the received light amount of the light receiving element Pk can be appropriately read from the recording unit 23.

  Further, according to the second embodiment, the amount of light received by the light receiving element PP corresponding to the color is also acquired and recorded in the recording unit 23. Therefore, natural image data can be generated using the amount of light received by the recording unit 23. The generated natural image data is displayed on a display device (not shown). Reading the amount of light received by the light receiving element PP separately from the light receiving element Pk may be performed in the same manner as the reading of the light receiving element Pk.

Third embodiment.
As described in the first embodiment, there is a time difference in the amount of light received by the light receiving element Pk in the frame data FD1 to FD4 (see, for example, FIG. 13). Therefore, when the measurement target is moving, the light receiving element Pk receives the reflected light L2 from a different part of the measurement target in the frame data FD1 to FD4. For example, as illustrated in FIG. 24, in the frame data FD1, a certain light receiving element Px receives reflected light L2 from a part to be measured (a part of a person's arm in FIG. 24). As the part moves, as shown in FIG. 25, in the frame data FD2, the light receiving element Px is reflected from another part (a part of the structure arranged behind the person in FIG. 25). L2 is received. Such a phenomenon causes a decrease in the accuracy of distance measurement. In FIGS. 24 and 25, although the movement amount of the measurement object is schematically shown, the movement amount is actually reduced by acquiring each frame data in a short period of time.

  In the third embodiment, an object is to reduce an error caused by the movement of the measurement target by a method different from the shortening of the acquisition period of the frame data.

  The distance image sensor illustrated in FIG. 26 further includes a motion compensation unit 26 as compared with the distance image sensor of FIG. As illustrated in FIG. 27, the motion compensation unit 26 includes a motion vector detection unit 261 and a corresponding unit 262.

  Further, as described in the first embodiment, the received light amount acquisition unit 10 has one of the received light amounts of the light receiving elements Pk in one frame data among a plurality of received light amounts required for calculating the phase difference. The amount of light received by the light receiving element Pk is acquired so as to correspond to the amount of received light. For example, as described with reference to FIG. 15, the amount of light received by the light receiving element Pk is sequentially acquired. In other words, the received light amount acquisition unit 10 acquires the received light amount with the odd-numbered rise of the clock signal CLK as a trigger, acquires the frame data FD1, and acquires the received light amount with the even-numbered rise of the clock signal CLK as a trigger. The frame data FD2 is acquired, the amount of received light is acquired with the odd-numbered falling edge of the clock signal CLK as a trigger, the frame data FD3 is acquired, and the light reception amount is acquired with the even-numbered falling edge of the clock signal CLK as a trigger. Frame data FD4 is acquired. The technical significance of this will become clear from the following explanation.

  The motion vector detection unit 261 detects a motion vector for the measurement target. A motion vector indicates from where to where a predetermined block (a block composed of a plurality of light receiving amounts (output signals of a plurality of light receiving elements) obtained by a plurality of light receiving elements) moves between different frame data. Is a vector. Such a motion vector can be detected by a known method, for example, using luminance data (so-called Y data). On the other hand, since the light receiving element Pk receives the modulated reflected light L2, the amount of light received by the light receiving element Pk does not strictly correspond to luminance data. However, the error is small even if the received light amount received by the light receiving element Pk is regarded as luminance data used for motion detection by increasing the direct current component of the irradiation light L1 as compared with the modulation component.

  Therefore, the motion vector detection unit 261 detects a motion vector by a known method using the received light amounts of the frame data FD1 to FD4. For example, the motion vector detection unit 261 searches for a block having high similarity between the frame data FD1 to FD4, and detects a motion vector between the frame data for each block. A known technique may be employed for detecting the motion vector, and a detailed description thereof will be omitted.

  The correspondence unit 262 associates the received light amounts of the light receiving elements Pk that receive the reflected light L2 from the same position of the measurement target with each other in the frame data FD1 to FD4 based on the motion vector. That is, the received light amount is made to correspond to each of the frame data FD1 to FD4 based on the motion vector.

  For example, description will be made assuming that FIGS. 24 and 25 correspond to the frame data FD1 and FD2, respectively. In the illustration of FIG. 24, in the frame data FD1, the light receiving element Px receives reflected light L2 from a portion (a part of an arm) to be measured. Referring to FIG. 25, in frame data FD2, light receiving element Py receives reflected light L2 from the portion. At this time, the motion vector detecting unit 261 detects a motion vector indicating this motion (that is, a vector starting from the light receiving element Px and ending at the light receiving element Py). Then, the correspondence unit 262 associates the received light amount of the light receiving element Px of the frame data FD1 with the received light amount of the light receiving element Py of the frame data FD2 based on the motion vector.

  28 and 29 are drawings corresponding to the frame data FD3 and FD4, respectively. In the illustration of FIG. 28, the light receiving element Pz receives the reflected light L2 from the portion in the frame data FD3, and in the illustration of FIG. 29, the light receiving element Pw receives the reflected light L2 from the portion in the frame data FD4. At this time, the motion vector detection unit 261 detects a motion vector having the light receiving element Py as the starting point and the light receiving element Pz as the end point between the frame data FD2 and FD3. Based on this motion vector, the correspondence unit 262 associates the received light amount of the light receiving element Py of the frame data FD2 with the received light amount of the light receiving element Pz of the frame data FD3. Similarly, the correspondence unit 262 associates the amount of light received by the light receiving element Pz of the frame data FD3 with the amount of light received by the light receiving element Pw of the frame data FD4.

  As described above, the light receiving element Px of the frame data FD1, the light receiving element Py of the frame data FD2, the light receiving element Pz of the frame data FD3, and the light receiving element Pw of the frame data FD4 are associated with each other.

  In the third embodiment, each of the light reception amounts of the light receiving element Pk in one frame data corresponds to one type of light reception amount among a plurality of light reception amounts required for calculating the phase difference. For example, the amount of received light is acquired as illustrated in FIG. Thus, the received light amount of an arbitrary light receiving element Pk in the frame data FD1 corresponds to the received light amount A (T0), and the received light amount of the arbitrary light receiving element Pk in the frame data FD2 corresponds to the received light amount A (T2). The received light amount of an arbitrary light receiving element Pk in the data FD3 corresponds to the received light amount A (T3), and the received light amount of the arbitrary light receiving element Pk in the frame data FD4 corresponds to the received light amount A (T1).

  Therefore, regardless of which light receiving element Pk corresponds to each frame data FD1 to FD4, the light receiving amounts of the corresponding four light receiving elements correspond to the light receiving amounts A (T0) to A (T3), respectively. Therefore, the phase difference φ can be calculated based on Expression (1) using the received light amounts of the four light receiving elements corresponding to each other. In the above example, the received light amount A (T0) of the light receiving element Px of the frame data FD1, the received light amount A (T2) of the light receiving element Py of the frame data FD2, and the received light amount A (T3) of the light receiving element Pz of the frame data FD3. ) And the received light amount A (T1) of the light receiving element Pw of the frame data FD4, the phase difference φ is calculated by the equation (1), and the distance is calculated based on the phase difference φ.

  The distance calculated here is the distance at the light receiving element of the reference frame data among the light receiving elements Px, Py, Pz, Pw corresponding to each other. Any one of the frame data FD1 to FD4 may be used as the reference frame data, for example, the frame data FD1. The other light receiving elements of the frame data FD1 serving as the reference are also associated with the light receiving elements of the frame data FD2 to FD4 as appropriate as described above, and the other light receiving elements are used by using the received light amounts of the four light receiving elements. The distance at the element is calculated.

  As described above, in the third embodiment, the distance can be calculated while performing motion compensation.

  In the above example, equation (1) is used, but equation (2) may be used. In this case, for example, all the light receiving elements Pk in the frame data FD1 correspond to the received light amount A (t0), all the light receiving elements Pk in the frame data FD2 correspond to the received light amount A (t1), and all the frame data FD3 The light receiving amount of the light receiving element Pk may be acquired so that the light receiving element Pk corresponds to the light receiving amount A (t2) and all the light receiving elements Pk in the frame data FD4 correspond to the light receiving amount A (t3). For example, when the clock signal CLK and the irradiation light L1 of FIG. 16 are used, all the frame data FD1 are acquired by sequentially acquiring the received light amount with the (4M-3) th rise as a trigger and acquiring the frame data FD1. The amount of light received by the light receiving element Pk corresponds to the amount of received light A (t0). Similarly, the received light amount can be acquired so that the frame data FD2 to FD4 become the received light amounts A (t1) to A (t3), respectively.

  Moreover, you may use Formula (3) and Formula (4). In this case, as described in the first embodiment, for example, all the light receiving elements Pk of the frame data FD1 ′ correspond to the received light amount A (T0 ′), and all the light receiving elements Pk of the frame data FD2 ′ receive light. What is necessary is just to acquire the light reception amount of the light receiving element Pk so that it may correspond to quantity A (T1 ').

<Other motion vector detection methods 1>
The motion vector detection unit 261 may detect a motion vector as follows, for example. That is, the frame data MFD1 and MFD2 for detecting the motion vector may be acquired separately from the frame data FD1 to FD4 (see, for example, FIG. 30), and the motion vector may be detected based on the frame data MFD1 and MFD2. . This will be described in detail below.

  Unlike the frame data FD1 to FD4, the frame data MFD1 and MFD2 are acquired in a state where the irradiation device 3 irradiates the irradiation light L1 of the DC component. At this time, since the reflected light L2 of the direct current component is received by the light receiving element Pk, the amount of received light is equivalent to the luminance data used for detecting the motion vector. Therefore, the motion vector detection accuracy can be improved as compared with the case where the motion vector is detected using the modulated irradiation light L1.

  In the example of FIG. 30, the frame data MFD1 is acquired prior to the acquisition of the frame data FD1 to FD4, and the frame data MFD2 is acquired after the acquisition of the frame data FD1 to FD4. A specific operation in this case will be described below.

  Prior to the acquisition of the frame data FD1 to FD4, the control arithmetic device 2 irradiates the irradiation light L1 of the direct current component from the irradiation device 3. This is realized, for example, when the irradiation light control unit 22 applies a constant voltage to the irradiation device 3.

  Further, the control arithmetic unit 2 outputs a clock signal CLK. The received light amount acquisition unit 10 acquires the received light amount of the light receiving element Pk based on the clock signal CLK in a state where the irradiation light L1 of the DC component is output, and acquires the frame data MFD1. The amount of light received by the light receiving element Pk is not limited to the amount of light received in one cycle of the clock signal CLK.

  Then, after acquiring the frame data MFD1, the frame data FD1 to FD4 are acquired in the same manner as in the first embodiment.

  Next, in order to acquire the frame data MFD2, the irradiation device 3 outputs the irradiation light L1 of the direct current component again, and in this state, the received light amount acquisition unit 10 sequentially acquires the received light amount and acquires the frame data MFD2.

  The motion vector detection unit 261 detects a motion vector by a known method based on the frame data MFD1 and MFD2. The motion vector detection unit 261 also applies the detected motion vector to the frame data FD1 to FD4. That is, it is assumed that the measurement object moves also in the frame data FD1 to FD4 according to the motion represented by the motion vector. Here, it is assumed that the frame data MFD1, MFD2, FD1 to FD4 are acquired at equal intervals. Therefore, the motion vector between the frame data MFD1, MFD2, FD1 to FD4 is calculated by dividing the magnitude of the detected motion vector by 5. For example, in FIG. 30, a motion vector starting from the light receiving element Pa in the frame data MFD1 and starting from the light receiving element Pf in the frame data MFD2 is detected. Then, the motion vector MV between the frame data is calculated by dividing the magnitude of this motion vector by 5.

  The correspondence unit 262 associates the light receiving elements in the frame data FD1 to FD4 based on the calculated motion vector MV. For example, the light receiving element Pb of the frame data FD1 indicated by the motion vector MV starting from the light receiving element Pa of the frame data MFD1 is associated with the light receiving element Pa. Similarly, the light receiving element Pc of the frame data FD2 indicated by the motion vector MV starting from the light receiving element Pb of the frame data FD1 is associated with the light receiving element Pb. Similarly, the light receiving element Pb of the frame data FD1, the light receiving element Pc of the frame data FD2, the light receiving element Pd of the frame data FD3, and the light receiving element Pe of the frame data FD4 correspond to each other.

  In the example of FIG. 30, the frame data MFD1, MFD2, FD1 to FD4 are acquired at equal intervals, but may be acquired at different intervals. In this case, the motion vectors may be calculated by weighting the motion vectors detected in the frame data MFD1 and MFD2 according to the intervals.

  Further, it is not always necessary to use only the two frame data MFD1 and MFD2. For example, frame data for detecting a motion vector (hereinafter referred to as a motion vector frame) may be acquired between the frame data FD1 to FD4. For example, a motion vector is detected using a motion vector frame before the frame data FD1 and a motion vector frame between the frame data FD1 and FD2, and this is used as a motion vector between the frame data FD1 and FD2. May be. The same applies to the motion vectors between other frame data.

<Other motion vector detection method 2>
When the light receiving element group 11 includes the camera light receiving element PP as in the second embodiment, the motion vector detecting unit 261 may detect a motion vector based on the amount of light received by the light receiving element PP.

  That is, the received light amount acquisition unit 10 sequentially acquires the received light amount of the light receiving element Pk and the received light amount of the camera light receiving element PP, and acquires the first to fourth frame data FD1 to FD4. Such an acquisition method is, for example, as described in the second embodiment.

  Then, the motion vector detection unit 261 detects a motion vector based on the amount of light received by the light receiving element PP in the first to fourth frame data FD1 to FD4. For example, the motion vector detection unit 261 calculates luminance data for each of the pixels PX1 and PX2 in the frame data FD1 to FD4. Then, a motion vector is calculated by a known method based on the luminance data. Note that the motion vector for the pixel PX1 may be used as the motion vector of the light receiving element Pk. On the other hand, the pixel PX2 does not have the light receiving element Pk, and a motion vector for the pixel PX2 is not necessary. Therefore, the pixel PX2 is not necessarily used.

  Then, the correspondence unit 262 associates the pixel PX1 (more specifically, the light receiving element Pk) in the frame data FD1 to FD4 based on the motion vector.

  According to this, it is not necessary to acquire a motion vector frame. Therefore, the amount of data for motion compensation can be reduced as compared with the case of further acquiring a motion vector frame by the light receiving element Pk. On the other hand, according to the above-described detection method for detecting a motion vector based on frame data obtained using the irradiation device 3 as a light source, the motion vector can be detected even in a dark environment.

Fourth embodiment.
As shown in FIG. 31, the distance image sensor according to the fourth embodiment further includes a correction unit 25 as compared with FIG. The light receiving element group 11 is provided with not only the light receiving element Pk but also a camera light receiving element PP as in the second embodiment. The correction unit 25 performs correction by subtracting a value obtained by multiplying the light reception amount of the light receiving element Pk by a positive coefficient from the light reception amount of the light receiving element PP. In the following, the light received by the light receiving element PP will be described first before describing the significance of the correction unit 25 in detail.

  For example, a filter provided in the light receiving element PP allows light having a wavelength corresponding to a color to pass therethrough and blocks light in other bands. For example, a filter provided in the light receiving element PR in FIG. 23 allows light of red wavelength to pass and blocks light in other bands. Therefore, the light receiving element PR mainly receives light having a red wavelength. However, in practice, the filter cannot always completely block light in other bands. Therefore, for example, light having the same wavelength as the irradiation light L1 (hereinafter referred to as distance measurement light) can be received to some extent by the light receiving element PP. Since the distance measuring light should not be received by the light receiving element PP, it is desired to reduce the light receiving amount of the distance measuring light out of the light receiving amount of the light receiving element PP.

  Therefore, it is considered to estimate the amount of distance measurement light received by the light receiving element PP from the amount of light received by the light receiving element Pk.

  Now, the filter provided in the light receiving element Pk allows the distance measuring light to pass therethrough. However, in practice, the filter does not always pass the distance measurement light without reducing it. It can be understood from the characteristics of the filter at what rate the distance measurement light passes. For example, if the distance measurement light is incident on the filter of the light receiving element Pk and the intensity thereof is defined as k1 (0 <k1 ≦ 1) times, the amount of distance measurement light incident on the filter of the light receiving element Pk Is calculated by a value (Pkk / k2) obtained by dividing the light receiving amount Pkk of the light receiving element Pk by the value k2.

  Then, it is assumed that this distance measuring light is also incident on the filter of the light receiving element PP. It can be known from the characteristics of the filter of the light receiving element PP how much the distance measurement light passes through the filter of the light receiving element PP. For example, when the distance measuring light is incident on the filter of the light receiving element PP, the intensity thereof is defined to be k2 (0 <k2 <k1) times. In this case, the amount of distance measuring light received by the light receiving element PP is calculated by a value (Pkk · k2 / k1) obtained by multiplying the amount (Pkk / k1) incident on the filter of the light receiving element PP by the value k2. Can do.

  As described above, the amount of distance measurement light received by the light receiving element PP can be estimated based on the amount of light received by the light receiving element Pk.

  Therefore, the correction unit 25 reduces the distance measurement light included in the light reception amount of the light receiving element PP by subtracting the value obtained by multiplying the light reception amount of the light receiving element Pk by a predetermined coefficient from the light reception amount of the light receiving element PP. It is. If (k2 / k1) is employed as the predetermined coefficient, theoretically, the distance measurement light included in the light receiving amount of the light receiving element PP can be reduced most.

  In the above correction, it is assumed that the same amount of distance measuring light is incident on the filters of the light receiving elements Pk and PP. Therefore, it is desirable that the amount of light received by the light receiving element Pk used for the correction is the amount of light received by the light receiving element Pk disposed at the position closest to the light receiving element PP to be corrected. For example, in FIG. 23, the received light amount of the light receiving element PR belonging to one pixel PX1 is corrected based on the received light amount of the light receiving element Pk belonging to the one pixel PX1. Thereby, the distance measurement light included in the amount of light received by the light receiving element PP can be accurately reduced.

  Note that the light receiving element Pk to be used for the light receiving element PP can be selected by the correction unit 25 as follows, for example. That is, for example, the light receiving element Pk used corresponding to the light receiving element PP is recorded in the recording unit 23 in advance, and the correction unit 25 refers to the recording unit 23 so that the light receiving element PP is corrected using the light receiving element Pk. I do.

  Further, when the acquisition of the image data of the distance to be measured and the acquisition of the image data of the natural image are not performed at the same time, it is not necessary to perform the irradiation by the irradiation device 3 when capturing the natural image. Therefore, the control arithmetic device 2 may output the clock signal CLK to the light receiving device 1 without outputting the irradiation light L1 to the irradiation device 3, and may capture a natural image. At this time, the light receiving element Pk receives distance measuring light existing in the natural world, not distance measuring light using the irradiation device 3 as a light source. Even in such a case, image data closer to the actual color can be obtained by performing the correction as described above.

  Further, it may be understood that the natural distance measuring light is incident almost uniformly on all the light receiving elements Pk, and the correction may be made as follows. That is, the correction unit 25 may correct the light reception amounts of all the light receiving elements PP based on the light reception amount of one light receiving element Pk. Alternatively, the received light amounts of all the light receiving elements PP may be corrected based on the average of the received light amounts of the plurality of light receiving elements Pk.

  Further, all of the amounts of light received by the light receiving elements PR, PG, and PB may be corrected. Alternatively, when infrared light (for example, near infrared light) is employed as the irradiation light L1, light having a wavelength closest to the wavelength of infrared light (here, red light) is received from the light receiving element PP. You may correct | amend only with respect to the light reception amount of the light receiving element PR. Usually, since the amount of light passing through the filter decreases as it moves away from the passband, only the light receiving element PR that is most likely to receive infrared light is the correction target. Thereby, it is possible to obtain image data closer to the actual color while reducing the arithmetic processing.

  Note that such a restriction on the correction target is achieved by, for example, previously recording information on the necessity of correction corresponding to each light receiving element PP in the recording unit 23 (or a recording unit (not shown) provided in the light receiving device 1). 25 can be realized by referring to the information and performing correction.

Fifth embodiment.
In FIG. 1, the amount of light received input to the AD conversion unit 17 is the amount of light received by the light receiving element Pk (or further, the light receiving element PP). However, the amount of light received by the light receiving element Pk includes not only the reflected light L2 using the irradiation device 3 as a light source but also a small amount of other light (hereinafter referred to as ambient light). Such ambient light is recognized as noise. Therefore, the SN ratio (to noise signal ratio) of the amount of received light input to the AD conversion unit 17 is reduced due to this. Therefore, the fifth embodiment aims to increase the SN ratio of the received light amount input to the AD conversion unit 17.

  The distance image sensor of FIG. 32 further includes a calibration unit 18 as compared with the distance image sensor of FIG. The calibration unit 18 is arranged closer to the column decoder 14 than the AD conversion unit 17 and acquires the amount of light received by the light receiving element Pk. In the example of FIG. 32, the calibration unit 18 is provided between the correlated double sample unit 15 and the AD conversion unit 17.

  The calibration unit 18 acquires the amount of received light from each of the light receiving elements Pk in a state where the irradiation device 3 does not irradiate the irradiation light L1, and records this as ambient light. More specifically, for example, the control arithmetic device 2 performs the following operation prior to acquisition of the frame data FD1 to FD4 used for distance measurement. That is, the control arithmetic device 2 notifies the calibration unit 18 that the operation is to acquire ambient light while transmitting the clock signal CLK to the light receiving device 1 without irradiating the irradiation device 3 with the irradiation light L1. Receiving the notification, the calibration unit 18 records the received light amount for each light receiving element Pk input to itself as ambient light, and acquires frame data in which the ambient light is recorded. Such ambient light is recorded in, for example, a recording unit (not shown) of the light receiving device 1.

  Next, similarly to the first or second embodiment, the control arithmetic device 2 outputs the clock signal CLK to the light receiving device 1 while irradiating the irradiation device 3 with the irradiation light L1. Furthermore, the control arithmetic unit 2 notifies the calibration unit 18 that the operation is to acquire the received light amount for distance measurement.

  The calibration unit 18 subtracts the ambient light of the light receiving element Pk from the light receiving amount of the light receiving element Pk acquired in the state irradiated with the irradiation light L1, and the received light amount after subtraction is the light receiving amount of the light receiving element Pk. Is output to the AD converter 17. The AD conversion unit 17 converts the amount of received light input from the calibration unit 18 from analog data to digital data.

  As described above, the calibration unit 18 reduces the ambient light from the amount of light received by the light receiving element Pk. Therefore, the SN ratio of the received light amount input to the AD conversion unit 17 can be improved. Thereby, the accuracy of analog / digital conversion can be improved.

Modified example.
In the illustration of FIG. 33, the distance image sensor further includes a reflector 4 whose reflectance is known.

  The reflector 4 is provided in a fixed positional relationship with respect to the light receiving element group 11 and is provided at a position where it can be imaged by the light receiving element group 11. For example, in the illustration of FIGS. 33 and 34, the distance image sensor includes a housing 5 to which the light receiving device 1 and the irradiation device 3 are attached. The housing 5 includes, for example, a main body 51 on which the light receiving device 1 is provided, and an extending portion 52 extending from the main body 51, and the reflector 4 is provided on the extending portion 52. Which light receiving element Pk receives the light of the reflector 4 is determined in advance by the fixed relative position between the reflector 4 and the light receiving element group 11.

  Further, the reflectance of the reflector 4 and the light receiving element Pk that receives the light of the reflector 4 are recorded in a recording unit (not shown) provided in the light receiving device 1 in advance. The calibration unit 18 calculates ambient light using the reflected light L2 from the reflector 4 in a state where the irradiation device 3 has irradiated the irradiation light L1. That is, the calibration unit 18 detects the amount of light incident on the reflector 4 from the amount of light received by the light receiving element that receives the reflected light L <b> 2 from the reflector 4 and the reflectance of the reflector 4. This amount of light can be calculated by dividing the reflectance from the amount of received light. Then, the ambient light is calculated by subtracting the light amount of the irradiation light L1 from the light amount.

  The calibration unit 18 subtracts the ambient light from the amount of light received by each light receiving element Pk and outputs the result to the AD conversion unit 17.

Sixth embodiment.
In the sixth embodiment, as shown in FIG. 35, each of the light receiving elements Pk includes a photodetector 11a, a plurality of capacitors 11c, and a switch 11d. The photodetector 11a passes a current according to the intensity of the received light. In FIG. 35, the photodetector 11a is equivalently shown as a current source.

  The plurality of capacitors 11c are connected to the photodetector 11a via the switch 11d. The current from the photodetector 11a flows into the capacitor 11c through the switch 11d. As a result, the capacitor 11c is charged with a voltage corresponding to the amount of received light. Therefore, the received light amount acquisition unit 10 acquires the voltage charged in the capacitor 11c as the received light amount.

  The switch 11d changes the connection relationship between the photodetector 11a and the plurality of capacitors 11c. Thereby, the synthetic | combination electrostatic capacitance of the capacitor | condenser 11c connected to the photodetector 11a can be varied.

  In the illustration of FIG. 35, four capacitors are illustrated as the plurality of capacitors 11c, and a switch 11d is provided corresponding to each of the four capacitors 11c. More specifically, the capacitor 11c and the switch 11d corresponding to each other are connected in series, and four of the series connection bodies are connected in parallel to each other. And this parallel connection body is connected in series with the photodetector 11a.

  Thus, for example, when all four switches 11d are turned on, the combined capacitance of the capacitor 11c connected to the photodetector 11a is equal to the combined capacitance of the capacitor 11c connected to the photodetector 11a when the three switches 11d are turned on. It differs from capacity.

  The combined electrostatic capacitance of the capacitor 11c that is electrically connected to the photodetector 11a affects the sensitivity of the light receiving element Pk. For example, if the synthetic capacitance is small, the voltage charged in the capacitor 11c with a little current increases. Therefore, even if the light received by the photodetector 11a is small, it can be detected as a relatively large voltage value. However, since the upper limit value of the voltage of the capacitor 11c is determined in advance, the range of the amount of received light that can be detected becomes small. That is, the dynamic range is reduced.

  According to the above-described structure, the sensitivity and dynamic range of the light receiving element Pk can be adjusted by controlling the switch 11d. Therefore, for example, an appropriate capacitor 11c can be selected according to the intensity of the ambient light or the irradiation light L1 output from the irradiation device 3.

  Further, the light receiving element PP may have a similar structure. In this case, the optimum capacitor 11c can be selected in each of the light receiving element PP and the light receiving element Pk, that is, in each acquisition of natural image and distance image data.

  The capacitances of the capacitor 11c may be different from each other. As a result, the type of combined capacitance of the capacitor 11c that is electrically connected to the photodetector 11a can be increased. For example, if the capacitances of the capacitors 11c are equal to each other, the composite capacitance takes the same value no matter which three switches are turned on. On the other hand, if the capacitances of the capacitors 11c are different from each other, the combined capacitances take different values if the three switches to be turned on are different. According to this, the sensitivity and the dynamic range can be finely adjusted.

  Hereinafter, an example of the light receiving device 1 employed in the light receiving element group 11 will be described in detail. In the illustration of FIG. 36, the light receiving device 1 further includes a switch control unit 19 that controls the switch 11d as compared with the light receiving device 1 of FIG. The switch control unit 19 controls the switch 11d so that the combined capacitance of the capacitor 11c that is electrically connected to the photodetector 11a in the light receiving element Pk increases as the light amount (DC component) of the irradiation light L1 increases. This can be realized, for example, as follows. That is, the relationship between the light amount of the irradiation light L1 and the switch 11d to be conducted is recorded in the recording unit, the control arithmetic device 2 notifies the switch control unit 19 of the light amount of the irradiation light L1, and the switch control unit 19 The switch 11d may be controlled based on the amount of light and the relationship.

  Thereby, when the light quantity of the irradiation light L1 is small, the sensitivity of the light receiving element Pk can be improved. Therefore, the amount of received light can be appropriately acquired even when the amount of the irradiation light L1 is small.

  If both the reflector 4 and the camera light receiving element PP are provided, the amount of ambient light can be calculated based on the amount of light received by the camera light receiving element PP that receives light from the reflector 4. For example, luminance data is calculated from the amount of light received by the light receiving element PP, and the amount of ambient light is calculated by dividing the reflectance of the reflector 4. In this case, the switch 11d may be controlled so that the combined capacitance of the capacitor 11c connected to the photodetector 11a in the light receiving element PP decreases as the amount of ambient light decreases. Thereby, when ambient light is small, the sensitivity of the light receiving element PP can be improved, and the amount of received light can be acquired appropriately.

Seventh embodiment.
In the first embodiment, the modulation period T has been described as being twice the period of the clock signal CLK. However, the modulation period T is not limited to twice, and may be an even number multiple of the period of the clock signal CLK.

  For example, the case where the modulation period T is four times the period of the clock signal CLK as shown in FIG. 37 will be described. In this case, periods T0 and T2, which are half periods of the modulation period T, are twice the period of the clock signal CLK.

  The timing for acquiring the amount of light received by the light receiving element Pk is the same as in the description of the first embodiment, and the frame data FD1 to FD4 are acquired and written in the recording unit 23. However, here, two cycles of the clock signal CLK correspond to a half cycle of the modulation cycle T. Therefore, the amount of received light in the frame data FD1 to FD4 of the light receiving element Pk does not correspond to the amount of received light in the periods T0 to T4. Therefore, as will be described in detail later, a plurality of light receiving elements are regarded as one light receiving element (hereinafter referred to as a virtual light receiving element), and the amount of light received by the virtual light receiving element is regarded as the amount of light received in the periods T0 to T4.

  The acquisition timing of the amount of light received by the light receiving element Pk will be specifically described with reference to FIG. Since the acquisition timing is the same as in the first embodiment, for example, triggered by each rising edge of the clock signal CLK, the received light amounts of the light receiving elements Pk are sequentially acquired in a predetermined order to obtain the frame data FD1. get.

  However, here, two cycles of the clock signal CLK coincide with a half cycle of the modulation cycle T. Therefore, in the illustration of FIG. 38, the light receiving amount of the light receiving element P1 matches the light receiving amount in the first half of the period T0, and the light receiving amount of the light receiving element P2 matches the light receiving amount in the latter half of the period T0. Similarly, the amount of light received by the light receiving elements P3 and P4 is equal to the amount of light received in the first half and the second half of the period T2, respectively. The same applies to the other light receiving elements Pk.

  Therefore, a pair of the light receiving element P2m-1 (m is a natural number, the same applies hereinafter) and the light receiving element P2m is regarded as one light receiving element (hereinafter referred to as a virtual light receiving element) IPm, and the amount of light received by the virtual light receiving element IPm is defined as Calculated as the sum of the amounts of light received by the light receiving elements P2m-1, P2m. Thereby, for example, the received light amount of the virtual light receiving element IP1 can be regarded as the received light amount A (T0) in the period T0.

  In the frame data FD2, as shown in FIG. 39, the received light amount of the virtual light receiving element IP1 corresponds to the received light amount A (T2). Similarly, in the frame data FD3 and FD4, the received light amount of the virtual light receiving element IP1 corresponds to A (T3) and A (T1), respectively.

  Then, the distance calculation unit 24 calculates the phase difference φ using the equation (1) based on the amount of received light in the frame data FD1 to FD4 of the virtual light receiving element IPm, and calculates the distance based on the phase difference φ.

  Thereby, the distance is calculated for each virtual light receiving element IPm. Therefore, the resolution is reduced as compared with the first embodiment. Thereby, distance data can be averaged.

  Moreover, according to the seventh embodiment, the cycle of the clock signal CLK can be shortened without shortening the modulation cycle T. Therefore, the clock signal CLK can be shortened without being limited by the performance of the irradiation device 3 (that is, the lower limit of the modulation period T). As described in the first embodiment, the time difference between the received light amounts A (T0), A (T2), A (T3), and A (T1) of the light receiving element Pk is the number of light receiving elements Pk. And the period of the clock signal CLK, the period of the clock signal CLK can be shortened, so that the time difference can be shortened.

  When the modulation period T is 2N (N is a natural number) times the clock signal CLK, the sum of the received light amounts of the N light receiving elements Pk adjacent to each other in the frame data FD1 is the received light amount A (T0), A (T2). Of the N light receiving elements Pk adjacent to each other in the frame data FD2 corresponds to the other of the received light amounts A (T0) and A (T2), and N adjacent to each other in the frame data FD3. The sum of the light receiving amounts of the light receiving elements Pk corresponds to one of the light receiving amounts A (T3) and A (T1), and the sum of the light receiving amounts of the N light receiving elements Pk adjacent to each other in the frame data FD4 is the light receiving amount A. It corresponds to the other of (T3) and A (T1). Therefore, these N light receiving elements Pk can be regarded as one virtual light receiving element IP.

  The first to seventh embodiments may be appropriately combined.

  In the above-described example, a plurality of frame data are acquired by repeatedly performing the step of acquiring the frame data by sequentially acquiring the amount of light received by the light receiving element Pk. Here, the operation of repeating the step is not performed. Instead of this, the step of acquiring the received light amount of a plurality of continuous light receiving periods for one light receiving element Pk is sequentially repeated for each light receiving element Pk. Hereinafter, the calculation method using Formula (3) and Formula (4) will be described in detail as an example.

  The irradiation light L1 is modulated in a rectangular wave shape and periodically as shown in FIG. For example, as shown in FIG. 18, the clock signal CLK rises at the beginning of periods T0 'and T1'.

  The received light amount acquisition unit 10 acquires the received light amount of the light receiving element Pk, for example, every time the clock signal CLK rises. However, the received light amount of the same light receiving element Pk is acquired at both the rising edge of the clock signal CLK and the subsequent rising edge.

  For example, as shown in FIG. 40, the amount of light received by the light receiving element P1 is acquired at the first rising UE1 ', and the amount of light received by the light receiving element P1 is acquired at the second rising UE2'. That is, by continuously acquiring the received light amount for the light receiving element P1, the received light amounts A (T0 ') and A (T1') for the light receiving element P1 are continuously acquired. Next, the light reception amounts A (T0 ′) and A (T1 ′) of the light receiving element P2 are acquired at the third rising UE3 ′ and the fourth rising UE4 ′ of the clock signal CLK, respectively. Similarly, the received light amounts A (T0 ′) and A (T1 ′) of the light receiving element Pk are sequentially obtained.

  This also makes it possible to obtain an amount of received light that is suitable for the distance calculation of Expression (3) and Expression (4) for each light receiving element Pk.

  For example, the following procedure is conceivable in order to continuously obtain the received light amounts A (T0 ′) and A (T1 ′) for the light receiving element Pk. That is, first, at the end of the period T0 ', the amount of received light (for example, the voltage value of the capacitor, and so on) is read to obtain the amount of received light A (T0'). Then, the received light amount is initialized (the capacitor is discharged). This initialization requires a predetermined time (for example, capacitor discharge time). Then, the accumulation of the received light amount (charging of the capacitor) is started again, and the received light amount is read at the end of the period T1 'to obtain the received light amount A (T10').

  However, according to this procedure, the time required for initialization occurs as a time lag, and an error may occur in the amount of received light A (T1 ').

  Therefore, the received light amount A (T0 ') in the period T0' and the received light amount A (T10 ') in the entire periods T0' and T1 'may be acquired. For example, the amount of received light is read at the end of the period T0 '. Thereby, the received light amount A (T0 ') in the period T0' is acquired. Then, the received light amount is continuously accumulated without initializing (resetting) the received light amount. Then, the amount of received light is read at the end of the period T1 '. Thereby, the received light amount A (T10 ') in the entire period T0', T1 'is acquired. The received light amount A (T1 ') can be calculated by subtracting the received light amount A (T0') from the received light amount A (T10 '). According to this, an error caused by initialization can be reduced, and distance measurement can be performed with higher accuracy.

  Although the present invention has been described in detail, the above description is illustrative in all aspects, and the present invention is not limited thereto. It is understood that countless variations that are not illustrated can be envisaged without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Light receiving device 3 Irradiation device 10 Received light quantity acquisition part 17 AD conversion part 18 Calibration part 21 Clock generation part 25 Correction part 26 Motion compensation part Pk Light receiving element PP Camera light receiving element

The first aspect of the distance image sensor according to the present invention is configured to receive the reflected light from the measurement target and the irradiation unit that irradiates the measurement target with the irradiation light while modulating the intensity of the irradiation light at a certain modulation period. A plurality of light receiving elements and each of the light receiving elements in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light. A clock generation unit that generates a changing clock signal, and a step of acquiring the frame data by sequentially acquiring the received light amount of the light receiving element based on the clock signal by a rolling shutter method, a plurality of frame data is obtained by repeating a plurality of times. By acquiring, the plurality of received light amounts at the same light receiving element in the plurality of frame data correspond to the received light amounts in the plurality of light receiving periods, respectively. And received light amount acquisition unit, wherein the plurality of by calculating the phase difference based on the frame data, a distance calculation unit for calculating a distance from the irradiation portion to the measurement target, the modulation period of the clock signal 2n (n is a natural number) times the cycle, the plurality of frame data is first to fourth frame data, and the received light amount acquisition unit performs the first to fourth steps in a predetermined order. In the first step, the received light amount is sequentially acquired at every (2r−1 + k · n) (r is a natural number, k = 0, 1, 2,...) Th rise of the clock signal. 1 frame data is acquired, and in the second step, the received light amount is sequentially acquired at every (2s + k · n) (s is a natural number) rise of the clock signal to acquire the second frame data. , The third step Obtains the third frame data by sequentially obtaining the received light amount at every (2t-1 + k · n) (t is a natural number) falling edge of the clock signal. In the fourth step, The fourth frame data is acquired by sequentially acquiring the received light amount at each (2u + k · n) (u is a natural number) falling edge of the signal .

A second aspect of the distance image sensor according to the present invention is the distance image sensor according to the first aspect, wherein the received light amount acquisition unit determines the received light amount of the light receiving element at each rising edge of the clock signal. Obtaining the first and second frame data sequentially in a predetermined order, obtaining the received light amount of the light receiving element sequentially in the order at each falling edge of the clock signal, The third and fourth frame data are acquired .

The third aspect of the distance image sensor according to the present invention is configured to receive the reflected light from the measurement target and the irradiation unit that irradiates the measurement target with the irradiation light while modulating the intensity of the irradiation light at a certain modulation period. A plurality of light receiving elements and each of the light receiving elements in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light. A clock generation unit that generates a changing clock signal, and a step of acquiring the frame data by sequentially acquiring the received light amount of the light receiving element based on the clock signal by a rolling shutter method, a plurality of frame data is obtained by repeating a plurality of times. By acquiring, the plurality of received light amounts at the same light receiving element in the plurality of frame data correspond to the received light amounts in the plurality of light receiving periods, respectively. A light receiving amount obtaining section calculates the phase difference based on the plurality of frame data, a distance calculation unit for calculating a distance from the irradiation portion to the measurement object
Each of the received light amounts of the light receiving element in one frame data of the plurality of frame data corresponds to a received light amount in one light receiving period of the plurality of light receiving periods, and the modulation period is The period of the clock signal is 2n (n is a natural number) times, the plurality of frame data is first to fourth frame data, and the received light amount acquisition unit performs the first to fourth steps in a predetermined manner In the first step, each time the (2r-1 + 2 · k · n) (r is a natural number, k = 0, 1, 2,...) Th rise of the clock signal, The received light amount is sequentially acquired to acquire the first frame data, and in the second step, the received light amount at every (2s + 2 · k · n) (s is a natural number) rise of the clock signal. In order The second frame data is acquired, and in the third step, the received light amount is sequentially acquired at every (2t-1 + 2 · k · n) (t is a natural number) fall of the clock signal. The third frame data is acquired, and in the fourth step, the received light amount is sequentially acquired at every (2u + 2 · k · n) (u is a natural number) falling edge of the clock signal. Get frame data .

A fourth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to third aspects, wherein the plurality of light receiving periods are four light receiving periods. The start of each light receiving period is shifted by a quarter of the modulation period .

A fifth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to fourth aspects, wherein the modulation period is twice the period of the clock signal, The distance calculation unit includes the received light amounts A (T0), A (T2), A (T3), A (T1) in the first to fourth frame data of the light receiving element, the irradiation light, and the Based on the relational expression with respect to the phase difference φ with respect to the reflected light, φ = tan ⁻¹ [{A (T0) −A (T2)} / {A (T3) −A (T1)}], from the irradiation unit The distance to the measurement object is calculated .

A sixth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to fourth aspects, wherein the modulation period is 2N (N is 2) of the period of the clock signal. The distance calculation unit calculates the sum A (T0) of the received light amounts in the first frame data of the N light receiving elements adjacent to each other and the N light receiving elements. The received light amount sum A (T2) in the second frame data, the received light amount sum A (T3) in the third frame data of the N light receiving elements, and the N received light amounts. The relational expression between the sum A (T1) of the received light amount in the fourth frame data of the element and the phase difference φ between the irradiation light and the reflected light, φ = tan ⁻¹ [{A (T0) −A ( T2)} / {A (T3) -A (T1)}] Zui and calculates the distance from the irradiation portion to the measurement target.

According to a seventh aspect of the distance image sensor of the present invention, the irradiation unit that irradiates the measurement target with the irradiation light while modulating the intensity of the irradiation light with a certain modulation period, and the reflected light from the measurement target are received. A plurality of light receiving elements and each of the light receiving elements in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light. A clock generation unit that generates a changing clock signal, and a step of acquiring the frame data by sequentially acquiring the received light amount of the light receiving element based on the clock signal by a rolling shutter method, a plurality of frame data is obtained by repeating a plurality of times. By acquiring, the plurality of received light amounts at the same light receiving element in the plurality of frame data correspond to the received light amounts in the plurality of light receiving periods, respectively. A light receiving amount obtaining section calculates the phase difference based on the plurality of frame data, a distance calculation unit for calculating a distance from the irradiation portion to the measurement object
The irradiation unit modulates the intensity of the irradiation light in a pulse shape, and the plurality of light receiving periods are a first period from a time point when the intensity of the irradiation light rises to a time point when the intensity falls, and the irradiation light Is a second period from the time when the intensity of the light falls to the time when it rises, the plurality of frame data is first and second frame data, and the received light amount acquisition unit is configured to receive the light reception based on the clock signal. The light reception amount of the light receiving element in the first period is sequentially acquired to acquire the first frame data, and the light reception light amount of the light receiving element in the second period is sequentially acquired based on the clock signal. To obtain the second frame data .

An eighth aspect of the distance image sensor according to the present invention is the distance image sensor according to the third or seventh aspect, wherein a motion vector detection unit that detects a motion vector for the measurement target, and the motion vector And a motion compensation unit that associates the light receiving elements with each other in the plurality of frame data.

A ninth aspect of the distance image sensor according to the present invention is the distance image sensor according to the eighth aspect, wherein the irradiation unit outputs second irradiation light of a DC component, and the motion vector detection unit is The motion vector is detected by using at least two motion vector frame data obtained by sequentially acquiring the amount of light received by the light receiving element in the state in which the second irradiation light is output.

A tenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the eighth aspect, wherein the distance image sensor is provided between the light receiving elements and receives a light corresponding to a color. The light receiving element further includes a light receiving element, the light receiving amount acquiring unit sequentially acquires the light receiving amount of the light receiving element and the light receiving amount of the camera light receiving element, acquires the plurality of frame data, and the motion vector detecting unit Detects the motion vector based on the amount of light received by the camera light receiving element in the plurality of frame data.

An eleventh aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to tenth aspects, wherein the distance image sensor is provided between the light receiving elements and corresponds to the color. And a correction unit that performs correction to subtract a value obtained by multiplying the received light amount of the light receiving element by a coefficient from the received light amount of the camera light receiving element.

A twelfth aspect of the distance image sensor according to the present invention is the distance image sensor according to the eleventh aspect, wherein the light receiving element used for the correction is the correction target among the light receiving elements. It is provided at a position closest to the camera light receiving element.

A thirteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the eleventh or twelfth aspect, wherein the irradiation light is infrared light, and each of the camera light receiving elements is red, The light corresponding to either blue or green is received, and the correction unit performs the correction only on the amount of light received by the camera light receiving element corresponding to the red.

A fourteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to thirteenth aspects, wherein the irradiation unit is acquired in a state where the irradiation light is not irradiated. A calibration unit that outputs the amount of light received by the light receiving element as ambient light, subtracts the ambient light from the amount of light received by the light receiving element, and outputs the amount of light received from the calibration unit from analog data to digital data. And a conversion unit for converting.

A fifteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to fourteenth aspects, wherein each of the light receiving elements has a current corresponding to the intensity of received light. And a plurality of capacitors through which the current flows, and a switch that changes a connection relationship between the photodetector and the plurality of capacitors, and the received light amount acquisition unit is connected to each of the light receiving elements. The voltage charged in the capacitor is acquired as the amount of received light.

A sixteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the fifteenth aspect, wherein the switch is controlled so that the smaller the irradiation light is, the more the condenser detects the light detector. It further includes a switch control unit that reduces the combined capacitance of the connected capacitors.

According to the first and fourth aspects of the distance image sensor according to the present invention, even in the method of acquiring the frame data by sequentially acquiring the received light amount based on the clock signal (for example, the global shutter method), the distance An image can be obtained.

Moreover, the case where the modulation period is twice the clock period will be described first. At this time, the clock cycle starting from the even-numbered rise corresponds to, for example, the first half of the modulation cycle (period T0), and the clock cycle starting from the odd-numbered rise is the half of the second half of the modulation cycle. This corresponds to (period T2).

According to the second aspect of the distance image sensor according to the present invention, the amount of received light is obtained every time the clock signal rises. For example, compared to the case where the amount of received light is obtained by skipping one rise of the clock signal, The first and second frame data can be acquired in a short period. Similarly, the third and fourth frame data can be acquired in a relatively short period.

According to the 3rd aspect of the distance image sensor concerning this invention, all of the light reception amount of the light receiving element in one frame data are equivalent to the light reception amount in one light reception period. Therefore, it is possible to correspond to the amount of received light in a plurality of light receiving periods for each frame data. Therefore, a received light amount suitable for motion compensation can be obtained.
According to the fifth aspect of the distance image sensor of the present invention, the distance can be measured.
According to the sixth aspect of the distance image sensor according to the present invention, since the cycle of the clock signal can be shortened regardless of the performance of the irradiation apparatus, the period for acquiring the first to fourth frame data can be shortened. it can. In addition, the sum of the amounts of light received by the N light receiving elements is used. Thereby, N light receiving elements can be grasped as one virtual light receiving element, and a distance can be calculated for each virtual light receiving element.

According to the seventh aspect of the distance image sensor of the present invention, for example, all the light receiving amounts of the light receiving elements in the first frame data correspond to the light receiving amount A (T0), and the light receiving elements in the second frame data All the received light amounts correspond to the received light amount A (T2), all the received light amounts of the light receiving elements in the third frame data correspond to the received light amount A (T3), and all the received light elements in the fourth frame data. The amount of received light corresponds to the amount of received light A (T1).

According to the eighth aspect of the distance image sensor of the present invention, motion compensation can be performed.

According to the ninth aspect of the distance image sensor of the present invention, since the motion vector is detected using the second irradiation light of the DC component, it is compared with the case where the motion vector is detected using the irradiation light of the modulation component. Detection accuracy is high.

According to the tenth aspect of the distance image sensor of the present invention, since the received light amount corresponding to the color exists in the plurality of frame data, it contributes to creation of image data of a natural image. In addition, it is not necessary to acquire motion vector frame data used only for detecting a motion vector. Therefore, the required data amount can be reduced as compared with the case of further acquiring motion vector frame data.

According to the eleventh aspect of the distance image sensor of the present invention, the manufacturing cost can be reduced as compared with the case where a filter that removes light having the same wavelength as the irradiation light is provided in the camera light receiving element.

According to the twelfth aspect of the distance image sensor of the present invention, the correction accuracy can be improved.

According to the thirteenth aspect of the distance image sensor according to the present invention, only the light receiving element for a camera corresponding to red that is most likely to receive light having the same wavelength as the irradiated light is corrected. Accordingly, it is possible to obtain image data closer to the actual color as compared with the case where only the other light receiving elements for cameras are corrected, while reducing the arithmetic processing.

According to the fourteenth aspect of the distance image sensor of the present invention, the S / N ratio of the received light amount input to the conversion unit can be improved.

According to the fifteenth aspect of the distance image sensor of the present invention, the combined capacitance of the capacitor connected to the photodetector can be changed. Therefore, the sensitivity in the amount of received light can be adjusted.

According to the sixteenth aspect of the distance image sensor of the present invention, the sensitivity of the light receiving element can be improved when the amount of irradiation light is small. Therefore, the amount of received light can be appropriately acquired even if the amount of irradiation light is small.

The first aspect of the distance image sensor according to the present invention is configured to receive the reflected light from the measurement target and the irradiation unit that irradiates the measurement target with the irradiation light while modulating the intensity of the irradiation light at a certain modulation period. A plurality of light receiving elements and each of the light receiving elements in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light. A clock generation unit that generates a changing clock signal, and a step of acquiring the frame data by sequentially acquiring the received light amount of the light receiving element based on the clock signal by a rolling shutter method, a plurality of frame data is obtained by repeating a plurality of times. By acquiring, the plurality of received light amounts at the same light receiving element in the plurality of frame data correspond to the received light amounts in the plurality of light receiving periods, respectively. A received light amount acquisition unit; and a distance calculation unit that calculates the phase difference based on the plurality of frame data and calculates a distance from the irradiation unit to the measurement target; and one of the plurality of frame data Each of the light receiving amounts of the light receiving elements in the frame data corresponds to a light receiving amount in one light receiving period among the plurality of light receiving periods, and the modulation period is 2n (n is a natural number) of the period of the clock signal. The plurality of frame data is first to fourth frame data, and the received light amount acquisition unit executes the first to fourth steps in a predetermined execution order. In the first step, , Sequentially obtaining the received light amount of the light receiving element at every rising edge of the clock signal (2r-1 + 2 · k · n) (r is a natural number, k = 0, 1, 2,...) First In the second step, the received light amount is sequentially acquired at every (2s + 2 · k · n) (s is a natural number) rise of the clock signal to obtain the second frame data, In the third step, the received light amount is sequentially acquired at every (2t-1 + 2 · k · n) (t is a natural number) falling edge of the clock signal, and the third frame data is acquired. In the fourth step, the received light amount is sequentially obtained at every (2u + 2 · k · n) (u is a natural number) falling edge of the clock signal to obtain the fourth frame data.

A second aspect of the distance image sensor according to the present invention is the distance image sensor according to the first aspect, wherein the plurality of light reception periods are four light reception periods, and the start periods of the four light reception periods are respectively It shifts by a quarter of the modulation period.

A third aspect of the distance image sensor according to the present invention is the distance image sensor according to the first or second aspect, wherein the modulation period is twice the period of the clock signal, and the distance calculator The received light amounts A (T0), A (T2), A (T3), A (T1) in the first to fourth frame data of the light receiving element, and the positions of the irradiation light and the reflected light Based on the relational expression with the phase difference φ, φ = tan ⁻¹ [{A (T0) −A (T2)} / {A (T3) −A (T1)}], from the irradiation unit to the measurement target Calculate the distance.

A fourth aspect of the distance image sensor according to the present invention is the distance image sensor according to the first or second aspect, wherein the modulation period is 2N (N is an integer of 2 or more) times the period of the clock signal. The distance calculation unit includes the sum A (T0) of the received light amounts in the first frame data of the N light receiving elements adjacent to each other and the second frame of the N light receiving elements. The sum A (T2) of the received light amounts in the data, the sum A (T3) of the received light amounts in the third frame data of the N light receiving elements, and the fourth of the N light receiving elements. The relational expression between the sum A (T1) of the received light amounts in the frame data and the phase difference φ between the irradiated light and the reflected light, φ = tan ⁻¹ [{A (T0) −A (T2)} / { A (T3) -A (T1)}] It calculates the distance from the irradiation portion to the measurement target.

According to a fifth aspect of the distance image sensor of the present invention, the irradiation unit that irradiates the measurement target with the irradiation light while modulating the intensity of the irradiation light with a certain modulation period, and the reflected light from the measurement target are received. A plurality of light receiving elements and each of the light receiving elements in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light. A clock generation unit that generates a changing clock signal, and a step of acquiring the frame data by sequentially acquiring the received light amount of the light receiving element based on the clock signal by a rolling shutter method, a plurality of frame data is obtained by repeating a plurality of times. By acquiring, the plurality of received light amounts at the same light receiving element in the plurality of frame data correspond to the received light amounts in the plurality of light receiving periods, respectively. A received light amount acquisition unit; and a distance calculation unit that calculates the phase difference based on the plurality of frame data and calculates a distance from the irradiation unit to the measurement target, the irradiation unit including the irradiation light The plurality of light receiving periods are a first period from the time when the intensity of the irradiated light rises to a time when it falls, and the time from when the intensity of the irradiated light falls to the time when it rises In the second period, the plurality of frame data are first and second frame data, and the received light amount acquisition unit calculates the received light amount of the light receiving element in the first period based on the clock signal. The first frame data is acquired sequentially, the received light amount of the light receiving element in the second period is sequentially acquired based on the clock signal, and the second frame data is acquired. To get the Mudeta.

A sixth aspect of the distance image sensor according to the present invention is the distance image sensor according to the first or fifth aspect, wherein a motion vector detection unit that detects a motion vector for the measurement target, and the motion vector And a motion compensation unit that associates the light receiving elements with each other in the plurality of frame data.

A seventh aspect of the distance image sensor according to the present invention is the distance image sensor according to the sixth aspect, wherein the irradiation unit outputs second irradiation light of a DC component, and the motion vector detection unit is The motion vector is detected by using at least two motion vector frame data obtained by sequentially acquiring the amount of light received by the light receiving element in the state in which the second irradiation light is output.

An eighth aspect of the distance image sensor according to the present invention is the distance image sensor according to the sixth aspect, wherein the distance image sensor is provided between the light receiving elements and receives a light corresponding to a color. The light receiving element further includes a light receiving element, the light receiving amount acquiring unit sequentially acquires the light receiving amount of the light receiving element and the light receiving amount of the camera light receiving element, acquires the plurality of frame data, and the motion vector detecting unit Detects the motion vector based on the amount of light received by the camera light receiving element in the plurality of frame data.

A ninth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to eighth aspects, wherein the distance image sensor is provided between the light receiving elements and corresponds to the color. And a correction unit that performs correction to subtract a value obtained by multiplying the received light amount of the light receiving element by a coefficient from the received light amount of the camera light receiving element.

A tenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the ninth aspect, wherein the light receiving element used for the correction is the correction target among the light receiving elements. It is provided at a position closest to the camera light receiving element.

An eleventh aspect of the distance image sensor according to the present invention is the distance image sensor according to the ninth or tenth aspect, wherein the irradiation light is infrared light, and each of the camera light receiving elements is red, The light corresponding to either blue or green is received, and the correction unit performs the correction only on the amount of light received by the camera light receiving element corresponding to the red.

A twelfth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to eleventh aspects, wherein the irradiation unit is acquired in a state where the irradiation light is not irradiated. A calibration unit that outputs the amount of light received by the light receiving element as ambient light, subtracts the ambient light from the amount of light received by the light receiving element, and outputs the amount of light received from the calibration unit from analog data to digital data. And a conversion unit for converting.

A thirteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to any one of the first to twelfth aspects, wherein each of the light receiving elements has a current corresponding to the intensity of the received light. And a plurality of capacitors through which the current flows, and a switch that changes a connection relationship between the photodetector and the plurality of capacitors, and the received light amount acquisition unit is connected to each of the light receiving elements. The voltage charged in the capacitor is acquired as the amount of received light.

A fourteenth aspect of the distance image sensor according to the present invention is the distance image sensor according to the thirteenth aspect, wherein the switch is controlled so that the smaller the irradiation light is, the more the condenser detects the photodetector. It further includes a switch control unit that reduces the combined capacitance of the connected capacitors.

According to the first and second aspects of the distance image sensor according to the present invention, even in the method of acquiring the frame data by sequentially acquiring the received light amount based on the clock signal (for example, the global shutter method), the distance An image can be obtained.

According to a first aspect of such a range image sensor in the present invention, it corresponds to the amount of light received in the light receiving period for any of one amount of light received by the light receiving element in one frame data. Therefore, it is possible to correspond to the amount of received light in a plurality of light receiving periods for each frame data. Therefore, a received light amount suitable for motion compensation can be obtained.
According to the third aspect of the distance image sensor of the present invention, the distance can be measured.
According to the 4th aspect of the distance image sensor concerning this invention, since the period of a clock signal can be shortened irrespective of the performance of an irradiation apparatus, the period which acquires the 1st-4th frame data can be shortened. it can. In addition, the sum of the amounts of light received by the N light receiving elements is used. Thereby, N light receiving elements can be grasped as one virtual light receiving element, and a distance can be calculated for each virtual light receiving element.

According to the fifth aspect of the distance image sensor of the present invention, for example, all the received light amounts of the light receiving elements in the first frame data correspond to the received light amount A (T0), and the light receiving elements in the second frame data All the received light amounts correspond to the received light amount A (T2), all the received light amounts of the light receiving elements in the third frame data correspond to the received light amount A (T3), and all the received light elements in the fourth frame data. The amount of received light corresponds to the amount of received light A (T1).

According to the sixth aspect of the distance image sensor of the present invention, motion compensation can be performed.

According to the seventh aspect of the distance image sensor of the present invention, since the motion vector is detected using the second irradiation light of the DC component, it is compared with the case where the motion vector is detected using the irradiation light of the modulation component. Detection accuracy is high.

According to the eighth aspect of the distance image sensor of the present invention, there is a received light amount corresponding to a color in a plurality of frame data, which contributes to the creation of natural image data. In addition, it is not necessary to acquire motion vector frame data used only for detecting a motion vector. Therefore, the required data amount can be reduced as compared with the case of further acquiring motion vector frame data.

According to the ninth aspect of the distance image sensor of the present invention, the manufacturing cost can be reduced as compared with the case where a filter for removing light having the same wavelength as the irradiation light is provided in the camera light receiving element.

According to the tenth aspect of the distance image sensor of the present invention, the correction accuracy can be improved.

According to the eleventh aspect of the distance image sensor of the present invention, only the light receiving element for a camera corresponding to red that is most likely to receive light having the same wavelength as the irradiated light is corrected. Accordingly, it is possible to obtain image data closer to the actual color as compared with the case where only the other light receiving elements for cameras are corrected, while reducing the arithmetic processing.

According to the twelfth aspect of the distance image sensor of the present invention, it is possible to improve the S / N ratio of the received light amount input to the conversion unit.

According to the thirteenth aspect of the distance image sensor of the present invention, the combined electrostatic capacitance of the capacitor connected to the photodetector can be changed. Therefore, the sensitivity in the amount of received light can be adjusted.

According to the fourteenth aspect of the distance image sensor of the present invention, the sensitivity of the light receiving element can be improved when the amount of irradiation light is small. Therefore, the amount of received light can be appropriately acquired even if the amount of irradiation light is small.

Claims (19)

An irradiation unit that irradiates the measurement object with the irradiation light while modulating the intensity of the irradiation light with a certain modulation period;
A plurality of light receiving elements for receiving reflected light from the measurement object;
Generates a clock signal that changes in synchronization with each of the start and end of a plurality of light receiving periods for obtaining a plurality of light receiving amounts used for calculating a phase difference between the irradiation light and the reflected light in each of the light receiving elements. A clock generator to
The step of obtaining the frame data by sequentially obtaining the received light amount of the light receiving element based on the clock signal is repeated a plurality of times to obtain a plurality of frame data, thereby obtaining the same light receiving element in the plurality of frame data. A plurality of received light amounts each of which corresponds to a received light amount in the plurality of received light periods,
A distance image sensor comprising: a distance calculation unit that calculates the phase difference based on the plurality of frame data and calculates a distance from the irradiation unit to the measurement target. The plurality of light receiving periods are four light receiving periods;
The range image sensor according to claim 1, wherein start times of the four light receiving periods are shifted by a quarter of the modulation period. The modulation period is an even multiple of the period of the clock signal;
The plurality of frame data are first to fourth frame data;
The amount of light received by each of the light receiving elements is:
In the first frame data, it is obtained in response to the rising of either the even or odd number of the clock signal,
In the second frame data is obtained triggered by the other rising edge of the clock signal,
In the third frame data, it is obtained in response to the fall of either the even or odd number of the clock signal,
3. The distance image sensor according to claim 1, wherein the fourth frame data is acquired in response to the other falling edge of the clock signal. The received light amount acquisition unit
The light receiving amount of the light receiving element is sequentially acquired in a predetermined order every time the clock signal rises, and the first and second frame data are acquired,
4. The distance image sensor according to claim 3, wherein the received light amount of the light receiving element is sequentially acquired in the order at each falling edge of the clock signal to acquire the third and fourth frame data.   3. The light receiving amount of the light receiving element in one frame data among the plurality of frame data corresponds to a light receiving amount in one light receiving period of the plurality of light receiving periods. Distance image sensor. The modulation period is an even multiple of the period of the clock signal;
The plurality of frame data are first to fourth frame data;
The received light amount acquisition unit
Triggered by each of the odd-numbered rising edges of the clock signal to sequentially obtain the amount of light received by the light receiving element, to obtain the first frame data,
Triggered by each of the even-numbered rising edges of the clock signal to sequentially obtain the amount of light received by the light receiving element, to obtain the second frame data,
Sequentially acquiring the amount of received light of the light receiving element triggered by each odd-numbered falling edge of the clock signal to obtain the third frame data;
6. The distance image sensor according to claim 5, wherein the fourth frame data is acquired by sequentially acquiring the received light amount of the light receiving element in response to each of the even-numbered falling edges of the clock signal. The irradiation unit modulates the intensity of the irradiation light into a pulse shape,
The plurality of light receiving periods are a first period from a time point when the intensity of the irradiation light rises to a time point when it falls, and a second period from a time point when the intensity of the irradiation light falls to a time point when it rises. The distance image sensor described in 1. The plurality of frame data are first and second frame data;
The received light amount acquisition unit
Based on the clock signal, sequentially obtain the amount of light received in the first period of the light receiving element to obtain the first frame data,
The distance image sensor according to claim 7, wherein the second frame data is acquired by sequentially acquiring the received light amount of the light receiving element in the second period based on the clock signal. A motion vector detection unit for detecting a motion vector for the measurement object;
The distance image sensor according to claim 5, further comprising a motion compensation unit that associates the light receiving elements with each other in the plurality of frame data based on the motion vector. The irradiation unit outputs a second irradiation light of a direct current component,
The motion vector detection unit detects the motion vector using at least two motion vector frame data obtained by sequentially acquiring the amount of light received by the light receiving element in a state where the second irradiation light is output. The range image sensor according to claim 9. A plurality of camera light-receiving elements that are provided between the light-receiving elements and receive light corresponding to colors;
The received light amount acquisition unit sequentially acquires the received light amount of the light receiving element and the received light amount of the camera light receiving element, and acquires the plurality of frame data,
The distance image sensor according to claim 9, wherein the motion vector detection unit detects the motion vector based on an amount of light received by the camera light receiving element in the plurality of frame data. A plurality of light receiving elements for a camera that are provided between the light receiving elements and receive light corresponding to a color;
The distance according to claim 1, further comprising: a correction unit that performs correction by subtracting a value obtained by multiplying the light reception amount of the light receiving element by a coefficient from the light reception amount of the camera light receiving element. Image sensor.   The distance image sensor according to claim 12, wherein the light receiving element used for the correction is provided at a position closest to the camera light receiving element to be corrected among the light receiving elements. The irradiation light is infrared light,
Each of the camera light receiving elements receives light corresponding to any of red, blue and green,
The distance image sensor according to claim 12 or 13, wherein the correction unit performs the correction only on the received light amount of the camera light receiving element corresponding to the red. A calibration unit for subtracting the ambient light from the received light amount of the light receiving element, and outputting the received light amount of the light receiving element obtained in a state where the irradiation unit does not irradiate the irradiation light; and
The distance image sensor according to claim 1, further comprising a conversion unit that converts the amount of received light from the calibration unit from analog data to digital data. Each of the light receiving elements is
A photodetector that passes a current according to the intensity of the received light;
A plurality of capacitors through which the current flows;
A switch for changing a connection relationship between the photodetector and the plurality of capacitors;
The distance image sensor according to any one of claims 1 to 15, wherein the received light amount acquisition unit acquires a voltage charged in the capacitor connected to each of the light receiving elements as the received light amount.   The distance image according to claim 16, further comprising a switch control unit that controls the switch to reduce a combined capacitance of a capacitor connected to the photodetector among the plurality of capacitors as the irradiation light is small. Sensor. The modulation period is twice the period of the clock signal;
The distance calculation unit
Phase difference between the received light amounts A (T0), A (T2), A (T3), and A (T1) in the first to fourth frame data of the light receiving element, and the irradiated light and the reflected light Relational expression with φ,
φ = tan ⁻¹ [{A (T0) −A (T2)} / {A (T3) −A (T1)}]
The distance image sensor according to any one of claims 3 to 6, wherein a distance from the irradiation unit to the measurement target is calculated on the basis of the distance. The modulation period is 2N (N is an integer of 2 or more) times the period of the clock signal,
The distance calculation unit
The sum A (T0) of the received light amounts in the first frame data of the N light receiving elements adjacent to each other, and the sum A (T) of the received light amounts in the second frame data of the N light receiving elements. T2), the sum A (T3) of the received light amounts in the third frame data of the N light receiving elements, and the sum of the received light amounts in the fourth frame data of the N light receiving elements. A relational expression between A (T1) and the phase difference φ between the irradiated light and the reflected light,
φ = tan ⁻¹ [{A (T0) −A (T2)} / {A (T3) −A (T1)}]
The distance image sensor according to any one of claims 3 to 6, wherein a distance from the irradiation unit to the measurement target is calculated on the basis of the distance.

Images