JP2017083243A - Distance sensor and system provided with the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distance sensor with which it is possible to suppress a reduction in the accuracy of distance measurement under a condition where background light is strong.SOLUTION: The distance sensor comprises a light source unit, a light receiving unit, and a control unit. The light source unit emits light to an object repeatedly. The light receiving unit receives light in synchronism with emission of light by the light source unit. The control unit controls the operation of the light source unit and light receiving unit, and generates distance information indicating the distance to the object on the basis of received luminous energy each time the received luminous energy (Q1, Q2, Q3) of received light is read from the light receiving unit. The control unit causes the cycle of each of emitted light by the light source unit and received light by the light receiving unit to change in accordance with the magnitude of the received luminous energy. Each time a received luminous energy read from the light receiving unit is repeated a prescribed number of times, the control unit generates distance information based on the received luminous energy for the prescribed number of times. The prescribed number of times corresponds to the cycle made to change by the control unit.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a distance sensor for measuring a distance to an object and a system including the distance sensor.

  As a distance sensor that measures a distance to an object, there is a distance sensor that uses a TOF (Time-Of-Flight) method that measures light based on a propagation period of reflected light by irradiating an object including the object. In a TOF type distance sensor, background light from the outside becomes an obstacle to distance measurement based on reflected light. Patent Document 1 discloses a distance sensor that considers removing the influence of background light in TOF distance measurement.

  The distance sensor of Patent Document 1 includes three charge storage nodes (capacitances) for each pixel circuit, and two of them have a charge distribution ratio that changes depending on the delay of reflected light with respect to light irradiated by pulse modulation. Next, the light reception timing is set. The remaining one is set to a timing at which the pulse-modulated light is always OFF, so that the received light amount of only the background light is accumulated. By using this, the influence of the background light is removed by subtracting the component due to the background light from the signal including the delay information of the reflected light.

Japanese Patent No. 4235729

  In Patent Document 1, after receiving the amounts of received light of the background light and the reflected light using the three capacitors of the pixel circuit, the component due to the background light is subtracted in the predetermined signal processing. However, when the amount of received light that is accumulated by strong background light such as sunlight increases, the effect of light shot noise, which is statistical noise proportional to the square root of the accumulated amount of received light, remains even if the component due to background light is subtracted. For this reason, the distance sensor of the prior art has a problem that the measurement accuracy of the distance is lowered under a strong background light.

  An object of the present invention is to provide a distance sensor and a system including the distance sensor that suppress a decrease in distance measurement accuracy under a situation where background light is strong in distance information indicating a distance to an object.

  The distance sensor which concerns on 1 aspect of this invention is provided with a light source part, a light-receiving part, and a control part. The light source unit repeatedly emits light to the object. The light receiving unit receives light in synchronization with light emission from the light source unit. The control unit controls the operations of the light source unit and the light receiving unit, and generates distance information indicating the distance to the object based on the received light amount every time the received light amount of the light received from the light receiving unit is read. The control unit changes each cycle of emission by the light source unit and light reception by the light receiving unit according to the magnitude of the received light amount. The control unit generates distance information based on the light reception amount for a predetermined number of times each time reading of the light reception amount from the light reception unit is repeated a predetermined number of times. The predetermined number of times is the number of times corresponding to the cycle changed by the control unit.

  A system according to one embodiment of the present invention includes a distance sensor and a processing device. The processing device acquires distance information from the distance sensor and executes predetermined processing. The control unit of the distance sensor changes each cycle of emission by the light source unit and light reception by the light receiving unit according to the magnitude of the received light amount. The processing apparatus includes an averaging processing unit that averages distance information for a predetermined number of times each time distance information is acquired from the distance sensor a predetermined number of times. The predetermined number of times is the number of times corresponding to the cycle changed by the control unit.

  According to the distance sensor and system of the present invention, the background light is generated by generating the distance information based on the received light amount for a predetermined number of times corresponding to the changed cycle while changing the operation cycle of the light source unit and the light receiving unit. This can suppress a decrease in distance measurement accuracy under a strong situation.

The block diagram which shows the structure of the distance sensor which concerns on Embodiment 1 of this invention. Schematic diagram showing a configuration example of the pixel circuit of the light receiving unit in the distance sensor Timing chart showing operation timing of emission and reception of pulsed light in distance sensor Schematic diagram for explaining the distance calculation method in the distance sensor The figure for demonstrating the production | generation operation | movement of the distance image by a distance sensor The figure for demonstrating operation | movement of the distance sensor according to the fluctuation | variation of background light The flowchart which shows the production | generation process of the distance image in Embodiment 1 The figure which shows the data table used in the production | generation process of a distance image Table to explain averaging process of range image FIG. 5 is a block diagram showing a configuration of a user interface system according to the second embodiment. The flowchart which shows the production | generation process of the distance image in Embodiment 2 Flow chart showing processing based on distance image by controller

  Hereinafter, a distance sensor according to the present invention and a system including the distance sensor will be described with reference to the accompanying drawings.

  Each embodiment is an exemplification, and needless to say, partial replacement or combination of configurations shown in different embodiments is possible. In the second and subsequent embodiments, description of matters common to the first embodiment is omitted, and only different points will be described. In particular, the same operational effects by the same configuration will not be sequentially described for each embodiment.

(Embodiment 1)
1. Configuration The configuration of the distance sensor according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration of the distance sensor according to the first embodiment.

  As shown in FIG. 1, the distance sensor 1 according to the present embodiment includes a light source unit 2, a light receiving unit 3, and a TOF signal processing unit 4. The distance sensor 1 is a sensor module that measures a distance in a measurement method such as an indirect TOF method. The distance sensor 1 emits light from the light source unit 2, receives reflected light from the object 5 by the light receiving unit 3, and generates a distance image indicating the distance to the object 5 in the TOF signal processing unit 4. The indirect TOF method is a method in which reflected light is received a plurality of times in a time-sharing manner, and the distance to the object is measured by comparing the obtained amounts of received light.

  The distance sensor 1 generates a distance image indicating the distance to the user's hand or finger, for example, as the object 5. The distance sensor 1 is mounted on an information terminal device such as a mobile terminal, for example. The distance sensor 1 and the controller 6 that controls the operation of the mounted information terminal device constitute a user interface system in the information terminal device. In the user interface system, the controller 6 detects a user operation such as a gesture operation by a user's hand or finger movement based on the distance image from the distance sensor 1. In order to enable such detection, the distance sensor 1 outputs a distance image having an accuracy of, for example, 20 mm or less in the detection range within a distance of 2000 mm to the controller 6. The information terminal device on which the distance sensor 1 is mounted may be a mobile terminal such as a smartphone, a tablet terminal, or a mobile phone, or a wearable terminal such as a glasses type or a watch type, a notebook PC, a digital camera, or a portable game. It may be a machine.

  The light source unit 2 includes a light source such as an LED (light emitting diode) and an LD (laser diode), and a drive circuit that drives the light source. The light source of the light source unit 2 emits light having a wavelength band of, for example, 750 nm to 1500 nm. Based on the timing signal from the timing generation unit 41, the light source unit 2 emits light by pulse-modulating the light from the light source. Hereinafter, the light emitted from the light source unit 2 by pulse modulation is referred to as “pulse light”.

  The light receiving unit 3 is configured by, for example, a CMOS (complementary metal oxide semiconductor) image sensor circuit. The light receiving unit 3 includes a plurality of pixel circuits 30 arranged in a matrix. The plurality of pixel circuits 30 have a light receiving surface that receives light including a wavelength band such as 750 nm or more and 1500 nm or less, and each stores a received light amount. The configuration of the pixel circuit 30 in the light receiving unit 3 will be described later. Further, although not particularly illustrated, the light receiving unit 3 converts the analog value of the received light amount accumulated in the pixel driving circuit that drives the pixel circuit 30 based on the timing signal from the timing generating unit 41 or the pixel circuit 30 into a digital value A. An A / D converter for / D (analog / digital) conversion is provided.

  The TOF signal processing unit 4 includes a circuit that performs various signal processing for generating a distance image in the TOF method, and includes a control unit 40, a timing generation unit 41, a distance image output unit 43, and a storage unit 44. Including. The TOF signal processing unit 4 is configured by, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The TOF signal processing unit 4 may be configured by a CPU or MPU.

  In the TOF signal processing unit 4, the control unit 40 is configured with a logic circuit or the like, and controls various circuits included in the TOF signal processing unit 4. The control unit 40 includes a distance calculation unit 42 and an averaging processing unit 45.

  The timing generation unit 41 includes an oscillation circuit and the like, and generates a timing signal having a predetermined cycle. The timing generation unit 41 supplies the generated timing signal to the light source unit 2 as a pulse light emission control signal for controlling the emission of pulsed light by the light source unit 2. The timing generation unit 41 also supplies the generated timing signal to the light receiving unit 3 to synchronously control the emission of the pulsed light from the light source unit 2 and the light reception by the light receiving unit 3. The operation timing of emission and reception of pulsed light in the distance sensor 1 will be described later. Note that the timing generation unit 41 may be integrated with the light receiving unit 3 to control the emission of pulsed light from the light receiving unit 3 by the light source unit 2.

  The distance calculation unit 42 is configured by an arithmetic circuit capable of performing four arithmetic operations. The distance calculation unit 42 calculates a distance based on the propagation period of the received reflected light based on the detection result of the reflected light by the light receiving unit 3. A method for calculating the distance will be described later. The distance calculation unit 42 calculates the distance for each pixel, and records, for example, distance data indicating the calculated distance for each pixel in the storage unit 44. A distance image is generated by calculating distance data for all pixels by the distance calculation unit 42. The distance calculation unit 42 is an example of a distance information generation unit that generates a distance image as distance information.

  The distance image output unit 43 includes an interface circuit that outputs information to an external device. The distance image output unit 43 outputs the distance image generated by the distance calculation unit 42 to an external device such as the controller 6. The distance image output unit 43 may output distance data for all the pixels recorded in the storage unit 44, or may output the distance data calculated by the distance calculation unit 42 as needed.

  The storage unit 44 is a storage medium that stores various information such as data and parameters for realizing the function of the distance sensor 1. The storage unit 44 is configured by a flash memory, for example. The storage unit 44 records, for example, a table in which various parameters for generating a distance image of one frame are associated with the number of frames for averaging the distance image (see FIG. 8).

  The averaging processing unit 45 includes, for example, an arithmetic circuit. The averaging processing unit 45 records, for example, image data indicating a distance image of a predetermined frame in the storage unit 44, and performs arithmetic processing for averaging the recorded distance images of a plurality of frames. Details of the averaging of the distance image will be described later.

  The controller 6 in the information terminal device is composed of, for example, a CPU and an MPU. The controller 6 includes, for example, an internal memory composed of a flash memory or a ROM, and implements various functions by executing predetermined programs recorded in the internal memory. For example, the controller 6 performs display control of a device on which the controller 6 is mounted. Further, the controller 6 detects the object 5 such as a hand based on the distance image from the distance sensor 1 and determines the user's operation on the information terminal device such as a mobile device on which the controller 6 is mounted. The controller 6 is an example of a processing device that performs predetermined processing based on the distance image generated by the distance sensor 1.

1-1. Configuration of Pixel Circuit Next, the configuration of the pixel circuit 30 in the light receiving unit 3 will be described with reference to FIG. FIG. 2A is a schematic diagram showing a pixel circuit 30 stacked on a semiconductor chip. FIG. 2B is a circuit diagram showing an equivalent circuit of FIG.

  As illustrated in FIG. 2A, the pixel circuit 30 includes a photodiode PD and three floating diffusions FD1, FD2, and FD3. In the pixel circuit 30, a photodiode PD is provided in an embedded manner on a p-type semiconductor substrate, and three floating diffusions FD1, FD2, and FD3 are provided around the photodiode PD. Further, transistors M1, M2, and M3 are formed from the region where the photodiode PD is provided to the floating diffusions FD1, FD2, and FD3, respectively.

  In the present embodiment, a charge distribution method is employed in the light receiving unit 3. In the light receiving unit 3, a pixel driving circuit (not shown) drives signals (for example, first to first) for driving various MOS transistors included in the pixel circuit 30 based on a timing signal from the timing generating unit 41 (see FIG. 1). Third gate signals Sg1 to Sg3, reset signals Sr1 to Sr3, selection signal Ss) are generated.

  In the three floating diffusions FD1, FD2, and FD3, capacitors (charge storage units) C1, C2, and C3 are formed as shown in FIG. The three capacitors C1, C2, and C3 are connected to the photodiode PD through transistors M1, M2, and M3, respectively. The transistors M1, M2, and M3 are controlled to open and close by ON / OFF of the first, second, and third gate signals Sg1, Sg2, and Sg3 input to the respective gates. The first, second, and third gate signals Sg1, Sg2, and Sg3 open and close the gates of the transistors M1, M2, and M3 based on the timing signal, and sequentially store the charges accumulated in the photodiode PD in the capacitors C1, C2, and C3. It is a signal to distribute to.

  The photodiode PD receives light in a predetermined wavelength band and performs photoelectric conversion. In the pixel circuit 30, portions other than the photodiode are shielded from light. The electric charge generated by the photoelectric conversion is accumulated in any one of the three capacitors C1, C2, and C3 through the transistor that is controlled to be open among the transistors M1, M2, and M3. As described above, the capacitors C1, C2, and C3 accumulate charges corresponding to the amount of light received by the photodiode PD. The light receiving unit 3 acquires the amount of received light by accumulating charges in the capacitors C1, C2, and C3 for each pixel circuit 30.

  The amount of received light acquired by the capacitors C1, C2, and C3 of the pixel circuit 30 is read from the analog signal line when the pixel circuit 30 is selected by the selection signal Ss. The selection signal Ss is a signal for selecting a pixel circuit 30 to be read from the plurality of pixel circuits 30. Further, the capacitors C1, C2, and C3 are reset by releasing the accumulated charges when the reference voltage VR is applied by the reset signals Sr1, Sr2, and Sr3.

2. Operation Next, the operation of the distance sensor 1 according to the present embodiment will be described.

2-1. First, a method for calculating the distance to the object by the distance sensor 1 will be described with reference to FIGS. FIG. 3 is a timing chart showing the operation timing of emission and reception of pulsed light in the distance sensor 1. FIG. 3A shows the supply timing of the pulse emission control signal. FIG. 3B shows the arrival timing of the reflected light that reaches the distance sensor 1 from the object. 3C, 3D, and 3E show the output timings of the first, second, and third gate signals Sg1, Sg2, and Sg3 input to the pixel circuit 30, respectively. FIG. 4 is a schematic diagram for explaining a distance calculation method by the distance sensor 1.

  The pulse emission control signal shown in FIG. 3A is supplied from the timing generation unit 41 to the light source unit 2 (see FIG. 1). Based on the pulse emission control signal, the light source unit 2 emits pulsed light having a pulse width of a predetermined period Tp from time t1. The period Tp is, for example, 10 ns (nanoseconds) or more and 20 ns or less, for example, 13 ns. When the pulsed light is applied to the object, it is reflected by the object to generate reflected light. The reflected light from the object reaches the distance sensor 1 with a delay from the time of emission of the pulsed light according to the distance to the distance sensor 1.

  The reflected light from the object shown in FIG. 3B has a delay period Td from the emission of the pulsed light, and reaches the distance sensor 1 at time t2 after the period Td from time t1. The waveform of the reflected light has a pulse width of the same period Tp as that of the pulsed light. In the present embodiment, it is assumed that the period Td is less than the period Tp.

  In the light receiving unit 3 of the present embodiment, as shown in FIGS. 3C to 3E, based on the first to third gate signals Sg1 to Sg3, as shown in FIG. While receiving external light such as background light, the reflected light from the object is received in a time-sharing manner in synchronization with the emission of the pulsed light.

  Specifically, the first, second, and third gate signals Sg1, Sg2, and Sg3 are sequentially generated in the pixel circuit 30 on the light receiving surface in synchronization with the pulse light emission control signal. In the pixel circuit 30, charges generated according to the amount of light received by the photodiode PD based on the first, second, and third gate signals Sg1, Sg2, and Sg3 are accumulated in the capacitors C1, C2, and C3 (FIG. 2 (b)). Note that the charge generated in the photodiode PD during a period in which the charge corresponding to the amount of received light is not accumulated in the capacitors C1, C2, and C3 is discharged to the outside through a drain gate (not shown).

  The first gate signal Sg1 shown in FIG. 3C is turned on before the emission of the pulsed light and for the period Tp. While the first gate signal Sg1 is ON, a charge corresponding to the amount of light received by the photodiode PD is accumulated in the capacitor C1. FIG. 4A shows the received light amount Q1 based on the first gate signal Sg1 accumulated in the capacitor C1. The amount of received light Q1 is the amount of received external light acquired in a state where no reflected light of pulsed light is generated. The amount of received light Q1 is acquired in order to confirm the influence of external light that is not related to pulsed light such as background light.

  The second gate signal Sg2 shown in FIG. 3 (d) is turned ON from time t1 when emission of pulsed light is started to time t3 when it is stopped. While the second gate signal Sg2 is ON, a charge corresponding to the amount of light received by the photodiode PD is accumulated in the capacitor C2. FIG. 4B shows the received light amount Q2 based on the second gate signal Sg2 accumulated in the capacitor C2. The amount of received light Q2 includes a reflected light component derived from reflected light that arrives within a period Tp from the time t1 when pulsed light emission starts. The received light quantity Q2 also includes external light components such as background light (see FIG. 4).

  The third gate signal Sg3 shown in FIG. 3E is turned on for a period Tp from time t3 after stopping emission of pulsed light. While the third gate signal Sg3 is ON, a charge corresponding to the amount of light received by the photodiode PD is accumulated in the capacitor C3. FIG. 4C shows the received light amount Q3 based on the third gate signal Sg3 accumulated in the capacitor C3. The received light amount Q3 includes a reflected light component derived from reflected light that continuously arrives from time t3 when emission of pulsed light is stopped to time t4 after the delay period Td. In this way, the entire amount of reflected light received is time-divided and distributed as reflected light components of the amounts of received light Q2 and Q3 according to the delay period Td. Note that it is assumed that the delay period Td of the reflected light is less than the period Tp, and the time t4 when the arrival of the reflected light ends is within the range of the charging period Tp of the capacitor C3. The received light amount Q3 includes an external light component as in the received light amount Q2. Note that the length of the period in which the pulsed light is emitted and the length of the period in which the gate signals Sg1, Sg2, and Sg3 are ON do not have to be the same length.

  As described above, in the distance sensor 1, the light receiving unit 3 receives light during a predetermined time from the start of the pulsed light emission period, and the received light amounts Q2, Q3 in the capacitors C2, C3 (first charge storage unit). The electric charge according to is accumulated. Further, light is received during the stop period of emission of the pulsed light, and charges corresponding to the amount of received light Q1 are accumulated in the capacitor C1 (second charge accumulation unit). By detecting the charges accumulated in the capacitors C1, C2, and C3, the received light amounts Q1, Q2, and Q3 can be obtained. According to the received light amounts Q1, Q2, and Q3 corresponding to the charges accumulated in the capacitors C1, C2, and C3, the ratio of the delay period Td and the period Tp of the reflected light from the object is as shown in FIG. Furthermore, this corresponds to the ratio (hereinafter referred to as “distribution ratio”) distributed to the received light quantity Q3 in the total received light quantity of the reflected light. Therefore, the delay period Td can be obtained based on the distribution of the received light amounts Q2 and Q3, that is, the distribution ratio.

  Here, as shown in FIGS. 4B and 4C, the received light amounts Q2 and Q3 include not only reflected light components but also external light components. Since the received light amounts Q2 and Q3 are acquired in the period Tp having the same length as the received light amount Q1 of only the external light, the external light component included in the received light amounts Q2 and Q3 is considered to be approximately the same as the received light amount Q1. It is done. Therefore, in the present embodiment, the received light amount of the reflected light component excluding the external light component is calculated by appropriately subtracting the received light amount Q1 from the received light amounts Q2 and Q3. Since the received light amount Q1 is acquired immediately before acquiring the received light amounts Q2 and Q3, the external light component in the received light amounts Q2 and Q3 can be accurately removed by the received light amount Q1.

The delay period Td is the time taken for the pulsed light to reach the object and return to the distance sensor 1 as reflected light. That is, this is the time taken when the distance between the object and the distance sensor 1 reciprocates at the speed of light c. Therefore, if the distance to the object is L, Td = 2L / c is established, and therefore the distance L to the object can be calculated based on the following equation.
L = (c / 2) * Tp * {(Q3-Q1) / (Q2 + Q3-2 * Q1)} (1)

  In the distance sensor 1 according to the present embodiment, the distance calculation unit 42 performs a predetermined calculation based on the principle of the above formula (1), and calculates a distance based on the distribution ratio of the received light amount. The calculation formula of the distance calculation unit 42 is defined by, for example, various signal timings and parameter settings (calibration) performed in advance under a specific environment.

2-2. Distance Image Generation Operation A distance image generation operation according to the present embodiment will be described with reference to FIG. FIG. 5A is a diagram for explaining an operation for generating a distance image of one frame. FIGS. 5B, 5C, and 5D are timing charts of the first, second, and third gate signals Sg1, Sg2, and Sg3 during the integration period of FIG. 5A. FIG. 5E shows a timing chart of the drain gate during the integration period of FIG. The drain gate is a gate for discharging charges from the photodiode PD.

  The distance sensor 1 according to the present embodiment repeatedly performs the series of operations of emitting the pulsed light and receiving the reflected light as described above, and sequentially generates a distance image each time the amount of received light is read. As shown in FIG. 5A, in order to generate a distance image once (one frame), an integration period and a readout period are required. The integration period is a period in which the light receiving unit 3 integrates the received light amount by repeating the above-described emission and reception operations of the pulsed light a predetermined number of times. The reading period is a period in which the control unit 40 reads the amount of light received integrated from the light receiving unit 3 following the integration period. Hereinafter, the number of frames refers to the number of distance images generated based on one integration period and readout period.

  The integration period includes a light reception period corresponding to the number of times of emitting pulsed light. The light receiving period is a period in which light is received in the pixel circuit 30. During one light receiving period, as shown in FIGS. 5B, 5C, and 5D, the first, second, and third gates are used. Signals Sg1, Sg2, and Sg3 are sequentially turned on (see FIG. 3). As shown in FIG. 5E, the drain gate is opened in a non-light-receiving period that is outside the light-receiving period in the integration period, and discharges the charge photoelectrically converted by the photodiode PD. In the integration period, based on the first, second, and third gate signals Sg1, Sg2, and Sg3, the received light amounts Q1, Q2, and Q3 (see FIG. 4) are accumulated by repeatedly accumulating charges in the charge accumulating units C1, C2, and C3. ) Is accumulated. Hereinafter, the number of times each of the received light amounts Q1, Q2, and Q3 during the integration period is referred to as “the number of integrations”. The number of integration is, for example, 300 to 30,000.

  In the readout period after the integration period, the control unit 40 of the TOF signal processing unit 4 reads out the digital values of the received light amounts Q1, Q2, and Q3 integrated by the pixel circuits 30 from the A / D converter of the light receiving unit 3. . In the control unit 40, the distance calculation unit 42 performs a calculation based on the formula (1) based on the digital values of the received light amounts Q1, Q2, and Q3 for each pixel circuit 30. By integrating the received light amounts Q1, Q2, and Q3, the statistical accuracy of the measured distance is increased, and the distance can be calculated with high accuracy.

  When the distance calculation unit 42 performs the above calculation on one pixel, distance data indicating the distance of the pixel is acquired. The TOF signal processing unit 4 generates a distance image of one frame by acquiring distance data for all pixels.

  The distance sensor 1 according to the present embodiment is mounted on a mobile device, for example, and repeatedly outputs a distance image once every predetermined time interval in order to detect an operation by a user's gesture or the like. The time interval is, for example, 1/60 seconds or more and 1/30 seconds or less, and is 1/30 seconds here. The distance sensor 1 is desired to output a distance image at a cycle within 1/30 seconds and reduce power consumption. Furthermore, it is assumed that mobile devices are taken out and used indoors, and measures against background light such as outdoor sunlight are important. Hereinafter, background light countermeasures by the distance image generation operation of this embodiment will be described.

2-2-1. About background light countermeasure The background light countermeasure in this embodiment is demonstrated using FIG. FIG. 6 is a diagram for explaining the operation of the distance sensor 1 according to the fluctuation of the background light. FIGS. 6A and 6B show the operations of the light source unit 2 and the light receiving unit 3 when generating a distance image at normal time (when the background light is relatively weak). 6C and 6D show operations of the light source unit 2 and the light receiving unit 3 when the background light increases from the normal time.

  In the outdoors, strong background light such as sunlight becomes an obstacle to distance measurement in a distance image. In the present embodiment, the intensity of the background light is monitored when the distance image is generated, and the parameter setting for newly generating the distance image is changed according to the amount of the received background light.

  First, as shown in FIGS. 6 (a) and 6 (b), from the viewpoint of power consumption and the like, the period of pulse light emission by the light source unit 2 and the period of light reception of the light receiving unit 3 synchronized therewith (hereinafter referred to as “pulse”) "Period") is set to a large value such as 1000 ns. In addition, the reading period is set to 4.48 ms (milliseconds) as an example.

Under circumstances where the background light is strong, the received light amount Q1 (see FIG. 4) of the background light is a signal amount Qs (= (Q2-Q1) obtained by removing the external light component (Q1) from the received light amounts Q2 and Q3 including the reflected light. ), (Q3-Q1)). When the received light amount Q1 of background light accumulated in the light receiving operation of one frame of the light receiving unit 3 (see FIG. 4) is much larger than the signal amount Qs, shot noise generated during the light receiving operation is approximately (Q1). ^The S / N ratio is proportional to Qs / (Q1) ^1/2 . For this reason, in order to reduce the influence of shot noise, it is necessary to increase the signal amount Qs while suppressing the increase in the received light amount Q1 of the background light accumulated in the light receiving unit 3 per frame. Even if the total light receiving amount is increased by simply extending the light receiving period in one frame, the noise components ((Q1) ^1/2 ) are increased and the charge storage units C1 to C3 may be saturated. Measurement accuracy will be worsened.

  Therefore, in the distance sensor 1 according to the present embodiment, when strong background light is detected, as shown in FIGS. 6C and 6D, the average of the distance images of a plurality of continuous frames is obtained while shortening the pulse period. Turn into. As a result, the signal amount Qs for a plurality of frames is reflected in the distance image output after the frame average, and the statistical measurement accuracy of the distance can be improved (that is, the deterioration of the distance accuracy due to shot noise) can be improved. .

  Here, if the distance images of a plurality of consecutive frames are simply averaged, the time interval for outputting the distance images may be doubled by the averaged plurality of frames, and the dynamic measurement accuracy of the distance to the object may be reduced. . In contrast, in the present embodiment, the frame rate (before averaging) is increased by shortening the pulse period when performing frame averaging. As a result, even if a plurality of frame periods are grouped by the frame average, the distance image after the frame average can be output once within a predetermined period such as 1/30 second, and the dynamic measurement accuracy of the distance can be maintained. .

In the present embodiment, when the pulse period is shortened, the emission stop period (and the non-light receiving period) is shortened without changing the pulse width (and the light receiving period). As a result, the amount of light received per frame can be maintained while shortening the pulse period, and the same amount of signal Qs as before the shortening of the pulse period can be maintained in each frame without increasing the noise component (Q1) ^1/2. Can be obtained at For this reason, in the distance sensor 1 which concerns on this embodiment, the influence of shot noise can be reduced, maintaining the dynamic measurement precision of distance. The distance image generation process in which the above background light countermeasure is taken will be described below.

2-2-2. Distance Image Generation Processing The distance image generation processing of the distance sensor 1 according to the present embodiment will be described with reference to FIGS. FIG. 7 is a flowchart showing a distance image generation process in the present embodiment. FIG. 8 shows an example of a data table used in the distance image generation process. FIG. 9 shows a table for explaining the distance image averaging process (frame average).

  The processing according to this flowchart is executed by the control unit 40 of the TOF signal processing unit 4.

  First, the control unit 40 controls the timing generation unit 41 to emit light repeatedly at a preset pulse period and receive light while integrating the received light amounts Q1, Q2, and Q3 (see FIGS. 4 and 5). Thus, the light source unit 2 and the light receiving unit 3 are on / off controlled (S1). In the present embodiment, setting of the pulse period and the like is performed based on the data table D1.

  As shown in FIG. 8, the data table D1 includes a mode number, a pulse period, a (pre-average) frame rate, a duty ratio per frame, an averaged number of frames, and the number of pulse emission during one output. to manage. The mode number is a mode management number provided in order to shorten the pulse period step by step. The frame rate is the number of frames per unit time of the distance image generated each time the received light amount is read out. The duty ratio per frame is the ratio of the period during which the light source unit 20 emits light during one frame to the period of one frame (before the average). The number of averaged frames is the number of frames of the distance image acquired as the target of the averaging process when the distance image is averaged. The number of pulse emission during one output is the number of times the pulsed light is emitted when receiving the received light amount used in the distance image that has been averaged and output once.

  In FIG. 8, a four-stage mode is defined as an example. Hereinafter, the mode having the mode number “n” (n = 1 to 4) is referred to as “mode n”. The states shown in FIGS. 6A and 6B show the mode 1 operation, and the states shown in FIGS. 6C and 6D show the mode 2 operation. For example, mode 1 is a normal operation mode, in which the pulse cycle is set to “1000 ns” and the number of pulse emission is set to “28000 times”. For this reason, in step S1 which is set to the mode 1, the pulse light emission by the light source unit 2 is repeated 28000 times once every 1000 ns.

  Returning to FIG. 7, the control unit 40 reads the digital values of the received light amounts Q1, Q2, and Q3 from the light receiving unit 3 by the on / off control in the mode being set (S2).

  Based on the read digital value of the received light amount, the control unit 40 calculates the distance based on the equation (1) in the distance calculation unit 42 and generates a distance image (S3).

  Next, the control unit 40 determines whether or not to average the generated distance image based on the number of average frames in the mode being set (S4). In the determination process of step S4, the control unit 40 proceeds to “No” when the number of averaged frames is “1”, and proceeds to “Yes” otherwise. This determination process considers the frame rate of the distance image. For example, in mode 1 in which the frame rate is “30.8 fps”, if the distance images of a plurality of frames are averaged, a period of 1/30 seconds or more is consumed until the averaged distance image is output. For this reason, in mode 1, it is determined not to average the distance image (No in S4), and the process proceeds to step S5. The case of averaging distance images will be described later.

  The control unit 40 outputs a distance image from the distance image output unit 43 to an external device such as the controller 6 (S5).

  In the present embodiment, as an example, the pulse width in step S1 is 13 ns, and the light reception amount reading period in step S2 is 4.48 ms. Therefore, in mode 1, the frame rate from when the light source unit 2 and the light receiving unit 3 are turned on / off in step S1 to when the distance image is output in step S5 is “30.8 fps” (see FIG. 8). . Further, the duty ratio per frame is “1.12%” in which the light receiving amount reading period is included in the period in which the light source unit 20 does not emit light.

  Next, the control unit 40 detects the amount of received background light based on the amount of received light read in step S2 (S8). In the present embodiment, the control unit 40 determines the amount of the received light amount of the background light using the received light amount Q1 accumulated in the charge storage unit C1 of each pixel circuit 30, and the received light amount Q1 of all the pixel circuits 30. The maximum value is used as the background light detection value.

  Next, the control unit 40 determines whether or not the detected value of the background light has exceeded the threshold value LA-n corresponding to the mode number “n” being set (S9). The threshold value LA-n is a threshold value indicating that a countermeasure for shot noise may be insufficient in mode n due to an increase in the amount of received light. In the present embodiment, the threshold value LA-n is set so as to increase as the mode number “n” increases. For example, the threshold value LA-1 is set to 70% of the maximum storage amount of the charge storage unit C1, and the threshold value LA-2 is set to 90% of the maximum storage amount of the charge storage unit C1.

  When the detected value of the background light exceeds the threshold value LA-n, the amount of light received by one of the pixel circuits 30 exceeds the threshold value LA-n, and light shot noise in the light receiving operation of the light receiving unit 3 increases. doing. Therefore, when it is determined that the detected value of the background light has exceeded the threshold value LA-n (Yes in S9), the control unit 40 performs a pulse cycle shortening process (S10).

  The pulse period shortening process is a process for shortening the pulse period of the distance image generated in the next control period and setting the averaging of the distance image in accordance with the increase in background light. In the present embodiment, the pulse cycle shortening process is performed by shifting the mode being set to a mode larger by one step in the data table D1 shown in FIG. For example, when the processing in the mode 1 shown in FIGS. 6A and 6B is performed and the detected value exceeds the threshold value LA-1 (Yes in S9), the control unit 40 uses the data table. With reference to D1, the mode 1 is shifted to the mode 2 (S10), and the processing after the step S1 is performed again in the mode 2 state (see FIGS. 6C and 6D).

  In the data table D1, each mode of mode number “2” or more is an operation mode for countermeasure against background light in which the pulse period is changed and the frame averaging is set under the condition where the background light is strong. In mode 2 of the data table D1, the number of averaged frames “2” is set. For this reason, in the state of mode 2, the process proceeds to “Yes” in step S4, and the distance images are averaged.

  When the distance images are averaged (Yes in step S4), the control unit 40 determines whether the number of frames of the generated distance image has reached the average number of frames in the data table D1 (S6). The control unit 40 repeats the processing after step S1 until the number of frames of the generated distance image reaches the average number of frames (No in S6), and records the generated distance image in the storage unit 44.

  When the number of frames of the generated distance image has reached the number of averaged frames (Yes in S6), the control unit 40 performs averaging of the distance images for the number of averaged frames in the averaging processing unit 45 (S7). .

  The table shown in FIG. 9 shows an example of distance data before and after averaging of distance images. As shown in FIG. 9, the distance image is averaged by calculating an average value of distance data corresponding to the number of averaged frames at each pixel (x1, y1), (x2, y2),. Done. The calculation of the average value of the distance data by the averaging processing unit 45 may be performed in units of actual distances or may be performed in units of the number of bits. By using the distance data of the distance image obtained by averaging the calculated average values, the variation of the distance data for each frame is flattened, and the distance measurement accuracy is improved.

  In the mode 2 state shown in FIG. 8, since the number of averaged frames is set to “2”, the distance images for two frames are averaged as shown in FIG. 9 (S7), and the averaged distance image is displayed. Is output (S5). Here, since the pulse period of mode 1 is shortened from “1000 ns” to mode “430 ns”, the frame rate of mode 2 (before average) is higher than the frame rate of mode 1 “30.8 fps”. It is a large “60.5 fps”. For this reason, even if the period for generating the distance image of 2 frames is put together, the distance image averaging process can be completed within 1/30 seconds.

  In addition, by the above averaging process, the distance image output in mode 2 is generated based on the received light amount for two readings every time the integrated received light amount is read twice. The amount of light received for two readings includes the amount of signal for the number of times of pulse emission “58000 times” that is twice the number of times of pulse emission for mode 1 “28000 times”. Can be reduced.

  In the pulse cycle shortening process, every time the detection value of background light increases, the number of modes in the data table D1 sequentially increases, and the number of averaged frames is increased while shortening the pulse cycle. Thereby, in each mode, a distance image can be output once within 1/30 seconds while performing an averaging process according to the number of average frames.

  Returning to FIG. 7, when the control unit 40 determines that the detected value of the background light does not exceed the threshold value LA-n in mode n (No in S9), the detected value of the background light is the threshold value LB-. It is determined whether or not it is less than n (S11). The threshold value LB-n is a threshold value indicating that a countermeasure against shot noise can be taken even if the mode n is canceled due to a decrease in the amount of received light, and is a value smaller than the threshold value LA-n. For example, the threshold value LB-1 is set to 65% of the maximum accumulation amount of the charge accumulation unit C1, and the threshold value LB-2 is set to 85% of the maximum accumulation amount of the charge accumulation unit C1.

  When the detected value of the background light falls below the threshold value LB-n, the amount of light received by all the pixel circuits 30 is below the threshold value LB-n, and the charge storage section is restored even if the integration count is reset. Saturation of C1, C2, C3 does not occur. For this reason, when it is judged that the detection value of background light has fallen below threshold value LB-n (Yes in S11), the control part 40 performs a pulse period extension process (S11).

  The pulse period extension process is a process for extending the pulse period of the distance image generated in the next control period and reducing the number of average frames in accordance with the decrease in background light. In the present embodiment, the pulse cycle extension process is performed by shifting the mode being set in the data table D1 shown in FIG. As a result, the distance sensor 1 can be operated with the minimum necessary power consumption when the background light becomes strong and then weakens.

  In addition, when the control unit 40 determines that the detected value of the background light is not lower than the threshold value LB-n (No in S11), the control unit 40 repeatedly performs the processing after step S1 in the mode setting state.

  According to the above processing, the pulse period is shortened stepwise in the mode of the data table D1 in a situation where the background light is strong, and the number of average frames of the distance image is increased. For this reason, it is possible to reduce the influence of shot noise and output a distance image with a single frequency within 1/30 seconds under a strong background light. Each output distance image is generated based on the received light amount read for the number of averaged frames, and the received light amount includes a signal amount corresponding to the number of pulse emission times (see FIG. 8) for the number of averaged frames. Therefore, the influence of shot noise is reduced. For this reason, it is possible to suppress a decrease in the accuracy of the distance image due to shot noise while maintaining a dynamic distance measurement accuracy under a situation where the background light is strong.

  In the above description, the detection of the amount of received background light (step S8) is performed after the output of the distance image in step S5. However, the timing for performing the processing after step S8 is not limited to after step S5. The processes of steps S8 to S12 may be performed before step S5, for example, after the received light amount is read in step S2.

  In the above description, the mode of the data table D1 shown in FIG. 10 is changed step by step in the processing of steps S10 and S12. However, the present invention is not limited to this, for example, the number of modes to be changed according to the detected value of background light. And a plurality of modes may be changed in one frame.

  In the above process, the data table D1 is referred to in the pulse period shortening process and the pulse period extending process in steps S10 and S12. Instead of using the data table D1, a pulse period set by calculation may be calculated. .

3. Summary As described above, the distance sensor 1 according to the present embodiment includes the light source unit 2, the light receiving unit 3, and the control unit 40. The light source unit 2 repeatedly emits pulsed light to the object 5. The light receiving unit 3 receives light in synchronization with the emission of pulsed light. The control unit 40 controls the operation of the light source unit 2 and the light receiving unit 3 and reads the received light amounts Q1 to Q3 of the light received from the light receiving unit 3 to determine the distance to the object based on the received light amounts Q1 to Q3. A distance image that is distance information to be shown is generated. The control unit 40 changes the pulse period which is the period of each of the emission by the light source unit 2 and the light reception by the light receiving unit 3 in accordance with the magnitude of the received light amount Q1. The control unit 40 generates a distance image based on the amount of received light for the number of averaged frames each time the reading of the amount of received light Q1 from the light receiving unit 3 is repeated for the number of averaged frames being set. The averaged frame number is the number of times corresponding to the pulse period changed by the control unit 40.

  According to the distance sensor 1, a distance image is generated based on the received light amount for the number of averaged frames corresponding to the pulse period while changing the pulse period according to the magnitude of the received light amount received by the light receiving unit 3. . For this reason, while using the amount of received light for the number of averaged frames to reduce the effect of shot noise that increases under strong background light conditions, dynamic measurement accuracy is maintained by changing the pulse period, A decrease in measurement accuracy can be suppressed.

  Further, in the distance sensor 1, the control unit 40 may shorten the pulse period in stages as the amount of received light Q1 increases. In this case, the number of averaged frames increases as the pulse period is shortened. As a result, when the average number of frames increases, the stronger the background light, the greater the number of averaged frames while shortening the pulse period, and the statistical measurement while suppressing the decrease in the dynamic measurement accuracy of distance. Decrease in accuracy can be suppressed.

  In the distance sensor 1, the control unit 40 does not change the time width (pulse width) in which the light source unit emits light per cycle while changing the pulse cycle. As a result, even if the pulse period is changed, the total amount of received light read per frame does not change, and the decrease in the statistical measurement accuracy of the distance is further suppressed.

  In the distance sensor 1, the control unit 40 may include an averaging processing unit 45 that averages the distance images based on the received light amounts in the received light amounts Q1 to Q3 for the number of averaged frames. The distance sensor 1 has a pulse so that the period from the start of the reception of the first received light amount of the received light amount for the number of averaged frames to the time when the averaging processing unit 45 averages the distance image is within a predetermined period. The period may be changed. Thereby, a distance image averaged by the averaging processing unit 45 is obtained once in a predetermined period, and a decrease in distance measurement accuracy can be better suppressed both dynamically and statistically.

  In the distance sensor 1, the predetermined period may be 1/60 seconds or more and 1/30 seconds or less. Thereby, for example, a distance image of a moving image that can follow a user's gesture or the like can be output.

  The distance sensor 1 may further include a storage unit 44 that stores a data table D1 in which the pulse period and the number of average frames are associated with each other. In this case, the control unit 40 changes the pulse period with reference to the data table D1 according to the magnitude of the received light amount Q1. Thereby, control which sets a pulse period and the number of frames which average a distance image based on data table D1 can be performed easily.

  Further, in the distance sensor 1, the light receiving unit 3 receives light by time division during a predetermined time from the start of emission of pulsed light, and accumulates charges corresponding to the light reception amounts Q2 and Q3 obtained by time division. A plurality of first charge storage units C2 and C3 may be provided. Thereby, the distance can be calculated using the received light amounts Q2 and Q3 which are time-divided.

  In the distance sensor 1, the light receiving unit 3 receives light during the stop period of emission of pulsed light, and accumulates charges corresponding to the received light amount Q <b> 1 received during the stop period of emission of pulsed light. An accumulation unit C1 may be provided. As a result, it is possible to obtain the received light amount Q1 of only the background light not including the reflected light of the pulsed light.

(Embodiment 2)
In the first embodiment, the distance image is averaged in the distance sensor 1. In the second embodiment, a distance image is averaged in a controller that acquires a distance image from a distance sensor. Hereinafter, a system including a distance sensor and a controller according to the present embodiment will be described.

  FIG. 10 is a block diagram illustrating a configuration of a user interface system according to the second embodiment of the present invention. In the first embodiment, the control unit 40 of the distance sensor 1 includes the averaging processing unit 45 (see FIG. 1). However, in this embodiment, the controller 6A is not the control unit 40A of the distance sensor 1A, but the averaging processing unit 61. Is provided.

  For example, similarly to the data table D1 of the first embodiment, the controller 6A stores information in which the mode set in the distance sensor 1A and the number of average frames are associated with each other in the internal memory. Based on the above information, the averaging processing unit 61 of the controller 6A performs frame averaging of the distance images of a plurality of frames acquired from the distance sensor 1A. The function of the averaging processing unit 61 is realized by a program executed on the controller 6A, for example.

  FIG. 11 is a flowchart showing a distance image generation process by the distance sensor 1A according to this embodiment. The distance sensor 1A performs the pulse period shortening process and the pulse period extending process (steps S9 to S12) for changing the pulse period according to the intensity of the background light, as in FIG. 7 of the first embodiment. On the other hand, in the first embodiment, the distance image averaging process is performed in steps S4 to S7 in FIG. 7, but in this embodiment, instead of this, the distance image of each frame is sequentially output to the controller 6A (step S5A). ).

  In step S5A, the distance sensor 1A outputs mode information indicating the mode being set (see FIG. 8) to the controller 6A together with the distance image. The mode information is information indicating a mode number in the data table D1, for example.

  FIG. 12 is a flowchart showing processing based on the distance image by the controller 6A. First, the controller 6A detects whether or not a distance image and mode information have been acquired from the distance sensor 1A at a predetermined cycle (for example, 1/30 seconds) (S21).

  When the controller 6A acquires the distance image and the mode information from the distance sensor 1 (Yes in S21), the controller 6A determines whether or not to average the distance image based on the acquired mode information (S23).

  When the distance image is averaged (Yes in step S23), the controller 6A determines whether or not the number of frames of the distance image acquired from the distance sensor 1A has reached the average number of frames based on the mode information (S24). ). The controller 6A repeats the processing after step S21 until the number of frames of the acquired distance image reaches the averaged number of frames (No in S24), and records the acquired distance image in the internal memory.

  When the number of frames of the distance image acquired from the distance sensor 1A reaches the number of averaged frames (Yes in S24), the controller 6A executes averaging of the distance images in the averaging processing unit 61 (S25). The process of step S25 is performed similarly to the process of step 7 of FIG.

  Next, the controller 6A detects a user operation such as a gesture based on the averaged distance image (S26). In the process of step S <b> 26, the controller 6 </ b> A performs, for example, a process for detecting the distance and motion to the object shown in the distance image. Further, the controller 6A may perform image processing for detecting the three-dimensional shape of the object shown in the distance image. The controller 6 performs, for example, display control of the mounted device using the detection result based on the distance image.

  The controller 6A repeatedly executes the above processing.

  According to the above processing, the distance sensor 1A sequentially outputs the distance images to the controller 6 while shortening the pulse period under a situation where the background light is strong. The controller 6A receives the distance images of a plurality of frames from the distance sensor 1A. Is averaged. Thereby, the controller 6A can obtain a distance image in which the influence of shot noise is reduced at a frequency of, for example, once every 1/30 seconds, and can detect a user operation or the like.

  As described above, the user interface system according to the present embodiment includes the distance sensor 1A and the controller 6A. The controller 6A acquires a distance image from the distance sensor 1A and executes predetermined processing. The control unit 40A of the distance sensor 1A changes the pulse cycle according to the magnitude of the received light amount Q1. The controller 6A includes an averaging processing unit 61 that averages the distance images for the number of averaged frames every time the averaged frame number and the distance image being set are acquired from the distance sensor 1A. The averaged frame number is the number of times corresponding to the pulse period changed by the control unit 40.

  According to this system, the distance image that the distance sensor 1A sequentially outputs while changing the pulse period under the strong background light is averaged by the number of average frames in the controller 6A, and the measurement accuracy of the distance is reduced. Can be suppressed.

  In the above description, the mode information is output from the distance sensor 1A to the controller 6A together with the distance image of one frame in step S5A of FIG. 11, but the mode information from the distance sensor 1A may not be output for each frame. . For example, the mode information may be output to the controller 6A when the mode is changed in the distance sensor 1A.

  Moreover, the setting of the number of average frames may be performed in the controller 6A. For example, each time the controller 6A acquires a distance image from the distance sensor 1A, the frame rate may be measured, and the average number of frames may be set based on the measurement result.

(Other embodiments)
In each of the above embodiments, when the control unit 40 of the distance sensor 1 or 1A determines that the detected value of the background light has exceeded the threshold value LA-n in step S9 of the distance image generation process (FIGS. 7 and 11). The pulse cycle shortening process was performed. However, the timing for executing the pulse period shortening process is not limited to this. For example, when the control unit 40 determines that the detected value of the background light continuously exceeds the threshold value LA-n in a predetermined plurality of frames. A pulse cycle shortening process may be executed. Also, with respect to the execution timing of the pulse cycle extension process, when the control unit 40 determines that the detected value of background light has fallen below the threshold value LB-n continuously in a predetermined plurality of frames, You may perform a period extension process.

  In each of the above embodiments, the received light amount Q1 accumulated in the charge storage unit C1 dedicated to external light is used as the background light monitor. However, instead of the received light amount Q1, the received light amount Q1 is integrated in the charge storage units C2 and C3. The received light amounts Q2 and Q3 may be used. For example, the received light amount including the reflected light component may be monitored using a larger received light amount of the received light amounts Q2 and Q3 for each frame. The pulse period of the next frame may be changed according to the magnitudes of the received light amounts Q1, Q2, and Q3 integrated for each frame. When only the background light is monitored using the received light amount Q1, it can be distinguished from, for example, an increase or decrease in the received light amount due to a change in the distance to the object, and a dedicated mode setting is performed when the background light is strong. It is effective in some cases.

  Further, in each of the above embodiments, the maximum value of the received light amount in all the pixel circuits 30 of the light receiving unit 3 is extracted as the detection value. However, the method of monitoring the received light amount is not limited to this, and for example, all the pixel circuits 30 The maximum value of the amount of light received in some of the pixel circuits 30 may be used as the detection value. Further, instead of extracting the maximum value of the amount of received light, a plurality of values may be sampled and compared with the threshold values LA-n and LB-n, respectively. For example, in comparison with the threshold value LB-n, the pulse period may be extended when the magnitude of a predetermined number of received light amounts of all the pixel circuits 30 falls below the threshold value LB-n.

  In each of the above embodiments, the pixel circuit 30 includes the charge storage unit C1 dedicated to external light. However, the pixel circuit in the distance sensor is not limited thereto, and the charge storage unit C1 dedicated to external light may be omitted. Good. In this case, in reading from the pixel circuit, the received light amount including reflected light and the received light amount of external light may be acquired in different frames, and a one-frame distance image may be generated by reading out two frames. For example, in two consecutive frames, the emission of pulsed light is repeated in one frame, and the charge is accumulated by time division in two charge accumulating units, read out to the TOF signal processing unit 4, and the received light amount including reflected light is calculated. get. In the other frame, by receiving light while stopping the emission of pulsed light, the amount of external light received can be acquired. In this case, the amount of received light may be monitored with the amount of received external light or with the amount of received light including reflected light.

  In each of the above-described embodiments, the pixel circuit 30 includes the three charge storage units C1, C2, and C3, and the three types of received light amounts Q1, Q2, and Q3 are acquired in a time division manner. The number of charge storage units included in the pixel circuit may be three or more. For example, the pixel circuit may include four or more charge storage units, and four or more received light amounts Q1, Q2,..., Qm (m is an integer of 4 or more) may be acquired in a time division manner.

  In each of the above embodiments, whether or not to average the distance image is determined based on the number of averaged frames. For switching whether or not to perform averaging, the number of average frames may not be used. For example, a predetermined flag may be used separately.

  In each of the above embodiments, the distance image is generated from the received light amount read in each frame and then averaged, so that the received light amount is averaged every time the average number of frames is read (predetermined number of times). A distance image based on the amount of light received by the number of frames was generated. However, the method for generating the distance image based on the light reception amount for the predetermined number of times is not limited to this, and for example, by calculating an average value for the data (light reception amount) acquired before the distance calculation, A distance image based on the amount of received light may be generated. For example, every time the distance sensor 1 repeats the reading of the received light amount for a plurality of frames, the received light amount for a plurality of frames read by the averaging processing unit 45 is first averaged, and then the received light amount that is averaged by the distance calculation unit 42 The distance image may be generated once based on the above.

  The calculation method for averaging the received light amount and the distance image (distance data) may be an arithmetic average for a plurality of frames, a weighted average or a geometric average.

  In each of the above embodiments, the control unit 40 includes the distance calculation unit 42 and the averaging processing unit 45 in the TOF signal processing unit 4. However, the present invention is not limited thereto, and the control unit 40 and the distance in the TOF signal processing unit 4 are not limited thereto. The calculation unit 42 and the averaging processing unit 45 may be configured separately.

  Further, in each of the above embodiments, the case where the distance measuring method of the distance sensors 1 and 1A is the indirect TOF method has been described. The distance measurement method of the distance sensor is not limited to this, and may be a direct TOF method that directly measures the propagation period of reflected light, for example. The direct TOF type distance sensor may be configured using, for example, a time digital converter or a single photon avalanche photodiode.

  Further, in each of the above-described embodiments, the case where the distance sensors 1 and 1A are mounted on the mobile device is illustrated. The device on which the distance sensors 1 and 1A are mounted is not limited to a mobile device, and may be, for example, a monitoring camera or an in-vehicle device. Even in such a case, according to the distance sensors 1 and 1A, it is possible to suppress a decrease in measurement accuracy of the distance to an object such as a person or a car even under a strong background light.

DESCRIPTION OF SYMBOLS 1 Distance sensor 2 Light source part 3 Light receiving part 40 Control part 41 Timing generation part 42 Distance calculating part 44 Storage part 45 Averaging process part 6 Controller 61 Averaging process part C1, C2, C3 Charge accumulation part

Claims (9)

A light source unit that repeatedly emits light to an object;
A light receiving unit that receives light in synchronization with emission of light by the light source unit;
A control unit that controls operations of the light source unit and the light receiving unit and generates distance information indicating a distance to the object based on the received light amount each time the received light amount of light received from the light receiving unit is read; With
The control unit changes each period of emission by the light source unit and light reception by the light receiving unit according to the magnitude of the received light amount,
The control unit generates distance information based on the received light amount for the predetermined number of times each time the reading of the received light amount from the light receiving unit is repeated a predetermined number of times.
The predetermined number of times is a distance sensor that is the number of times corresponding to the cycle changed by the control unit. The controller shortens the cycle step by step as the amount of received light increases.
The distance sensor according to claim 1, wherein the predetermined number of times increases as the cycle is shortened. The distance sensor according to claim 1, wherein the control unit does not change a time width during which the light source unit emits light per cycle while changing the cycle. The controller is
An averaging processing unit that averages distance information based on each received light amount in the predetermined number of received light amounts,
The period is changed so that a period from the start of light reception of the first light reception amount of the predetermined number of light receptions to the time when the averaging processing unit averages the distance information is within a predetermined period. Item 4. The distance sensor according to any one of Items 1 to 3. The distance sensor according to claim 4, wherein the predetermined period is 1/60 seconds or more and 1/30 seconds or less. A storage unit for storing a table in which the period and the predetermined number of times are associated;
The distance sensor according to any one of claims 1 to 5, wherein the control unit changes the cycle with reference to the table according to the magnitude of the amount of received light. The light receiving unit is
Receives light in a time-sharing manner for a predetermined time from the start of emission by the light source unit,
The distance sensor according to any one of claims 1 to 6, further comprising a plurality of first charge accumulation units each accumulating charges corresponding to the amount of light received by time division. The light receiving unit is
Receiving light during a stop period of emission by the light source unit,
The distance sensor according to claim 1, further comprising a second charge accumulation unit that accumulates charges corresponding to the amount of light received during the stop period. A light source unit that repeatedly emits light to an object;
A light receiving unit that receives light in synchronization with emission of light by the light source unit;
A control unit that controls operations of the light source unit and the light receiving unit, and generates distance information indicating a distance to the object based on the received light amount each time the received light amount of light received from the light receiving unit is read. A distance sensor with
A processing device that acquires the distance information from the distance sensor and executes predetermined processing;
The control unit of the distance sensor changes each period of emission by the light source unit and light reception by the light receiving unit according to the magnitude of the received light amount,
The processing device includes an averaging processing unit that averages distance information for the predetermined number of times each time the distance information is acquired from the distance sensor a predetermined number of times.
The predetermined number of times is a number of times corresponding to the period changed by the control unit.

Images