WO2016208215A1

WO2016208215A1 - Distance sensor and user interface device

Info

Publication number: WO2016208215A1
Application number: PCT/JP2016/054636
Authority: WO
Inventors: 勝宏田淵
Original assignee: 株式会社村田製作所
Priority date: 2015-06-24
Filing date: 2016-02-18
Publication date: 2016-12-29
Also published as: JP2018136123A

Abstract

This distance sensor is provided with a light source unit, a light receiving unit, and a distance information generating unit. The light source unit irradiates a subject with irradiation light. The light receiving unit receives light in a predetermined period of time from the start of an irradiation period of the irradiation light, and receives light during an irradiation stop period. On the basis of a first light receiving quantity, i.e., the quantity of light received in the predetermined period of time from the start of the irradiation period of the irradiation light, and a second light receiving quantity, i.e., the quantity of light received in the irradiation stop period, a distance image generating unit generates distance information indicating the distance to the subject. On the basis of the first and second light receiving quantities, the distance information generating unit determines whether a third light receiving quantity based on a light receiving quantity excluding the second light receiving quantity from the first light receiving quantity is equal to or more than a predetermined threshold value (SH). In the cases where the quantity of received third reflection light is less than the threshold value, the distance information generating unit does not calculate the distance, and in the cases where the quantity of the received third reflection light is equal to or more than the threshold value, the distance information generating unit calculates the distance.

Description

Distance sensor and user interface device

The present invention relates to a distance sensor for measuring a distance to an object and a user interface device including the distance sensor.

The distance sensor that measures the distance to the object generates a distance image by the TOF (Time-Of-Flight) method that measures the distance based on the propagation period of reflected light by irradiating the object including the object. There is a distance image sensor. In the range image sensor of the TOF method, it is known that noise that becomes an obstacle to detection of an object occurs in the generated range image. Patent Document 1 discloses a distance measuring device intended to remove a noise component included in a distance image.

In the distance measuring device of Patent Document 1, first, reflected light of light irradiated on a subject including a measurement object is received by a light receiving element, and a distance image representing a distance between the subject and a signal level is acquired. Next, the obtained distance image is divided into a number of predetermined areas, and when the fluctuation of the signal level indicating the distance in the divided area exceeds a predetermined threshold, the distance image of the area is regarded as a noise component and rejected. Only the region where the signal level fluctuation does not exceed the threshold value is left as a distance image. As a result, a distance image showing only the measurement object is obtained in the remaining area without being rejected, and the distance to the measurement object and its external shape can be measured with reduced influence of noise. .

JP 2002-277239 A

In Patent Document 1, the entire distance image is acquired once, and for each predetermined area obtained by dividing the acquired distance image, it is determined whether or not it is a noise component based on a change in signal level representing the distance in the predetermined area. Is called. For this reason, the noise component cannot be determined unless processing such as distance calculation for obtaining the entire distance image is performed, and the distance image of the area regarded as the noise component is eventually rejected. There is a problem that the processing until the noise in the image is removed is not efficient.

An object of the present invention is to provide a distance sensor that can efficiently reduce noise in distance information indicating a distance to an object.

The distance sensor according to the present invention includes a light source unit, a light receiving unit, and a distance information generating unit. The light source unit irradiates the object with irradiation light. The light receiving unit receives light during a predetermined time from the start of the irradiation period of the irradiation light, and receives light during the irradiation stop period. The distance image generation unit is a first received light amount that is the amount of light received during a predetermined time from the start of the irradiation period of the irradiation light, and a received light amount of light that is received during the irradiation stop period. Based on the second received light amount, distance information indicating the distance to the object is generated. The distance information generation unit determines whether the third received light amount based on the first received light amount and the second received light amount is equal to or greater than a predetermined threshold based on the first and second received light amounts. Determine whether or not. The distance information generation unit does not calculate the distance when the amount of received third reflected light is not greater than or equal to the threshold, and calculates the distance when the amount of received third reflected light is greater than or equal to the threshold. .

According to the distance sensor of the present invention, the noise is determined based on the third received light amount in consideration of the influence of outside light before the distance is calculated, and if it is regarded as noise, the distance is not calculated and the noise is determined. Since the distance is calculated later, noise in the distance information can be efficiently reduced.

The block diagram which shows the structure of the distance image sensor which concerns on Embodiment 1 of this invention. Perspective view showing appearance and assembled state of range image sensor Block diagram showing a configuration example of a sensor circuit in a distance image sensor Schematic diagram showing a configuration example of a pixel circuit in a sensor circuit Timing chart showing the operation timing of irradiation and light reception in the distance image sensor Schematic diagram for explaining the distance calculation method in the distance image sensor The figure which shows an example of three types of brightness | luminance images by a distance image sensor The figure which shows an example of the distance image based on three types of brightness | luminance images The flowchart which shows the distance image generation process in a distance image sensor The figure which shows the example of the distance image after the noise removal by a distance image sensor The figure for demonstrating an example of the setting method of the threshold value in a distance image sensor The figure for demonstrating the user interface apparatus which concerns on Embodiment 2. FIG. The block diagram which shows the structure of the user interface apparatus provided with the distance image sensor The figure which shows an example of the setting operation | movement of the threshold value of a distance image sensor Sequence diagram showing flow of threshold setting operation in user interface device The figure which shows the modification of the threshold value setting operation | movement of a distance image sensor

Hereinafter, a distance image sensor according to the present invention will be described with reference to the accompanying drawings.

Each embodiment is an exemplification, and it is needless to say that partial replacement or combination of configurations shown in different embodiments is possible. In the second and subsequent embodiments, description of matters common to the first embodiment is omitted, and only different points will be described. In particular, the same operational effects by the same configuration will not be sequentially described for each embodiment.

(Embodiment 1)
1. Configuration The configuration of the distance image sensor according to the first embodiment will be described with reference to FIGS. FIG. 1 is a block diagram illustrating a configuration of the distance image sensor according to the first embodiment. FIG. 2A is a perspective view showing an appearance of the distance image sensor. FIG. 2B is an exploded view showing the distance image sensor of FIG.

The distance image sensor 1 according to the present embodiment includes an LED (light emitting diode) 2, a sensor circuit 3, and a TOF signal processing unit 4 as shown in FIG. The distance image sensor 1 is a sensor device that measures a distance in the TOF method, and is an example of a distance sensor that generates a distance image as distance information indicating the distance to the object 5. The distance image sensor 1 is mounted on, for example, a mobile device or an information terminal, and outputs a distance image used for detecting a user's hand or the like as the object 5 on the host side. The distance image sensor 1 emits light from the LED 2, receives reflected light from the object 5 by the sensor circuit 3, and generates a distance image indicating the distance to the object 5 in the TOF signal processing unit 4.

As shown in FIGS. 2A and 2B, the distance image sensor 1 further includes a lens 11, a holder 12, and a circuit board 13.

The LED 2 is attached to the outer surface of the holder 12 as shown in FIG. The LED 2 emits light having a wavelength band in the infrared region (hereinafter referred to as “LED light”) toward the outside of the holder 12. The LED light is irradiated with pulse modulation under the control of the TOF signal processing unit 4. The LED 2 is an example of a light source unit that performs irradiation and stops irradiation using LED light as irradiation light.

The sensor circuit 3 is composed of a CMOS (complementary metal oxide semiconductor) image sensor circuit having a light receiving surface. As shown in FIG. 2B, the sensor circuit 3 is integrated on one semiconductor chip and attached to the circuit board 13 inside the holder 12. A lens 11 such as a barrel lens is attached to the outer surface of the holder 12 so as to cover the light receiving surface of the sensor circuit 3. The lens 11 condenses light from the outside of the holder 12 on the light receiving surface of the sensor circuit 3. The sensor circuit 3 is an example of a light receiving unit that receives light in synchronization with the irradiation of LED light. Details of the configuration of the sensor circuit 3 will be described later.

The TOF signal processing unit 4 is a circuit group that performs various signal processing for generating a distance image in the TOF method, and includes a timing generation unit 41, a distance calculation unit 42, and a distance image output unit 43. The TOF signal processing unit 4 is composed of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), and is integrated on the circuit board 13. The TOF signal processing unit 4 is an example of a distance information generation unit that generates a distance image based on the amount of light received by the sensor circuit 3 as distance information.

In the TOF signal processing unit 4, the timing generation unit 41 includes an oscillation circuit and the like, and generates a timing signal having a predetermined cycle. The timing generation unit 41 supplies the generated timing signal to the LED 2 as an irradiation control signal for pulse-modulating and emitting LED light. The timing generation unit 41 also supplies the generated timing signal to the sensor circuit 3 to synchronously control irradiation by the LED 2 and light reception by the sensor circuit 3. The operation timing of irradiation and light reception in the distance image sensor 1 will be described later.

The distance calculation unit 42 includes an arithmetic circuit capable of performing four arithmetic operations and an internal memory such as a flash memory. The distance calculation unit 42 calculates a distance based on the propagation period of the received reflected light based on the detection result of the reflected light by the sensor circuit 3. A method for calculating the distance will be described later. The distance calculation unit 42 calculates the distance for each pixel, and records, for example, distance data indicating the calculated distance for each pixel in the internal memory. A distance image is generated by calculating distance data for all pixels by the distance calculation unit 42.

The distance image output unit 43 includes an interface circuit that outputs information to an external device. The distance image output unit 43 outputs the distance image generated by the distance calculation unit 42 to an external device. The distance image output unit 43 may output the distance data for all the pixels recorded in the internal memory, or may output the distance data calculated by the distance calculation unit 42 at any time.

1-1. Configuration of Sensor Circuit Next, details of the configuration of the sensor circuit 3 will be described with reference to FIGS. FIG. 3 is a block diagram showing a configuration of the sensor circuit 3 in the distance image sensor 1. FIG. 4 is a schematic diagram illustrating a configuration of a pixel circuit in the sensor circuit 3.

As shown in FIG. 3, the sensor circuit 3 includes a plurality of pixel circuits 30 and peripheral circuits such as a gate drive circuit 31, a vertical scanning circuit 32, and a horizontal readout circuit 33. In the present embodiment, the sensor circuit 3 employs a charge distribution method.

The gate drive circuit 31 is a drive circuit for driving various MOS transistors included in the pixel circuit 30 based on a timing signal from the timing generator 41 (see FIG. 1). The gate driving circuit 31 sequentially outputs the first, second, and third gate signals Sg1, Sg2, and Sg3 to one pixel circuit 30. The first, second, and third gate signals Sg1, Sg2, and Sg3 are output to the plurality of pixel circuits 30 at the same timing.

The plurality of pixel circuits 30 are arranged in a matrix in the horizontal direction and the vertical direction on the light receiving surface. The plurality of pixel circuits 30 are an example of a plurality of pixels in the light receiving unit of the distance image sensor. FIG. 4A is a schematic diagram showing the pixel circuit 30 stacked on the semiconductor chip. FIG. 4B is a circuit diagram showing an equivalent circuit of FIG.

The pixel circuit 30 includes a photodiode PD and three floating diffusions FD1, FD2, and FD3, as shown in FIG. In the pixel circuit 30, a photodiode PD is provided in an embedded manner on a p-type semiconductor substrate, and three floating diffusions FD1, FD2, and FD3 are provided around the photodiode PD. Further, MOS transistors M1, M2, and M3 are formed from the region where the photodiode PD is provided to the floating diffusions FD1, FD2, and FD3, respectively.

In the three floating diffusions FD1, FD2, and FD3, capacitors C1, C2, and C3 are formed as shown in FIG. The three capacitors C1, C2, and C3 are connected to the photodiode PD through MOS transistors M1, M2, and M3, respectively. The MOS transistors M1, M2, and M3 are controlled to be opened / closed by ON / OFF of the first, second, and third gate signals Sg1, Sg2, and Sg3 inputted to the respective gates from the gate driving circuit 31.

The photodiode PD receives light from the outside and photoelectrically converts it. The electric charge generated by the photoelectric conversion is stored in any one of the three capacitors C1, C2, and C3 through the MOS transistor that is controlled to be open among the MOS transistors M1, M2, and M3. As described above, the capacitors C1, C2, and C3 accumulate charges corresponding to the amount of light received by the photodiode PD. The sensor circuit 3 acquires the received light amount by accumulating charges in the capacitors C1, C2, and C3 for each pixel circuit 30.

The amount of light received in the capacitors C1, C2, C3 between the pixel circuits 30 is read from the analog signal line when the pixel circuit 30 is selected by the selection signal Ss. The selection signal Ss is a signal for selecting a pixel circuit 30 to be read from the plurality of pixel circuits 30. Further, the capacitors C1, C2, and C3 are reset by releasing the accumulated charges when the reference voltage VR is applied by the reset signals Sr1, Sr2, and Sr3. The reset signals Sr1, Sr2, and Sr3 are input from the gate drive circuit 31, for example.

Referring back to FIG. 3, the vertical scanning circuit 32 is a circuit for vertically scanning the pixel circuits 30 arranged in a matrix when the amount of light received from the pixel circuit 30 is read. The vertical scanning circuit 32 sequentially outputs a selection signal Ss for each pixel circuit 30 arranged in a row.

The horizontal readout circuit 33 is a circuit for reading out the received light amount of the pixel circuit 30 scanned by the vertical scanning circuit 32 to the TOF signal processing unit 4. The horizontal readout circuit 33 includes a plurality of A / D (analog / digital) converters 35, and converts the received light amount of the analog value from the pixel circuit 30 into a digital value (A / D conversion). A plurality of A / D converters 35 are provided, for example, three for each pixel circuit 30 in each column, and A / D convert the received light amounts acquired by the capacitors C1, C2, and C3 of the pixel circuit 30, respectively. The digital value of the received light amount after A / D conversion is output to the distance calculation unit 42 (see FIG. 1) of the TOF signal processing unit 4.

2. Operation Next, the operation of the distance image sensor 1 according to the present embodiment will be described.

2-1. Distance Calculation Method First, a method for calculating the distance to the object by the distance image sensor 1 will be described with reference to FIGS. FIG. 5 is a timing chart showing the operation timing of irradiation and light reception in the distance image sensor 1. FIG. 5A shows the timing of an irradiation control signal for controlling the irradiation of LED light. FIG. 5B shows the arrival timing of the reflected light that reaches the distance image sensor 1 from the object. FIGS. 5C, 5D, and 5E show timings of the first, second, and third gate signals Sg1, Sg2, and Sg3 that are input to the pixel circuit 30, respectively. FIG. 6 is a schematic diagram for explaining a distance calculation method by the distance image sensor 1.

The irradiation control signal shown in FIG. 5A is supplied from the timing generator 41 to the LED 2 (see FIG. 1). Based on the irradiation control signal, LED light having a pulse waveform with a predetermined period Tp as a pulse width is irradiated from time t1. The period Tp is, for example, not less than 10 nanoseconds (ns) and not more than 20 nanoseconds. When the LED light is irradiated onto the object, reflected light is generated from the object. Reflected light from the object reaches the distance image sensor 1 with a delay from the time of irradiation of the LED light according to the distance to the distance image sensor 1.

5B, the reflected light from the object has a delay period Td with respect to the LED light at the time of irradiation, and reaches the distance image sensor 1 at time t2 after the delay period Td from time t1. The waveform of the reflected light has a pulse width of the same period Tp as that of the LED light. In the present embodiment, it is assumed that the delay period Td is less than the period Tp.

In the sensor circuit 3 of the present embodiment, external light such as background light is received during the stop period of LED light irradiation based on the gate signals Sg1 to Sg3 shown in FIGS. In addition, the reflected light from the object is received in a time-sharing manner in synchronization with the irradiation period of the LED light.

The gate signals Sg1, Sg2, and Sg3 are sequentially output from the gate drive circuit 31 of the sensor circuit 3 to the pixel circuit 30 on the light receiving surface in synchronization with the irradiation control signal (see FIG. 3). In the pixel circuit 30, the photodiode PD receives light based on the gate signals Sg1, Sg2, and Sg3, and charges corresponding to the received light amount are accumulated in the capacitors C1, C2, and C3 (see FIG. 4B). Note that photoelectrons generated in the photodiode PD in a period in which charges corresponding to the amount of received light are not accumulated in the capacitors C1, C2, and C3 are discharged to the outside.

The first gate signal Sg1 shown in FIG. 5C is turned on before the irradiation with the LED light and for the period Tp. While the first gate signal Sg1 is ON, a charge corresponding to the amount of light received by the photodiode PD is accumulated in the capacitor C1. FIG. 6A shows the received light quantity Q1 based on the first gate signal Sg1 accumulated in the capacitor C1. The received light amount Q1 is the received light amount of external light acquired in a state where no reflected light of the LED light is generated. The received light quantity Q1 is acquired in order to confirm the influence of external light that is not related to LED light such as background light.

The second gate signal Sg2 shown in FIG. 5 (d) is turned on from the time t1 when the LED light irradiation is started to the time t3 when the LED light irradiation is stopped. While the second gate signal Sg2 is ON, a charge corresponding to the amount of light received by the photodiode PD is accumulated in the capacitor C2. FIG. 6B shows the received light amount Q2 based on the second gate signal Sg2 accumulated in the capacitor C2. The amount of received light Q2 includes a reflected light component derived from reflected light that arrives within a period Tp from the time t1 of the start of LED light irradiation. The received light quantity Q2 also includes external light components such as background light.

The third gate signal Sg3 shown in FIG. 5 (e) is turned ON for a period Tp from time t3 after the stop of LED light irradiation. While the third gate signal Sg3 is ON, a charge corresponding to the amount of light received by the photodiode PD is accumulated in the capacitor C3. FIG. 6C shows the received light amount Q3 based on the third gate signal Sg3 accumulated in the capacitor C3. The amount of received light Q3 includes a reflected light component derived from reflected light that continues from time t3 when the LED light irradiation is stopped to time t4 after the delay period Td. In this way, the entire amount of reflected light received is time-divided and distributed as reflected light components of the amounts of received light Q2 and Q3 according to the delay period Td. Note that it is assumed that the delay period Td of the reflected light is less than the period Tp, and the time t4 when the arrival of the reflected light ends is within the range of the charging period Tp of the capacitor C3. The received light amount Q3 includes an external light component as in the received light amount Q2.

As described above, in the distance image sensor 1, the sensor circuit 3 receives light during a predetermined time from the start of the LED light irradiation period, and the received light amounts Q2, Q3 in the capacitors C2, C3 (first capacitor). Charges corresponding to (first received light amount) are accumulated. In addition, light is received during the stop period of the LED light irradiation, and a charge corresponding to the received light amount Q1 (second received light amount) is accumulated in the capacitor C1 (second capacitor). By detecting the charges accumulated in the capacitors C1, C2, and C3, the received light amounts Q1, Q2, and Q3 can be obtained. According to the received light amounts Q1, Q2, and Q3 corresponding to the charges accumulated in the capacitors C1, C2, and C3, the ratio between the delay period Td and the period Tp of the reflected light from the object is as shown in FIG. Furthermore, this corresponds to the proportion of the total amount of received reflected light that is distributed to the amount of received light Q3. Therefore, the delay period Td can be obtained based on the distribution of the received light amounts Q2 and Q3.

Here, as shown in FIGS. 6B and 6C, the received light amounts Q2 and Q3 include not only reflected light components but also external light components. Since the received light amounts Q2 and Q3 are acquired in the period Tp having the same length as the received light amount Q1 of only the external light, the external light component included in the received light amounts Q2 and Q3 is considered to be approximately the same as the received light amount Q1. It is done. Therefore, in the present embodiment, the received light amount of the reflected light component excluding the external light component is calculated by appropriately subtracting the received light amount Q1 from the received light amounts Q2 and Q3. Since the received light amount Q1 is acquired immediately before acquiring the received light amounts Q2 and Q3, the external light component in the received light amounts Q2 and Q3 can be accurately removed by the received light amount Q1.

The delay period Td is the time taken for the LED light to reach the object and return to the distance image sensor 1 as reflected light. That is, this is the time required when the distance between the object and the distance image sensor 1 reciprocates at the speed of light c. Therefore, if the distance to the object is L, Td = 2L / c is established, and the distance L to the object can be calculated by calculating the following equation.
L = (c / 2) × Tp × {(Q3-Q1) / (Q2 + Q3-2 × Q1)} (1)

2-2. Distance Image Generation Operation A distance image generation operation according to the present embodiment will be described. In the present embodiment, the distance calculation unit 42 of the distance image sensor 1 reads the received light amounts Q1, Q2, and Q3 (see FIG. 6) of each pixel circuit 30 from the sensor circuit 3, and receives the received light amounts Q1 and Q2 of each pixel circuit 30. , Q3, the above equation (1) is calculated.

Reading of the received light amounts Q1, Q2, and Q3 is performed after repeating the above-described series of operations of irradiating the pulsed LED light and receiving the reflected light for a predetermined number of times, for example, 10,000 times to 20,000 times. Thereby, the statistical accuracy of the received light amounts Q1, Q2, and Q3 used in the calculation of the above equation (1) is increased, and the distance can be calculated with high accuracy.

When the distance calculation unit 42 calculates the above expression (1) for one pixel, distance data indicating the distance of the pixel is acquired. The TOF signal processing unit 4 generates a distance image by acquiring distance data for all pixels.

2-2-1. Noise in Distance Image Here, in the TOF type distance image as described above, noise that is distance data indicating a distance different from the actual distance may occur. Hereinafter, noise in the distance image will be described with reference to FIGS. FIG. 7 is a diagram illustrating an example of three types of luminance images in the distance image sensor 1. FIG. 8 is a diagram illustrating an example of a distance image based on the three types of luminance images in FIG.

7 (a), (b), and (c) are luminance images having received light amounts Q1, Q2, and Q3 as luminances, respectively. 7A, 7B, and 7C, the mannequin hand 51 and the two

cylinders

52 and 53 are arranged in the detection area of the distance image sensor 1, and the irradiation of the LED light and the received light quantity Q1, Images were taken by receiving Q2 and Q3. In the depth direction of the drawing from the position where the distance image sensor 1 is installed, the mannequin hand 51 is located at a distance of 35 cm, and the

cylinders

52 and 53 arranged on the left and right of the hand 51 are located at a distance of 70 cm and 50 cm from the imaging position, respectively. . Further, the infrared reflectance of the cylinder 52 is higher than that of the cylinder 53.

Based on the luminance images of FIGS. 7A, 7B, and 7C, the distance image shown in FIG. 8 was obtained by performing the above equation (1) for all the pixels. The distance image in FIG. 8 is a shaded image that represents the distance. The distance image in FIG. 8 indicates that the hand 51 is the closest distance between the hand 51 and the left and

right cylinders

52 and 53, and is located farther in the order of the right cylinder 53 and the right cylinder 52. ing.

However, it can be seen that noise (distance data representing a distance different from the actual distance) is generated in the distance image of FIG. For example, the

areas

61 and 62 are areas far away from the hand 51 and the

cylinders

52 and 53, and the whole area should be displayed with a uniform darkness. Has been. Since the noise in FIG. 8 includes a distance similar to or closer to the hand 51, when detecting the hand 51 or the like as an object in an external device based on such a distance image, the object is detected. The processing amount of processing may increase or erroneous detection of an object may occur. Such noise in the distance image is generated in the calculation of the above equation (1) even though external light components such as background light, which is one of the main causes of noise, are removed.

Therefore, in the present embodiment, before calculating the distance, the net amount of received reflected light (third amount of received light) excluding the influence of external light from the acquired amount of received light is used (FIG. 7D). (Refer to the above), and make a decision to remove noise. Based on this determination, the distance is not calculated when the amount of net reflected light received is too small, and the distance is calculated only when the amount of net reflected light received is sufficiently large. Therefore, if it is considered that noise is likely to occur even if the distance is calculated based on the magnitude of the amount of net reflected light received after the influence of outside light, the distance calculation itself is not performed. Since the distance image is generated, noise in the distance image can be efficiently reduced.

2-2-2. Distance Image Generation Processing The distance image generation processing according to this embodiment will be described below with reference to FIGS. The distance image generation process is a process of generating a distance image after making a determination for removing noise based on the amount of received light acquired by the sensor circuit 3. FIG. 9 is a flowchart showing distance image generation processing in the distance image sensor 1. FIG. 10 is a diagram illustrating an example of a distance image after noise removal by the distance image sensor 1.

The processing described below is executed by the TOF signal processing unit 4 in the distance image sensor 1.

First, the TOF signal processing unit 4 reads three types of luminance images (see FIGS. 7A to 7C) based on the received light amounts Q1, Q2, and Q3 (see FIG. 6) from the sensor circuit 3 (S2).

Next, the TOF signal processing unit 4 selects one pixel in the read image (S4). Hereinafter, the horizontal position of the selected pixel is represented by x, and the vertical position of the selected pixel is represented by y (see FIG. 3).

Next, the distance calculation unit 42 of the TOF signal processing unit 4 removes the external light component corresponding to the received light amount Q1 from the total received light amount Q2, Q3 based on the received light amounts Q1, Q2, Q3 of the selected pixel. The received light amount I of the net reflected light is calculated (S6). Specifically, the distance calculation unit 42 calculates the following expression.
I (x, y) = Q2 (x, y) + Q3 (x, y) −2 × Q1 (x, y) (2)

In the above equation (2), the subscript (x, y) represents the position of the selected pixel. Q1 (x, y), Q2 (x, y), and Q3 (x, y) are the received light amounts Q1, Q2, and Q3 at the selected pixel, respectively, and I (x, y) is the net for the selected pixel. Is the amount of received light I of the reflected light. FIG. 7 (d) shows an image based on the received light amount I of the net reflected light with respect to the luminance images of FIGS. 7 (a) to 7 (c).

Next, the distance calculation unit 42 determines whether or not the calculated amount of received net reflected light I (x, y) is equal to or greater than a predetermined threshold value SH (S8). The threshold value SH is a threshold value for removing noise in the distance image based on the received light amount I of the net reflected light. A method for setting the threshold value SH will be described later.

When the calculated net received light amount I (x, y) of the reflected light is equal to or greater than the threshold value SH, the distance calculation unit 42 receives the received light amounts Q1 (x, y) and Q2 (x, y) of the selected pixel. , Q3 (x, y) to calculate the expression (1) and calculate the distance to the selected pixel (S10). Here, the denominator on the right side in Equation (1) coincides with the received light amount I (x, y) of the net reflected light. Therefore, the distance calculation unit 42 performs the calculation of Expression (1) using the value of the net reflected light reception amount I (x, y) calculated using Expression (2) in Step S6. Thereby, the calculation resource in the process of step S6 is not wasted and the process can be executed efficiently.

On the other hand, when the calculated amount of received net reflected light I (x, y) is not equal to or greater than the threshold value SH, the distance calculation unit 42 calculates the distance to the selected pixel without calculating the distance to the selected pixel. A predetermined distance is set (S12). The predetermined distance is a distance indicating a long distance such as infinity.

Next, the TOF signal processing unit 4 records the distance acquired in Step S10 or Step S12 in the internal memory as the distance data of the selected pixel (S14).

Next, the TOF signal processing unit 4 determines whether or not the distance data of all the pixels has been acquired (S16). When the distance data of all the pixels has not been acquired (NO in S16), the TOF signal processing unit 4 sequentially performs the processes from step S4 on the pixels for which the distance data has not been acquired.

When the TOF signal processing unit 4 acquires the distance data of all the pixels (YES in S16), the TOF signal processing unit 4 generates a distance image based on the acquired distance data of all the pixels (S18). The generated distance image is output from the distance image output unit 43 to an external device.

The above processing is repeatedly executed at a predetermined cycle such as 1/30 second.

According to the above distance image generation processing, the net reflected light reception amount I (x, y) is calculated for each pixel, and the net reflected light reception amount I (x, y) is less than the threshold value SH. The calculation of the distance is omitted for. On the other hand, the distance is calculated only for pixels whose net reflected light reception amount I (x, y) is equal to or greater than the threshold value SH. Thereby, even if the net reflected light reception amount I (x, y) is too small and the distance is calculated, the calculation of the distance is omitted for a pixel that is statistically prone to noise. Can be efficiently reduced.

In addition, when the amount of received net reflected light I (x, y) is small, the distance is often a long distance, and therefore a distance indicating a long distance, such as infinity, for a pixel from which the distance calculation is omitted. Set the data. As a result, the image quality of the distance image can be improved as compared with the case where the distance is sequentially calculated for all the pixels.

Further, in the determination process in step S6, determination for noise removal is performed by simple addition and subtraction as shown in the above equation (2). Therefore, it is possible to realize a noise removal determination function with a simple hardware configuration. Furthermore, since the value calculated in step S6 is used in the distance calculation in step S10, it is possible to efficiently perform the noise removal determination process without wasting calculation resources.

FIGS. 10A, 10B, and 10C show distance images generated by setting the threshold value SH to various values based on the luminance image shown in FIG. FIG. 10A shows a distance image when the threshold value SH = 30 is set. FIG. 10B shows a distance image when the threshold value SH = 100 is set. FIG. 10C shows a distance image when threshold value SH = 255 is set. In the above case, the settable range of the threshold value SH is a digital value range of 0 to 8190.

10A, most of the noise generated in the

areas

61 and 62 in FIG. 8 is removed, and the entire area is almost uniformly represented. As described above, by removing noise before the distance calculation using the threshold value SH, it is possible to suppress erroneous distance data from being mixed in the distance image and to improve the image quality of the distance image. it can. Moreover, the power consumption of the distance image sensor 1 can be reduced by setting the threshold value SH and limiting the pixels for calculating the distance.

In the distance image of FIG. 10B, the hand 51 (distance 35 cm from the distance image sensor 1) and the cylinder 52 (distance 70 cm) are shown as in FIG. 10A, whereas the distance of FIG. The cylinder 53 (distance 50 cm) that was shown in the image is no longer shown. The cylinder 53 is an object that is located at a shorter distance than the cylinder 52 but has a lower infrared reflectance than the cylinder 52. The amount of net reflected light I that is determined by the threshold SH varies depending on the distance of the object that reflects the LED light and the reflectance of the LED light. According to the distance image sensor 1, when a specific object is to be detected using the distance image, by adjusting the setting of the threshold value, the object that is not the object of detection is regarded as the background, and the distance image is displayed. It can be prevented from being reflected. Thereby, it is possible to easily detect the object in the external device.

In the distance image of FIG. 10C, the cylinder 52 is not shown, and only the hand 51 is shown. By setting the threshold value to a large value, the number of pixels for calculating the distance is reduced, and the power consumption of the distance image sensor 1 can be further reduced.

Further, in the distance image of FIG. 10C, the contour of the hand 51 is shown more clearly than the distance images of FIGS. 10A and 10B. The distance image in FIG. 10C is in the range of a distance of 10 cm or more and 40 cm or less, for example, because the

cylinders

52 and 53 that are not to be detected are not shown in the image and only the hand 51 is shown except for the background. When a hand is a detection target, the object and the background area can be easily separated. As described above, by setting the threshold value, space separation between the object and the background in the distance image is facilitated.

2-2-3. Threshold Value Setting Method Hereinafter, an example of a threshold value SH setting method in the distance image generation process will be described with reference to FIG. FIG. 11 is a diagram for explaining an example of a threshold value setting method in the distance image sensor 1.

The setting method described below is a method for setting a threshold value in advance before using the distance image sensor 1 for acquiring a distance image of an object. For example, it is performed at the time of manufacture and shipment of the distance image sensor 1 in a factory.

First, the object 54 is arranged at the maximum distance Lmax within the range of distances that are assumed to be detected using the distance image sensor 1, and LED light irradiation and light reception amounts Q1, Q2, and Q3 are received. . The object 54 is an object having the lowest reflectance among the infrared reflectances of objects that are supposed to be detected using the distance image sensor 1. The object 54 is arranged so as to cover the entire detection range of the distance image sensor 1.

Next, the above equation (2) is calculated for the received light amounts Q1, Q2, and Q3 of all received pixels, and the received light amount I of the net reflected light is obtained. This calculation is performed in the distance calculation unit 42 of the distance image sensor 1, for example. Note that the external device may read out the received light amounts Q1, Q2, and Q3 of all the pixels from the distance image sensor 1 and perform the above calculation on the external device.

Next, based on the acquired amount of received net reflected light, the threshold SH is set to be equal to or less than the value of the net reflected light amount I of all pixels. For example, the threshold value SH is set so as to coincide with the lowest value of the received light amount I of the net reflected light of all the acquired pixels.

With the above threshold value setting, even when the reflected light from the target is the weakest within the range where the target is supposed to be detected, the amount of received light I of the net reflected light from the target is reduced. It becomes the threshold value SH or more. For this reason, it is ensured that the pixel receiving the reflected light from the object is reflected in the distance image without considering it as a background, and a distance image that can easily detect the object can be acquired.

In the above threshold value setting method, the threshold value SH may be acquired by changing the irradiation intensity of the LED light and acquiring a plurality of patterns. Thereby, for example, the setting values of a plurality of patterns are stored in the internal memory, and the threshold value is changed together with the irradiation intensity of the LED light from the outside according to the use environment, so that the distance of the LED light irradiation operation is also adjusted. The power consumption of the entire image sensor 1 can be adjusted.

3. Summary As described above, the distance image sensor 1 according to the present embodiment includes the LED 2, the sensor circuit 3, and the TOF signal processing unit 4. The LED 2 irradiates the object 5 with LED light. The sensor circuit 3 receives light during a predetermined time from the start of the LED light irradiation period, and receives light during the irradiation stop period. The TOF signal processing unit 4 is based on the received light amounts Q2 and Q3 which are received light amounts during a predetermined time from the start of the LED light irradiation period, and the received light amount received during the irradiation stop period. A distance image indicating the distance to the object 5 is generated based on a certain amount of received light Q1. Based on the received light amounts Q1, Q2, and Q3, the TOF signal processing unit 4 determines that the received light amount I of the net reflected light based on the received light amount obtained by removing the received light amount Q1 from the received light amounts Q2 and Q3 is a predetermined threshold SH. It is determined whether it is above. The TOF signal processing unit 4 does not calculate the distance when the net reflected light reception amount I is not equal to or greater than the threshold value SH, but does not calculate the distance, and when the net reflected light reception amount I is equal to or greater than the threshold value SH. Is calculated.

According to the distance image sensor 1, noise is determined based on the amount of received light I of the net reflected light by subtracting the influence of external light such as background light. Noise is determined before the distance is calculated, and the calculation of the distance is omitted when the amount of net reflected light I is less than the threshold SH and is regarded as noise, so noise in the distance image is efficiently reduced. can do.

In the distance image sensor 1, the threshold value SH may be set based on the reflectance of the LED light on the object 5 and the detection range of the distance of the object 5. Thereby, it is ensured that the distance to the object 5 is calculated, and calculation of other distances can be omitted as appropriate, so that a distance image capable of efficiently detecting the object 5 can be obtained.

Further, in the distance image sensor 1, the sensor circuit 3 may include a plurality of pixel circuits 30 that respectively acquire the amount of external light received Q1 and the time-divided reflected light received amounts Q2, Q3. The TOF signal processing unit 4 may determine whether or not the received light amount I (x, y) of the net reflected light with respect to each pixel is greater than or equal to the threshold value SH. The TOF signal processing unit 4 calculates a distance with respect to a pixel whose net reflected light reception amount I (x, y) is equal to or greater than the threshold value SH, and the net reflected light reception amount I (x, y). A distance of a predetermined value may be set for pixels that are not equal to or greater than the threshold value SH.

As a result, the amount of net reflected light received I (x, y) is not greater than or equal to the threshold value SH, and for pixels that are prone to noise, a predetermined distance is set for each pixel sequentially. The image quality of the distance image can be improved as compared with the case where the distance is calculated.

In the distance image sensor 1, the TOF signal processing unit 4 calculates the received light amount I of the net reflected light, determines whether or not the calculated received light amount I of the reflected net light is equal to or greater than the threshold value SH. The distance may be calculated using the calculated net received light amount I of the reflected light. Thereby, since the distance is calculated using the received light amount I of the net reflected light calculated for the noise determination, it is possible to efficiently remove the noise without wasting calculation resources.

In the distance image sensor 1, the sensor circuit 3 may include a plurality of capacitors C2 and C3 that respectively store charges corresponding to the received light amounts Q2 and Q3 of the reflected light that are time-divided. Accordingly, the received light amounts Q2 and Q3 of the reflected light that are time-divided based on the charges accumulated in the capacitors C2 and C3 can be acquired.

In the distance image sensor 1, the sensor circuit 3 may include a capacitor C1 that accumulates electric charge corresponding to the amount of external light received Q1. Thereby, the received light quantity Q1 of external light can be acquired based on the electric charge accumulated in the capacitor C1.

(Embodiment 2)
In the second embodiment, an example of a user interface device that detects a user's hand or the like as an object using the distance image sensor 1 according to the first embodiment will be described. According to the present embodiment, by acquiring a distance image from which noise has been removed from the distance image sensor 1, the user interface device can efficiently detect an object such as a user's hand. Furthermore, in the user interface device, the threshold value SH for noise removal of the distance image sensor 1 is updated based on a user instruction, so that the object can be detected more efficiently according to the usage environment. Can be performed.

1. Configuration of User Interface Device The configuration of the user interface device according to the second embodiment will be described with reference to FIGS. FIG. 12 is a diagram for explaining the user interface device according to the second embodiment. FIG. 12A is an external view showing a user interface device mounted state by a user. FIG. 12B is a schematic diagram illustrating an operation state of the user interface device by the user. FIG. 13 is a block diagram illustrating a configuration of the user interface device.

As shown in FIG. 13, the user interface device 8 according to the present embodiment includes a distance image sensor 1 according to the first embodiment, a display unit 80, a control unit 81, a storage unit 82, an operation unit 83, And a communication unit 84. The user interface device 8 is a glasses-type wearable terminal. As shown in FIG. 12A, the user can turn his / her line of sight inside the display unit 80 by wearing the user interface device 8.

The display unit 80 is a transmissive display including a half mirror. The display unit 80 displays a predetermined image by projecting a virtual image via a half mirror, and allows the user to visually recognize the displayed image so as to overlap the user's visual field. Thereby, the user can visually recognize as if an operation member etc. exist in the space in front of eyes. The image displayed by the display unit 80 is an image showing operation members such as a switch, a button, a keyboard, a cursor, and an icon.

The image displayed by the display unit 80 is visually recognized in a region R1 within a range of, for example, 10 cm to 1 m in the depth direction of the line of sight when the user interface device 8 is mounted, as shown in FIG. In the user interface device 8 according to this embodiment, the distance image sensor 1 is used to detect a gesture that touches or pushes the image visually recognized by the user with the fingertip 55 of the hand in the region R1, and uses the gesture. Accept user operations.

The distance image sensor 1 is arranged in the user interface device 8 so that the distance from the extension line in the direction of the user's line of sight to the user interface device 8 can be acquired. The distance image sensor 1 generates a distance image using the hand including the fingertip 55 in the region R1 as an object, and outputs the distance image to the control unit 81 of the user interface device 8.

The control unit 81 is composed of, for example, a CPU and an MPU, and controls the operation of the entire user interface device 8. The control unit 81 implements various functions by executing a predetermined program. The control unit 81 may be realized by a hardware circuit (ASIC, FPGA, etc.) such as a dedicated electronic circuit or a reconfigurable electronic circuit.

The control unit 81 detects an object based on the distance image from the distance image sensor 1 and determines a user operation. Specifically, in the distance image input from the distance image sensor 1, the control unit 81 detects the fingertip 55 in the hand and the movement of the fingertip 55 with respect to the region where the object (user's hand) is shown. The detection process is performed. Here, in the distance image as shown in FIG. 8, for example, when the hand 51 is detected as an object, noise in the

areas

61 and 62 becomes an obstacle, and it is difficult to separate the object and the background area. On the other hand, in this embodiment, since the distance image (see FIG. 10) from which noise has been removed is output from the distance image sensor 1, the controller 81 can easily perform spatial separation of the object and the background area, It becomes easy to detect an object.

The storage unit 82 is a storage medium that stores parameters, data, and programs necessary for realizing various functions of the control unit 81. The storage unit 82 stores control programs executed by the control unit 81 and various types of data. ing. The storage unit 82 is configured by a ROM or a flash memory, for example.

The operation unit 83 is a device for a user to give an instruction to the user interface device 8, and is configured by providing a touch pad or the like on the side of the user interface device 8, for example.

The communication unit 84 is an interface circuit for performing information communication with external devices by wireless signals. The communication unit 84 performs wireless communication with an external device according to a communication method such as Wi-Fi, Bluetooth (registered trademark), 3G, or LTE.

2. Threshold Setting Operation Next, the threshold SH setting operation for removing noise of the distance image sensor 1 according to the present embodiment will be described with reference to FIGS. FIG. 14 is a diagram illustrating an example of threshold value setting operation in the user interface device 8. FIG. 15 is a sequence diagram showing the flow of the threshold setting operation.

First, the distance image sensor 1 in the user interface device 8 generates a distance image using a default threshold value SH (for example, SH = 0) (S22). The generated distance image is output to the control unit 81 of the user interface device 8.

The control unit 81 acquires a distance image from the distance image sensor 1 (S24), and displays the acquired distance image on the display unit 80. In the user interface device 8, for example, a distance image including noise as shown in FIG.

The control unit 81 receives a user instruction for noise removal on the acquired distance image (S26). For example, as shown in FIG. 14B, the user's instruction is performed by designating a region Ra that the user desires to remove as a noise in the distance image visually recognized by the user, as shown in FIG. Is called.

The control unit 81 designates a region Ra that is a reference for setting the threshold value SH in the distance image based on a user instruction, and notifies the distance image sensor 1 (S28). Hereinafter, the horizontal position of the pixels included in the region Ra is xa (xa1 ≦ xa ≦ xa2), and the vertical position is ya (ya1 ≦ ya ≦ ya2).

The TOF signal processing unit 4 of the distance image sensor 1 acquires the received light amount I (xa, ya) of the net reflected light of each pixel in the designated region Ra based on the notification from the control unit 81 of the user interface device 8. (S30). For the net reflected light reception amount I (xa, ya), for example, the net reflected light reception amount I of all the pixels is stored in the internal memory in advance in step S22, and in step S30, each pixel in the region Ra is stored. Acquired by reading the value.

Next, the TOF signal processing unit 4 receives the net reflected light reception amount I of all the pixels in the region Ra in which the threshold value SH is designated based on the acquired net reflected light reception amount I (xa, ya). The threshold value SH is updated so as to be larger than (xa1, ya1),..., I (xa2, ya2) (S32). The threshold value SH is updated, for example, by comparing the net received light amounts I (xa1, ya1),..., I (xa2, ya2) of all the pixels with the threshold value SH. When the received light amount I (xa, ya) of the net reflected light is present, the threshold value SH is rewritten so as to increase sequentially. The updated threshold value SH is recorded in the internal memory of the TOF signal processing unit 4.

Through the above threshold setting operation, the net received light amounts I (xa1, ya1),..., I (xa2, ya2) of all the pixels in the region Ra designated by the user are updated. It is less than the value SH. For this reason, in the distance image generation process based on the updated threshold value SH, as shown in FIG. 14C, the distance data in the region Ra is removed as noise, and the distance of the image quality (accuracy) desired by the user. Images can be acquired.

In the threshold setting operation described above, the user's instruction is performed by designating a region Ra that is desired to be removed as noise in the distance image. It may be done by specifying. This case will be described with reference to FIG. FIG. 16 is a diagram showing a modification of the threshold setting operation.

In the example shown in FIG. 16, in step S <b> 26 of FIG. 15, instead of the region Ra shown in FIG. 14B, the distance image shown in FIG. The region Rb overlapping the hand 51 is designated. The region Rb is designated as a region where the user desires to acquire distance data without removing it as noise. In this case, the TOF signal processing unit 4 of the distance image sensor 1 receives the net reflected light amount I (xb, yb) (xb1 ≦ xb ≦ xb2) of each pixel in the designated region Rb in step S32 of FIG. , Yb1 ≦ yb ≦ yb2), the threshold value SH is updated so as to be less than or equal to the net received light amount I (xb1, yb1),..., I (xb2, yb2) of all pixels.

Thereby, the amount of received light I (xb1, yb1),..., I (xb2, yb2) of the net reflected light of all the pixels in the region Rb designated by the user becomes equal to or greater than the updated threshold value SH. For this reason, in the updated distance image generation process, as shown in FIG. 16C, it is guaranteed that the distance data in the region Rb is not removed as noise. On the other hand, noise is removed from a region where the amount of net reflected light I received is smaller than that of the region Rb. In this way, the image quality of the distance image can be changed according to the user's request, and a more easily handled distance image can be acquired according to the use environment.

In the above threshold value setting operation, the user's instruction is performed by designating an area in the distance image, but may be performed by designating one point (one pixel) in the distance image. . 14 and 16, the areas Ra and Rb according to the user's instruction are illustrated as rectangles. However, the areas are not limited to rectangles, and the areas may be specified in any shape.

3. Summary As described above, the user interface device 8 according to the present embodiment includes the distance image sensor 1 and the control unit 81. The control unit 81 detects an object based on the distance image generated by the distance image sensor 1 and accepts a user operation. According to the user interface device 8, a distance image from which noise that may be an obstacle to detection of an object has already been removed is acquired from the distance image sensor 1, so that the control unit 81 of the user interface device 8 efficiently detects the object. Can do well.

In the user interface device 8, the control unit 81 may set a threshold value SH for the distance image sensor 1 based on a user instruction. Thereby, the image quality of the distance image can be changed according to the use environment, and the object can be detected more efficiently.

(Other embodiments)
In each of the embodiments described above, the pixel circuit 30 includes the three capacitors C1, C2, and C3, and the case where the three kinds of received light amounts Q1, Q2, and Q3 are acquired in a time division manner has been described. However, the pixel circuit of the distance image sensor The number of capacitors included in is not limited to three. For example, the pixel circuit may include four or more capacitors, and four or more received light amounts Q1, Q2,..., Qn (n is an integer of 4 or more) may be acquired in a time division manner. In this case, the determination process in step S8 of FIG. 9 is performed based on the following equation.
Q2 + Q3 + ... + Qn− (n−1) × Q1 ≧ SH (3)

In the above equation (3), the left side indicates the amount of net reflected light received with respect to four or more received light amounts Q1, Q2,. Here, similarly to the case shown in FIGS. 5 and 6, the amount of received light Q1 is the amount of external light received by the external light dedicated capacitor during the period Tp during which no LED light is irradiated and no reflected light is generated. It is. The remaining light receiving amounts Q2, Q3,..., Qn are sequentially received in separate capacitors in synchronization with the irradiation of the LED light by the same gate signals as the second gate signal Sg2 and the third gate signal Sg3 shown in FIG. This is the amount of received reflected light. In the distance image generation process using the above equation (3), the distance calculation is not performed for the pixels for which the above equation (3) is not satisfied, and the distance calculation is performed only for the pixels for which the above equation (3) is satisfied. I do. Thereby, the noise in a distance image can be reduced efficiently.

Further, the number of capacitors provided in the pixel circuit of the distance image sensor may be two. In this case, the external light dedicated capacitor is omitted, and the amount of received external light and the amount of reflected light are acquired in different frames in reading from the pixel circuit. For example, in two consecutive frames, pulsed LED light irradiation is repeated in one frame, and electric charge corresponding to the amount of reflected light received is accumulated by time division with two capacitors, and stored in the TOF signal processing unit 4. Reads and receives the amount of reflected light. In the other frame, the amount of external light received can be acquired by receiving light while stopping the irradiation of LED light.

Further, even when the number of capacitors included in the pixel circuit of the distance image sensor is three or more, as in the case of two, the external light dedicated capacitor is omitted, and the received light amount of the external light and the received light amount of the reflected light And may be acquired in different frames.

In each of the above embodiments, in step S4 of the distance image generation process (FIG. 9), the TOF signal processing unit 4 selects one pixel in the image read from the sensor circuit 3, and performs the processes after step S4. went. The pixel selection method is not limited to this. For example, pixels may be read out from the sensor circuit 3 pixel by pixel, and the processing from step S4 onward may be sequentially performed from the read out pixels. Reading from the sensor circuit 3 may be performed pixel by pixel in a predetermined order, or pixel by pixel may be randomly read.

In each of the above embodiments, in the distance image generation process, distance data is recorded in the internal memory for each pixel (S14), distance data for all images is acquired (S16), and the distance image generated thereafter is acquired. Was output to an external device. However, the distance data for all images need not be output to the external device after being recorded in the internal memory. For example, the distance acquired in step S10 or step S12 is sequentially output to the external device as the distance data of a specific pixel. May be.

In the second embodiment, the case where the user interface device 8 including the distance image sensor 1 is a glasses-type wearable terminal is illustrated, but the user interface device including the distance image sensor 1 is not limited thereto. For example, it may be another wearable terminal such as a watch, or may be a PC (personal computer), a tablet terminal, a digital camera, a smartphone, a mobile phone, or the like.

Further, the device on which the distance image sensor 1 is mounted is not limited to the user interface device, and may be a monitoring camera or an in-vehicle device, for example. Even in such a case, it is possible to efficiently detect an object such as a person or a car by removing noise in the distance image.

1 Distance image sensor 2 LED
DESCRIPTION OF SYMBOLS 3 Sensor circuit 30 Pixel circuit 4 TOF signal processing part 41 Timing generation part 42 Distance calculation part 43 Distance image output part 8 User interface apparatus PD Photodiode FD1, FD2, FD3 Floating diffusion C1, C2, C3 Capacitor

Claims

A light source unit for irradiating the object with irradiation light;
A light receiving unit that receives light during a predetermined time from the start of the irradiation period of the irradiation light, and receives light during the irradiation stop period;
A first received light amount that is the amount of light received during a predetermined time from the start of the irradiation period of the irradiated light, and a second received light amount that is the amount of light received during the irradiation stop period. A distance information generating unit that generates distance information indicating the distance to the object based on the quantity;
The distance information generation unit
Based on the first and second received light amounts, whether or not a third received light amount based on the received light amount obtained by subtracting the second received light amount from the first received light amount is equal to or greater than a predetermined threshold value. Judgment,
If the third received light amount is not greater than or equal to the threshold value, the distance is not calculated,
A distance sensor that calculates the distance when the third received light amount is equal to or greater than the threshold value.
The distance sensor according to claim 1, wherein the threshold value is set based on a reflectance of the irradiation light on the object and a detection range of a distance of the object.
The light receiving unit includes a plurality of pixels for obtaining first and second received light amounts, respectively.
The distance information generation unit
Determining whether the third amount of received light for each pixel is equal to or greater than the threshold;
Calculating the distance with respect to a pixel having the third received light amount equal to or greater than the threshold;
The distance sensor according to claim 1 or 2, wherein a distance of a predetermined value is set for a pixel in which the third received light amount is not equal to or greater than the threshold value.
The distance information generation unit
Calculating the third amount of received light;
Determining whether the calculated third received light amount is equal to or greater than the threshold value;
The distance sensor according to any one of claims 1 to 3, wherein the distance is calculated using the calculated third received light amount.
The light receiving unit is
Receiving light in a time-sharing manner for a predetermined time from the start of the irradiation period of the irradiation light,
The distance sensor according to any one of claims 1 to 4, further comprising a plurality of first capacitors that respectively store electric charges corresponding to the first received light amount divided in time.
The distance sensor according to any one of claims 1 to 5, wherein the light receiving unit includes a second capacitor that accumulates electric charge corresponding to the second amount of received light.
The distance sensor according to any one of claims 1 to 6,
A user interface device comprising: a control unit that detects the object based on distance information generated by the distance sensor and receives a user operation.
The user interface device according to claim 7, wherein the control unit sets the threshold value for the distance sensor based on a user instruction.