WO2021149360A1

WO2021149360A1 - Distance measurement device, distance measurement method, and program

Info

Publication number: WO2021149360A1
Application number: PCT/JP2020/044529
Authority: WO
Inventors: 大木　光晴
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2020-01-20
Filing date: 2020-11-30
Publication date: 2021-07-29
Also published as: JP2021113743A

Abstract

The purpose of this invention is to reduce distance measurement device processing load.　This distance measurement device comprises: a detection unit for repeatedly acquiring distance indexes using a direct ToF method; a histogram generation unit for using the distance indexes repeatedly obtained by the detection unit in each frame period to generate, for each frame period, a distance histogram representing the distance indexes as frequency information for each distance; and a control unit for using the distance histogram generated for a past frame period to set a control parameter for distance histogram generation for frame periods from the current frame period onward.

Description

Distance measuring device, distance measuring method, program

This technology relates to a distance measuring device that measures a distance by a direct ToF (Time of Flight) method, a distance measuring method, and a program.

Various distance measurement technologies for measuring the distance to an object are known, and in recent years, for example, distance measurement technology using the direct ToF method has attracted attention.

In the direct ToF method, the results of multiple distance measurements are treated as frequency information for each distance. Specifically, it is treated as frequency distribution information (histogram information) with the distance as a class, such as the frequency at which the distance A is measured, the frequency at which the distance B is measured, the frequency at which the distance C is measured, and so on. It is a thing. Hereinafter, the frequency distribution information in which the results of a plurality of distance measurements are expressed as frequency information for each distance is referred to as a “distance histogram”.

By using the distance histogram, even if multiple objects overlap in the depth direction, it can be expected that frequency peaks will appear at different distances on the histogram, and it is possible to obtain the distance of each object that overlaps in the depth direction. Therefore, accurate distance measurement can be realized.

Here, it is always desired to improve the efficiency of control in distance measurement. For example, in Patent Document 1 below, when performing distance measurement in the direct ToF method, the light emitting unit is compared with the peak value of the generated histogram and other values. A technique for determining the number of exposures is disclosed.

Japanese Unexamined Patent Publication No. 2010-091377

When the above-mentioned distance histogram is used in the direct ToF method, it may not be necessary to generate a distance histogram for distance measurement in a region that is not targeted for distance measurement, such as a background.
Further, when it is highly probable that the distance of the object does not change, for example, when the object to be distance-measured is in a stationary state, there is often no problem even if the distance-measurement is not performed and the distance histogram is not generated.

This technology was created in view of this situation, and aims to reduce the processing load in distance measurement.

The distance measuring device according to the present technology includes a detection unit that acquires the distance index by the direct ToF method a plurality of times based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period, and the frame. Based on the distance index obtained a plurality of times by the detection unit for each period, a histogram generation unit that generates a distance histogram representing the distance index as frequency information for each distance for each frame period and a histogram generation unit generated in the past frame period are generated. Based on the distance histogram, the control unit for setting the control parameters related to the generation of the distance histogram in the frame period after the present is provided.
As a result, control related to distance measurement from the present onward is executed based on the control parameters that reflect the situation that can be read from the distance histogram generated in the past frame period.
The distance index referred to here is a numerical value that correlates with the distance, and examples thereof include a time difference from light emission to light reception, and a value obtained by multiplying the time difference by c / 2 (c represents the speed of light).

In the distance measuring device according to the present technology described above, the light receiving unit that obtains the light receiving signal has a plurality of light receiving elements, and the control unit is based on the distance histogram generated for each light receiving element in the past frame period. Therefore, it is conceivable to set the control parameters related to the generation of the distance histogram in the frame period after the present.
As a result, control related to distance measurement from the present onward is executed for each light receiving element based on the control parameters that reflect the situation for each light receiving element that can be read from the distance histogram generated in the past frame period.

In the distance measuring device according to the present technology described above, it is conceivable that the control unit sets the class range to be the target of the frequency count in the distance histogram as the setting of the control parameters.
As a result, the distance histogram is generated only for a part of the class range considered to be necessary for performing distance measurement from the present onward, depending on the situation that can be read from the distance histogram generated in the past.

In the distance measuring device according to the present technology described above, it is conceivable that the control unit sets a predetermined class range including the class of the peak value of the distance histogram in the past as the class range to be the target of the frequency count. ..
The peak value is a value estimated to be the distance to the object. Therefore, according to the above configuration, only a predetermined class range including the distance of the object in the past frame period such as the immediately preceding frame period is set as the class range to be counted by the frequency in the distance histogram after the present. Is possible.

In the distance measuring device according to the present technology described above, it is conceivable that the control unit sets the number of exposures in the frame period as the setting of the control parameters.
This makes it possible to set the number of exposures in a frame period suitable for the surrounding environment when performing distance measurement.

In the distance measuring device according to the present technology described above, the control unit continuously performs processing with the same control parameters from the present onward based on the distance histogram generated in the past frame period. It is conceivable to set the adaptive control number as the number of times.
As a result, in response to the case where it is estimated that it is not necessary to sequentially set the control parameters for each frame period from the tendency of the past distance histogram, the same control parameters can be set in the frame period within the adaptive control number of times after the present. It becomes possible to use.

In the distance measuring device according to the present technology described above, it is conceivable that the control unit sets the adaptive control number based on the amount of change of the distance histogram generated in the past continuous frame period.
The speed of movement of the object can be estimated from the amount of change in the distance histogram over the past consecutive frame periods. Therefore, according to the above configuration, the adaptive control number can be set according to the estimated speed of movement of the object.

In the distance measuring device according to the present technology described above, it is conceivable that the control unit sets a larger number of adaptive controls as the amount of change is smaller.
As a result, for an object in which the amount of change in the distance histogram is small in the past consecutive frame periods, the frame period in which the same control parameter is used after the present increases.

In the distance measuring device according to the present technology described above, whether the control unit generates the distance histogram in the frame period after the present based on the amount of change in the distance histogram generated in the past consecutive frame periods. It is possible to decide whether or not.
This makes it possible to determine whether or not to generate the distance histogram in the frame period after the present according to the movement of the object in the past.

In the distance measuring device according to the present technology described above, whether the control unit emits light in the frame period after the present based on the amount of change in the distance histogram generated in the past consecutive frame periods. It is possible to decide whether or not.
This makes it possible to prevent the light emitting unit from emitting light in response to the case where the distance histogram is not generated in the frame period after the present.

In the distance measuring device according to the present technology described above, the light receiving unit that obtains the light receiving signal has a plurality of light receiving elements, and the control unit uses the reflected light from the same object as one of the control parameters. It is conceivable to generate segment information indicating a group of light receiving elements that have received light, and to add and average the distance histograms of the light receiving elements in the group of light receiving elements indicated by the segment information.
As a result, noise reduction between the distance histograms generated in the light receiving elements belonging to the same segment is performed.

The distance measurement method according to the present technology includes a detection unit that acquires the distance index by the direct ToF method a plurality of times based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period, and the frame. Distance of a distance measuring device including a histogram generating unit that generates a distance histogram representing the distance index as frequency information for each distance based on the distance index obtained a plurality of times by the detecting unit for each period. This is a measurement method for setting control parameters related to the generation of the distance histogram in the current and subsequent frame periods based on the distance histogram generated in the past frame period.
Even with such a distance measuring method, the same operation as that of the distance measuring device according to the present technology can be obtained.

It is explanatory drawing of the structure of the distance measuring apparatus in embodiment of this technique. It is explanatory drawing of the distance histogram in embodiment of this technique. It is a figure which shows the control state of the distance measurement for every frame period in the 1st Embodiment of this technique. It is a figure which showed the distance histogram in the 1st Embodiment of this technique as 3D data in (x, y, n) space. It is explanatory drawing about input / output data to a discriminator part in 1st Embodiment of this technique. It is explanatory drawing about the setting of the class range which generates the distance histogram in the 1st Embodiment of this technique. It is a flowchart which shows the processing of the control part in the 1st Embodiment of this technique. It is a flowchart which shows the processing of the control part in the 1st Embodiment of this technique. It is a figure which shows the object in the 1st Embodiment of this technique. It is explanatory drawing about input / output data to a discriminator part in 2nd Embodiment of this technique. It is a flowchart which shows the processing of the control part in the 2nd Embodiment of this technique. It is explanatory drawing about input / output data to a discriminator part in 3rd Embodiment of this technique. It is a flowchart which shows the processing of the control part in the 3rd Embodiment of this technique.

An embodiment of the present technology will be described with reference to the above drawings. In the description of the drawings, the configuration once described may be referred to with the same reference numerals and the description thereof may be omitted. In addition, the drawings are schematic, and the configuration of the main parts and their surroundings necessary for explaining the present technology shall be extracted and shown.

Hereinafter, embodiments according to the present technology will be described in the following order.
<1. Configuration of distance measuring device>
<2. Overview of distance histogram>
<3. First Embodiment>
<4. Second Embodiment>
<5. Third Embodiment>
<6. Summary and variant examples>
<7. This technology>

<1. Configuration of distance measuring device>
FIG. 1 is a diagram showing an example of a configuration of a distance measuring device 10 according to an embodiment of the present technology.
The distance measuring device 10 supports distance measurement by a direct ToF (Time of Flight) method. The direct ToF method referred to here is a method in which the time difference from light emission to light reception is measured and the distance to the object is obtained based on the measured time difference.

The distance measuring device 10 includes a light emitting unit 12, a light receiving unit 13, a detection unit 14, a histogram generation unit 15, a distance measuring unit 16, a control unit 11, an output terminal 17, a detection mode input terminal 18, and the like. It includes an image input terminal 19.

The light emitting unit 12 has at least one light emitting element, and emits pulse light under the control of the control unit 11. The light emitting unit 12 in this example has, for example, a light emitting element by a VCSEL (Vertical Cavity Surface Emitting LASER), and these light emitting elements are arranged in a two-dimensional array, for example.
The light emitting unit 12 emits light several thousand to tens of thousands of times during the frame period. The number of times the light emitting unit 12 emits light during the frame period will be described below as the number of exposures. Let the number of exposures be M (M is an integer). The frame period refers to a period set as a measurement period of data used by the distance measuring device 10 for performing distance measurement, and is continuously provided with the passage of time.

The light receiving unit 13 has a plurality of light receiving elements in which the light 40 emitted from the light emitting unit 12 acquires the light 41 reflected by the object Tg as a light receiving signal. The light receiving unit 13 has a SPAD (Single Photon Avalanche Diode) element as a light receiving element.
The light receiving unit 13 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor in which SPAD elements are arranged in a two-dimensional array. A lens (not shown) is attached to the front surface of the light receiving unit 13, and the light 41 is focused by this lens so that each SPAD element can efficiently receive light.

The light 40 emitted from the light emitting unit 12 reaches the object Tg as active light, the active light is reflected by the object Tg, and is received as light 41 by one SPAD element in the light receiving unit 13.

The light 40 emitted from the light emitting unit 12 is always reflected by the object Tg and is not received by the light receiving unit 13 as the light 41. In some cases, the light 41 is stochastically received, or the light is received. It may not be done.
Further, apart from the light 40 emitted from the light emitting unit 12, the light 42 from the sun 400 also reaches the object Tg as ambient light. Then, the light 42 is reflected by the object Tg and reaches one SPAD element in the light receiving unit 13 as the light 43.

The detection unit 14 acquires the distance index by the direct ToF method a plurality of times based on the received signal of the light 41 that the light emitting unit 12 emits light and is reflected by the object Tg for each continuous frame period. The distance index referred to here is a numerical value that correlates with the distance, and examples thereof include a time difference from light emission to light reception, and a value obtained by multiplying the time difference by c / 2 (c represents the speed of light).
The detection unit 14 measures the time rt from the light emission timing of the light emitting unit 12 to the light reception timing by the SPAD element for each SPAD element of the light receiving unit 13 under the control of the control unit 11. The detection unit 14 measures the time rt M times, which is the number of exposures of the light emitting unit 12 for each frame period. In this example, the detection unit 14 acquires this time rt as a distance index.
The detection unit 14 is composed of, for example, a TDC (Time To Digital Converter). Then, in order to measure the time for each SPAD element, the TDC generally has the same number as the SPAD element.
The detection unit 14 cannot determine whether the light received by the SPAD element is the light 41 based on the light 40 from the light emitting unit 12 or the light 43 based on the light 42 from the sun 400.

The histogram generation unit 15 has a memory (not shown), and based on the measurement result of the time rt of a plurality of times (M times) for each frame period acquired from the detection unit 14, the measurement result of the plurality of times is the frequency information for each bin. The distance histogram expressed as is generated in the memory.
Here, the bin is each class in the frequency distribution information as such a distance histogram. The distance histogram is frequency distribution information in which the class is bin (distance) and the frequency for each class is frequency.
The histogram generation unit 15 increments the frequency in the bin of the distance histogram corresponding to the value of the time rt acquired from the detection unit 14 by 1. This updates the distance histogram. The histogram generation unit 15 generates one distance histogram for each SPAD element. The details of the distance histogram will be described later.

The distance measuring unit 16 calculates the distance to the object Tg based on the distance histogram generated by the histogram generating unit 15. The distance measuring unit 16 calculates the distance by multiplying the process of detecting the peak value from the distance histogram and the time corresponding to the peak value by c / 2 (hereinafter, c is the value of the speed of light). Perform processing. The peak value referred to here is a value estimated as a value corresponding to the distance to the object Tg in the distance histogram.

Each SPAD element in the light receiving unit 13 is a sensor that detects the first incoming light (photon). Therefore, if the light 41 based on the light 40 emitted from the light emitting unit 12 arrives earlier than the light 43 based on the light 42 from the sun 400, the distance measurement can be performed correctly. That is, the time rt detected by the detection unit 14 is the round-trip time to the object Tg, and the distance to the object Tg can be calculated by multiplying by c / 2. On the other hand, when the light 43 arrives earlier than the light 41 in time, the correct time for distance measurement cannot be measured.

The light reception of the light 43 (reception of ambient light) always occurs with a certain probability regardless of the time. On the other hand, the light reception (light reception of active light) of the light 41 is concentrated at a certain time (more specifically, a value obtained by dividing twice the distance to the object Tg by c). Therefore, the distance to the object Tg can be obtained by detecting the peak on the distance histogram and obtaining the time corresponding to the peak value.
A distance measuring method for measuring the distance to the object Tg based on such time is called a direct ToF method.

The distance measuring unit 16 outputs the distance data to the object Tg obtained by the processing to an external device or the like via the output terminal 17.
Further, in calculating the distance data to the object Tg, the distance measuring unit 16 displays a distance histogram of the SPAD element in the SPAD element group for each SPAD element group composed of a plurality of SPAD elements specified by the control of the control unit 11. Add and average.

The control unit 11 is composed of a microcomputer equipped with, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and is based on various programs based on the discrimination unit 21 and the operation control unit. The process for realizing the various functions of 22 is executed.

The discrimination unit 21 sets the control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated for each SPAD element in the past frame period. The details of the control parameters will be described later.

In the present embodiment, the RGB camera 200 is used in combination so that the image data captured by the RGB camera 200 can be input to the discrimination unit 21 from the image input terminal 19, but the RGB camera 200 and the image input terminal The present technology can be realized even if 19 is not provided.
Further, information for designating an object Tg to be measured, such as a person, a face, a hand, or a car, is input to the discrimination unit 21 via the detection mode input terminal 18.

The operation control unit 22 controls the light emitting unit 12, the light receiving unit 13, the detection unit 14, the histogram generation unit 15, and the distance measuring unit 16 based on the control parameters set by the discrimination unit 21.

<2. Overview of distance histogram>
FIG. 2 is a diagram showing an example of a distance histogram generated by the histogram generation unit 15 in the embodiment of the present technology.

In this distance histogram, time (horizontal axis) is expressed in units of width D. That is, if the value of the time rt detected by the detection unit 14 is in the range of time 0 to time D, it is added to the frequency of the 0th bin, and the value of time rt is in the range of time D to time 2D. If there is, it is added to the frequency of the first bin. If the value of time rt is in the range of time 2D to time 3D, it is added to the frequency of the second bin. After that, if it is in the range from time (N-1) × D to time N × D, it is added to the frequency of the N-1th bin (N is a constant). Here, the time width D is the resolution of the TDC as the detection unit 14.

The measurement time is limited to N × D in one measurement by the detection unit 14. That is, if no light reception is detected by the light receiving unit 13 even after N × D has elapsed from the light emission by the light emitting unit 12, the detection unit 14 ends the measurement there. In this case, the detection unit 14 does not output the time rt value, and the histogram generation unit 15 does not update the distance histogram.
Therefore, the total frequency of all bins may be less than M for the final distance histogram obtained after the measurement of M times (the number of exposures of the light emitting unit 12 for each frame period) by the detection unit 14. ..

It is assumed that a SPAD element is arranged at each position (x, y) of the light receiving unit 13. Here, x = 1 to X, y = 1 to Y, and the total number of SPAD elements in the light receiving unit 13 is XY. As described above, the distance measurement is performed by light emission and light reception M times, and the distance histogram is generated by the histogram generation unit 15. That is, a distance histogram is generated for each SPAD element corresponding to each position (x, y) by M times of light emission and light reception. Therefore, the total number of distance histograms is XY.

Furthermore, it is assumed that distance measurement is performed continuously in time. That is, the process of "creating a histogram with M times of light emission and light reception" is continuously performed for each frame period. In the present embodiment, the frame period is set to the time required for the process of "creating a histogram with M times of light emission and light reception". By defining the frame period in this way, at time t, the distance histogram created in the t-th frame period in time is output by the histogram generation unit 15. Here, t is an integer of 1 or more.

Let the distance histogram at time t and position (x, y) be H (t, x, y, n). Here, t is an integer of 1 or more, x is an integer of 1 or more and X or less, and y is an integer of 1 or more and Y or less. And n represents a bin number, which is an integer of 0 or more and N-1 or less.

In the present technology, the control unit 11 determines the situation at the present time (time t) from the histogram H (t, x, y, n) at time t (tth frame period), and the distance histogram after time t + 1. Set the control parameters related to the generation of.

<3. First Embodiment>
A first embodiment of the present technology will be described with reference to FIGS. 3 to 8.
In the first embodiment, as an example, when the remainder obtained by dividing the time t by 3 is 1, the normal distance measurement operation control is executed by the control from the control unit 11, and at the same time, the distance histogram at the time t is analyzed. By doing so, the control parameters related to the generation of the distance histogram at the time t + 1 and the time t + 2 are set. Then, at time t + 1 and time t + 2, distance measurement operation control based on the control parameters set at time t is executed by control from the control unit 11.
Hereinafter, the distance measurement operation control based on the control parameters will be described as adaptive control, and the normal distance measurement operation control not based on the control parameters will be described as normal control.

FIG. 3 shows a control state in distance measurement for each frame period (time t). In FIG. 3, the time t (horizontal axis) is continuously provided as t1, t2, t3, ... With the passage of time for each frame period.
Between time 0 and time t1, distance measurement is performed so that the number of exposures becomes a predetermined number of times (referred to as M1) (“normal distance measurement” in the figure). As a result, distance measurement is performed at time t = 1. At the same time, by analyzing the histogram H (1, x, y, n), control parameters related to the generation of the distance histogram for efficient distance measurement operation control (adaptive control) at time t + 1 and t + 2 are set. (Here, the time t is 1). A specific example of setting control parameters will be described later.

Between time t1 and time t2, a distance histogram is generated based on the control parameter set at time t = 1, and distance measurement is performed based on the distance histogram (“distance measurement under adaptive control” in the figure. "). As a result, distance measurement is performed at time t = 2.

Between time t2 and time t3, a distance histogram is generated based on the control parameter set at time t = 1, and distance measurement is performed based on the distance histogram (“distance measurement under adaptive control” in the figure. "). As a result, distance measurement is performed at time t = 3.

Between time t3 and time t4, the remainder obtained by dividing time t (t = 4) by 3 is 1, so normal distance measurement is performed. In this way, after time t4, as shown in FIG. 3, the same processing as described above is repeated.

Next, the above-mentioned "analysis of the distance histogram H (t, x, y, n) (when t is divided by 3 and the remainder is 1)" will be described with reference to FIG.

As shown in FIG. 4, the distance histogram H (t, x, y, n) is three-dimensional data in the (x, y, n) space with respect to "time t when the remainder divided by 3 is 1". Is. Here, x is an integer of 1 or more and X or less, and y is an integer of 1 or more and Y or less. And n is an integer of 0 or more and N-1 or less. The distance histogram H (t, x, y, n) in this three-dimensional data is input to the discrimination unit 21. Then, the discrimination unit 21 sets the control parameters related to the generation of the distance histogram for performing efficient distance measurement operation control (adaptive control) at the time t + 1 and the time t + 2.

Specifically, as shown in FIG. 5, the discrimination unit 21 (control unit 11)
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
as well as,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
Control parameters such as are set, and these values are passed to the operation control unit 22. Here, x is an integer of 1 or more and X or less, and y is an integer of 1 or more and Y or less.

Here, the value of W is set as a binary value of 0 or 1. W (t + 1, x, y) is a signal that specifies whether or not to operate the SPAD element at the position (x, y) of the light receiving unit 13 at time t + 1. At time t + 1, the SPAD element at the position W (t + 1, x, y) = 1 operates, and the SPAD element at the position W (t + 1, x, y) = 0 does not operate. Naturally, power consumption can be suppressed if the SPAD element is not operated. Further, the light emitting unit 12 does not need to emit light in the direction corresponding to the position (x, y) of the light receiving unit 13, and the power consumption can be further suppressed.

S (t + 1, x, y) and E (t + 1, x, y) are the data acquisition start positions (frequency count start positions) of the distance histogram of the position (x, y) of the light receiving unit 13 at time t + 1. ) And a signal to specify the end position. By specifying the start position and the end position, the class range to be counted in the distance histogram is set.
An example of the distance histogram in the direct ToF format is shown in FIG. FIG. 6 shows an example of S (t + 1, x, y) = 10 and E (t + 1, x, y) = 20. In this case, the distance histogram is not generated in the class range in which bins are 0 to 9th and 21st to N-1st. The distance histogram is generated only for 11 bins whose bin values are 10th to 20th. As a matter of course, the time from time 0 to time 10D and the time from time 21D to time N × D do not perform the distance histogram generation processing, so that compared with the conventional method in which the distance histogram generation processing is performed at all times. , Power consumption can be suppressed.

M (t + 1) is the value of the number of exposures at time t + 1 (frame period). Optimal control can be performed by setting the number of exposures necessary and sufficient for distance measurement.

G (t + 1, x, y) is information on the position (x, y) on which the same object Tg is projected at time t + 1. If G (t + 1, x1, y1) and G (t + 1, x2, y2) in the frame period at time t + 1 are the same, the object projected at the two positions (x1, y1) and the position (x2, y2). The thing Tg is the same. Therefore, noise can be removed by adding and averaging the distance histograms of the two positions (x1, y1) and the positions (x2, y2). That is, noise can be removed by adding and averaging the distance histograms at the positions (x, y) where G (t + 1, x, y) is the same.

W (t + 2, x, y) for time t + 2,
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
The same applies to the above time t + 1.

W (t + 1, x, y) determined by the determination unit 21 (control unit 11),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
as well as,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
The control parameters related to the generation of the distance histogram in the above are input to the operation control unit 22, and the operation control unit 22 controls the distance measurement at the time t + 1 and the time t + 2 based on the control parameters.

An example of processing executed by the control unit 11 for realizing the first embodiment will be described with reference to FIG. 7.

First, the control unit 11 sets 1 at time t and 0 at Mode in step S1. Then, the control unit 11 proceeds to the process in step S2.

The control unit 11 determines the Mode value in step S2. If the Mode value is 0, the process proceeds to step S3.

In step S3, the determination unit 21 of the control unit 11 determines.
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
as well as,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
Find the control parameters for. Further, the control unit 11 determines the value of Mode to 3, and proceeds to step S4. The details of the process of step S3 performed by the control unit 11 will be described later.
Here, as an example, the value of Mode is determined to be 3, but the value of Mode can be arbitrarily set and can be set to 2 or 4 or more.

In step S4, the determination unit 21 of the control unit 11 sets the control parameter related to the generation of the distance histogram to EN (x, y) = 1.
Start (x, y) = 0,
End (x, y) = N-1,
M = predetermined constant (M1),
ID (x, y) = all different IDs
And.
Here, EN (x, y) is W (t, x, y) at time t, Start (x, y) is S (t, x, y) at time t, and End (x, y) is. E (t, x, y) at time t is indicated, and M indicates M (t) at time t. Further, at time t, the IDs (x, y) of each SPAD element in the light receiving unit 13 are all set to different IDs.
The control unit 11 sets the control parameters for performing the distance measurement based on the normal control at the time t by performing the process of step S4. Then, the control unit 11 proceeds to the process in step S6.

In step S6, the control unit 11 emits light and receives light M1 times (the number of exposures is M1) for distance measurement by normal control at time t, and operates only the SPAD element having EN (x, y) = 1. .. At this time, the data acquisition (frequency count) start position and end position of the distance histogram of the position (x, y) are Start (x, y) = 0 and End (x, y) = N-1, respectively. Is.

In step S7, the control unit 11 causes the distance measuring unit 16 to perform an averaging process between the distance histograms of the positions (x, y) having the same ID (x, y). The equation used in the averaging process by the ranging unit 16 is represented by the following [Equation 1].

At time t, the IDs (x, y) of each SPAD element in the light receiving unit 13 are all set to different IDs, so that the averaging process by the distance measuring unit 16 may be omitted.

In step S8, the control unit 11 causes the distance measuring unit 16 to detect the peak value of the distance histogram at each position (x, y) of the light receiving unit 13, and multiplies the time corresponding to the peak value by c / 2. The distance to the object Tg is calculated by the method.

In step S9, the control unit 11 outputs the distance obtained in step S8 for the position (x, y) where EN (x, y) = 1 as the distance measurement result of the position (x, y) at time t. Output from terminal 17. In the normal control at time t, the control unit 11 outputs the distance measurement results for all the positions (x, y).

In step S10, the control unit 11 increments the time t by 1 and decrements the Mode by 1. After that, the control unit 11 returns the process from step S10 to step S2.

The control unit 11 determines the Mode value in step S2. At time t + 1, the mode is set to 3 by the process in step S3 at time t, and only 1 is decremented by the subsequent process in step S10, and the mode becomes 2.
Therefore, the control unit 11 determines that Mode = 0, and proceeds to step S5.

In step S5, the determination unit 21 of the control unit 11 determines for each SPAD element.
EN (x, y) = W (t + 1, x, y),
Start (x, y) = S (t + 1, x, y),
End (x, y) = E (t + 1, x, y),
M = M (t + 1),
ID (x, y) = G (t + 1, x, y)
And set.
The values of W (t + 1, x, y), S (t + 1, x, y), E (t + 1, x, y), M (t + 1), and G (t + 1, x, y) are the current values. It is determined as a control parameter for distance measurement by applicable control by the process of step S3 at time t before time t + 1.

When Start (x, y) = 0 and End (x, y) = N-1 are set in step S5, the distance histogram corresponding to the position (x, y) of the light receiving unit 13 is usually Similar to the controlled distance measurement, accumulation is performed for all bins from the 0th bin to the N-1th bin.

Then, in step S6, the control unit 11 emits light and receives light M (t + 1) times for distance measurement by applicable control at time t + 1, and operates only the SPAD element with EN (x, y) = 1. At this time, the data acquisition (frequency count) start position and end position of the distance histogram of the position (x, y) are, respectively, Start (x, y) = S (t + 1, x, y), End (x). , Y) = E (t + 1, x, y).

In step S7, the control unit 11 causes the distance measuring unit 16 to perform an averaging process between the distance histograms of the positions (x, y) having the same ID (x, y). The equation used in the averaging process by the ranging unit 16 is represented by the above [Equation 1].

In step S9, the control unit 11 outputs the distance obtained in step S8 for the position (x, y) where EN (x, y) = 1 as the distance measurement result of the position (x, y) at time t. Output from terminal 17. The distance measurement result is not output for the position (x, y) where EN (x, y) = 0.

In step S10, the control unit 11 increments the time t by 1 and decrements the Mode by 1. As a result, the time t + 1 becomes the time t + 2, and the mode changes from 2 to 1. The control unit 11 returns the process from step S10 to step S2.

After that, the control unit 11 determines the Mode value in step S2. Since the Mode is 1 at time t + 2, the control unit 11 proceeds to step S3, and is based on the control parameters for distance measurement by the applicable control at time t + 2 set in step S3 at time t. Then, the processes from step S5 to step S9 are executed.

Then, in step S10, the control unit 11 increments the time t by 1 and decrements the Mode by 1. As a result, the time t + 2 becomes the time t + 3, and the mode changes from 1 to 0.
Therefore, the control unit 11 proceeds from step S2 to step S3, and executes the same processing as described above at the time after time t + 3.

By performing the processing of FIG. 7, the control unit 11 executes the distance measurement in the normal control at the time t as shown in FIG. 3, and applies the distance measurement based on the control parameters set at the time t at the time t + 1 and the time t + 2. Distance measurement can be performed by target control.

Next, the details of the process of step 3 in FIG. 7 executed by the control unit 11 (discrimination unit 21) will be described with reference to FIG.
That is, the histogram H (t, x, y, n) at the time of "time t when the remainder divided by 3 is 1" is used as an input.
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
as well as,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
It is a detailed description of the process of determining the control parameters such as.

First, in step S101, the control unit 11 detects the object Tg from the image at time t. Here, the image is, for example, an image captured at time t in the RGB camera 200. The captured image is input to the control unit 11 (discrimination unit 21) of the distance measuring device 10 via the image input terminal 19.
As an image used for detecting the object Tg, a luminance image calculated from the distance histogram at time t by the calculation shown in FIGS. 13 and 14 of Japanese Patent Application No. 2017-224451 (when illuminated by ambient light). (Brightness image) can also be used, in which case the RGB camera 200 can be eliminated.

The object Tg referred to here is a designated object input via the detection mode input terminal 18 before the control unit 11 performs the process of FIG. 7. It should be noted that the process of detecting the object Tg (person, face, hand, car, etc.) from the image has been conventionally performed and is known, so a detailed description thereof will be omitted.

In the present embodiment, the case where the object Tg is a person will be described as an example with reference to FIG. FIG. 9A is a captured image at time t.
In this case, the control unit 11 detects two people as the object Tg1 and the object Tg2 based on the captured image at time t. In the following description, when the object Tg1 and the object Tg2 are not particularly distinguished, they are described as the object Tg in the same manner as described above.

Then, in step S102, the control unit 11 segments the detected object Tg1 and the object Tg2.
FIG. 9B shows image data segmented for each object Tg. The shaded area in FIG. 9B shows the region of the first object Tg1, the black region shows the region of the second object Tg2, and the white region shows the region other than the objects Tg1 and Tg2 (persons).

In step S103, the control unit 11
W (t + 1, x, y),
S (t + 1, x, y, n),
E (t + 1, x, y, n),
M (t + 1),
G (t + 1, x, y)
Is determined for each SPAD element.

The control unit 11 determines the value of W (t + 1, x, y) in each SPAD element at time t + 1.
Specifically, the control unit 11 sets W (t + 1, x, y) = 1 for the SPAD element corresponding to the detected target Tg1 and Tg2 regions (hatched region and black region in FIG. 9B). Then, W (t + 1, x, y) = 0 for the SPAD element corresponding to the region other than the detected objects Tg1 and Tg2 (white region in FIG. 9B).
As described with reference to FIG. 7, the control unit 11 does not generate a distance histogram for the SPAD element at the position (x, y) where W (t + 1, x, y) = 0 at time t + 1, so that the distance measurement is performed. Not performed. Since the region of W (t + 1, x, y) = 0 is not the object Tg, there is no inconvenience even if distance measurement is not performed.

Further, the control unit 11 sets a class range (bin range) to be counted by the frequency at time t + 1. The control unit 11 determines the values of S (t + 1, x, y) and E (t + 1, x, y), and the frequency ranges from S (t + 1, x, y) to E (t + 1, x, y). Set as the class range to be counted.

Here, the predetermined value is Constant. Const is a value that estimates the maximum amount that the object Tg can move in the depth direction (distance direction) within the frame period. For example, if the object Tg specified from the detection mode input terminal 18 is a person, it is estimated that the amount of change in movement is not so large, and the Const is set to a relatively small value. It is estimated that the amount of change is large, and Const is set to a relatively large value.

The peak of the distance histogram is detected at each position (x, y) at time t, and a value smaller by Const than the peak value is defined as S (t + 1, x, y). Let E (t + 1, x, y) be a value larger than that value by Const. At time t, the distance to the object Tg at the position (x, y) is the distance based on the detection of the peak value. Therefore, at time t + 1, it exists at any of the distances of "the distance-Const" or more and "the distance + Const" or less, and does not exist at any other distance. Therefore, the class range (bin to be counted on the distance histogram) for distance measurement is sufficient only in the range of S (t + 1, x, y) or more and E (t + 1, x, y) or less.

Let Pav be the average value of the peak values of the distance histogram at each position (x, y) satisfying W (t + 1, x, y) = 1. That is, Pav is represented by the following [Equation 2].

Let P1 be a predetermined constant. This value P1 is a peak value that makes detection reliable and easy in peak detection. At this time, M (t + 1) is represented by the following [Equation 3].

The control unit 11 determines the number of exposures M (t + 1) at time t + 1 based on the above [Equation 3].
At time t, the peak value due to M1 light emission and light reception (the number of exposures is M1) is about Pav. Therefore, if the number of exposures M (t + 1) at time t + 1 is set to the value shown in [Equation 3], the peak value at time t + 1 is about P1. Therefore, the detection of the peak value at time t + 1 becomes reliable and easy. In [Equation 2], the position (x, y) is limited to the position (x, y) satisfying W (t + 1, x, y) = 1 only to the object Tg to be distance-measured. Because it is.

Further, the control unit 11 generates segment information at time t + 1 based on the segmentation process in step S102.
The segment information referred to here is information indicating a group of SPAD elements that have received the reflected light from the same object Tg, and for example, only those obtained by segmenting the detected objects Tg1 and Tg2 as shown in FIG. 9B. However, as shown in FIG. 9C, segmented objects Tg1 and Tg2 are also included.

Specifically, the control unit 11 generates segment information to which one ID is assigned as G (t + 1, x, y) for each segmented SPAD element group. For example, G (t + 1, x, y) = 1 is assigned to the position (x, y) in the shaded area of FIG. 9B. Further, G (t + 1, x, y) = 2 is assigned to the position (x, y) in the black region. Further, G (t + 1, x, y) = 3 is assigned to the position (x, y) in the white region. As already described with reference to FIG. 7, at time t + 1, the position (x, y) where G (t + 1, x, y) = 1 is the region where the same object Tg1 is measured, and the step. Noise can be removed by performing the averaging process in this region in S7. The same applies to G (t + 1, x, y) = 2.

In step S104, the control unit 11
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
Is determined for each SPAD element.

First, the control unit 11 generates segment information at time t + 2. The control unit 11 has G (t + 1, x, y) = 1 and G (t + 1, x, y) in each segment within W (t + 1, x, y) = 1 obtained in step S103 (in the case of FIG. 9B, G (t + 1, x, y) = 1. ) = 2), dilation (expansion processing of the image) is performed. In FIG. 9C, dilation is performed on the object Tg1 (diagonal region) of G (t + 1, x, y) = 1 in FIG. 9B and the object Tg2 (black region) of G (t + 1, x, y) = 2. It shows the state.
Then, the control unit 11 assigns one ID to the dilated area of each segment. That is, the control unit 11 has G (t + 2, x, y) = 1 for the position (x, y) in the shaded area of FIG. 9C, and G (t + 2, x) for the position (x, y) in the black area. , Y) = 2 is assigned. The control unit 11 assigns another ID, G (t + 2, x, y) = 3, to the remaining region (position (x, y) in the white region of FIG. 9C).

Further, the control unit 11 determines the value of W (t + 1, x, y) in each SPAD element at time t + 2.
The control unit 11 sets W (t + 2, x, y) = 1 for the position (x, y) belonging to any of the dilated regions. For the other regions, W (t + 2, x, y) = 0. That is, the control unit 11 sets W (t + 2, x, y) = 1 for the shaded area and the black area in FIG. 9C. Then, the region other than the detected objects Tg1 and Tg2 (white region in FIG. 9C) is set to W (t + 2, x, y) = 0.

At time t + 2, since time has passed by 2 with respect to time t, there is a possibility that the object Tg is moving, and the position of the object Tg detected at time t (diagonal area in FIG. 9B). And the black area), the entire area (the shaded area and the black area in FIG. 9C) that is one size larger than the area (and the black area) should be measured. Therefore, the dilated region as described above is set to W (t + 2, x, y) = 1. Since it is considered that the object Tg is at almost the same position as the time t at the time t + 1, dilation is not performed for the determination of W (t + 1, x, y).

As a result, as described with reference to FIG. 7, the control unit 11 does not generate a distance histogram for the SPAD element at the position (x, y) where W (t + 2, x, y) = 0 at time t + 2. Distance measurement is not performed. Since the region of W (t + 2, x, y) = 0 is not the object Tg, there is no inconvenience even if distance measurement is not performed.

As already described with reference to FIG. 7, at time t + 2, the position (x, y) where G (t + 2, x, y) = 1 is the region where the same object Tg1 is measured, and the step. Noise can be removed by performing the averaging process in this region in S7. Strictly speaking, the region where G (t + 2, x, y) = 1 includes a region that is not the object Tg due to dilation, but there is no problem because the area is small. The same applies to G (t + 2, x, y) = 2.

Further, the control unit 11 detects the peak value of the distance histogram at each position (x, y) at time t when setting the class range to be the target of the frequency count at time t + 2, and only 2 × Const from that value. Let S (t + 2, x, y) be a small value, and E (t + 2, x, y) be a value 2 × Const larger than that value.
At time t, the distance to the object Tg at the position (x, y) is the distance based on the detection of the peak value. Therefore, at time t + 2, it exists at any of the distances of "the distance-2 x Const" or more and "the distance + 2 x Const" or less, and does not exist at any other distance. Therefore, the range to be measured (that is, the bin to be counted on the histogram) is sufficient only in the range of S (t + 2, x, y) or more and E (t + 2, x, y) or less.

Let Pav2 be the average value of the peak sizes of the histogram at each position (x, y) satisfying W (t + 2, x, y) = 1. That is, Pav2 is represented by the following [Equation 4].

At this time, M (t + 2) is represented by the following [Equation 5].

The control unit 11 determines the number of exposures M (t + 2) at time t + 2 based on the above [Equation 5].
At time t, the peak value is about Pav2 due to M1 light emission and light reception (the number of exposures is M1). Therefore, if the number of exposures M (t + 2) at time t + 2 is set to the value represented by [Equation 4], the peak value at time t + 2 is about P1. Therefore, the detection of the peak value at time t + 2 becomes reliable and easy. In [Equation 5], the position (x, y) is limited to the position (x, y) satisfying W (t + 2, x, y) = 1 only to the object Tg to be distance-measured. Because it is.

After step S104, the control unit 11 ends the process of FIG. 8 and proceeds to step S4 of FIG. 7, and executes the same process as described above.

In the first embodiment, as shown in FIG. 3, one "distance measurement by normal control" and two "distance measurement under adaptive control" are repeated in three time cycles. It was the case. The present technology does not limit the period to 3. For example, one "normal distance measurement" and one "distance measurement under adaptive control" may be repeated in two cycles in time. For example, when the "designated object Tg" input from the detection mode input terminal 18 is an object with relatively little movement such as a person, face, or hand, one "distance measurement by normal control" is performed. And, it is preferable to have three cycles in which "distance measurement under adaptive control" is repeated twice. On the other hand, when the "designated object Tg" input from the detection mode input terminal 18 is an object with relatively large movement such as a car, one "distance measurement by normal control" and one "adaptation". It is advisable to set two cycles in which "distance measurement under target control" is repeated.

<4. Second Embodiment>
A second embodiment of the present technology will be described with reference to FIGS. 10 and 11.
In the present embodiment, the control unit 11 (discrimination unit 21) executes the process shown in FIG. 7 as in the first embodiment, but executes a different process in step S3. Also in the second embodiment, the value of Mode determined in step S3 is always 3.

In the second embodiment, the control unit 11 (discrimination unit 21) has a histogram H (t, x, y, n) at "time t when the remainder divided by 3 is 1" as shown in FIG. ) And the histogram H (t-3, x, y, n) at time t-3, which is 3 earlier than that time.
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
as well as,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
The process of determining the control parameters such as is executed.

An example of processing executed by the control unit 11 (discrimination unit 21) for realizing the second embodiment will be described with reference to FIG.

In steps S201, S202, S203, and S204 of FIG. 11, the control unit 11 detects and segments the object Tg, respectively.
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
Decision,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
Performs processing such as determining. Since these processes are the same as steps S101 to S104 shown in FIG. 8 of the first embodiment, the description thereof will be omitted.

In step S205, the control unit 11 determines whether or not to cause the histogram generation unit 15 to generate the distance histogram in the frame period after the present, based on the amount of change in the distance histogram generated in the past consecutive frame periods. .. That is, the control unit 11 executes a process of overwriting the values of W (t + 1, x, y) and W (t + 2, x, y) as needed.

Specifically, the control unit 11 detects the peak of the histogram H (t, x, y, n) at each position (x, y) satisfying W (t + 2, x, y) = 1, and W ( The peak of the histogram H (t-3, x, y, n) at each position (x, y) satisfying t + 2, x, y) = 1 is detected. Then, the control unit 11 sets the peak value of the histogram H (t-3, x, y, n) and the histogram H (t) at all the positions (x, y) satisfying W (t + 2, x, y) = 1. , X, y, n) It is determined whether the peak values are the same (change).

If there is even one difference in the position (x, y) that satisfies W (t + 2, x, y) = 1, the control unit 11 has W (t + 1, x, y) and W (t + 2, x, y). Do not overwrite the value of. On the other hand, if all the positions (x, y) satisfying W (t + 2, x, y) = 1 are the same, the control unit 11 has W (t + 1, x, y) and W (t + 2, x) at all the positions. , Y) is overwritten with 0.
The positions overwritten with 0 are the positions of all SPAD elements regardless of the value of W (t + 2, x, y) before the update. At all positions (x, y) satisfying W (t + 2, x, y) = 1, the peak value of the histogram H (t-3, x, y, n) and the peak value of H (t, x, y, n) The fact that the peak values are the same means that the object Tg does not move from time t-3 to time t. Therefore, it may be determined that the device is not moving at time t + 1 and t + 2. Since the object Tg is not moving, it is not necessary to measure the distance at time t + 1 and t + 2. By setting W (t + 1, x, y) = W (t + 2, x, y) = 0 for the positions of all the SPAD elements in step S205, the times t + 1 and t + 2 are shown in step S6 of FIG. In, the distance measurement is not performed in all the SPAD elements. This is because the distance was normally measured every frame period, but the distance is adaptively measured at a time interval of 3, that is, the time interval for adaptive distance measurement is variable. Become.

Further, in step S205, the control unit 11 determines whether or not to make the light emitting unit 12 emit light in the frame period after the present, based on the amount of change in the distance histogram generated in the past consecutive frame periods.
Specifically, when the control unit 11 has the same (unchanged) positions (x, y) satisfying W (t + 2, x, y) = 1, the control unit 11 has W (t + 1) at all positions. , X, y) and W (t + 2, x, y) can be overwritten with 0, and the light emitting unit 12 can be controlled not to emit light at time t + 1 and time t + 2. By not causing the light emitting unit 12 to emit light during the time when the distance measurement is not performed, the power consumption of the distance measuring device 10 can be reduced.

In the second embodiment, the control unit 11 determines whether the peak value of the histogram H (t-3, x, y, n) and the peak value of the histogram H (t, x, y, n) are the same. Although it was decided to make a judgment, the amount of change as the difference between the peak value of the histogram H (t-3, x, y, n) and the peak value of the histogram H (t, x, y, n) is equal to or less than a predetermined threshold value. It may be determined whether or not it is.
For example, in step S205, when the amount of change from time t-3 to time t is equal to or less than a predetermined threshold value, the control unit 11 determines that the object Tg is hardly moving, and W (t + 1, x, Overwrite y) and W (t + 2, x, y) with 0.

After step S205, the control unit 11 ends the process of FIG. 11 and proceeds to step S4 of FIG. 7, and executes the same process as described above.

<5. Third Embodiment>
A third embodiment of the present technology will be described with reference to FIGS. 12 and 13.
In the third embodiment, the control unit 11 (discrimination unit 21) executes the process shown in FIG. 7 as in the first embodiment, but executes a different process in step S3.

The third embodiment is an example in which the adaptive control number of times is set as the number of times of the frame period in which the same control parameter is continuously processed after the present based on the distance histogram generated in the past frame period. Is. That is, the control unit 11 determines the situation at the present time (time t) from the histogram H (t, x, y, n) at time t, and "adaptively controls only time t + 1" (adaptive). Whether the number of times of control is one) or "the number of times of adaptive control is adaptively controlled for two times of time t + 1 and time t + 2" (whether the number of times of adaptive control is two) is determined. Here, t is the time when Mode = 0.

In the first embodiment and the second embodiment, the situation at the present time (time t) is determined from the histogram H (t, x, y, n) at time t, and the time is always "time." The process of "adaptively controlling two times of t + 1 and time t + 2 (the number of adaptive controls is 2)" has been described. That is, the value of Mode set in step S3 was always 3. In the third embodiment, the Mode value set in step S3 is 2 or 3, which is adaptively determined by the situation at time t. This is the difference between the first and second embodiments and the third embodiment.

In the third embodiment, as shown in FIG. 12, the control unit 11 (discrimination unit 21) has a histogram H (t, x, y, n) at "time t when the mode is 0" and Enter the histogram H (ta, x, y, n) of the latest time (referred to as ta) that was earlier than that time and whose Mode was 0.
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
as well as,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
The process of determining the control parameters such as is executed. The value of Mode is also determined.

An example of processing executed by the control unit 11 (discrimination unit 21) for realizing the third embodiment will be described with reference to FIG.

In steps S301, S302, S303, and S304 of FIG. 13, detection and segmentation of the object Tg, respectively,
W (t + 1, x, y),
S (t + 1, x, y),
E (t + 1, x, y),
M (t + 1),
G (t + 1, x, y),
Decision,
W (t + 2, x, y),
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
Performs processing such as determining. Since these processes are the same as steps S101 to S104 shown in FIG. 8 of the first embodiment, the description thereof will be omitted.

The control unit 11 determines the Mode value in step S305. Specifically, the control unit 11 detects the peak of the histogram H (t, x, y, n) at each position (x, y) satisfying W (t + 2, x, y) = 1. Further, the peak detection of the histogram H (ta, x, y, n) at each position (x, y) satisfying W (t + 2, x, y) = 1 is also performed. Here, the time ta means the frame period during which the normal control was performed immediately before.
Then, at all the positions (x, y) satisfying W (t + 2, x, y) = 1, the peak value of the histogram H (ta, x, y, n) and the peak value of H (t, x, y, n) Determine if the peak values are the same. At the position (x, y) that satisfies W (t + 2, x, y) = 1, if the peak value is different in more than half, Mode = 1 is set. If the number of positions (x, y) with different peak values is less than half, Mode = 2.

Here, the determination of the mode in step S305 will be described.
The peak value of the histogram H (ta, x, y, n) and the peak of H (t, x, y, n) at almost all positions (x, y) satisfying W (t + 2, x, y) = 1. The fact that the values are the same means that the object Tg does not move (the amount of change is small) from time ta to time t. Therefore, it may be determined that the device is hardly moving even at time t + 1 and t + 2. Since the object Tg is almost stationary, even if adaptive control (that is, some SPADs are not operated or the data acquisition part of the histogram is limited) is performed at time t + 1 and t + 2. , Distance measurement never fails. Therefore, there is no problem even if Mode = 3 is set.

On the other hand, at almost all positions (x, y) satisfying W (t + 2, x, y) = 1, the peak value of the histogram H (ta, x, y, n) and H (t, x, y, n) The fact that the peak values of the objects are different means that the object Tg moves considerably (the amount of change is large) from the time ta to the time t. Therefore, adaptive control may be performed at time t + 1, but adaptive control may fail at time t + 2. That is, the movement of the object Tg is large, and the prediction at time t + 2, which is a time advanced by 2 in time, may be incorrect, and W (t + 2, x, y) and S (t + 2,) obtained in step S304 may be incorrect. x, y) and E (t + 2, x, y) may not be appropriate. Therefore, by setting Mode = 2, adaptive control is not performed at time t + 2. In this case, W (t + 2, x, y) obtained in step S304,
S (t + 2, x, y),
E (t + 2, x, y),
M (t + 2),
G (t + 2, x, y)
Is never used.

After step S305, the control unit 11 ends the process of FIG. 13 and proceeds to step S4 of FIG. 7, and executes the same process as described above.

<6. Summary and modification examples>
The distance measuring device 10 of the above embodiment acquires the distance index by the direct ToF method a plurality of times based on the received signal of the light emitted by the light emitting unit 12 and reflected by the object Tg for each continuous frame period. The detection unit 14, and the histogram generation unit 15 that generates a distance histogram representing the distance index as frequency information for each distance based on the distance index obtained a plurality of times by the detection unit 14 for each frame period. A control unit 11 for setting control parameters related to the generation of the distance histogram in the current and subsequent frame periods based on the distance histogram generated in the past frame period is provided (see S3 and FIG. 8 in FIG. 7). ).
As a result, control related to distance measurement from the present onward is executed based on the control parameters that reflect the situation that can be read from the distance histogram generated in the past frame period.
Therefore, distance measurement can be efficiently performed in the frame period after the present. Therefore, it is possible to reduce the processing load of the distance measuring device 10 in the distance measurement and reduce the power consumption.

In the distance measuring device 10 of the embodiment, the light receiving unit 13 for obtaining a light receiving signal has a plurality of light receiving elements (SPAD elements), and the control unit 11 is generated for each light receiving element (SPAD element) in the past frame period. Based on the distance histogram, the control parameters related to the generation of the distance histogram in the frame period after the present are set (see S3 and FIG. 8 in FIG. 7).
As a result, control related to distance measurement from the present onward is executed for each SPAD element based on the control parameters that reflect the situation for each SPAD element that can be read from the distance histogram generated in the past frame period.
Therefore, for example, depending on the situation of each SPAD element, only some of the SPAD elements generate the distance histogram after the present, and the control related to the distance measurement after the present reflecting the past situation is realized for each element. can do. Further, by not performing the processing on the SPAD element that does not require control, it is possible to reduce the processing load of the distance measuring device 10 in the distance measurement and reduce the power consumption.

In the distance measuring device 10 of the embodiment, the control unit 11 sets the class range to be counted in the distance histogram as the setting of the control parameters (S3 in FIGS. 6 and 7 and S103 in FIG. 8). , S104, etc.).
As a result, the distance histogram is generated only for a part of the class range considered to be necessary for performing distance measurement from the present onward, depending on the situation that can be read from the distance histogram generated in the past.
Therefore, it is possible to omit the process of generating the distance histogram for the class range presumed to be unnecessary for the distance measurement, reduce the processing load of the distance measuring device 10 in the distance measurement, and reduce the power consumption. ..

In the distance measuring device 10 of the embodiment, the control unit 11 sets a predetermined class range including the class of the peak value of the distance histogram in the past as the class range to be the target of the frequency count (S3 in FIG. 7). See S103, S104, etc. in FIG. 8).
The peak value is a value estimated to be the distance to the object Tg. Therefore, according to the above configuration, only a predetermined class range including the distance of the object Tg in the past frame period such as the immediately preceding frame period is set as the class range to be counted by the frequency in the distance histogram after the present. It becomes possible.
Therefore, it is possible to improve the accuracy in setting the class range that is presumed to be unnecessary for performing distance measurement. Further, by improving the setting accuracy of the class range that does not generate the distance histogram, the processing load of the distance measuring device 10 in the distance measurement can be further reduced, and the power consumption can be further reduced.

In the distance measuring device 10 of the embodiment, the control unit 11 sets the number of exposures M (t) in the frame period as the setting of the control parameter (see S3 in FIG. 7, S103, S104 in FIG. 8 and the like).
This makes it possible to set the number of exposures in a frame period suitable for the surrounding environment when performing distance measurement.
Therefore, for example, when the ambient light is low, the number of exposures in the frame period can be reduced, the processing load of the distance measuring device in distance measurement can be reduced, and the power consumption can be reduced.

In the distance measuring device 10 of the third embodiment, the control unit 11 continuously performs processing with the same control parameters from the present onward based on the distance histogram generated in the past frame period. The adaptive control number (Mode) is set as the number of times (see S3 in FIG. 7, S305 in FIG. 13 and the like).
As a result, in response to the case where it is estimated that it is not necessary to sequentially set the control parameters for each frame period from the tendency of the past distance histogram, the same control parameters can be set in the frame period within the adaptive control number of times after the present. It becomes possible to use.
Therefore, it is possible to reduce the processing load of the distance measuring device 10 caused by creating the control parameters for distance measurement. In addition, the power consumption of the distance measuring device 10 can be reduced.

In the distance measuring device 10 of the third embodiment, the control unit 11 sets the adaptive control number (Mode) based on the amount of change in the distance histogram generated in the past consecutive frame periods (FIG. 7). See S3, S305 of FIG. 13 and the like).
The speed of movement of the object Tg can be estimated from the amount of change in the distance histogram in the past consecutive frame periods. Therefore, according to the above configuration, the adaptive control number can be set according to the estimated moving speed of the object Tg.
Therefore, it is possible to set a large number of adaptive controls depending on the speed of movement of the object Tg for distance measurement. Therefore, the processing load of the distance measuring device in distance measurement can be further reduced, and the power consumption can be further reduced.

In the distance measuring device 10 of the third embodiment, the control unit 11 sets a larger number of adaptive control times (Mode) as the amount of change in the generated distance histogram is smaller (S3 and 13 in FIG. 7). See S305).
As a result, the object Tg in which the amount of change in the distance histogram in the past consecutive frame periods is small increases the frame period in which the same control parameter is used after the present.
Since the object Tg in which the amount of change in the distance histogram is small is stationary or moves slowly, the distance due to distance measurement does not change unpredictably. Therefore, even if a large number of frame periods in which control parameters are not generated after the present are provided, it is unlikely that the distance measurement of the object Tg will be hindered.
Therefore, for the object Tg having little movement, the processing load of the distance measuring device in the distance measurement can be further reduced and the power consumption can be further reduced by providing a large frame period in which the control parameters are not generated after the present.

In the distance measuring device 10 of the second embodiment, the control unit 11 causes the distance histogram to be generated in the frame period after the present based on the amount of change of the distance histogram generated in the past consecutive frame periods. Whether or not it is determined (see S3 in FIG. 7 and S205 in FIG. 11).
This makes it possible to determine whether or not to generate the distance histogram in the frame period after the present according to the movement of the object Tg in the past.
Therefore, for example, when the amount of change in the distance histogram is small, the processing load of the distance measuring device in the distance measurement can be reduced and the power consumption can be reduced by not generating the distance histogram in the frame period after the present.

In the distance measuring device 10 of the second embodiment, the control unit 11 causes the light emitting unit 12 to emit light in the frame period after the present based on the amount of change of the distance histogram generated in the past consecutive frame periods. Whether or not it is determined (see S3 in FIG. 7 and S205 in FIG. 11).
This makes it possible to prevent the light emitting unit from emitting light in response to the case where the distance histogram is not generated in the frame period after the present.
This is because when the distance histogram is not generated, it is not necessary to measure the time from the light emission to the light reception of the light emitting unit 12. By not emitting light from the light emitting unit 12 when the distance histogram is not generated in this way, the processing load of the distance measuring device in distance measurement can be further reduced, and the power consumption can be further reduced.

In the distance measuring device 10 of the embodiment, the light receiving unit 13 that obtains a light receiving signal has a plurality of light receiving elements (SPAD elements), and the control unit 11 uses the same object Tg as one of the control parameters. Segment information indicating a light receiving element group (SPAD element group) that has received reflected light is generated, and the distance histogram of the light receiving element (SPAD element) in the light receiving element group (SPAD element group) indicated by the segment information is added and averaged ( See S3 in FIG. 7, S103, S104 in FIG. 8 and the like).
As a result, noise reduction between the distance histograms generated in the light receiving elements belonging to the same segment is performed.
Therefore, even distance measurement can be realized for the segmented object Tg.

Further, in the embodiment, at least a part of the processing described with reference to FIGS. 8, 11 and 13 may be realized by AI (Artificial Intelligence) using, for example, CNN (Convolutional Neural Network) or the like. .. For example, it is conceivable to realize the detection process of the object Tg shown in step S101 or the like in FIG. 8 or the segmentation process of the SPAD element shown in step S102 or the like by CNN.

Further, in the distance measuring method as an embodiment, the distance index is acquired a plurality of times by the direct ToF method based on the received signal of the light 41 emitted by the light emitting unit 12 and reflected by the object Tg for each continuous frame period. The detection unit 14, and the histogram generation unit 15 that generates a distance histogram representing the distance index as frequency information for each distance based on the distance index obtained a plurality of times by the detection unit 14 for each frame period. The distance measuring method of the distance measuring device 10 including the above, which sets control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated in the past frame period. be.
Even by such a distance measuring method, the same operation and effect as the distance measuring device 10 according to the present technology can be obtained.

The program of the embodiment is a program for causing, for example, a CPU, a DSP (Digital Signal Processor), or a device including these to execute the processes of FIGS. 7, 8, 11, and 13.
That is, the program of the embodiment includes a detection unit that acquires the distance index by the direct ToF method a plurality of times based on the received signal of the light emitted by the light emitting unit and reflected by the object Tg for each continuous frame period, and the frame. A distance measuring device comprising a histogram generating unit for generating a distance histogram representing the distance index as frequency information for each distance based on the distance index obtained a plurality of times by the detecting unit for each period. This is a program that executes a process of setting control parameters related to the generation of the distance histogram in the current and subsequent frame periods based on the distance histogram generated in the past frame period. With such a program, the above-mentioned distance control device can be realized.

Such a program can be recorded in advance in an HDD (Hard Disk Drive) as a recording medium built in a device such as a computer device, a ROM in a microcomputer having a CPU, or the like.
Alternatively, flexible discs, CD-ROMs (Compact Disc Read Only Memory), MO (Magnet optical) discs, DVDs (Digital Versatile Discs), Blu-ray discs (registered trademarks), magnetic discs, semiconductor memories, memory It can be temporarily or permanently stored (recorded) on a removable recording medium such as a card. Such a removable recording medium can be provided as so-called package software.
In addition to installing such a program from a removable recording medium on a personal computer or the like, it can also be downloaded from a download site via a network such as a LAN or the Internet.

Further, such a program is suitable for a wide range of provision of the communication control device of the embodiment. For example, by downloading a program to a personal computer, a portable information processing device, a mobile phone, a game device, a video device, a PDA, or the like, the personal computer or the like can function as the distance measuring device of the present disclosure.

<7. This technology>
The present technology can also adopt the following configurations.
(1)
A detection unit that acquires the distance index multiple times by the direct ToF method based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period.
A histogram generator that generates a distance histogram that represents the distance index as frequency information for each distance based on the distance index obtained a plurality of times by the detection unit for each frame period.
A distance measuring device including a control unit for setting control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated in the past frame period.
(2)
The light receiving unit that obtains the light receiving signal has a plurality of light receiving elements and has a plurality of light receiving elements.
The control unit sets control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated for each light receiving element in the past frame period. Distance measuring device.
(3)
The distance measuring device according to (1) or (2) above, wherein the control unit sets a class range for frequency counting in the distance histogram as a setting of the control parameters.
(4)
The distance measuring device according to (3) above, wherein the control unit sets a predetermined class range including the class of the peak value of the distance histogram in the past as the class range to be the target of the frequency count.
(5)
The distance measuring device according to any one of (1) to (4) above, wherein the control unit sets the number of exposures in a frame period as a setting of the control parameters.
(6)
Based on the distance histogram generated in the past frame period, the control unit sets the adaptive control number as the number of frame periods in which the same control parameter is continuously processed from the present time onward. The distance measuring device according to any one of 1) to (5).
(7)
The distance measuring device according to (6) above, wherein the control unit sets the adaptive control number of times based on the amount of change of the distance histogram generated in the past consecutive frame periods.
(8)
The distance measuring device according to (7) above, wherein the control unit sets a larger number of adaptive control times as the amount of change is smaller.
(9)
The control unit determines whether or not to generate the distance histogram in the frame period after the present based on the amount of change of the distance histogram generated in the past consecutive frame periods (1) to (8). ). The distance measuring device according to any one of.
(10)
The control unit determines whether or not to make the light emitting unit emit light in the frame period after the present based on the amount of change of the distance histogram generated in the past consecutive frame periods. Distance measuring device.
(11)
The light receiving unit that obtains the light receiving signal has a plurality of light receiving elements and has a plurality of light receiving elements.
As one of the control parameters, the control unit generates segment information indicating a light receiving element group that has received the reflected light from the same object, and the distance of the light receiving element in the light receiving element group indicated by the segment information. The distance measuring device according to any one of (1) to (10) above, which adds and averages histograms.
(12)
A detection unit that acquires the distance index by the direct ToF method multiple times based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period, and the detection unit multiple times for each frame period. A distance measuring method of a distance measuring device including a histogram generating unit that generates a distance histogram representing the distance index as frequency information for each distance based on the obtained distance index for each frame period.
A distance measuring method for setting control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated in the past frame period.
(13)
A detection unit that acquires the distance index by the direct ToF method multiple times based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period, and the detection unit multiple times for each frame period. A distance measuring device including a distance histogram generating unit that generates a distance histogram representing the distance index as frequency information for each distance based on the obtained distance index for each frame period.
A program that executes a process of setting control parameters related to the generation of the distance histogram in the current and subsequent frame periods based on the distance histogram generated in the past frame period.

Finally, the effects described in the present disclosure are exemplary and not limited, and may have other effects or are part of the effects described in the present disclosure. You may.
Further, the embodiments described in the present disclosure are merely examples, and the present technology is not limited to the above-described embodiments. Therefore, it goes without saying that various changes can be made depending on the design and the like as long as the technical idea of the present technology is not deviated from the above-described embodiment. It should be noted that not all combinations of configurations described in the embodiments are essential for solving the problem.

10 Distance measuring device 11 Control unit 12 Light emitting unit 13 Light receiving unit 14 Detection unit 15 Histogram generation unit 16 Distance measuring unit 21 Discriminating unit 22

Operation control unit

40, 41 Optical Tg Object

Claims

A detection unit that acquires the distance index multiple times by the direct ToF method based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period.
A histogram generator that generates a distance histogram that represents the distance index as frequency information for each distance based on the distance index obtained a plurality of times by the detection unit for each frame period.
A distance measuring device including a control unit for setting control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated in the past frame period.
The light receiving unit for obtaining the light receiving signal has a plurality of light receiving elements and has a plurality of light receiving elements.
The distance according to claim 1, wherein the control unit sets control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated for each light receiving element in the past frame period. measuring device.
The distance measuring device according to claim 1, wherein the control unit sets a class range for frequency counting in the distance histogram as a setting of the control parameters.
The distance measuring device according to claim 3, wherein the control unit sets a predetermined class range including the class of the peak value of the distance histogram in the past as the class range to be the target of the frequency count.
The distance measuring device according to claim 1, wherein the control unit sets the number of exposures in a frame period as a setting of the control parameters.
A claim that the control unit sets an adaptive control number as the number of frame periods in which the same control parameter is continuously processed from the present onward based on the distance histogram generated in the past frame period. The distance measuring device according to 1.
The distance measuring device according to claim 6, wherein the control unit sets the adaptive control number of times based on the amount of change of the distance histogram generated in the past consecutive frame periods.
The distance measuring device according to claim 7, wherein the control unit sets a larger number of adaptive control times as the amount of change is smaller.
The distance according to claim 1, wherein the control unit determines whether or not to generate the distance histogram in the frame period after the present based on the amount of change of the distance histogram generated in the past consecutive frame periods. measuring device.
The distance according to claim 9, wherein the control unit determines whether or not to make the light emitting unit emit light in the frame period after the present based on the amount of change of the distance histogram generated in the past consecutive frame periods. measuring device.
The light receiving unit that obtains the light receiving signal has a plurality of light receiving elements and has a plurality of light receiving elements.
As one of the control parameters, the control unit generates segment information indicating a light receiving element group that has received the reflected light from the same object, and the distance of the light receiving element in the light receiving element group indicated by the segment information. The distance measuring device according to claim 1, wherein the histograms are added and averaged.
A detection unit that acquires the distance index by the direct ToF method multiple times based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period, and the detection unit multiple times for each frame period. A distance measuring method of a distance measuring device including a histogram generating unit that generates a distance histogram representing the distance index as frequency information for each distance based on the obtained distance index for each frame period.
A distance measuring method for setting control parameters related to the generation of the distance histogram in the frame period after the present based on the distance histogram generated in the past frame period.
A detection unit that acquires the distance index by the direct ToF method multiple times based on the received signal of the light emitted by the light emitting unit and reflected by the object for each continuous frame period, and the detection unit multiple times for each frame period. A distance measuring device including a distance histogram generating unit that generates a distance histogram representing the distance index as frequency information for each distance based on the obtained distance index for each frame period.
A program that executes a process of setting control parameters related to the generation of the distance histogram in the current and subsequent frame periods based on the distance histogram generated in the past frame period.