WO2024095626A1 - Ranging device - Google Patents

Abstract

A ranging device according to one embodiment of the present disclosure is provided with a light-emitting unit for emitting irradiation light, and a ranging unit for calculating data pertaining to the distance to a subject on the basis of incident light obtained by irradiating the subject with the irradiation light. The light-emitting unit has a plurality of light-emitting elements arranged in an array, and a drive unit for causing the plurality of light-emitting elements to sequentially emit light in prescribed unit groups within one frame period. Within one frame period, the drive unit causes a plurality of light-emitting elements in a unit group to emit light at a first light emission intensity and then emit light at a second light emission intensity different from the first light emission intensity.

Description

Distance measuring device

This disclosure relates to a distance measuring device.

In recent years, optical detection devices using SPADs (Single Photon Avalanche Diodes) have been attracting attention in fields such as distance measurement sensors (see, for example, Patent Document 1).

JP 2021-120630 A

In the field of distance measuring sensors, there is a demand for further improvements in distance measuring accuracy. Therefore, it is desirable to provide a distance measuring device that can achieve further improvements in distance measuring accuracy.

A distance measuring device according to one aspect of the present disclosure includes a light emitting unit that emits illumination light, and a distance measuring unit that calculates data regarding the distance to a subject based on incident light obtained by irradiating the subject with the illumination light. The light emitting unit has a plurality of light emitting elements arranged in an array, and a driving unit that sequentially causes the plurality of light emitting elements to emit light for each predetermined unit group within one frame period. The driving unit causes the plurality of light emitting elements in the unit group to emit light at a first emission intensity and then at a second emission intensity different from the first emission intensity within one frame period.

In a distance measuring device according to one aspect of the present disclosure, during one frame period, multiple light-emitting elements in a unit group emit light at a first emission intensity, and then emit light at a second emission intensity different from the first emission intensity. As a result, even if the subject is at a close distance, data can be obtained that has little measurement error due to the pile-up phenomenon and is almost completely free of the effects of dead time due to reflection on the cover glass. Furthermore, even if the subject is at a long distance, data can be obtained that allows the distance to the subject to be calculated.

FIG. 1 is a diagram illustrating an example of functional blocks of a distance measuring device and an information processing device according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of output data output from the communication unit in FIG. FIG. 3 is a diagram showing an example of a planar configuration of the light-emitting element array of FIG. FIG. 4 is a diagram showing an example of the planar configuration of the DOE of FIG. 1 and an example of the distribution of light emission spots generated by the DOE of FIG. FIG. 5 is a diagram illustrating an example of functional blocks of the light receiving unit in FIG. FIG. 6 is a diagram illustrating an example of a circuit configuration of the light-receiving pixel in FIG. FIG. 7 is a diagram showing an example of light emission by light-emitting pixels and light reception by light-receiving pixels within one frame period. FIG. 8 is a diagram illustrating an example of a histogram generated by the histogram generating unit of FIG. FIG. 9 is a diagram for explaining the Pile-Up phenomenon. FIG. 10 is a diagram for explaining the influence of reflection on the cover glass. FIG. 11 is a diagram showing a modified example of the functional blocks of the distance measuring device and the information processing device in FIG. FIG. 12 is a diagram illustrating an example of data stored in the storage unit of FIG. FIG. 13 is a diagram showing an example of the various drive modes shown in FIG. FIG. 14 is a diagram showing an example of how light is received in the light receiving element array of FIG. FIG. 15 is a diagram showing an example of how light is received in the light receiving element array of FIG. FIG. 16 is a diagram showing an example of how light is received in the light receiving element array of FIG. FIG. 17 is a diagram showing an example of various drive modes. FIG. 18 is a diagram showing an example of a procedure for switching between the various drive modes shown in FIG. FIG. 19 is a diagram showing a modified example of the functional blocks of the distance measuring device and the information processing device in FIG. FIG. 20 is a diagram illustrating an example of a histogram generated by the histogram generating unit of FIG. FIG. 21 is a diagram illustrating an example of a procedure for calculating the reliability. FIG. 22 is a diagram illustrating an example of output data output from the communication unit in FIG. 19. In FIG. FIG. 23 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 24 is an explanatory diagram showing an example of the installation positions of the outside-vehicle information detection unit and the imaging unit.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The description will be made in the following order.

1. Example of embodiment: Example of switching light emission intensity within one frame period (FIGS. 1 to 10)
2. Modifications Modification A: Setting the switching of the light emission intensity for each driving mode,
Examples of setting a light receiving area for each driving mode (Figures 11 to 16)
Modification B: Example of switching driving modes using distance data (Figs. 17 and 18)
Modification C: Example of calculating reliability (FIGS. 19 to 22)
3. Application examples (Figures 23 and 24)

1. Preferred embodiment
[composition]
1 illustrates an example of functional blocks of a distance measuring device 100 according to an embodiment of the present disclosure and an information processing device 200 that processes data output from the distance measuring device 100. The distance measuring device 100 and the information processing device 200 are connected by an FPC 300.

The ranging device 100 and the information processing device 200 are electrically connected by a data bus included in the FPC 300. The data bus is a signal transmission path that connects the ranging device 100 and the information processing device 200. Transmission data sent from the ranging device 100 is transmitted from the ranging device 100 to the information processing device 200 via the data bus. The ranging device 100 and the information processing device 200 may be electrically connected by a control bus included in the FPC 300. The control bus is another signal transmission path that connects the ranging device 100 and the information processing device 200, and is a transmission path different from the data bus. Control data sent from the information processing device 200 is transmitted from the information processing device 200 to the ranging device 100 via the control bus.

(Information processing device 200)
The information processing device 200 includes a communication unit 210, a processor 220, and a storage unit 230. The communication unit 210 is a communication interface configured to be able to communicate with the distance measuring device 100 via the FPC 300. The processor 220 is configured with one or more processors configured with an arithmetic circuit such as an MPU (Micro Processing Unit), various processing circuits, etc. The processor 220 performs various processes, such as a process related to the recording control of data in the storage unit 230 and a process of executing any application software. The processor 220 may control the functions of the distance measuring device 100, for example, by transmitting control information to the distance measuring device 100. The processor 220 may also control the driving mode of the distance measuring device 100, for example, by transmitting driving mode setting information to the distance measuring device 100.

(Packet structure)
Next, a description will be given of an example of a packet structure used for transmitting transmission data from the distance measuring device 100 to the information processing device 200. The transmission data generated in the distance measuring device 100 is divided into partial data for each row, for example, and the partial data for each row is transmitted using one or more packets.

FIG. 2 shows an example of transmission data transmitted from the distance measuring device 100 to the information processing device 200. FIG. 2 shows an example of transmission data used when transmitting data according to the MIPI CSI-2 standard or the MIPI CSI-3 standard.

As shown in FIG. 2, a packet used to transmit transmission data contains a packet header PH, payload data, and a packet footer PF, arranged in that order. The payload data (hereinafter also simply referred to as "payload") contains pixel data of partial data in row units.

The packet header PH is, for example, the packet header of the PayloadData of a LongPacket. A LongPacket refers to a packet that is placed between the packet head PH and the packet footer PF. The PayloadData of a LongPacket refers to the main data transmitted between devices.

The transmission data is composed of a data frame, for example, as shown in Figure 2. A data frame usually has a header area, a packet area, and a footer area. In a data frame, the header area contains EmbeddedData. EmbeddedData refers to additional information that can be embedded in the header or footer of a data frame. In this case, EmbeddedData contains the frame number, channel number, and emission intensity information.

Also, as shown in FIG. 2, in the data frame, the packet area contains the payload data of the LongPacket for each line, and further contains a packet header PH and a packet footer PF at the position sandwiching the payload data of the LongPacket. The packet area also contains, for example, histogram data described below, or distance data calculated based on a histogram. The histogram data or distance data calculated based on a histogram corresponds to a specific example of "data relating to the distance to the subject" in this disclosure. The payload data of the LongPacket for each line contains pixel data for one line in the histogram data described below, or distance data calculated based on a histogram.

(Range measuring device 100)
The distance measuring device 100 includes a light emitting unit 110, a light receiving unit 120, a cover glass 130, a histogram generating unit 140, a communication unit 150, and a controller 160. In the distance measuring device 100, the light receiving unit 120, the histogram generating unit 140, the communication unit 150, and the controller 160 are provided, for example, in one sensor chip.

The light emitting unit 110 has a drive circuit 111, a light emitting element array 112, and a DOE 113.

The light-emitting element array 112 has a configuration in which a plurality of light-emitting pixels Pa are arranged in an array, as shown in FIG. 3, for example. Each light-emitting pixel Pa includes a light-emitting element. Driven by the drive circuit 111, each light-emitting element outputs light of a predetermined wavelength (also called irradiation light) at a predetermined pulse repetition period (also called PRI period). For each light-emitting element, for example, a vertical cavity surface-emitting laser (VCSEL) can be used. However, various light sources capable of emitting light of a predetermined wavelength may be used for each light-emitting element.

In the light-emitting element array 112, for example, as shown in FIG. 3, a number of light-emitting pixels Pa are divided into N channels Ch1 to ChN. Each channel Ch corresponds to a specific unit group that is driven simultaneously by the drive circuit 111.

The drive circuit 111 drives each pixel Pa of the light-emitting element array 112. The drive circuit 111 may be configured to be capable of driving each pixel Pa independently, or may be configured to be capable of driving each pixel Pa in a predetermined unit group. When enabling independent driving of each pixel Pa, the drive circuit 111 may include, for example, a plurality of drive circuits provided in a one-to-one correspondence with each of the plurality of pixels Pa. When enabling driving of the plurality of pixels Pa in a predetermined unit group, the drive circuit 111 may include, for example, a plurality of drive circuits provided one for each channel Ch. The drive circuit 111 is configured to sequentially drive the plurality of pixels Pa for each channel Ch. In other words, the drive circuit 111 is configured to sequentially cause the plurality of pixels Pa (light-emitting elements) to emit light for each channel Ch.

The DOE 113 is a diffractive optical element. The DOE 113 splits the light emitted from each light-emitting pixel Pa into multiple beams by using a diffraction grating that utilizes the diffraction phenomenon of light. For example, as shown in FIG. 4, the DOE 113 splits each light beam incident from the light-emitting element array 112 into three beams. This allows the DOE 113 to generate three times the number of light-emitting spots Sp as the number of light-emitting pixels Pa included in the light-emitting element array 112. Note that the number of beams split by the DOE 113 is not limited to three, and may be two, or four or more.

The light generated by the DOE 113 is emitted to the outside through the cover glass 130. The light emitted to the outside through the cover glass 130 becomes irradiation light La and reaches the subject. In other words, the light emitter 110 emits irradiation light La to the subject. The irradiation light La is reflected by the subject, and the reflected light (incident light Lb) generated by the reflection on the subject is incident on the light receiver 120 through the cover glass 130. The DOE 113 may be omitted if necessary.

The light receiving unit 120 receives reflected light (incident light Lb) obtained by irradiating the subject with the irradiation light La. The light receiving unit 120 has a drive circuit 121, a light receiving element array 122, and a TDC (Time-to-Digital Converter) 123.

The light receiving element array 122 has a configuration in which a plurality of light receiving pixels Pb are arranged in an array, as shown in FIG. 5, for example. Each light receiving pixel Pb has a light receiving element (SPAD element 21), a readout circuit 22, a latch 26, and an output buffer 27, as shown in FIG. 6, for example.

The SPAD element 21 operates in Geiger mode, and generates an avalanche current when a photon is incident while a negative bias voltage VSPAD (e.g., about -10V to -20V) equal to or greater than the breakdown voltage is applied between the anode and cathode.

The readout circuit 22 is composed of three transistors 23 to 25. Transistor 23 is, for example, a quench resistor and may be composed of a P-type MOS (Metal-Oxide-Semiconductor) transistor. Transistors 24 and 25 are, for example, selection transistors for selecting the light-receiving pixel Pb. Transistor 24 may be, for example, a P-type MOS transistor, and transistor 25 may be, for example, an N-type MOS transistor.

For example, a preset bias voltage (also called a quench voltage) is applied to the gate of transistor 23, which is a quench resistor, to make transistor 23 act as a quench resistor. For example, an output from latch 26, which will be described later, is applied to the gates of transistors 24 and 25.

The latch 26 is composed of a latch circuit 261, a NAND circuit 262, and a buffer 263. The latch circuit 261 holds the selection/non-selection information of the pixel. The NAND circuit 262 calculates a negative logical product of the selection/non-selection information of the pixel held in the latch circuit 261 and the column selection signal YE. The result of this negative logical product is applied to the gates of the selection transistors 24 and 25 in the readout circuit 22, and is also input to the control terminal OE of the output buffer 27 via the buffer 263.

In the above configuration, when the light-receiving pixel Pb is in a non-selected state ('0' is stored in the latch circuit 261), the pixel selection signal PXSEL = 0. At this time, the output PXE of the NAND circuit 262 is PXE = 1, so that the transistor 24 is in an off state and the transistor 25 is in an on state, causing the cathode potential VS of the SPAD element 21 to become 0V. In other words, only the breakdown voltage is applied to the SPAD element 21, and the device enters a mode in which photons are not detected.

When the light-receiving pixel Pb is in the selected state ('1' is stored in the latch circuit 261), the pixel selection signal PXSEL becomes 1. At this time, if the column selection signal YE = 1, the output of the NAND circuit 262 becomes PXE = 0, the transistor 24 becomes on, the transistor 25 becomes off, and a voltage equal to or higher than the breakdown voltage is applied to the SPAD element 21. This makes it easier for the charge generated by photoelectric conversion when a photon is incident on the SPAD element 21 to reach the avalanche multiplication region.

When the charge generated by the incidence of a photon reaches the avalanche multiplication region and an avalanche current is generated, the cathode potential VS of the SPAD element 21 is discharged to the 0V side, and the cathode potential VS is recharged by the current source of the quench resistor 23.

However, even if the light-receiving pixel Pb is in a selected state in the latch circuit 261, if the region to which the light-receiving pixel Pb belongs is not selected (column selection signal YE=0), although a breakdown voltage is applied between both electrodes of the SPAD element 21, the light-receiving pixel Pb itself is in a non-selected state, and photon detection is not performed.

When the light-receiving pixel Pb detects a photon, a detection signal PXOUT whose rising edge coincides with the timing of the detection of this photon is output from the output buffer 27. The output buffer 27 corresponds to the output circuit 126 shown in FIG. 5.

The TDC 123 converts the timing of the rising edge of the detection signal PXOUT into a digital value. For example, the TDC 123 measures the flight time of a photon from when the light emitting unit 110 emits light until the reflected light (incident light Lb) is detected by the light receiving pixel Pb, and outputs the measured time as a digital output value TDCOUT. For example, the TDC 123 has a TDC circuit for each light receiving pixel Pb, and measures the elapsed time from the output timing of the irradiated light La to the detection timing of the incident light Lb for each light receiving pixel Pb with high resolution (for example, a period of about 100 ps (picoseconds)), and outputs the time obtained thereby as a digital output value TDCOUT.

The driving circuit 121 controls the light receiving element array 122 and the TDC 123. The driving circuit 121 includes, for example, a timing control circuit 124 and a driving circuit 125, as shown in FIG. 5.

The timing control circuit 124 includes a timing generator that generates various timing signals, and controls the drive circuit 125 and output circuit 126 based on the various timing signals generated by the timing generator.

The drive circuit 125 includes a shift register, an address decoder, and the like, and drives each light-receiving pixel Pb of the light-receiving element array 122. The drive circuit 125 includes at least a circuit that applies a bias voltage (quench voltage) (described later) to each selected light-receiving pixel Pb in the light-receiving element array 122.

The detection signal PXOUT output from each light receiving pixel Pb selected by the drive circuit 125 is input to the output circuit 126 through each output signal line LS. The output circuit 126 outputs the detection signal PXOUT input from each light receiving pixel Pb selected by the drive circuit 125 to the TDC 123.

(sampling period)
Incidentally, in the TDC 123, a period for measuring the flight time of photons from when the light-emitting unit 110 emits light until the reflected light (incident light Lb) is detected by the light-receiving pixel Pb is called a sampling period. The sampling period is set to a period shorter than the PRI period of the light-emitting unit 110. For example, by shortening the sampling period, it becomes possible to estimate or calculate the flight time of photons emitted from the light-emitting unit 110 and reflected by the subject with a higher time resolution. This means that by increasing the sampling frequency, it becomes possible to estimate or calculate the distance to the subject with a higher ranging resolution.

For example, if the flight time from when the light-emitting unit 110 emits irradiation light La, when this irradiation light La is reflected by the subject, until this reflected light (incident light Lb) is incident on the light-receiving unit 120 is t, then since the speed of light C is constant (C ≈ 300,000,000 m (meters)/s (seconds), the distance L to the subject can be estimated or calculated as shown in the following formula (1).
L = C × t / 2 (1)

If the sampling frequency is 1 GHz, the sampling period is 1 ns (nanosecond). In that case, one sampling period corresponds to 15 cm (centimeter). This indicates that the distance measurement resolution when the sampling frequency is 1 GHz is 15 cm. Furthermore, if the sampling frequency is doubled to 2 GHz, the sampling period becomes 0.5 ns (nanosecond), and one sampling period corresponds to 7.5 cm (centimeter). This indicates that the distance measurement resolution can be halved when the sampling frequency is doubled. In this way, by increasing the sampling frequency and shortening the sampling period, it is possible to estimate or calculate the distance to the subject with greater accuracy.

The histogram generating unit 140 generates a histogram for each light receiving pixel Pb based on the output value TDCOUT of each light receiving pixel Pb output from the TDC 123. The histogram relates to, for example, the time from when the light emitting pixel Pa is driven until the incidence of light on one or more light receiving pixels Pb is detected. The histogram generating unit 140 generates a histogram for each light receiving pixel Pb by, for example, adding the output value TDCOUT output from the TDC 123 for each light receiving pixel Pb to a count value (accumulated value) stored in a BIN corresponding to the sampling period. The histogram generating unit 140 may calculate distance data based on the generated histogram as necessary.

The communication unit 150 is a communication interface configured to be able to communicate with the information processing device 200 via the FPC 300. The communication unit 150 transmits the histogram generated for each light receiving pixel Pb in the histogram generation unit 140, or distance data calculated based on the generated histogram, as transmission data to the information processing device 200 via the FPC 300. The communication unit 150 receives drive mode setting information from the information processing device 200 via the FPC 300. The communication unit 150 outputs the received drive mode setting information to the controller 160.

The controller 160 controls the light-emitting unit 110 and the light-receiving unit 120. The controller 160, for example, controls the light emission of the light-emitting unit 110 and the light reception of the light-receiving unit 120. The controller 160 controls the light-emitting unit 110 and the light-receiving unit 120 based on control information (e.g., drive mode setting information) received from the information processing device 200 via the FPC 300, for example.

Next, the switching of the light emission intensity in the light-emitting unit 110 will be described with reference to FIG. 7. FIG. 7 shows an example of the light emission of the light-emitting pixel Pa and the light reception of the light-receiving pixel Pb within one frame period.

The driving unit 111, in accordance with the control of the controller 160, causes a plurality of light-emitting pixels Pa (light-emitting elements) in a unit group (channel ch) to emit light at a first emission intensity and then at a second emission intensity different from the first emission intensity during one frame period. The driving unit 111 further causes a plurality of light-emitting pixels Pa (light-emitting elements) to emit light sequentially for each of channels ch1 to chN during one frame period, in accordance with the control of the controller 160. The first emission intensity is, for example, a relatively weaker emission intensity than the second emission intensity. The second emission intensity is, for example, a relatively stronger emission intensity than the first emission intensity. The driving unit 111 sets the first emission intensity and the second emission intensity according to control information (for example, drive mode setting information) received from the information processing device 200.

When the driving unit 111 switches the light emission intensity as described above within one frame period, in the light emitting unit 110, for example, as shown in FIG. 7, within one frame period, multiple light emitting pixels Pa (light emitting elements) in a unit group (channel ch) emit light at a first light emission intensity and then emit light at a second light emission intensity. In the light emitting unit 110, for example, as shown in FIG. 7, within one frame period, multiple light emitting pixels Pa (light emitting elements) emit light sequentially for each of channels ch1 to chN.

At this time, the drive unit 121, in accordance with the control of the controller 160, causes the multiple light receiving pixels Pb corresponding to the emitting channel ch to sequentially perform exposure, readout, and standby in synchronization with the light emission (weak light emission) of the channel ch in the light emitting unit 110. Furthermore, the drive unit 121, in accordance with the control of the controller 160, causes the multiple light receiving pixels Pb corresponding to the emitting channel ch to sequentially perform exposure, readout, and standby in synchronization with the light emission (strong light emission) of the channel ch in the light emitting unit 110.

The driving unit 111 further causes the multiple light receiving pixels Pb (light receiving elements) to sequentially perform exposure, readout, and standby for each of the multiple light receiving pixels Pb corresponding to the channel ch that emits light (weak light emission) during one frame period in accordance with the control of the controller 160.The driving unit 111 further causes the multiple light receiving pixels Pb (light receiving elements) to sequentially perform exposure, readout, and standby for each of the multiple light receiving pixels Pb corresponding to the channel ch that emits light (strong light emission) during one frame period in accordance with the control of the controller 160.

When the driving unit 121 causes multiple light receiving pixels Pb (light receiving elements) corresponding to a channel ch that emits light (weak light emission) to perform exposure, readout, and standby, the multiple light receiving pixels Pb (light receiving elements) corresponding to a unit group (channel ch) in the light receiving unit 120, for example, as shown in FIG. 7, perform exposure, readout, and standby. When the driving unit 121 causes multiple light receiving pixels Pb corresponding to a channel ch that emits light (strong light emission) to perform exposure, readout, and standby, the multiple light receiving pixels Pb (light receiving elements) corresponding to a unit group (channel ch) in the light receiving unit 120, for example, perform exposure, readout, and standby.

The histogram generating unit 140 generates a histogram (first data related to the distance to the subject) for each light receiving pixel Pb based on the output value TDCOUT of the multiple light receiving pixels Pb (light receiving elements) corresponding to the channel ch that emits light (weak light emission) output from the TDC 123. The histogram generating unit 140 generates a histogram (first data related to the distance to the subject) for each light receiving pixel Pb by, for example, adding the output value TDCOUT output from the TDC 123 for each light receiving pixel Pb (light receiving element) corresponding to the channel ch that emits light (weak light emission) to a count value (accumulated value) stored in a BIN corresponding to the sampling period. The histogram generating unit 140 may calculate distance data based on the generated histogram (first data related to the distance to the subject) as necessary.

The histogram generating unit 140 generates a histogram (second data related to the distance to the subject) for each light receiving pixel Pb based on the output values TDCOUT of the multiple light receiving pixels Pb (light receiving elements) corresponding to the channel ch that emits light (strong light emission) output from the TDC 123. The histogram generating unit 140 generates a histogram (second data related to the distance to the subject) for each light receiving pixel Pb by, for example, adding the output values TDCOUT output from the TDC 123 for each light receiving pixel Pb (light receiving elements) corresponding to the channel ch that emits light (strong light emission) to a count value (accumulated value) stored in a BIN corresponding to the sampling period. The histogram generating unit 140 may calculate distance data based on the generated histogram (second data related to the distance to the subject) as necessary.

The communication unit 150 outputs at least one of the first data and the second data calculated by the histogram generation unit 140 and data on the light emission intensity of the light-emitting unit 110 as transmission data in a predetermined format. If necessary, the communication unit 150 may output distance data corresponding to at least one of the first data and the second data and data on the light emission intensity of the light-emitting unit 110 as transmission data in a predetermined format.

(Pile-Up phenomenon)
Fig. 8 shows an example of a histogram generated by the histogram generating unit 140. When the subject is relatively close, the histogram obtained with weak light emission has a peak value at a BIN number smaller than the BIN number corresponding to a distance of 1 m, as shown in Fig. 8(A), for example. When the peak value is greater than a predetermined threshold value at this time, the distance corresponding to the peak value corresponds to the distance to the subject. Note that Figs. 8(A) and 8(B) show 1 m as an example of the boundary between close and not close distances, but the boundary is not limited to 1 m.

When the subject is relatively close, the histogram obtained with strong light emission has a peak value at a BIN number smaller than the BIN number corresponding to a distance of 1 m, as shown in FIG. 8B, for example. However, at this time, the so-called pile-up phenomenon occurs, so even if the peak value at this time is greater than a predetermined threshold, the distance corresponding to the peak value is shorter than the actual distance to the subject. In this pile-up phenomenon, for example, as shown in FIG. 9, when comparing the BIN number (first BIN number) corresponding to the peak value obtained when incident light Lb of appropriate intensity is incident on the light receiving element array 122 with the BIN number (second BIN number) corresponding to the peak value obtained when incident light Lb of strong intensity is incident on the light receiving element array 122, the second BIN number is smaller than the first BIN number by a predetermined magnitude (ΔX). Therefore, when the subject is relatively close, it is desirable to derive the distance to the subject based on the histogram obtained with weak light emission, not the histogram obtained with strong light emission.

When incident light Lb enters the light receiving element array 122 and a light receiving element reacts, the reacting light receiving element then enters a certain period of time during which it is in principle unable to detect light (dead time). For example, when strong intensity illumination light La is emitted from the light emitting pixel Pa, the light reflected by the cover glass 130 also becomes strong and enters the light receiving element. At this time, there is a high probability that the light receiving element will react and then enter the dead time, so for example, as shown in Figure 10 (A), the light receiving element will not be able to detect light reflected from a nearby subject.

On the other hand, when irradiation light La of a moderate intensity is emitted from the light-emitting pixel Pa, the probability of the light receiving element reacting to the light reflected by the cover glass 130 is also reduced, so in distance measurement control that repeats light emission and distance measurement many times, it becomes possible to measure distances at relatively short distances, for example, as shown in Figure 10 (B).

In light of the above, when the subject is relatively close, it is desirable to derive the distance to the subject based on a histogram obtained with light of moderate intensity (e.g., weak intensity) rather than a histogram obtained with strong light.

When the subject is at a relatively long distance, the histogram obtained with weak light emission does not have a peak value exceeding the threshold, as shown in FIG. 8(C), for example. This is because the intensity of the reflected light is insufficient. On the other hand, the histogram obtained with strong light emission has a peak value exceeding the threshold beyond the boundary (1 m) between close and not close distances, as shown in FIG. 8(D), for example. Note that although 1 m is shown as an example of the boundary between close and not close distances in FIG. 8(C) and FIG. 8(D), the boundary is not limited to 1 m. Therefore, when the subject is at a relatively long distance, it is desirable to derive the distance to the subject based on the histogram obtained with strong light emission, rather than the histogram obtained with weak light emission. Note that when the subject is at a relatively long distance, there is almost no effect from the Pile-Up phenomenon.

[effect]
Next, the effects of the distance measuring device 100 will be described.

In this embodiment, in one frame period, multiple light-emitting pixels Pa (light-emitting elements) in a unit group (channel ch) emit light at a first emission intensity, and then emit light at a second emission intensity different from the first emission intensity. As a result, even if the subject is at a close distance, for example, data can be obtained that has little measurement error due to the Pile-Up phenomenon and is almost free of the effects of dead time due to reflection on the cover glass. Furthermore, even if the subject is at a long distance, for example, data can be obtained that allows the distance to the subject to be calculated. Therefore, the distance to the subject can be calculated with greater accuracy than when light is emitted at a single intensity.

In this embodiment, a histogram (first data related to the distance to the subject) is generated for each light receiving pixel Pb based on the output value TDCOUT of the multiple light receiving pixels Pb (light receiving elements) corresponding to the channel ch that emits light (weak light emission) output from the TDC 123. Furthermore, a histogram (second data related to the distance to the subject) is generated for each light receiving pixel Pb based on the output value TDCOUT of the multiple light receiving pixels Pb (light receiving elements) corresponding to the channel ch that emits light (strong light emission) output from the TDC 123. As a result, for example, even if the subject is at a close distance, data (first data) is obtained that has little measurement error due to the Pile-Up phenomenon and is almost free of the influence of dead time due to reflection on the cover glass. Also, for example, even if the subject is at a long distance, data (second data) that can be used to calculate the distance to the subject is obtained. Therefore, the distance to the subject can be calculated with higher accuracy than when light is emitted at a single intensity.

In this embodiment, at least one of the first data and the second data calculated by the histogram generating unit 140 and data on the light emission intensity of the light emitting unit 110 are output as output data (transmission data) in a predetermined format. This allows the information processing device 200 to accurately calculate the distance to the subject based on the transmission data.

2. Modified Examples
Next, a modification of the distance measuring device 100 according to the above embodiment will be described.

[Modification A]
In the above embodiment, the distance measuring device 100 may include a storage unit 170, for example, as shown in Fig. 11. The storage unit 170 may be configured, for example, with a random access memory (RAM), a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage unit 170 may store a data set in which MP setting data corresponding to a drive mode is defined, for example, as shown in Fig. 12. This data set will be described in detail later.

The various drive modes illustrated in FIG. 12 have, for example, the conditions and features shown in FIG. 13. As shown in FIG. 13, when the drive mode is Dual1 or Dual2, the controller 160 instructs the drive unit 111 to switch the light emission intensity. Also, as shown in FIG. 13, when the drive mode is Single1 or Single2, the controller 160 instructs the drive unit 111 to drive at a single light emission intensity.

In this modified example, the information processing device 200 (processor 220) transmits drive mode setting information to the distance measuring device 100. When the distance measuring device 100 (controller 160) receives the drive mode setting information from the information processing device 200 (processor 220), it instructs the drive unit 111 to set the light emission intensity according to the received drive mode setting information. When the drive mode setting information acquired from the information processing device 200 (processor 220) is a mode (e.g., Dual1, Dual2) that assumes that the subject is located at a relatively long distance, the distance measuring device 100 (controller 160) instructs the drive unit 111 to make the multiple light emitting pixels Pa (light emitting elements) in the unit group (channel ch) emit light at a first light emission intensity and then at a second light emission intensity during one frame period. This makes it possible to select a drive mode according to the distance to the subject, and therefore to accurately calculate the distance to the subject.

Next, we will explain the data set stored in the memory unit 170.

In this data set, the MP setting data refers to, for example, address data of the light receiving pixels Pb to be activated according to the drive mode. This address data refers to address data of some of the light receiving pixels Pb (light receiving elements) among the light receiving pixels Pb (light receiving elements) included in the light receiving element array 122. Address data that is different from the address data assigned to a mode that is assigned to a subject that is relatively close (for example, Dual1, Single2, Dual2) is assigned to a mode that is assigned to a subject that is relatively far away (for example, Single1).

When the drive mode is Dual1, the controller 160 reads out the MP setting data data1 corresponding to Dual1 from the data set in the memory unit 170, and instructs the drive unit 121 to activate the multiple light receiving pixels Pb (light receiving elements) corresponding to the read MP setting data data1. The drive unit 121 activates the multiple light receiving pixels Pb that correspond to the MP setting data data1 among the multiple light receiving pixels Pb, thereby causing the multiple active light receiving pixels Pb to output light receiving data.

When the drive mode is Single1, the controller 160 reads out the MP setting data data2 corresponding to Single1 from the data set in the memory unit 170, and instructs the drive unit 121 to activate the multiple light receiving pixels Pb (light receiving elements) corresponding to the read MP setting data data2. The drive unit 121 activates the multiple light receiving pixels Pb that correspond to the MP setting data data2 among the multiple light receiving pixels Pb, thereby causing the multiple active light receiving pixels Pb to output light receiving data.

When the drive mode is Single2, the controller 160 reads out the MP setting data data3 corresponding to Single2 from the data set in the memory unit 170, and instructs the drive unit 121 to activate the multiple light receiving pixels Pb (light receiving elements) corresponding to the read MP setting data data3. The drive unit 121 activates the multiple light receiving pixels Pb that correspond to the MP setting data data3 among the multiple light receiving pixels Pb, thereby causing the multiple active light receiving pixels Pb to output light receiving data.

When the drive mode is Dual2, the controller 160 reads out the MP setting data data4 corresponding to Dual2 from the data set in the memory unit 170, and instructs the drive unit 121 to activate the multiple light receiving pixels Pb (light receiving elements) corresponding to the read MP setting data data4. The drive unit 121 activates the multiple light receiving pixels Pb that correspond to the MP setting data data4 among the multiple light receiving pixels Pb, thereby causing the multiple active light receiving pixels Pb to output light receiving data.

As described above, in the light receiving element array 122, when only a portion of the multiple light receiving pixels Pb (light receiving elements) are active, the region (active pixel region MP) that includes the multiple active light receiving pixels Pb (light receiving elements) is fixed regardless of the distance to the subject, for example, as shown in FIG. 14. In that case, the position of the region (illumination spot Rb) onto which the incident light Lb is irradiated will be displaced depending on the distance to the subject due to the effects of parallax, for example, as shown in FIG. 14, and the illumination spot Rb may fall outside the active pixel region MP. If the illumination spot Rb falls outside the active pixel region MP, the subject cannot be detected.

As a result, it is possible to consider, for example, expanding the area of the active pixel region MP in the light receiving element array 122, as shown in FIG. 15. However, doing so would increase the amount of power consumed by the light receiving element array 122 by the amount of expansion of the area of the active pixel region MP.

On the other hand, in this modified example, when the controller 160 receives drive mode setting information from the information processing device 200 (processor 220), it reads out MP setting data corresponding to the received drive mode setting information from the data set in the storage unit 170. The controller 160 instructs the drive unit 121 to activate a plurality of light receiving pixels Pb (light receiving elements) that correspond to the read MP setting data, among the plurality of light receiving pixels Pb (light receiving elements) included in the light receiving element array 122. This allows the controller 160 to output light receiving data from the plurality of active light receiving pixels Pb (light receiving elements).

The driving unit 121 sets an active pixel region MP according to the driving mode in accordance with control from the controller 160. This allows selective activation of multiple light receiving pixels Pb included in the area where the incident light Lb is incident in the light receiving element array 122. As a result, power consumption in the light receiving element array 122 can be kept low compared to when all light receiving pixels Pb included in the light receiving element array 122 are activated. Furthermore, regardless of the distance to the subject, at least a portion of the irradiation spot Rb can be made to overlap at least a portion of the active pixel region MP. As a result, the subject can be reliably detected regardless of the distance to the subject while keeping power consumption in the light receiving element array 122 low. Therefore, the distance to the subject can be derived with high accuracy.

Furthermore, in this modified example, when the controller 160 receives drive mode setting information from the information processing device 200 (processor 220), it reads MP setting data corresponding to the received drive mode setting information from the data set in the storage unit 170, and instructs the drive unit 121 to activate multiple light receiving pixels Pb (light receiving elements) corresponding to the read MP setting data. This allows the controller 160 to determine multiple light receiving pixels Pb (light receiving elements) to be activated according to the drive mode obtained from the information processing device 200 (processor 220).

If the drive mode acquired from the information processing device 200 (processor 220) is a mode (e.g., Dual1, Single1) that assumes that the subject is located at a relatively long distance, the controller 160 sets the active pixel area MP, for example, at the position shown in FIG. 16(A). If the drive mode acquired from the information processing device 200 (processor 220) is a mode (e.g., Dual2) that assumes that the subject is located at a relatively medium distance, the controller 160 sets the active pixel area MP, for example, at the position shown in FIG. 16(B). If the drive mode acquired from the information processing device 200 (processor 220) is a mode (e.g., Single2) that assumes that the subject is located at a relatively short distance, the controller 160 sets the active pixel area MP, for example, at the position shown in FIG. 16(C).

In this case, at least a portion of the irradiation spot Rb can be made to overlap at least a portion of the active pixel region MP regardless of the drive mode. As a result, the power consumption in the light receiving element array 122 can be kept low and the subject can be reliably detected regardless of the distance to the subject. Furthermore, even if the drive mode is dynamically changed under control of the information processing device 200 (processor 220), the power consumption in the light receiving element array 122 can be kept low and the subject can be reliably detected regardless of the drive mode. Therefore, the distance to the subject can be derived with high accuracy.

[Modification B]
In the above embodiment and its modified examples, the histogram generating unit 140 may calculate distance data based on the generated histogram. In this case, the controller 160 may dynamically set the drive mode based on the distance data calculated by the histogram generating unit 140.

FIG. 17 shows an example of various drive modes. Three drive modes are shown in FIG. 17: basic mode, close distance mode, and medium distance mode. The controller 160 dynamically sets, for example, one of the three drive modes shown in FIG. 17 based on the distance data calculated by the histogram generating unit 140.

FIG. 18 shows an example of a procedure for dynamically setting one of the three drive modes shown in FIG. 17. First, the controller 160 controls the drive units 111 and 121 in the basic mode shown in FIG. 17 (step S101). At that time, the controller 160 acquires the distance data D calculated by the histogram generating unit 140 (step S102). If the distance data D is shorter than 3 m and shorter than 1 m (step S103: Y, step S104: Y), the controller 160 controls the drive units 111 and 121 in the medium distance mode shown in FIG. 17 (step S105). If the distance data D is shorter than 3 m and longer than 1 m (step S103: Y, step S104: N), the controller 160 controls the drive units 111 and 121 in the short distance mode shown in FIG. 17 (step S106).

In this manner, in this modified example, the drive mode is dynamically set based on the distance data calculated by the histogram generating unit 140. This makes it possible to derive the distance to the subject with high accuracy.

[Modification C]
In the above embodiment and its modified examples, the distance measuring device 100 may further include a reliability calculation unit 180 that calculates a reliability C (score value) as shown in Fig. 19. The reliability C is useful when the subject is at a close distance.

For example, as shown in FIG. 20(A), suppose that the distance corresponding to the peak of the histogram (first data) obtained by weak light emission when the subject is close is just slightly larger than the boundary between close and not close (for example, 1 m). Furthermore, as shown in FIG. 20(B), suppose that the peak of the histogram (second data) obtained by strong light emission when the subject is close is just slightly smaller than the boundary between close and not close (for example, 1 m) due to the pile-up phenomenon.

At this time, the distance to the subject cannot be derived from the first data and the second data. To avoid such problems, the reliability calculation unit 180 calculates the reliability C (= C1) of the first data and the reliability C (= C2) of the second data to determine whether the first data and the second data can be regarded as having been obtained by correct measurement.

Specifically, first, the distance measuring device 100 performs driving that switches between strong and weak light emission within one frame period, and acquires a histogram (first data, second data) of each light receiving pixel Pb (light receiving element) based on the light received at that time (strong and weak light emission distance measuring, step S201). Next, the reliability calculation unit 180 calculates the difference (first S/N ratio) between the peak value and the noise value contained in the histogram (first data), and calculates the reliability C (= C1) based on the calculated first S/N ratio (step S202).

If the first S/N ratio is greater than a predetermined threshold TH1 (step S203: Y), the reliability calculation unit 180 determines that the first data is reliable and outputs the reliability C (= C1) (step S205). If the first S/N ratio is equal to or less than the predetermined threshold TH1 (step S203: N), the reliability calculation unit 180 determines that the first data is not reliable. At this time, the reliability calculation unit 180 calculates the difference (second S/N ratio) between the peak value and the noise value included in the histogram (second data), and calculates the reliability C (= C2) based on the calculated second S/N ratio (step S204). If the second S/N ratio is greater than the predetermined threshold TH2, the reliability calculation unit 180 determines that the first data is reliable and outputs the reliability C (= C1) (step S205). If the second S/N ratio is equal to or less than a predetermined threshold TH2, the reliability calculation unit 180 determines that the second data is not reliable and outputs the reliability C (= C2) (step S205).

The communication unit 150 outputs at least one of the first data and the second data, data on the light emission intensity of the light-emitting unit 110, and the reliability C as transmission data in a predetermined format. If necessary, the communication unit 150 may output distance data corresponding to at least one of the first data and the second data, data on the light emission intensity of the light-emitting unit 110, and the reliability C as transmission data in a predetermined format.

In this modified example, transmission data including the reliability C (score value) is output. This makes it possible to derive the distance to the subject from at least one of the first data and the second data. As a result, it is possible to avoid a situation in which the distance to the subject cannot be derived due to the peak positions in the first data and the second data.

<3. Application Examples>
The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving object, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, a construction machine, or an agricultural machine (tractor).

23 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology disclosed herein can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. In the example shown in FIG. 23, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an outside vehicle information detection unit 7400, an inside vehicle information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be, for example, an in-vehicle communication network conforming to any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).

Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores the programs executed by the microcomputer or parameters used in various calculations, and a drive circuit that drives various devices to be controlled. Each control unit includes a network I/F for communicating with other control units via a communication network 7010, and a communication I/F for communicating with devices or sensors inside and outside the vehicle by wired or wireless communication. In FIG. 23, the functional configuration of the integrated control unit 7600 includes a microcomputer 7610, a general-purpose communication I/F 7620, a dedicated communication I/F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle device I/F 7660, an audio/image output unit 7670, an in-vehicle network I/F 7680, and a storage unit 7690. Other control units also include a microcomputer, a communication I/F, a storage unit, and the like.

The drive system control unit 7100 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 functions as a control device for a drive force generating device for generating a drive force for the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force for the vehicle. The drive system control unit 7100 may also function as a control device such as an ABS (Antilock Brake System) or ESC (Electronic Stability Control).

The drive system control unit 7100 is connected to a vehicle state detection unit 7110. The vehicle state detection unit 7110 includes at least one of the following: a gyro sensor that detects the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle, or a sensor for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine speed, or the rotation speed of the wheels. The drive system control unit 7100 performs arithmetic processing using the signal input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

The body system control unit 7200 controls the operation of various devices installed in the vehicle body according to various programs. For example, the body system control unit 7200 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, tail lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals from various switches transmitted from a portable device that replaces a key can be input to the body system control unit 7200. The body system control unit 7200 accepts the input of these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

The battery control unit 7300 controls the secondary battery 7310, which is the power supply source for the drive motor, according to various programs. For example, information such as the battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from a battery device equipped with the secondary battery 7310. The battery control unit 7300 performs calculations using these signals, and controls the temperature regulation of the secondary battery 7310 or a cooling device or the like equipped in the battery device.

The outside vehicle information detection unit 7400 detects information outside the vehicle equipped with the vehicle control system 7000. For example, at least one of the imaging unit 7410 and the outside vehicle information detection unit 7420 is connected to the outside vehicle information detection unit 7400. The imaging unit 7410 includes at least one of a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. The outside vehicle information detection unit 7420 includes at least one of an environmental sensor for detecting the current weather or climate, or a surrounding information detection sensor for detecting other vehicles, obstacles, pedestrians, etc., around the vehicle equipped with the vehicle control system 7000.

The environmental sensor may be, for example, at least one of a raindrop sensor that detects rain, a fog sensor that detects fog, a sunshine sensor that detects the level of sunlight, and a snow sensor that detects snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) device. The imaging unit 7410 and the outside vehicle information detection unit 7420 may each be provided as an independent sensor or device, or may be provided as a device in which multiple sensors or devices are integrated.

24 shows an example of the installation positions of the imaging unit 7410 and the outside vehicle information detection unit 7420. The imaging units 7910, 7912, 7914, 7916, and 7918 are provided, for example, at least one of the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 7900. The imaging unit 7910 provided on the front nose and the imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 7900. The imaging units 7912 and 7914 provided on the side mirrors mainly acquire images of the sides of the vehicle 7900. The imaging unit 7916 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 7900. The imaging unit 7918 provided on the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

Note that FIG. 24 shows an example of the imaging ranges of the imaging units 7910, 7912, 7914, and 7916. Imaging range a indicates the imaging range of the imaging unit 7910 provided on the front nose, imaging ranges b and c indicate the imaging ranges of the imaging units 7912 and 7914 provided on the side mirrors, respectively, and imaging range d indicates the imaging range of the imaging unit 7916 provided on the rear bumper or back door. For example, an overhead image of the vehicle 7900 viewed from above is obtained by superimposing the image data captured by the imaging units 7910, 7912, 7914, and 7916.

External information detection units 7920, 7922, 7924, 7926, 7928, and 7930 provided on the front, rear, sides, corners, and upper part of the windshield inside the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. External information detection units 7920, 7926, and 7930 provided on the front nose, rear bumper, back door, and upper part of the windshield inside the vehicle 7900 may be, for example, LIDAR devices. These external information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

The explanation will be continued by returning to FIG. 23. The outside-vehicle information detection unit 7400 causes the imaging unit 7410 to capture an image outside the vehicle and receives the captured image data. The outside-vehicle information detection unit 7400 also receives detection information from the connected outside-vehicle information detection unit 7420. If the outside-vehicle information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the outside-vehicle information detection unit 7400 transmits ultrasonic waves or electromagnetic waves and receives information on the received reflected waves. The outside-vehicle information detection unit 7400 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface based on the received information. The outside-vehicle information detection unit 7400 may perform environmental recognition processing for recognizing rainfall, fog, road surface conditions, etc. based on the received information. The outside-vehicle information detection unit 7400 may calculate the distance to an object outside the vehicle based on the received information.

The outside vehicle information detection unit 7400 may also perform image recognition processing or distance detection processing to recognize people, cars, obstacles, signs, or characters on the road surface based on the received image data. The outside vehicle information detection unit 7400 may perform processing such as distortion correction or alignment on the received image data, and may also generate an overhead image or a panoramic image by synthesizing image data captured by different imaging units 7410. The outside vehicle information detection unit 7400 may also perform viewpoint conversion processing using image data captured by different imaging units 7410.

The in-vehicle information detection unit 7500 detects information inside the vehicle. For example, a driver state detection unit 7510 that detects the state of the driver is connected to the in-vehicle information detection unit 7500. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, or a microphone that collects sound inside the vehicle. The biosensor is provided, for example, on the seat or steering wheel, and detects the biometric information of a passenger sitting in the seat or a driver gripping the steering wheel. The in-vehicle information detection unit 7500 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off. The in-vehicle information detection unit 7500 may perform processing such as noise canceling on the collected sound signal.

The integrated control unit 7600 controls the overall operation of the vehicle control system 7000 according to various programs. The input unit 7800 is connected to the integrated control unit 7600. The input unit 7800 is realized by a device that can be operated by the passenger, such as a touch panel, a button, a microphone, a switch, or a lever. Data obtained by voice recognition of a voice input by a microphone may be input to the integrated control unit 7600. The input unit 7800 may be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or a PDA (Personal Digital Assistant) that supports the operation of the vehicle control system 7000. The input unit 7800 may be, for example, a camera, in which case the passenger can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the passenger may be input. Furthermore, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on information input by the passenger using the above-mentioned input unit 7800 and outputs the input signal to the integrated control unit 7600. Passengers and others can operate the input unit 7800 to input various data and instruct processing operations to the vehicle control system 7000.

The memory unit 7690 may include a ROM (Read Only Memory) that stores various programs executed by the microcomputer, and a RAM (Random Access Memory) that stores various parameters, calculation results, sensor values, etc. The memory unit 7690 may also be realized by a magnetic memory device such as a HDD (Hard Disc Drive), a semiconductor memory device, an optical memory device, or a magneto-optical memory device, etc.

The general-purpose communication I/F 7620 is a general-purpose communication I/F that mediates communication between various devices present in the external environment 7750. The general-purpose communication I/F 7620 may implement cellular communication protocols such as GSM (registered trademark) (Global System of Mobile communications), WiMAX (registered trademark), LTE (registered trademark) (Long Term Evolution) or LTE-A (LTE-Advanced), or other wireless communication protocols such as wireless LAN (also called Wi-Fi (registered trademark)) and Bluetooth (registered trademark). The general-purpose communication I/F 7620 may connect to devices (e.g., application servers or control servers) present on an external network (e.g., the Internet, a cloud network, or an operator-specific network) via, for example, a base station or an access point. In addition, the general-purpose communication I/F 7620 may connect to a terminal located near the vehicle (e.g., a driver's, pedestrian's, or store's terminal, or an MTC (Machine Type Communication) terminal) using, for example, P2P (Peer To Peer) technology.

The dedicated communication I/F 7630 is a communication I/F that supports a communication protocol developed for use in a vehicle. The dedicated communication I/F 7630 may implement a standard protocol such as WAVE (Wireless Access in Vehicle Environment), DSRC (Dedicated Short Range Communications), or a cellular communication protocol, which is a combination of the lower layer IEEE 802.11p and the higher layer IEEE 1609. The dedicated communication I/F 7630 typically performs V2X communication, which is a concept that includes one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

The positioning unit 7640 performs positioning by receiving, for example, GNSS signals from GNSS (Global Navigation Satellite System) satellites (for example, GPS signals from GPS (Global Positioning System) satellites), and generates position information including the latitude, longitude, and altitude of the vehicle. The positioning unit 7640 may determine the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, PHS, or smartphone that has a positioning function.

The beacon receiver 7650 receives, for example, radio waves or electromagnetic waves transmitted from radio stations installed on the road, and acquires information such as the current location, congestion, road closures, and travel time. The functions of the beacon receiver 7650 may be included in the dedicated communication I/F 7630 described above.

The in-vehicle device I/F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle devices 7760 present in the vehicle. The in-vehicle device I/F 7660 may establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), or WUSB (Wireless USB). The in-vehicle device I/F 7660 may also establish a wired connection such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), or MHL (Mobile High-definition Link) via a connection terminal (and a cable, if necessary) not shown. The in-vehicle device 7760 may include, for example, at least one of a mobile device or wearable device owned by a passenger, or an information device carried into or attached to the vehicle. The in-vehicle device 7760 may also include a navigation device that searches for a route to an arbitrary destination. The in-vehicle device I/F 7660 exchanges control signals or data signals with these in-vehicle devices 7760.

The in-vehicle network I/F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I/F 7680 transmits and receives signals in accordance with a specific protocol supported by the communication network 7010.

The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired through at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle device I/F 7660, and the in-vehicle network I/F 7680. For example, the microcomputer 7610 may calculate the control target value of the driving force generating device, the steering mechanism, or the braking device based on the acquired information inside and outside the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 may perform cooperative control for the purpose of realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 may control the driving force generating device, steering mechanism, braking device, etc. based on the acquired information about the surroundings of the vehicle, thereby performing cooperative control for the purpose of automatic driving, which allows the vehicle to travel autonomously without relying on the driver's operation.

The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures and people based on information acquired via at least one of the general-purpose communication I/F 7620, the dedicated communication I/F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle equipment I/F 7660, and the in-vehicle network I/F 7680, and may create local map information including information about the surroundings of the vehicle's current position. The microcomputer 7610 may also predict dangers such as vehicle collisions, the approach of pedestrians, or entry into closed roads based on the acquired information, and generate warning signals. The warning signals may be, for example, signals for generating warning sounds or turning on warning lights.

The audio/image output unit 7670 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of FIG. 23, an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include, for example, at least one of an on-board display and a head-up display. The display unit 7720 may have an AR (Augmented Reality) display function. The output device may be other devices such as headphones, a wearable device such as a glasses-type display worn by the passenger, a projector, or a lamp, in addition to these devices. When the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, graphs, etc. When the output device is an audio output device, the audio output device converts an audio signal consisting of reproduced audio data or acoustic data into an analog signal and audibly outputs it.

23, at least two control units connected via the communication network 7010 may be integrated into one control unit. Alternatively, each control unit may be composed of multiple control units. Furthermore, the vehicle control system 7000 may include another control unit not shown. In the above description, some or all of the functions performed by any control unit may be provided by another control unit. In other words, as long as information is transmitted and received via the communication network 7010, a predetermined calculation process may be performed by any control unit. Similarly, a sensor or device connected to any control unit may be connected to another control unit, and multiple control units may transmit and receive detection information to each other via the communication network 7010.

Note that computer programs for implementing the functions of the distance measuring device 100 and the information processing device 200 described above can be implemented in any of the control units, etc. Also, a computer-readable recording medium on which such a computer program is stored can also be provided. The recording medium can be, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. Also, the above computer program can be distributed, for example, via a network, without using a recording medium.

In the vehicle control system 7000 described above, the above-mentioned distance measuring device 100 and information processing device 200 can be used, for example, as a light source steering unit of a LIDAR as an environmental sensor. In addition, image recognition in the imaging unit can be performed by an optical computing unit using the above-mentioned distance measuring device 100 and information processing device 200.

Furthermore, at least some of the components of the distance measuring device 100 and the information processing device 200 described above may be realized in a module (e.g., an integrated circuit module configured on a single die) for the integrated control unit 7600 shown in FIG. 23. Alternatively, the distance measuring device 100 and the information processing device 200 described above may be realized by multiple control units of the vehicle control system 7000 shown in FIG. 23.

The present disclosure has been described above by giving embodiments, their modifications, and application examples, but the present disclosure is not limited to the above-described embodiments, and various modifications are possible. Note that the effects described in this specification are merely examples. The effects of the present disclosure are not limited to the effects described in this specification. The present disclosure may have effects other than those described in this specification.

Furthermore, for example, the present disclosure can have the following configuration.
(1)
A light emitting unit that emits irradiation light;
a distance measuring unit that calculates data on a distance to the subject based on incident light obtained by irradiating the subject with the irradiation light,
the light-emitting section includes a plurality of light-emitting elements arranged in an array, and a first driving section that sequentially causes the plurality of light-emitting elements to emit light for each predetermined unit group within one frame period;
the first driving section causes the plurality of light-emitting elements in the unit group to emit light at a first emission intensity and then at a second emission intensity different from the first emission intensity during the one frame period.
(2)
The distance measuring device described in (1), wherein the distance measuring unit calculates first data regarding the distance to the subject obtained when the multiple light-emitting elements are made to emit light at the first emission intensity, and second data regarding the distance to the subject obtained when the multiple light-emitting elements are made to emit light at the second emission intensity.
(3)
The distance measuring device described in (2), further comprising an output unit that outputs at least one of the first data and the second data calculated by the distance measuring unit and data regarding the light emission intensity of the light emitting unit as output data in a predetermined format.
(4)
An acquisition unit that acquires a control signal for setting a drive mode,
The distance measuring device according to any one of (1) to (3), wherein the first driving unit sets the first emission intensity and the second emission intensity according to the driving mode acquired by the acquisition unit.
(5)
The distance measuring device described in (4), wherein when the drive mode acquired by the acquisition unit is a mode assuming that the subject is located at a relatively long distance, the first drive unit causes the multiple light-emitting elements in the unit group to emit light at the first emission intensity and then at the second emission intensity during the one frame period.
(6)
A light receiving unit that receives the incident light is further provided,
the light receiving unit has a plurality of light receiving elements arranged in an array, and a second drive unit that activates at least a portion of the plurality of light receiving elements to output light receiving data from the activated light receiving elements,
The distance measuring device according to (4) or (5), wherein the second driving unit determines which of the plurality of light receiving elements is to be activated in accordance with the driving mode acquired by the acquisition unit.
(7)
A light receiving unit that receives the incident light is further provided,
the light receiving unit has a plurality of light receiving elements arranged in an array, and a second drive unit that activates at least a portion of the plurality of light receiving elements to output light receiving data from the activated light receiving elements,
The light receiving element is a single photon avalanche diode (SPAD),
The distance measuring unit generates a histogram for each pixel that includes one or more of the light receiving elements, regarding the time from when the light emitting element is driven to when light is detected as being incident on the one or more light receiving elements, and uses the generated histogram or distance information calculated based on the generated histogram as data regarding the distance to the subject.
(8)
The distance measurement unit calculates a reliability of the histogram based on the generated histogram,
The ranging device described in (7) further includes an output unit that outputs at least one of the first data and the second data calculated by the ranging unit, data on the light emission intensity of the light emission unit, and the reliability as output data in a predetermined format.

In a distance measuring device according to one aspect of the present disclosure, during one frame period, multiple light emitting elements in a unit group emit light at a first emission intensity, and then emit light at a second emission intensity different from the first emission intensity. This makes it possible to obtain data with little measurement error caused by the Pile-Up phenomenon and with almost no effect of dead time caused by reflection on the cover glass, even when the subject is at a close distance. Furthermore, for example, even when the subject is at a long distance, it is possible to obtain data that allows the distance to the subject to be calculated. As a result, it is possible to improve distance measurement accuracy, particularly at close distances, compared to when light is emitted at a single intensity.

Those skilled in the art may conceive of various modifications, combinations, subcombinations, and variations depending on design requirements and other factors, and it is understood that these are within the scope of the appended claims and their equivalents.

This application claims priority based on Japanese Patent Application No. 2022-174231, filed on October 31, 2022 in the Japan Patent Office, the entire contents of which are incorporated herein by reference.

Those skilled in the art may conceive of various modifications, combinations, subcombinations, and variations depending on design requirements and other factors, and it is understood that these are within the scope of the appended claims and their equivalents.

Claims (8)

1. A light emitting unit that emits irradiation light;
   a distance measuring unit that calculates data on a distance to the subject based on incident light obtained by irradiating the subject with the irradiation light,
   the light-emitting section includes a plurality of light-emitting elements arranged in an array, and a first driving section that sequentially causes the plurality of light-emitting elements to emit light for each predetermined unit group within one frame period;
   the first driving section causes the plurality of light-emitting elements in the unit group to emit light at a first emission intensity and then at a second emission intensity different from the first emission intensity during the one frame period.
2. The distance measuring device of claim 1 , wherein the distance measuring unit calculates first data regarding the distance to the subject obtained when the multiple light-emitting elements are made to emit light at the first emission intensity, and second data regarding the distance to the subject obtained when the multiple light-emitting elements are made to emit light at the second emission intensity.
3. The distance measuring device according to claim 2 , further comprising an output unit that outputs at least one of the first data and the second data calculated by the distance measuring unit and data regarding the light emission intensity of the light emitting unit as output data in a predetermined format.
4. An acquisition unit that acquires a control signal for setting a drive mode,
   The distance measuring device according to claim 1 , wherein the first driving section sets the first emission intensity and the second emission intensity in accordance with the driving mode acquired by the acquisition section.
5. 5. The distance measuring device according to claim 4, wherein when the drive mode acquired by the acquisition unit is a mode assuming that the subject is located at a relatively long distance, the first drive unit causes the multiple light-emitting elements in the unit group to emit light at the first emission intensity and then at the second emission intensity during the one frame period.
6. A light receiving unit that receives the incident light is further provided,
   the light receiving unit has a plurality of light receiving elements arranged in an array, and a second drive unit that activates at least a portion of the plurality of light receiving elements to output light receiving data from the activated light receiving elements,
   The distance measuring device according to claim 4 , wherein the second driving section determines which of the plurality of light receiving elements is to be activated in accordance with the driving mode acquired by the acquisition section.
7. A light receiving unit that receives the incident light is further provided,
   the light receiving unit has a plurality of light receiving elements arranged in an array, and a second drive unit that activates at least a portion of the plurality of light receiving elements to output light receiving data from the activated light receiving elements,
   The light receiving element is a single photon avalanche diode (SPAD),
   The distance measuring device of claim 2, wherein the distance measuring unit generates a histogram for each pixel that includes one or more of the light receiving elements, regarding the time from when the light emitting element is driven to when incidence of light on the one or more light receiving elements is detected, and the generated histogram or distance information calculated based on the generated histogram is used as data regarding the distance to the subject.
8. The distance measuring unit calculates a reliability of the histogram based on the generated histogram,
   The distance measuring device of claim 7, further comprising an output unit that outputs at least one of the first data and the second data calculated by the distance measuring unit, data regarding the light emission intensity of the light emitting unit, and the reliability as output data in a predetermined format.

Images