JP2024004276A - object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection device capable of detecting a low-reflection object.

SOLUTION: An object detection device 1 detects an object in a measurement area MA. The object detection device 1 includes: a light-emitting section 10 that is a light-emitting section for emitting a light projection beam to the measurement area MA and is configured to be able to change a light emission mode of the light projection beam between a reference mode and a high accuracy mode in which object detection accuracy can be enhanced more than the reference mode; a light-receiving section 30 that receives a reflected beam obtained by reflecting the light projection beam on the object and outputs an electric signal corresponding to the light reception; and a light emission control section 52 that, to detect an object in a weak signal area in which a weak signal having intensity equal to or less than a preset threshold echo_th is detected as an electric signal, sets a light emission mode of light source light projected to at least the weak signal area of the measurement area MA to the high accuracy mode.

SELECTED DRAWING: Figure 7

COPYRIGHT: (C)2024,JPO&INPIT

Description

The disclosure herein relates to techniques for detecting objects.

Patent Document 1 discloses that the arrangement of a light emitting section, a light receiving section, and a filter are controlled in conjunction with each other according to environmental information acquired by an environmental sensor. The light emitting section emits light source light toward the measurement area. The light receiving section receives reflected light caused by the light source light being reflected by an object in the measurement area. The filter filters the light that enters the light receiving section. Specifically, the light source that emits light from among the plurality of light sources capable of emitting light source light in different wavelength ranges is selected according to the temperature measured by a temperature sensor serving as an environmental sensor.

JP2019-78748A

Now, an object (for example, a black object) having a low reflectance for all wavelength ranges may exist in the measurement region. The technique disclosed in Patent Document 1, which selects light sources with different wavelength ranges, cannot obtain high reflectance for such low-reflection objects, so it is difficult to detect the objects.

One of the objects of the disclosure of this specification is to provide an object detection device capable of detecting low reflection objects.

One of the object detection devices of the present disclosure is an object detection device that detects an object in a measurement area (MA),
A light emitting unit that emits light source light toward a measurement area, and is configured such that the emission mode of the light source light can be changed between a reference mode and a high precision mode that can improve object detection accuracy more than the reference mode. a light emitting part (10),
a light receiving unit (30) that receives reflected light caused by the light source light being reflected by an object and outputs an electrical signal in accordance with the received light;
Light source light that is projected toward at least the weak signal region of the measurement region in order to detect objects in the weak signal region where a weak signal with an intensity equal to or lower than a preset threshold signal strength is detected as an electrical signal. and a light emission control unit (52) that sets the light emission mode to a high precision mode.

According to such a configuration, the light source light is emitted toward the weak signal region of the measurement region in a highly accurate manner. By increasing object detection accuracy in weak signal areas where there is a possibility that low-reflection objects exist, low-reflection objects can be detected.

Another object detection device of the present disclosure is an object detection device that detects an object in a measurement area (MA),
A light emitting unit that emits light source light toward a measurement area, and is configured such that the emission mode of the light source light can be changed between a reference mode and a high precision mode that can improve object detection accuracy more than the reference mode. a light emitting part (10),
a light receiving unit (30) that receives reflected light caused by the light source light being reflected by an object in the measurement area, and outputs an electrical signal in accordance with the received light;
a photographed image information acquisition unit (54) that acquires information on a photographed image from an imaging device (90) that photographs the measurement area, and identifies from the information a weak area in which the light emitting unit and the light receiving unit are weak in detecting objects;
It includes a light emission control unit (52) that sets the light emission mode of the light source light projected toward the weak area to a high precision mode.

According to such a configuration, a weak area in which the light emitting unit and the light receiving unit are weak in detecting an object is identified using an image taken by the imaging device, and then a light source is directed toward the weak area in a highly accurate manner. Light is emitted. Even if a low-reflection object exists in a weak area, the low-reflection object can be detected by increasing object detection accuracy in the weak area.

Note that the symbols in parentheses exemplarily indicate correspondence with parts of the embodiment described later, and are not intended to limit the technical scope.

FIG. 1 is a configuration diagram showing a schematic configuration of an object detection device. FIG. 3 is a diagram illustrating scanning of a projection beam. FIG. 3 is a configuration diagram illustrating the functions of a control unit. A diagram explaining a histogram waveform. A diagram showing an example of a low reflection object. A diagram showing an example of a low reflection object. A flowchart showing an example of an object detection method. FIG. 3 is a configuration diagram illustrating the functions of a control unit. A flowchart showing an example of an object detection method. A flowchart showing an example of an object detection method. A diagram illustrating synthesis of raw waveforms. A flowchart showing an example of an object detection method. A flowchart showing an example of an object detection method. A flowchart showing an example of an object detection method.

Hereinafter, a plurality of embodiments will be described based on the drawings. Note that redundant explanation may be omitted by assigning the same reference numerals to corresponding components in each embodiment. When only a part of the configuration is described in each embodiment, the configuration of the other embodiments previously described can be applied to other parts of the configuration. Furthermore, in addition to the combinations of configurations specified in the description of each embodiment, it is also possible to partially combine the configurations of multiple embodiments even if the combinations are not explicitly stated. .

(First embodiment)
As shown in FIG. 1, an object detection device 1 according to a first embodiment of the present disclosure is mounted on a vehicle as a moving body, and detects objects around the vehicle. The object detection device 1 includes a lidar (LiDAR, Light Detection and Ranging/Laser imaging Detection and Ranging) 2 and an image processing unit 3. The object detection device 1 may be called a lidar unit. Note that in the following description, each direction indicated by front, rear, top, bottom, left, and right is defined with respect to the vehicle on a horizontal plane.

The lidar 2 is a light detection device that projects light from light sources 11a to 11d and detects reflected light from an object to be measured. The lidar 2 includes a light emitting section 10, a scanning section 20, a light receiving section 30, and a controller 40.

The light emitting unit 10 emits a projection beam toward the measurement area MA. The light emitting unit 10 has a configuration including, for example, a light source array 11 and a light projection optical system 12. As shown in FIG. 2, the light source array 11 is formed by arranging a plurality of light sources 11a to 11d, for example, along the vertical direction. The number of light sources 11a-d may be arbitrary. Each of the light sources 11a to 11d may be a laser oscillation element such as a laser diode (LD), for example. Each light source 11a-d may be an LED. Each of the light sources 11a to 11d is capable of emitting light at a light emission timing according to an electric signal from the controller 40.

The light projection optical system 12 collects the light emitted from the light sources 11a to 11d, and projects a beam-shaped projection beam toward the measurement area MA. The projection optical system 12 includes one or more lenses.

As shown in FIG. 2, the scanning unit 20 scans the projected beam from the light emitting unit 10 within the measurement area MA. The scanning unit 20 has a configuration including a scanning mirror 21. The scanning mirror 21 includes a drive motor and a reflector. The drive motor is, for example, a voice coil motor, a brushed DC motor, a stepping motor, or the like. The drive motor drives the rotation shaft of the reflector at a rotation amount and rotation speed according to an electric signal from the controller 40. The reflector is a mirror having a reflective surface that reflects the projected beam toward the measurement area MA. The reflective surface is, for example, formed in a planar shape.

In particular, in this embodiment, the reflector has an elongated shape with a longitudinal dimension along the rotation axis direction. Thereby, the reflector faces both the light projecting optical system 12 of the light emitting section 10 and the light receiving optical system 31 of the light receiving section 30. Therefore, the reflector is capable of simultaneously reflecting not only the projected beam but also the reflected beam that is the projected beam reflected from the object in the measurement area MA, and allows the reflected beam to enter the light receiving section 30.

The light receiving section 30 includes a light receiving optical system 31, a light receiving element 32, and a decoder 33. The light-receiving optical system 31 collects the reflected beam reflected by the reflector of the scanning unit 20 and makes it enter the light-receiving element 32 . The light receiving optical system 31 includes one or more lenses.

The light receiving element 32 receives light from the light receiving optical system 31. The light receiving element 32 is, for example, a single photon avalanche photodiode (SPAD) sensor. The light receiving element 32 is formed by two-dimensionally arranging a plurality of SPADs in a highly integrated state on a rectangular detection surface.

When SPAD receives one photon, it generates one electric pulse by electron doubling operation by avalanche doubling (so-called Geiger mode). That is, the SPAD can directly generate electric pulses as digital signals without going through an AD conversion circuit that converts analog signals into digital signals. Therefore, the light reception results can be read out at high speed.

The decoder 33 is provided to output the electric pulses generated by the SPAD, and includes a selection circuit and a clock oscillator. The selection circuit sequentially selects SPADs from which electric pulses are to be extracted. The selected SPAD outputs electrical pulses to controller 40. When the selection circuit finishes selecting SPADs one by one, one round of sampling ends. This sampling period corresponds to the clock frequency output from the clock circuit.

Note that at the timing when the light emitting unit 10 does not emit a light beam, that is, when no light is emitted, the light receiving unit 30 can receive and detect disturbance light from a direction according to the angle of the reflector. . Furthermore, by forming an image of the disturbance light on the light receiving element 32, it is also possible to capture an object in the measurement area MA. The disturbance light referred to here is also referred to as background light.

The controller 40 controls the operation of the rider 2. Specifically, the controller 40 controls the light emission of each of the light sources 11a to 11d in the light emitting section 10 and the direction of the scanning mirror 21 in the scanning section 20 so that they are linked. Further, the controller 40 outputs detection data obtained by processing the electric pulses output from the light receiving section 30 to the image processing unit 3. The controller 40 may be configured by a dedicated computer having at least one memory and one processor, or may be realized by an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or the like.

Here, the controller 40 may have a function of calculating the distance from the rider 2 with respect to the object in the measurement area MA. The distance calculated here may be included in the detection data. On the other hand, the distance may be calculated by the image processing unit 3. Distance may also be referred to as depth. This distance may be measured by the TOF (Time of Flight) method, which measures the so-called flight time of light.

The image processing unit 3 generates and processes images based on the detection data acquired from the lidar 2. The image processing unit 3 may be realized by a dedicated computer. The image processing unit 3 may include at least one memory 3a and at least one processor 3b. The memory 3a is at least one type of non-transitional physical storage medium, such as a semiconductor memory, a magnetic medium, an optical medium, etc., that non-temporarily stores programs, data, etc. that can be read by the processor 3b. good. Further, as the memory 3a, a rewritable volatile storage medium such as a RAM (Random Access Memory) may be provided. The processor 3b includes, as a core, at least one type of, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a RISC (Reduced Instruction Set Computer)-CPU.

The image processing unit 3 generates various images such as a reflection intensity image, a depth image, a pulse width image, and a background light image based on the detection data obtained from the controller 40.

The reflection intensity image is a two-dimensional image of the received light intensity of the reflected beam in the detection data corresponding to the light emission timing of the light emitting unit 10, which shows the reflection intensity from each direction of the measurement area MA.

The depth image is obtained by converting the distance (depth) measured by the TOF method from the reflected beam in the detection data corresponding to the light emission timing of the light emitting unit 10 into a two-dimensional image of the measurement area MA.

The pulse width image is a two-dimensional image of the measurement area MA of the pulse width of the reflected beam in the detection data corresponding to the light emission timing of the light emitting unit 10.

The background light image is a two-dimensional image of the measurement area MA of the received light intensity of the background light in the detection data corresponding to the timing when the light emitting unit 10 does not emit light.

Furthermore, the image processing unit 3 may output a feedback signal to the controller 40 for the controller 40 to control the light emitting section 10 and the light receiving section 30 based on these generated images or the results of image processing.

FIG. 3 is a diagram showing the functional architecture of the object detection device 1. Here, the controller 40 and the image processing unit 3 cooperate with each other to function as a control section 50 for controlling the light emitting section 10, the scanning section 20, and the light receiving section 30. The control unit 50 includes a data processing unit 51, a light emission control unit 52, and a frame rate control unit 53 as functional blocks implemented by at least one processor that executes a program.

The data processing section 51 processes detection data from the light receiving section 30. Specifically, the data processing unit 51 statistically processes the data of the electric pulse output from the light receiving element 32. The statistical processing here may include calculation of distance (depth) using the TOF method. The data processing unit 51 generates various images such as a reflection intensity image, a depth image, a pulse width image, and a background light image based on the data of the electric pulse output from the light receiving element 32. Furthermore, the data processing section 51 may cooperate with the light emission control section 52 and the frame rate control section 53 to change the frequency and timing of distance (depth) calculation and image generation.

As shown in FIG. 4, the data processing unit 51 sets two threshold values for the waveform obtained from the electric pulse. The waveform obtained from the electric pulse here may be a histogram waveform obtained by integrating photon detection events.

Of the two thresholds, the peak time calculation threshold half_th is a threshold for detecting the time when the waveform shows the peak intensity, in other words, the time when the reflected beam returns. For example, the peak time calculation threshold half_th may be set to half the peak value.

Of the two thresholds, the echo determination threshold echo_th is a threshold for determining whether the electric pulse indicates that the light receiving unit 30 has received a reflected beam. That is, if the intensity of the electric pulse is equal to or greater than the echo determination threshold echo_th, the electric pulse is considered to be an electric signal indicating a reflected beam. On the other hand, when the intensity of the electric pulse is less than the echo determination threshold echo_th, the electric pulse is considered to be a weak signal, that is, a weak signal. A weak signal here is a concept that includes not only a case where the signal strength is weak but also a case where the signal strength is not substantially detected.

The light emission control section 52 controls the emission of the projected beam in the light emission section 10 based on the processing result of the data processing section 51 . The light emission control section 52 controls the light emitting section 10 so as to change the light emitting mode of the projected beam in the light emitting section 10 between the reference mode and the high precision mode. The light emission mode includes, for example, setting of the light emission intensity of the projected beam.

The reference mode is a mode that serves as a reference in control. For example, the reference mode is a mode in which the light emission intensity is set as default. The high-accuracy mode is a mode in which object detection accuracy can be higher than that in the reference mode for low-reflection objects with low reflectance. For example, the high-precision mode is a mode in which the emission intensity is greater than the emission intensity under the standard mode.

The low reflection object is, for example, a vehicle painted black. The low-reflection object may be, for example, an object whose energy reflectance at the wavelength of the projected beam is 10% or less. The low-reflection object may include an object in a positional relationship that makes it difficult to reflect the projected beam toward the lidar 2. For example, as shown in FIG. 5, another vehicle TV that is oriented diagonally in front of the host vehicle EV equipped with the object detection device 1 in a direction orthogonal to the host vehicle EV may correspond to a low reflection object. This positional relationship is, for example, a positional relationship that occurs in a parking lot, an intersection, or the like. For example, as shown in FIG. 6, another vehicle TV diagonally in front of the own vehicle EV and oriented along a direction parallel to the own vehicle EV may correspond to a low reflection object. This positional relationship is, for example, a positional relationship that occurs in relation to other vehicles TV in adjacent lanes and oncoming lanes.

The light emission control unit 52 sets the light emission mode of the light beam directed toward the weak signal area to a high precision mode in order to detect an object in the weak signal area where the above-mentioned weak signal has been detected in the measurement area MA. do. At this time, the light emission control unit 52 may set the light emission mode to the reference mode for areas other than the weak signal area in the measurement area MA.

The frame rate control unit 53 controls the frame rate of various images when the data processing unit 51 generates the various images. The frame rate control unit 53 reduces the frame rate in accordance with power consumption constraints when the light emission intensity increases due to the light emission mode being changed to the high precision mode. That is, as the light emission intensity increases, the power consumption of the object detection device 1 increases. When the power supplied by the power supply becomes insufficient or is expected to become insufficient due to an increase in power consumption, the frame rate is lowered to suppress power consumption per unit time.

Further, the frame rate control unit 53 lowers the frame rate when the image processing load in the weak signal region increases due to the light emission mode being changed to the high precision mode. This suppresses the delay in providing the image or information about the object detected using the image to the outside.

Next, an example of a detection method that improves detection accuracy of low-reflection objects will be described using the flowchart of FIG. 7. The series of processes shown in FIG. 7 are executed at a predetermined execution cycle or based on a predetermined trigger. A series of processes may be executed mainly by at least one processor included in the control unit 50 so that the functions of the control unit 50 are realized.

In S101, the light emission control unit 52 controls the light emission unit 10 to emit a projected beam. After processing in S101, the process advances to S102.

In S102, the data processing unit 51 acquires detection data based on the reception of the reflected beam. After processing in S102, the process advances to S103.

In S103, the data processing unit 51 determines whether there is an area in the measurement area MA where the signal strength in the histogram waveform of the detection data is equal to or greater than the threshold echo_th. If an affirmative determination is made in S103, the process advances to S104. If a negative determination is made in S103, the process advances to S105.

In S104, the data processing unit 51 generates an image such as a depth image. The series of processing ends at S104.

In S105, the light emission control unit 52 determines whether the light emission intensity is equal to or less than the threshold value emit_th. The threshold value emit_th may be a value corresponding to the output limit of the light sources 11a to 11d, or may be a value set based on power consumption constraints or the like. If an affirmative determination is made in S105, the process advances to S106. If a negative determination is made in S105, the process advances to S104.

In S106, the light emission control unit 52 increases the light emission intensity of the projected beam toward the region where the signal strength is less than the threshold echo_th, that is, the weak signal region. After processing in S106, the process returns to S101.

The first embodiment described above will be summarized below. According to the first embodiment, the projection beam as the light source light is emitted toward the weak signal region in the measurement area MA in a highly accurate manner. By increasing object detection accuracy in weak signal areas where there is a possibility that low-reflection objects exist, low-reflection objects can be detected.

Further, according to the first embodiment, since the intensity of the projected beam directed toward the weak signal region is increased, it is possible to increase the possibility that the intensity of the reflected beam reflected by the low reflection object will be equal to or higher than the threshold echo_th. .

Note that the projected light beam corresponds to "light source light." The reflected beam corresponds to "reflected light." The echo determination threshold corresponds to "threshold signal strength."

(Second embodiment)
The second embodiment is a modification of the first embodiment. The second embodiment will be described focusing on the differences from the first embodiment.

In the second embodiment, the light emission mode includes not only the setting of the light emission intensity of the projected beam but also other settings. In one example, the light emission mode includes setting the number of times of light emission per unit measurement operation. The unit measurement operation is, for example, one scan of the measurement area MA. In one scan, if the projection beam is emitted n times in the standard mode, the projection beam is emitted n+1 times or more in the high precision mode. In this case, the number of times the light is emitted to the weak signal region may be increased intensively.

In another example, the light emitting aspect includes, when the number of unit measurement operations themselves to generate one frame of an image increases, the number of light emissions increases in accordance with this increase. That is, in the high precision mode, the number of scans to generate one frame is increased. In this case, even if the setting of the number of times of light emission per scan is maintained, the number of times the projection beam is projected onto a low reflection object substantially increases by the increase in the number of scans.

These aspects can improve the detection accuracy of low reflection objects.

(Third embodiment)
The third embodiment is a modification of the first embodiment. The third embodiment will be described focusing on the differences from the first embodiment.

As shown in FIG. 8, the object detection device 1 of the third embodiment is configured to further include a photographed image information acquisition unit 54 that acquires information on a photographed image from an imaging device 90 mounted on a vehicle. The imaging device 90 is, for example, a front camera that photographs the front of the vehicle. The imaging area of the imaging device 90 at least partially overlaps the measurement area MA by the lidar 2. The information on the photographed image acquired by the photographed image information acquisition unit 54 may include the photographed image itself. The information on the photographed image may include information obtained by analyzing the photographed image using semantic segmentation or the like.

The light emission control unit 52 of the third embodiment does not emit light to the entire weak signal region in a highly accurate manner. The light emission control unit 52 sets the light emission mode to a high-precision mode by limiting the weak signal area to a dark color area corresponding to a dark color portion of the photographed image. The dark area may include a black area with crushed black.

Furthermore, the light emission control unit 52 excludes detection-unnecessary areas corresponding to parts recognized as sky or clouds among dark-colored parts of the captured image from the areas set in the high-precision mode, and also directs light emission toward detection-unnecessary areas. The projection of the beam may be regulated.

By setting these areas, object detection accuracy can be focused on areas where there is a high possibility that a low reflection object exists.

While continuing such control, if a portion recognized as the sky, clouds, or road surface disappears from the latest captured image, the light emission control unit 52 directs the light beam toward the area where detection is not required. Light projection may be resumed. By quickly restarting light projection, new objects can be immediately detected in areas where detection was no longer necessary.

Further, the light emission mode set by the light emission control unit 52 may include setting the number of times of light emission per scan, as in the second embodiment. Here, the light emission control unit 52 may set the number of times of light emission according to the estimated distance (estimated depth) of the object existing in the weak signal area, which is estimated based on the captured image. For example, the setting may be such that the number of times the light is emitted increases as the estimated distance increases.

Here, in particular, a method for dealing with detection-unnecessary areas will be explained using the flowchart of FIG.

In S201, the photographed image information acquisition unit 54 determines whether there is an area that does not require detection. If an affirmative determination is made in S201, the process advances to S202. If a negative determination is made in S201, the process skips to S203.

In S202, the photographed image information acquisition unit 54 stops emitting light to the detection-unnecessary area. Thereby, power consumption and heat generation of the light emitting section 10 can be suppressed. After processing in S202, the process advances to S203.

In S203, the data processing unit 51 generates a depth image using only pixels for which data has been detected. The series of processing ends at S203.

(Fourth embodiment)
The fourth embodiment is a modification of the third embodiment. The fourth embodiment will be described focusing on the differences from the third embodiment.

Next, an example of a detection method that improves the detection accuracy of low-reflection objects will be explained using the flowchart of FIG. 10. The series of processes shown in FIG. 10 are executed at a predetermined execution cycle or based on a predetermined trigger. A series of processes may be executed mainly by at least one processor included in the control unit 50 so that the functions of the control unit 50 are realized. S301, S302, and S304 are the same as S101, S102, and S104 of the first embodiment.

In S303, the photographed image information acquisition unit 54 uses the photographed image to determine whether the rider 2 is encountering a scene that is difficult for him or not. In particular, areas that are difficult for the lidar 2 to detect, such as areas where low-reflection objects are present, are identified. "Not good at detection" means that it tends to be difficult to detect. If an affirmative determination is made in S303, the process advances to S305. The process advances to S304 where a negative determination is made in S303.

S305 and S306 are contents in which "weak signal area" in S105 and S106 of the first embodiment is replaced with "weak signal area".

According to the fourth embodiment described above, after identifying a weak area in which the light emitting unit 10 and the light receiving unit 30 are weak in detecting an object using the captured image from the imaging device 90, A projected beam as a light source light is emitted in a highly accurate manner. Even if a low-reflection object exists in a weak area, the low-reflection object can be detected by increasing object detection accuracy in the weak area.

(Fifth embodiment)
The fifth embodiment is a modification of the third embodiment. The fifth embodiment will be described with a focus on the points that are different from the third embodiment.

The fifth embodiment adopts the architecture of the first embodiment shown in FIG. This is the configuration used.

That is, the light emission control unit 52 sets the light emission mode to a high precision mode, limited to a low light reception intensity region corresponding to a low light reception intensity portion of the background light image, among the weak signal regions. Here, the low received light intensity portion and low received light intensity region correspond to the dark colored portion and dark colored region of the third embodiment. The low received light intensity refers to an intensity that is substantially 0 and is less than a threshold for background light determination. The background light determination threshold may be set even smaller than the echo determination threshold echo_th.

(Sixth embodiment)
The sixth embodiment is a modification of the first embodiment. The sixth embodiment will be described focusing on the differences from the first embodiment.

In the sixth embodiment, as shown in FIG. 11, the data processing unit 51 adds up or averages the waveforms of the signals output from the light receiving element 32 that have been measured multiple times with time differences. I take the. Thereby, the data processing unit 51 removes noise in the waveform of the weak signal and obtains a reflected light signal.

That is, by statistically processing a plurality of raw waveforms before being tracked, it is possible to suppress the reflected beam from a low-reflection object from being determined as noise. In FIG. 11, waveforms obtained by projecting a projection beam at adjacent angles in time series during scanning are statistically processed.

(Seventh embodiment)
The seventh embodiment is a modification of the first embodiment. The seventh embodiment will be described with a focus on differences from the first embodiment.

The light emission control unit 52 of the seventh embodiment changes the light emission mode when a certain pixel has not been able to detect a signal exceeding the threshold signal intensity continuously for a preset threshold number of times in the time series information of the latest detection data. Set to high precision mode.

The time series information in the latest detection data is detection data for a predetermined number of times in the recent past, measured according to the frame rate. The threshold signal strength here may be an echo determination threshold. A pixel in which a signal exceeding the threshold signal strength is not detected may correspond to a pixel in which a signal is not substantially detected.

Next, an example of a detection method that improves the detection accuracy of low-reflection objects will be described using the flowchart of FIG. 12. The series of processes shown in FIG. 12 are executed at a predetermined execution cycle or based on a predetermined trigger. A series of processes may be executed mainly by at least one processor included in the control unit 50 so that the functions of the control unit 50 are realized.

In S401, the data processing unit 51 determines whether there is a pixel whose signal is not substantially detected. If an affirmative determination is made in S401, the process advances to S402. If a negative determination is made in S401, the process advances to S404.

In S402, the data processing unit 51 stores, in the memory 3a, the pixels whose signals were determined to be substantially undetectable in S401. After processing in S402, the process advances to S403.

In S403, the data processing unit 51 adds up the stored number of undetected pixels. After processing in S403, the process advances to S405.

In S404, the data processing unit 51 resets the number of undetected times of the pixel whose signal was determined to be detected in S401. After processing in S404, the process advances to S405.

In S405, the data processing unit 51 generates a depth image using only pixels for which signals have been detected. After processing in S405, the process advances to S406.

In S406, the data processing unit 51 determines whether the number of consecutive undetected pixels for which it is determined that the signal is not substantially detected is equal to or greater than the threshold number of times. The threshold number of times is set in advance. After processing in S406, the process advances to S407.

In S407, the S/N ratio at the pixel is improved. Improving the S/N ratio includes the light emission control unit 52 setting the light emission mode of the projected light beam projected to the region corresponding to the pixel, that is, the weak signal region, to a high precision mode. After processing in S407, the process advances to S408.

In S408, the data processing unit 51 determines whether the object can now be detected as a result of dealing with the pixels for which it has been determined that the signal has not been substantially detected continuously. If an affirmative determination is made in S408, the process advances to S409.

In S409, the data processing unit 51 generates a depth image including pixels that can be detected. The series of processing ends at S409.

In S410, emission of the projection beam to the area corresponding to the pixel for which no signal is still substantially detected is stopped. The series of processing ends at S410.

(Eighth embodiment)
The eighth embodiment is a modification of the seventh embodiment. The eighth embodiment will be described focusing on the points that are different from the seventh embodiment.

The data processing unit 51 of the eighth embodiment bins pixels for which a signal exceeding a threshold signal intensity has not been detected when an increasing tendency of background light intensity is obtained in the time series information of the latest detection data. . Further, the light emission control unit 52 sets the light emission mode to a high precision mode when a certain binned macro pixel cannot detect a signal exceeding the threshold signal strength in the time series information.

Here, binning means configuring one macro pixel on an image from detection data from a plurality of mutually adjacent SPADs in the light receiving element 32. As a result of increasing the number of SPADs included in one macro pixel, the dynamic range can be widened although the spatial resolution is reduced.

An increasing tendency in the intensity of background light may mean, for example, that the intensity of background light in a certain background light image is increasing relative to the intensity of background light in the previous background light image. . The increasing tendency may be obtained by statistically processing the intensities of three or more background lights having different timings. This tendency for the intensity of background light to increase may mean, for example, that noise has simply increased. Such an increasing tendency in the intensity of the background light may mean that the vicinity of the vehicle in which the object detection device 1 is mounted or the object has moved from the shade to the sunlight.

It may not be possible to determine whether the distance of the object to be detected has become closer or further away, but if the distance has not changed from the last time it was detected, the image can be binned to improve the S/N ratio. This allows objects to be detected.

Further, the light emission control unit 52 sets the light emission mode to a high precision mode when a tendency to maintain or decrease the intensity of the background light is obtained. Furthermore, after setting to this high precision mode, if a certain pixel has not been able to detect a signal exceeding the threshold signal strength continuously for a preset threshold number of times in the time series information, binning.

A tendency to maintain or decrease the intensity of the background light may mean, for example, that the distance of the object has become greater. Assuming that the distance of the object has become longer, the S/N ratio should be improved without performing binning, since binning the image will reduce the spatial resolution and make it difficult to detect the object. If the object still cannot be detected, you can try binning the image. Furthermore, if the object cannot be detected even after that, the light emission should be stopped to suppress power consumption and heat generation.

Next, an example of a detection method that improves the detection accuracy of low-reflection objects will be described using flowcharts in FIGS. 13 and 14. The series of processes shown in FIGS. 13 and 14 are executed at a predetermined execution cycle or based on a predetermined trigger. A series of processes may be executed mainly by at least one processor included in the control unit 50 so that the functions of the control unit 50 are realized.

In S501 shown in FIG. 13, the data processing unit 51 determines whether there are any pixels whose signals are not substantially detected. If an affirmative determination is made in S501, the process advances to S502. If a negative determination is made in S501, the process advances to S505.

In S502, the data processing unit 51 determines whether a pixel whose signal has not been substantially detected could have been detected before a preset threshold frame. If an affirmative determination is made in S502, the process advances to S512 shown in FIG. 14. If a negative determination is made in S502, the process advances to S503.

S503 to S511 are similar to S402 to S410 of the seventh embodiment.

In S512 shown in FIG. 14, the data processing unit 51 generates a depth image using only pixels for which signals have been detected. After processing in S512, the process advances to S513.

In S513, the data processing unit 51 determines whether or not the intensity of background light has increased since the previous time for pixels whose signals have not been substantially detected. If an affirmative determination is made in S513, the process advances to S514. If a negative determination is made in S513, the process advances to S516.

In S514, the data processing unit 51 bins the image according to the distance of the previous object. After processing in S514, the process advances to S515.

In S515, the data processing unit 51 determines whether the object can be detected. If an affirmative determination is made in S515, the process advances to S510 shown in FIG. 13. If a negative determination is made in S515, the process advances to S508 shown in FIG. 13.

In S516, the S/N ratio at the pixel is improved. Improving the S/N ratio includes the light emission control unit 52 setting the light emission mode of the projected light beam projected to the region corresponding to the pixel, that is, the weak signal region, to a high precision mode. After processing in S516, the process advances to S517.

In S517, the data processing unit 51 determines whether the object can be detected. If an affirmative determination is made in S517, the process advances to S510 shown in FIG. 13. If a negative determination is made in S517, the process advances to S518.

In S518, the data processing unit 51 bins the image according to the distance of the previous object. After processing in S518, the process advances to S519.

In S519, the data processing unit 51 determines whether the object can be detected. If an affirmative determination is made in S519, the process advances to S510 shown in FIG. 13. If a negative determination is made in S519, the process advances to S511 shown in FIG. 13.

(Ninth embodiment)
The ninth embodiment is a modification of the first embodiment. The ninth embodiment will be described focusing on the differences from the first embodiment.

The frame rate control unit 53 of the ninth embodiment causes the data processing unit 51 to reduce the load of image processing in other areas when the light emission control unit 52 sets the light emission mode to the high precision mode. request. This request is in response to an increase in the load of image processing in a weak signal region set in a high precision mode.

Thereby, the frame rate control unit 53 offsets increases and decreases in processing load between the weak signal area and other areas, and maintains the apparent frame rate when generating an image. Therefore, even if the weak signal region is set in a highly accurate manner, the operation timing or operation interval of the scanning mirror 21 etc. can be maintained. As a result, it is possible to suppress the occurrence of control troubles due to timing shifts and the like.

(Other embodiments)
Although multiple embodiments have been described above, the present disclosure is not to be construed as being limited to those embodiments, and may be applied to various embodiments and combinations within the scope of the gist of the present disclosure. Can be done.

For example, the object detection device 1 does not need to be mounted on a vehicle. The object detection device 1 may be mounted on a roadside unit in infrastructure, and may transmit information about detected objects to a vehicle via V2X communication.

The configurations of the light emitting section 10, the scanning section 20, and the light receiving section 30 can be changed as appropriate as long as control by the control section 50 of the present disclosure can be applied. For example, the light emitting unit 10 may include a single light source instead of the light source array 11.

The control unit and techniques described in this disclosure may be implemented by a special purpose computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the apparatus and techniques described in this disclosure may be implemented with dedicated hardware logic circuits. Alternatively, the apparatus and techniques described in this disclosure may be implemented by one or more special purpose computers configured by a combination of a processor executing a computer program and one or more hardware logic circuits. The computer program may also be stored as instructions executed by a computer on a computer-readable non-transitory tangible storage medium.

1: Object detection device, 10: Light emitting unit, 30: Light receiving unit, 52: Light emission control unit, 54: Photographed image information acquisition unit, 90: Imaging device, MA: Measurement area

Claims (19)

An object detection device that detects an object in a measurement area (MA),
A light emitting unit that emits light source light toward the measurement area, the light emitting mode of the light source light being changed between a reference mode and a high precision mode that can increase object detection accuracy more than the reference mode. a light emitting unit (10) configured to allow
a light receiving unit (30) that receives reflected light caused by the light source light being reflected by the object and outputs an electrical signal in accordance with the received light;
In order to detect the object in a weak signal region in which a weak signal having an intensity equal to or lower than a preset threshold signal strength is detected as the electric signal, light is projected toward at least the weak signal region of the measurement region. An object detection device, comprising: a light emission control unit (52) that sets the light emission mode of the light source light to the high precision mode. The object detection device according to claim 1, wherein the light emission mode includes setting of the light emission intensity of the light source light. The object is detected using an image generated based on detection data,
The object detection device according to claim 2, further comprising a frame rate control unit (53) that reduces the frame rate at which the image is generated according to power consumption constraints when increasing the light emission intensity. 2. The object detection device according to claim 1, wherein the light emission mode includes a setting of the number of times of light emission per unit measurement operation. The object detection device according to claim 4, wherein the light emission control unit sets the number of times of light emission according to an estimated distance of the object estimated to exist in the weak signal region. The object is detected using an image generated based on detection data,
The object detection device according to claim 1, wherein, when the number of unit measurement operations themselves for generating one frame of the image increases, the light emission mode includes increasing the number of times of light emission in accordance with this increase. . further comprising a photographed image information acquisition unit (54) that acquires information on a photographed image from an imaging device that photographs the measurement area,
The object detection according to claim 1, wherein the light emission control unit sets the light emission mode to the high precision mode only in a dark color area corresponding to a dark color portion of the captured image in the weak signal area. Device. The light emission control unit excludes a detection-unnecessary area corresponding to a portion recognized as the sky or clouds among the dark-colored portions from the area set in the high-precision mode, and directs the light source toward the detection-unnecessary area. The object detection device according to claim 7, wherein projection of light is regulated. According to claim 8, the light emission control unit restarts projecting the light source light toward the detection-unnecessary area when the portion recognized as the sky, clouds, or road surface disappears from the latest captured image. The object detection device described. The apparatus further includes a data processing section (51) that generates a background light image in which the received light intensity of the background light in the detection data corresponding to the timing when the light emitting section does not emit light is converted into a two-dimensional image of the measurement area. ,
1 . The light emission control unit sets the light emission mode to the high precision mode only in a low light reception intensity region corresponding to a low light reception intensity portion of the background light image in the weak signal region. The object detection device described in . The signal of the reflected light is obtained by adding or averaging the waveforms of the signals output from the light receiving section, which are measured multiple times with time differences, to remove noise in the waveform of weak signals. The object detection device according to claim 1, further comprising a data processing unit that obtains. The object is detected using an image generated based on the detection data and in which a plurality of pixels are arranged in a two-dimensional manner,
In the time-series information of the most recent detection data, when a certain pixel has not been able to detect a signal exceeding the threshold signal intensity continuously for a preset threshold number of times, the light emission control unit changes the light emission mode. The object detection device according to claim 1, wherein the object detection device is set to the high precision mode. If the received light intensity in the detection data corresponding to the timing when the light emitting part does not emit light is defined as the intensity of background light,
The object detection device according to claim 12, further comprising a data processing unit that bins the image when an increasing tendency of the intensity of the background light is obtained in the time series information of the latest detection data. In the time series information, when a certain binned macro pixel cannot detect a signal exceeding the threshold signal intensity, the light emission control unit sets the light emission mode to the high precision mode. 14. The object detection device according to 13. If the received light intensity in the detection data corresponding to the timing when the light emitting part does not emit light is defined as the intensity of background light,
13. The object according to claim 12, wherein the light emission control unit sets the light emission mode to the high precision mode when a tendency to maintain or decrease the intensity of the background light is obtained in the time series information. Detection device. After setting to the high precision mode, in the time series information, if a certain pixel has not been able to detect a signal exceeding the threshold signal strength continuously for a preset threshold number of times, the image is binned. The object detection device according to claim 15, further comprising a data processing section. The object is detected using an image generated based on detection data,
The object according to claim 1, further comprising a frame rate control unit that reduces a frame rate at which the image is generated in accordance with an increase in image processing load in the weak signal region set to the high precision mode. Detection device. The object is detected using an image generated based on detection data,
In response to an increase in the image processing load in the weak signal area set to the high precision mode, the image processing load in other areas is reduced, and the apparent frame when generating the image is reduced. The object detection device according to claim 1, further comprising a frame rate control unit that maintains the rate. An object detection device that detects an object in a measurement area (MA),
A light emitting unit that emits light source light toward the measurement area, the light emitting mode of the light source light being changed between a reference mode and a high precision mode that can increase object detection accuracy more than the reference mode. a light emitting unit (10) configured to allow
a light receiving unit (30) that receives reflected light caused by the light source light being reflected by an object in the measurement area, and outputs an electrical signal in accordance with the received light;
a photographed image information acquisition unit that acquires information on a photographed image from an imaging device (90) that photographs the measurement area, and identifies from the information a weak area in which the light emitting unit and the light receiving unit are weak in detecting the object; (54) and
An object detection device comprising: a light emission control unit (52) that sets the light emission mode of the light source light projected toward the weak area to the high precision mode.

Images