JP7236835B2 - Output device and output method - Google Patents

Description

The present invention relates to an output device and an output method.

2. Description of the Related Art Conventionally, for example, there are technologies such as detecting a surrounding situation such as a target with a sensor, executing vehicle control based on the detection result, or notifying a driver of the detected target. For example, sensor detection results are processed, such as by reducing the amount of raw data detected by the sensor to some extent, in order to make it easier for humans to understand, or to lighten the processing load in the later stage. may be generated by For example, processing such as lowering the sampling rate or frame rate, lowering the resolution, reducing the number of colors, applying a filter (narrowing the frequency band), reducing the dynamic range, making judgment processing results by light processing, etc. applied.

JP 2017-74887 A

However, if the detection results are generated by processing such as reducing the raw data to some extent, there is a risk that errors will occur in the detection results, which are the output of the sensor, due to the reduction of detailed information that cannot be understood by humans. rice field.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an output device and an output method capable of improving the reliability of sensor output.

In order to solve the above problems and achieve the object, an output device according to the present invention includes an acquisition section, a determination section, and an output section. The acquisition unit acquires raw data, which is data before processing of the sensor detection result. The determination unit determines reliability of the sensor detection result. The output unit outputs the raw data instead of the sensor detection result to a subsequent processing device when the determination unit determines that the reliability of the sensor detection result is low.

According to the present invention, the reliability of sensor output can be enhanced.

FIG. 1 is a diagram showing an outline of an output method according to an embodiment. FIG. 2 is a block diagram showing the configuration of the output device according to the embodiment. FIG. 3 is a diagram showing an example of correlation information. FIG. 4 is a diagram showing the processing contents when comparing between a plurality of sensors. FIG. 5 is a flowchart illustrating a processing procedure of processing executed by the output device according to the embodiment;

Hereinafter, embodiments of an output device and an output method disclosed in the present application will be described in detail with reference to the accompanying drawings. It should be noted that the present invention is not limited by this embodiment.

First, the outline of the output method according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an outline of an output method according to an embodiment. In FIG. 1, it is assumed that a target T, which is an obstacle, exists in front of the vehicle C. As shown in FIG. In FIG. 1, the vehicle C is equipped with a target detection system 100, and the target T is detected by the target detection system 100. As shown in FIG.

It should be noted that the target object detection system 100 is not limited to the vehicle C, and may be other moving bodies such as motorcycles, bicycles, ships, and aircraft. Alternatively, the target detection system 100 may be mounted on a stationary object such as a streetlight or a roadside object (guardrail, traffic light, etc.), not limited to a mobile object.

As shown in FIG. 1, the target system 100 includes an output device 1, a sensor 10, and a target detection device 50 (an example of a subsequent processing device). The sensor 10 is a sensor that detects the circumstances around the vehicle C, and includes, for example, a camera, a radar device, and a LiDAR (Laser Imaging Detection and Ranging).

The target detection device 50 acquires the output of the sensor 10 via the output device 1 and detects the target T based on the output of the sensor 10 . For example, when a plurality of sensors 10 are provided, the target detection device 50 detects the target T by combining the outputs of the plurality of sensors 10 .

The output device 1 determines the reliability of the sensor detection result (hereinafter referred to as the detection result) that is the output of the sensor 10, and if the reliability of the detection result is low, the data of the detection result before processing. An output method for outputting raw data from the sensor 10 is executed.

Specifically, the output device 1 according to the embodiment first acquires raw data, which is data before processing of the detection result output from the sensor 10 (step S1). Subsequently, the output device 1 according to the embodiment determines the reliability of the detection result generated from the acquired raw data (step S2). A method for determining the reliability of detection results will be described later.

Further, the detection result used for reliability determination may be generated by the output device 1 based on raw data, or may be obtained from the sensor 10 . In addition, the raw data referred to here may be the detection signal itself detected by the sensor 10 (that is, unprocessed data), or intermediate data generated in the process of generating the detection result (data in process of processing). ), the details of the raw data will be described later.

Subsequently, when the output device 1 according to the embodiment determines that the reliability of the detection result is low as a result of the determination, instead of the detection result, raw data is output from the sensor 10, and sent to the target detection device 50 Output (step S3). As a result, the target detection apparatus 100 can perform detailed analysis using the raw data in the target detection process, so that the target T can be detected with high accuracy.

As described above, according to the output method according to the embodiment, when the reliability of the detection result of the sensor 10 is low, by using the raw data as the output of the sensor 10, the reliability of the output of the sensor 10 can be improved. can.

Raw data has a problem that communication is burdened due to its large data volume, and detailed analysis of raw data has a problem that processing burden increases. Since the information is transmitted, and processing such as detailed analysis is performed, it is possible to reduce the burden on communication and processing.

Further, FIG. 1 shows a case where the processing device in the latter stage of the output device 1 is the target detection device 50, but the processing device is not limited to the target detection device 100, and various processing is performed based on the sensor detection result. Any processing device can be employed as long as it is a processing device.

Next, the configuration of the output device 1 according to the embodiment will be described in detail with reference to FIG. FIG. 2 is a block diagram showing the configuration of the output device 1 according to the embodiment. As shown in FIG. 2, the output device 1 according to the embodiment is connected to a camera 10a, a radar device 10b, a LiDAR 10c, and a target detection device 50. The camera 10a, the radar device 10b and the LiDAR 10c are specific examples of the sensor 10 described above.

The camera 10a is an imaging device that captures an image of the external situation of the vehicle C. As shown in FIG. The camera 10a is provided in the windshield of the vehicle C, for example, and images the front of the vehicle C. As shown in FIG. In addition, the camera 10a may be provided at a position for imaging the left and right sides of the vehicle C and a position for imaging the rear of the vehicle.

The radar device 10b detects targets around the vehicle C using radio waves such as millimeter waves. Specifically, the radar device 10b detects a target by transmitting radio waves around the vehicle C and receiving reflected waves reflected by the target.

The LiDAR 10c detects targets around the vehicle C using laser light. Specifically, the LiDAR 10c detects a target by transmitting laser light to the vicinity of the vehicle C and receiving reflected light reflected by the target.

The target detection device 50 integrates detection results or raw data of the camera 10a, the radar device 10b, and the LiDAR 10c to detect the final target. Specifically, the target detection device 100 detects the final target based on the result of assigning a predetermined weight to the detection result of each sensor.

Alternatively, when the target detection device 100 acquires the raw data of the sensor 10 , the final target is detected based on the analysis result of the raw data and the detection result of the other sensors 10 .

An output device 1 according to the embodiment includes a control section 2 and a storage section 3 . The control unit 2 includes an acquisition unit 21 , a determination unit 22 and an output unit 23 . The storage unit 3 stores correlation information 31 .

Here, the output device 1 includes, for example, a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a data flash, an input/output port, and various circuits.

The CPU of the computer functions as an acquisition unit 21, a determination unit 22, and an output unit 23 of the control unit 2 by reading and executing programs stored in the ROM, for example.

In addition, at least one or all of the acquisition unit 21, the determination unit 22, and the output unit 23 of the control unit 2 may be configured by hardware such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). can.

Also, the storage unit 3 corresponds to, for example, a RAM or a data flash. The RAM and data flash can store the correlation information 31, various program information, and the like. Note that the output device 1 may acquire the above-described programs and various information via other computers or portable recording media connected via a wired or wireless network.

The correlation information 31 is information indicating the result of learning the correlation between the raw data of the sensor 10 and the detection results generated from the raw data. Specifically, the correlation information 31 is information that associates a detection result exceeding the detection limit of the sensor 10 with raw data of the detection result.

The correlation information 31 may be generated by learning in advance by an external device and stored in the storage unit 3, or may be generated by the control unit 2 sampling raw data and detection results and learning. may FIG. 3 is a diagram showing an example of the correlation information 31. As shown in FIG.

As shown in FIG. 3, the correlation information 31 is associated with items such as "sensor ID", "sensor name", "detection result", and "raw data". “Sensor ID” is identification information for identifying each sensor. "Sensor name" is the name of the sensor. “Detection result” indicates the content of the detection result exceeding the detection limit of the sensor 10 . “Raw data” is unprocessed raw data of detection results exceeding the detection limit of the sensor 10 . That is, the correlation information 31 includes, for each sensor 10, a set of the detection result when the detection limit of the sensor 10 is exceeded and the raw data of the detection result.

Based on the result of comparison between the “raw data” of the correlation information 31 and the raw data acquired by the acquisition unit 21, the determination unit 22, which will be described later, determines whether the detection result generated from the acquired raw data exceeds the detection limit of the sensor 10. Reliability is determined by determining whether or not By learning the correlation information 31 in this way, it is possible to improve the accuracy of determination by the determination unit 22, which will be described later. In addition, the processing content of the determination part 22 is mentioned later.

The control unit 2 determines the reliability of the detection result that is the output of the sensor 10 , and outputs raw data of the detection result before processing when the reliability of the detection result is low.

The acquisition unit 21 acquires raw data from the sensors 10 such as the camera 10a, the radar device 10b, and the LiDAR 10c.

In the case of the camera 10a, the raw data is the image information of the captured image as it is. For example, it is a color image with the number of pixels of the captured image as it is, and each pixel includes color information such as RGB values. The raw data for the camera 10a may be, for example, half-processed data such as a black-and-white image obtained by binarizing the above-described color image, or image information subjected to edge detection processing.

Further, in the case of the radar device 10b, the raw data is, for example, information on received signals and transmitted signals. In addition, as raw data in the case of the radar device 10b, the beat signal obtained by mixing the received signal and the transmitted signal, the information of the frequency spectrum obtained by two-dimensional fast Fourier transform of the beat signal, the information derived from one target obtained from the frequency spectrum Data in the process of processing, such as information on a plurality of peaks of .

Further, in the case of the LiDAR 10c, the raw data is, for example, laser light information when received and transmitted. Raw data in the case of the LiDAR 10c may be data in the process of being processed, such as the difference between the reception time and the transmission time of the laser light.

The determination unit 22 determines the reliability of the detection result generated from the raw data acquired by the acquisition unit 21 . Specifically, the determination unit 22 compares the “raw data” of the correlation information 31 with the raw data acquired by the acquisition unit 21 so that the detection result generated from the acquired raw data is detected by the sensor 10 . Reliability is determined by determining whether or not the limit is exceeded.

For example, if the raw data of the camera 10a acquired by the acquiring unit 21 is similar to “#1”, which is the “raw data” of “backlight”, which is the “detection result” of the correlation information 31, the determination unit 22 determines that the camera It is determined that the raw data (image) 10a was taken under backlight, that is, the detection result exceeds the detection limit of the sensor 10, and the reliability is low.

In other words, the determining unit 22 determines that the reliability of the detection result is low because the target object may be hidden in the detection result because the image in backlight has a large difference in brightness. Thus, the determination unit 22 can accurately determine reliability by comparing the correlation information 31 and the raw data.

Further, the determination unit 22 may determine that the reliability is low when, for example, there is a difference in detection results among the plurality of sensors 10. This point will be described later with reference to FIG.

Also, the determination unit 22 may determine the reliability based on the output of another sensor that detects the surrounding conditions. For example, the determination unit 22 detects the brightness of the surroundings with a brightness sensor, and determines that the reliability of the camera is low when the surroundings are dark. In addition, it is conceivable to determine reliability based on weather, speed, and vibration, and to determine reliability by detecting contamination of the sensor, displacement of the mounting position, and the like.

When the determination unit 22 determines that the reliability of the detection result is low, the output unit 23 outputs raw data from the sensor 10 instead of the detection result. For example, when the determination unit 22 determines that the reliability is low due to the difference in the detection results among the plurality of sensors 10, the output unit 23 outputs the detection result for at least one sensor 10 out of the plurality of sensors 10. , the output is the raw data. This point will be described with reference to FIG.

FIG. 4 is a diagram showing the contents of processing when comparing between a plurality of sensors 10. As shown in FIG. FIG. 4 shows a case of comparing the presence/absence of an obstacle, which is the detection result of the camera 10a, the radar device 10b, and the LiDAR 10c. In FIG. 4, an example will be described in which the camera 10a detects no obstacle, the radar device 10b detects an obstacle, and the LiDAR 10c detects an obstacle.

As shown in FIG. 4, when the detection results differ among the camera 10a, the radar device 10b, and the LiDAR 10c, the determination unit 22 determines that the reliability of these three detection results is low.

Then, the output unit 23 outputs the raw data of the camera 10a instead of the detection result of the camera 10a for the camera 10a out of the camera 10a, the radar device 10b and the LiDAR 10c. Note that the output unit 23 outputs the detection results of the respective sensors 10 for the radar device 10b and the LiDAR 10c.

That is, in FIG. 4, among the plurality of sensors 10, the sensor 10 that outputs raw data is determined by a majority vote of the detection results. In FIG. 4, the sensors 10 that output raw data are determined by majority vote.

Alternatively, assuming that the detection result of the radar device 10b is relatively reliable, the output unit 23 may output the raw data of the sensor 10 other than the radar device 10b, that is, the camera 10a and the LiDAR 10c. good.

In this way, when there is a difference in detection results among the plurality of sensors 10, by using the outputs of the sensors 10 as raw data, erroneous target detection by the target detection device 50 can be suppressed.

Next, a processing procedure of processing executed by the output device 1 according to the embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing a processing procedure of processing executed by the output device 1 according to the embodiment.

As shown in FIG. 5, first, the acquisition unit 21 acquires raw data, which is data before processing of the detection result that is the output of the sensor that detects the surrounding situation (step S101).

Subsequently, the determination unit 22 determines the reliability of the detection result generated from the raw data acquired by the acquisition unit 21 (step S102).

Subsequently, the determination unit 22 determines whether or not the reliability of the detection result is low through determination processing (step S103).

When the determination unit 22 determines that the reliability of the detection result is low (step S103, Yes), the output unit 23 outputs raw data from the sensor 10 instead of the detection result (step S104), and ends the process. do.

On the other hand, in step S103, when the determination unit 22 determines that the reliability of the detection result is low (step S103, No), the output unit 23 outputs the detection result from the sensor 10 (step S105), and ends the process. do.

As described above, the output device 1 according to the embodiment includes the acquisition unit 21, the determination unit 22, and the output unit 23. The acquisition unit 21 acquires raw data, which is data before processing of the detection result that is the output of the sensor 10 . The determination unit 22 determines reliability of the detection result of the sensor 10 . When the determination unit 22 determines that the reliability of the detection result is low, the output unit 23 outputs raw data to the subsequent processing device instead of the detection result. Thereby, the reliability of the output of the sensor 10 can be improved.

In the above-described embodiment, the output device 1 is configured separately from the sensor 10 and the target detection device 50, but the output device 1, the sensor 10 and the target detection device 50 are integrated. It may be a single device configured to

Moreover, the output device 1 may be incorporated in each of the plurality of sensors 10 or may be incorporated in the target detection device 50 .

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments so shown and described. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

1 output device 2 control unit 3 storage unit 10 sensor 10a camera 10b radar device 10c LiDAR
21 acquisition unit 22 determination unit 23 output unit 31 correlation information 50 target detection device 100 target detection system C vehicle

Claims (8)

an acquisition unit that acquires raw data that is data before processing of sensor detection results;
a determination unit that determines the reliability of the sensor detection result;
an output unit that outputs the raw data to a subsequent processing device instead of the sensor detection result when the determination unit determines that the reliability of the sensor detection result is low ,
The determination unit
When the sensor detection result exceeds the detection limit of the sensor, it is determined that the reliability is low.
An output device characterized by: The acquisition unit
Acquire raw data from each of multiple sensors,
The determination unit
If the sensor detection results are different between the plurality of sensors, it is determined that the reliability is low,
The output unit
2. The output device according to claim 1, wherein, for at least one of said plurality of sensors, raw data is used as said output instead of a sensor detection result. an acquisition unit that acquires raw data that is data before processing of sensor detection results;
a determination unit that determines the reliability of the sensor detection result;
an output unit configured to output the raw data instead of the sensor detection result to a subsequent processing device when the determination unit determines that the reliability of the sensor detection result is low;
with
The acquisition unit
Acquire raw data from each of multiple sensors,
The determination unit is
If the sensor detection results are different between the plurality of sensors, it is determined that the reliability is low,
The output unit
For at least one sensor among the plurality of sensors, raw data is used as the output instead of the sensor detection result.
An output device characterized by: Further comprising a storage unit for storing raw data and correlation information of sensor detection results generated from the raw data,
The determination unit
The output device according to any one of claims 1 to 3, wherein reliability is determined based on the raw data acquired by the acquisition unit and the correlation information stored in the storage unit. an acquisition unit that acquires raw data that is data before processing of sensor detection results;
a determination unit that determines the reliability of the sensor detection result;
an output unit configured to output the raw data instead of the sensor detection result to a subsequent processing device when the determination unit determines that the reliability of the sensor detection result is low;
a storage unit that stores raw data and correlation information of sensor detection results generated from the raw data;
with
The determination unit is
Determining reliability based on the raw data acquired by the acquisition unit and the correlation information stored in the storage unit
An output device characterized by: an acquisition step of acquiring raw data, which is data before processing of sensor detection results;
a determination step of determining the reliability of the sensor detection result;
an output step of outputting the raw data to a subsequent processing device instead of the sensor detection result when the reliability of the sensor detection result is determined to be low by the determination step ,
The determination step includes
When the sensor detection result exceeds the detection limit of the sensor, it is determined that the reliability is low.
An output method characterized by an acquisition step of acquiring raw data, which is data before processing of sensor detection results;
a determination step of determining the reliability of the sensor detection result;
an output step of outputting the raw data to a subsequent processing device instead of the sensor detection result when the reliability of the sensor detection result is determined to be low by the determination step;
with
The obtaining step includes
Acquire raw data from each of multiple sensors,
The determination step includes
If the sensor detection results are different between the plurality of sensors, it is determined that the reliability is low,
The output step includes
For at least one sensor among the plurality of sensors, raw data is used as the output instead of the sensor detection result.
An output method characterized by an acquisition step of acquiring raw data, which is data before processing of sensor detection results;
a determination step of determining the reliability of the sensor detection result;
an output step of outputting the raw data to a subsequent processing device instead of the sensor detection result when the reliability of the sensor detection result is determined to be low by the determination step;
a storage step of storing raw data and correlation information of sensor detection results generated from the raw data;
with
The determination step includes
Determining reliability based on the raw data acquired in the acquiring step and the correlation information stored in the storing step
An output method characterized by

Images