WO2023021756A1 - Information processing system, information processing device, and information processing method - Google Patents

Abstract

An information processing system according to one embodiment of the present technology comprises a plurality of radar devices, an information processing device, and an output device. The information processing device has a distance calculation unit and a processing unit. The distance calculation unit uses at least one of the plurality of radar devices to calculate a distance to an object. The processing unit carries out a process in which the plurality of radar devices detect different reflection points of an object when the distance to the object is less than a prescribed threshold, and in which a digital radar signal of the plurality of radar devices is synthesized to detect common reflection points of objects when the distance to the objects is not less than the prescribed threshold. The output device has an output unit that outputs reflection point information pertaining to a reflection point on the basis of the detection process of the processing unit.

Description

Information processing system, information processing device, and information processing method

The present technology relates to an information processing system, an information processing device, and an information processing method applicable to radar devices and the like.

In Patent Document 1, based on the detected position of the reflection point on the target and the received signal intensity of the reflected wave from the reflection point, a partial area where a part of the target exists is estimated, and the estimated one or more parts A radar device that estimates the occupied area of a target based on the area is described. As a result, the occupied area of the target can be estimated satisfactorily (paragraphs [0040] to [0051] in FIG. 6 of Patent Document 1, etc.).

JP-A-2020-3336

There is a demand for a technology that can achieve highly accurate object detection in such object detection by radar equipment.

In view of the circumstances as described above, an object of the present technology is to provide an information processing system, an information processing apparatus, and an information processing method capable of realizing highly accurate object detection.

To achieve the above object, an information processing system according to one aspect of the present technology includes a plurality of radar devices, an information processing device, and an output device.
The information processing device has a distance calculation unit and a processing unit.
The distance calculator calculates a distance to an object using at least one of the plurality of radar devices.
The processing unit detects, when the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and the distance to the object is greater than or equal to the predetermined threshold. In this case, a process of detecting a common reflection point of the object is executed by synthesizing the digital radar signals of the plurality of radar devices.
The output device has an output unit that outputs reflection point information about the reflection point based on the detection processing of the processing unit.

In this information processing system, distances from a plurality of radar devices to an object are calculated, and when the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object. and detecting a common reflection point of the object by synthesizing the digital radar signals of the plurality of radar devices to form a large-aperture array antenna when the distance to the object is equal to or greater than a predetermined threshold. be done. This makes it possible to achieve highly accurate object detection.

The information processing device has a first estimating unit that estimates reflection point information of each of the different reflection points based on a radar signal related to the object when the distance to the object is less than the predetermined threshold. You may In this case, the output device includes a second estimating unit that estimates reflection point information of the common reflection point based on the radar signal related to the object when the distance to the object is equal to or greater than the predetermined threshold. may have.

The reflection point information may include at least one of distance, speed, or direction.

The predetermined threshold may be determined based on the positional relationship of the plurality of radar devices.

The information processing device, based on a first estimation result of the reflection point information by the first estimation unit and a second estimation result of the reflection point information by the second estimation unit, the plurality of radar devices Each of may have a switching unit for switching to processing for detecting different reflection points of the object, or processing for detecting a common reflection point of the object by synthesizing digital radar signals of the plurality of radar devices. .

When a plurality of objects are detected at different angles according to the second estimation result, the switching unit detects a common reflection point of the object by synthesizing the digital radar signals of the plurality of radar devices. You can switch.

When the number of reflection points detected by the second estimation result is larger than the number of reflection points detected by the first estimation result, the switching unit synthesizes the digital radar signals of the plurality of radar devices. The process may be switched to detect a common reflection point of the object.

The switching unit causes each of the plurality of radar devices to detect a different reflection point of the object based on the distance and speed of the reflection point when the reflection point is close according to the first estimation result. processing, or processing for detecting a common reflection point of the object by synthesizing the digital radar signals of the plurality of radar devices.

When the distance to the object is less than the predetermined threshold, the processing unit outputs the reflection point information estimated by the first estimation unit to the output unit via a first network, A radar signal related to the object may be output to the second estimator via a second network when the distance to the object is equal to or greater than the predetermined threshold.

An information processing device according to an aspect of the present technology includes a distance calculation unit and a processing unit.
The distance calculator calculates a distance to an object using at least one of the plurality of radar devices.
The processing unit detects, when the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and the distance to the object is greater than or equal to the predetermined threshold. In this case, a process of detecting a common reflection point of the object is executed by synthesizing the digital radar signals of the plurality of radar devices.

An information processing method according to an embodiment of the present technology is an information processing method executed by a computer system, and includes calculating a distance to an object using at least one of a plurality of radar devices.
When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of A process of detecting a common reflection point of the object is performed by synthesizing the digital radar signals of the radar equipment.

It is a figure which shows the outline|summary of an information processing system typically. 1 is a block diagram showing a configuration example of an information processing system; FIG. It is a block diagram which shows the structural example of a detection process part. FIG. 10 is a schematic diagram showing a process from a digital radar signal to peak detection; 4 is a graph showing a chirp signal in the case of the FCM method; It is a schematic diagram in the case of performing short-distance and long-distance object detection using a plurality of radar devices. 4 is a flow chart showing the operation of a detection processing unit; 1 is a block diagram showing a configuration example of a vehicle control system, which is an example of an information processing system to which the present technology is applied; FIG. FIG. 9 is a diagram showing an example of sensing areas by the camera, radar device, LiDAR, ultrasonic sensor, etc. of the external recognition sensor in FIG. 8 ; It is a block diagram which shows the hardware structural example of a detection process part.

Hereinafter, embodiments according to the present technology will be described with reference to the drawings.

FIG. 1 is a diagram schematically showing an overview of an information processing system according to the present technology.

As shown in FIG. 1, the information processing system 100 has a plurality of radar devices 10. For example, the plurality of radar devices 10 are mounted on mobile objects such as automobiles, drones, and autonomous mobile robots. That is, the information processing system 100 can be applied to recognition of obstacles, map generation of the surrounding environment, tracking of surrounding moving objects, and the like. The present invention is not limited to this, and a plurality of radar devices may be installed at an intersection or the like, and applied to traffic monitoring such as traffic volume observation and danger prediction.

A plurality of radar devices 10 generate transmission signals and radiate them into space as transmission waves. Further, the plural radar devices 10 receive the transmission wave reflected by the reflection point 2 of the object 1 as a reflected wave. Note that the positional relationship and the number of installed radar devices are not limited.

The information processing system 100 switches detection processing for the reflection point 2 based on the distance to the object 1 . In this embodiment, when the distance to the object 1 is short, each of the plurality of radar devices 10 detects different reflection points of the object 1 . For example, in the left diagram of FIG. 1, reflected waves from reflection points 2A, 2B, and 2C of objects are received by radar devices 10A, 10B, and 10C, respectively.

Further, as shown in the right diagram of FIG. 1, when the distance from the object 1 is long, the detection processing of the reflection point 3 of the object 1 can be performed as one radar device by synthesizing the digital radar signals of a plurality of radar devices 10. done.

That is, when the distance to the object 1 is short, many reflection points are detected by the detection processing by each of the plurality of radar devices 10, so that it is possible to obtain a large amount of reflection point information about the reflection points. When the distance from the object 1 is long, a large aperture array antenna is formed by synthesizing the digital radar signals of a plurality of radar devices 10, and detection processing is performed as one radar device. Reflection point information can be obtained.

Reflection point information includes the distance, speed, and direction of the reflection point. For example, the distance until the reflected wave from the object 1 is received by the receiving antenna of the radar device, the relative speed between the object 1 and the moving body equipped with the radar device, the direction in which the moving body equipped with the radar device moves On the other hand, the angle at which the object 1 exists may be included.

Note that the method of synthesizing a plurality of radar devices 10 is not limited, and includes a synthetic aperture obtained by high-precision synchronization of a plurality of radar devices 10, a synthesis method in which a phase error between a plurality of radar devices 10 is corrected, an array of a plurality of radar devices 10. A combination of averaged antenna correlation matrices, etc. may be used.

FIG. 2 is a block diagram showing a configuration example of the information processing system 100. As shown in FIG.

As shown in FIG. 2, the information processing system 100 has a plurality of radar devices 10, a network 30, and an output device 40. Note that in this embodiment, object detection by two radar devices will be described as an example. When distinguishing two radar devices, they are described as a first radar device and a second radar device.

The radar device 10 has a radio signal transmission section 11 , a radio signal reception section 12 , a demodulation section 13 , an A/D conversion section 14 and a detection processing section 20 .

The radio signal transmission unit 11 generates a transmission signal and radiates it into space as a transmission wave. For example, in the case of an FCM (Fast Chirp Modulation) type radar apparatus, a transmission signal is generated in which a chirp signal whose frequency changes linearly is repeated at high speed. Also, the radio signal transmitter 11 outputs the transmission signal to the demodulator 13 .

The radio signal receiving section 12 receives the transmission wave reflected by the object as a reflected wave and outputs the received signal to the demodulation section 13 . For example, the radio signal receiving unit 12 receives reflected waves using a single or multiple antennas.

The demodulator 13 demodulates the received signal with the transmitted signal and outputs the demodulated signal to the A/D converter 14 . In this embodiment, the demodulator 13 generates a radar signal including the position and velocity information of the reflection point based on the received signal and the transmitted signal. In the case of the FCM system, the received signal and the transmitted signal are mixed to obtain a radar signal containing the difference frequency. Also, the frequency of the radar signal is proportional to the distance between the reflection point and the radar device 10 . The amount of phase change between repetitive chirps is proportional to the relative velocity between the reflection point and the radar device 10 .

The A/D converter 14 converts the demodulated signal into a digital value and outputs the digital radar signal to the detection processor 20 . In this embodiment, the A/D converter 14 samples and quantizes the demodulated signal to generate a digital radar signal. Depending on the radar system, the transmission signal and the reception signal may be converted into digital signals by the A/D converter 14 first, and then the digital radar signal may be generated by the demodulator 13 .

The detection processing unit 20 determines whether the radar device 10 or the output device 40 performs the detection processing of the reflection point of the object. A specific configuration will be described with reference to FIG.

In this embodiment, the detection processing unit 20 performs reflection point detection processing in the radar device 10 when the distance to the object is less than a predetermined threshold. That is, the detection processing unit 20 outputs the reflection point information to the output device 40 via the first network 31 when the distance to the object is less than a predetermined threshold.

Also, the detection processing unit 20 causes the output device 40 to perform the reflection point detection processing when the distance to the object is equal to or greater than a predetermined threshold. Specifically, the detection processing unit 20 generates an extracted radar signal in which components corresponding to the distance and speed of the reflection point are extracted from the input digital radar signal, and transmits the extracted radar signal to the second network 32. output to the output device 40 via.

The network 30 transmits reflection point information and extracted radar signals output from the plurality of radar devices 10 to the output device 40 . In this embodiment, the network 30 includes a first network 31 that transmits reflection point information from the plurality of radar devices 10 to the output device 40 and a second network 31 that transmits extracted radar signals from the plurality of radar devices 10 to the output device 40 . network 32 .

The output device 40 has an integrated detection processing unit 41 and a reflection point information output unit 42 .

The integrated detection processing unit 41 performs reflection point detection processing based on the extracted radar signals output from the plurality of radar devices 10 . In this embodiment, the integrated detection processing unit 41 estimates the direction in which the reflection point exists based on the extracted radar signal, and outputs reflection point information to the reflection point information output unit.

The reflection point information output unit 42 receives reflection point information from a plurality of radar devices 10 and the integrated detection processing unit 41 and outputs it to the outside.

FIG. 3 is a block diagram showing a configuration example of the detection processing unit 20. As shown in FIG.

As shown in FIG. 3, the detection processing unit 20 has a distance distribution calculation unit 21, a speed distribution calculation unit 22, a peak detection processing unit 23, a signal extraction unit 24, a detection processing switching unit 25, and a direction estimation processing unit 26. .

The detection processing unit 20 has hardware necessary for configuring a computer, such as processors such as CPU, GPU, and DSP, memories such as ROM and RAM, and storage devices such as HDD (see FIG. 10). For example, the information processing method according to the present technology is executed by the CPU loading a program according to the present technology pre-recorded in the ROM or the like into the RAM and executing the program.

For example, the detection processing unit 20 can be realized by any computer such as a PC. Of course, hardware such as FPGA and ASIC may be used.

In this embodiment, the detection processing switching unit as a functional block is configured by the CPU executing a predetermined program. Of course, dedicated hardware such as an IC (integrated circuit) may be used to implement the functional blocks.

The program is installed in the detection processing unit 20 via various recording media, for example. Alternatively, program installation may be performed via the Internet or the like.

The type of recording medium on which the program is recorded is not limited, and any computer-readable recording medium may be used. For example, any computer-readable non-transitory storage medium may be used.

The distance distribution calculator 21 converts the digital radar signal output from the A/D converter 14 into a distance spectrum. In this embodiment, the converted distance spectrum is supplied to the velocity distribution calculator 22 .

FIG. 4 is a schematic diagram showing the steps from digital radar signals to peak detection. FIG. 4A is a graph showing a demodulated digital radar signal. FIG. 4B is a graph showing the distance spectrum. FIG. 4C is a graph showing distance and velocity spectra.

4A and 4B, the horizontal axis is time and the vertical axis is frequency. 4A and 4B, the supplied digital radar signals 1 to L are shown for simplification. That is, it is assumed that each frame displays a digital radar signal generated based on the transmission and reception of chirp signals as shown in FIG.

FIG. 5 is a graph showing chirp signals in the case of the FCM method. In the graph shown in FIG. 5A, the horizontal axis indicates time and the vertical axis indicates RF frequency. In the graph shown in FIG. 5B, the horizontal axis indicates time and the vertical axis indicates amplitude.

For example, in the case of an FCM radar system, the frequency of the digital radar signal is proportional to the distance between the radar system and the reflection point. Obtainable.

The velocity distribution calculator 22 converts the distance spectrum into a distance and velocity spectrum. In this embodiment, the transformed distance and velocity spectra are supplied to the peak detection processing section 23 and the signal extraction section 24 .

In the case of FCM radar equipment, as shown in FIG. 4B, the distance spectrum corresponding to each chirp signal repeatedly transmitted is arranged in a chirp transmission timing sequence, and FFT (Fast Fourier Transform) is performed in the direction of the chirp transmission timing. and velocity spectra can be obtained.

The peak detection processing unit 23 detects reflection points by detecting power peaks in the distance and velocity spectra. In this embodiment, information on the distance and speed of the detected reflection point is supplied to the signal extractor 24 .

Reflection points exist at distances and velocities where the power intensity on the distance and velocity spectrum is large. In the graph shown in FIG. 4C, the horizontal axis indicates distance and the vertical axis indicates speed. As shown in FIG. 4, for example, the peak detection processing unit 23 calculates the square of the absolute value of the complex signal at each distance and each speed of the distance and velocity spectrum, which is a complex signal, that is, the power level, and then calculates the CFAR (Constant false alarm rate) The power level peak position is detected by detection processing or the like.

The signal extraction unit 24 extracts components corresponding to the distance and velocity of the reflection point from the distance and velocity spectrum. In addition, when the radio signal transmission unit 11 or the radio signal reception unit 12 uses a plurality of antennas, the signal extraction unit 24 generates a plurality of distance and speed spectra, so that a plurality of complex signals are extracted at a certain distance and speed. do. In this embodiment, the signal extraction unit 24 supplies the extracted radar signal to the detection processing switching unit 25 .

The detection processing switching unit 25 outputs the extracted radar signal to the direction estimation processing unit 26 or the output device 40 according to a predetermined standard. In this embodiment, the detection process switching unit 25 switches the detection process based on whether the object is less than a predetermined threshold.

FIG. 6 is a schematic diagram of a case where short-range and long-range object detection is performed using a plurality of radar devices. FIG. 6A is a schematic diagram for detecting an object at a short distance. FIG. 6B is a schematic diagram when long-distance object detection is performed.

As shown in FIG. 6A, when the distance between the object 6 and the first radar device 16 and the second radar device 17 is short, reflections from different reflection points (7 and 8) of the same object 6 in each radar device Waves may be observed. In this case, detection processing of each reflection point is performed on the digital radar signal in each radar device. Information on a plurality of different reflection points 7 and 8 of the object 6 can be obtained by aggregating the reflection point information of the reflection points. That is, it is possible to obtain useful information for estimating the object occupied area.

As shown in FIG. 6B, when the distance between the object 6 and the first radar device 16 and the second radar device 17 is long, reflected waves from the same reflection point 9 of the object 6 are observed in each radar device. there is a possibility. In this case, since the reflection point information of each radar device indicates the same reflection point, a plurality of different reflection point information cannot be obtained even if the reflection point information is aggregated.

In this embodiment, when the distance between the object 6 and the first radar device 16 and the second radar device 17 is long (when a plurality of radar devices observe the reflected wave from the same reflection point 9), each Detection accuracy can be improved by collecting the digital radar signals of the radar devices and then performing the detection processing of the reflection point 9 .

In particular, when high-precision phase synchronization is ensured between a plurality of radar devices, coherent processing in which the plurality of radar devices 10 are regarded as a single array antenna enables highly accurate and high-resolution direction estimation processing. .

The direction estimation processing unit 26 estimates the direction in which the reflection point exists from the extracted radar signal. That is, the direction estimation processing unit 26 generates reflection point information including the distance, speed, and direction of the reflection point, and outputs the reflection point information to the output device 40 .

In the present embodiment, the detection processing unit 20 includes a distance calculation unit that calculates the distance to an object using at least one of a plurality of radar devices, and a distance calculation unit that calculates the distance to the object. In this case, each of the plurality of radar devices detects a different reflection point of the object, and if the distance to the object is equal to or greater than a predetermined threshold, the digital radar signals of the plurality of radar devices are combined to detect a common reflection point of the object. It corresponds to an information processing apparatus including a processing unit that executes a process of detecting .
In this embodiment, the distance distribution calculator 21 corresponds to a distance calculator that calculates the distance to an object using at least one of a plurality of radar devices.
Note that in the present embodiment, the detection processing switching unit 25 performs a plurality of Each of the radar devices corresponds to a switching unit for switching to processing for detecting different reflection points on an object or processing for detecting a common reflection point on an object by synthesizing digital radar signals from a plurality of radar devices.
In this embodiment, when the distance to the object is less than a predetermined threshold, the direction estimation processing unit 26 estimates reflection point information for each of the different reflection points based on the radar signal related to the object. It corresponds to the estimation part.
In the present embodiment, the output device 40 corresponds to an output device having an output section that outputs reflection point information regarding reflection points based on the detection processing of the processing section.
In the present embodiment, when the distance to the object is equal to or greater than the predetermined threshold, the integrated detection processing unit 41 performs a second process for estimating reflection point information of a common reflection point based on radar signals related to the object. It corresponds to the estimation part.

FIG. 7 is a flowchart showing the operation of the detection processing unit 20. FIG.

As shown in FIG. 7, the distance distribution calculator 21 acquires a digital radar signal (step 101) and calculates a distance spectrum (step 102).

A distance and speed spectrum are calculated from the distance spectrum by the speed distribution calculation unit 22 (step 103).

A peak is detected by the peak detection processing unit 23 to obtain the distance and speed of the reflection point (step 104). Typically at step 104, multiple peaks are detected. That is, the processing of steps 105 to 111 is executed for each detected peak.

The signal extraction unit 24 extracts a complex signal of distance and velocity corresponding to the peak position in the distance and velocity spectrum (step 106).

The detection processing switching unit 25 determines whether the reflection point direction estimation processing is performed by the radar device 10 or the output device 40 (step 107). In this embodiment, the detection processing switching unit 25 switches processing based on whether or not the distance from the object to the radar device is less than a predetermined threshold value Rth.

If the distance at which the reflection point peak exists is equal to or greater than the threshold value Rth (YES in step 107), the extracted radar signal is transmitted to the output device 40 (step 108). That is, the direction estimation processing of the reflection point is executed by the integrated detection processing section 41 .

If the distance at which the reflection point peak exists is less than the threshold value Rth (NO in step 107), the extracted radar signal is transmitted to the direction estimation processing unit 26. The direction estimation processing unit 26 performs direction estimation processing of the reflection point based on the extracted radar signal (step 109). Specifically, the direction estimation processing unit 26 utilizes phase difference information between complex signals corresponding to a plurality of antennas, and uses a direction-of-arrival estimation algorithm such as a digital beamforming method or MUSIC (Multiple Signal Classification) to estimate the existence of a reflection point. Estimate direction.

The method for setting the predetermined threshold Rth is not limited, and may be set by any method. For example, it may be set based on the installation width of the radar device, such as by multiplying the installation width of the radar device by a predetermined coefficient.

Also, the detection process may be switched to other than the distance to the object. For example, when a plurality of objects are detected at different angles as a result of the direction estimation processing of the integrated detection processing unit 41 of the output device 40 , the output device 40 may perform the direction estimation processing of the reflection point.

Further, for example, when the number of reflection points detected by the direction estimation by the integrated detection processing unit 41 is larger than the average value of the number of reflection points detected by each radar device by the direction estimation by the direction estimation processing unit 26, the reflection point may be performed by the output device 40 . For example, in FIG. 1, the average number of detections is 2 because there are 6 reflection points and 3 radar devices in the left diagram of FIG. Become. In this case, since the average values are the same, the direction estimation processing unit 26 may perform the direction estimation processing of the reflection point.

In addition to this, overlap region Rth1<R (distance to object)<Rth2 may be set for Rth, and detection processing may be switched based on the setting. Further, for example, when there is a combination of reflection points having close distances (Δd) and velocities (Δv) from the detection result of the direction estimation processing unit 26, it is determined that the same reflection point may be observed. , the detection process may be switched according to the combination of the reflection points. That is, the detection process may be switched based on the magnitude relationship between the predetermined distance threshold (Dth) and Δd and the predetermined speed threshold (Vth) and Δv. Further, for example, control may be executed to switch detection processing in all detection areas of a plurality of radar devices.

The direction estimation processing unit 26 transmits the reflection point information to the output device 40 (step 110). Reflection point information is supplied to the reflection point information output unit 42 through steps 108 and 110 .

As described above, in the information processing system 100 according to the present embodiment, the distances from the plurality of radar devices 10 to the object 1 are calculated, and when the distance to the object 1 is less than the predetermined threshold value Rth, each of the plurality of radar devices 10 detects different reflection points 2 of the object 1, and the distance to the object 1 is equal to or greater than a predetermined threshold value Rth, the common reflection point 3 of the object 1 is detected by synthesizing the digital radar signals of a plurality of radar devices 10. Processing to detect is performed. This makes it possible to achieve highly accurate object detection.

Conventionally, it was attempted to improve the monitoring accuracy by installing multiple radar devices on mobile objects. However, when an object is far away in object detection, all of the plurality of radar devices observe the reflection point from the same point on the object, and it is difficult to improve the estimation accuracy even if the reflection point information is aggregated.

With this technology, when an object is at a short distance, each of multiple radar devices detects a different reflection point of the same object to improve accuracy, and when an object is at a long distance, multiple radar devices are integrated. Therefore, it is possible to detect the position of the object with high accuracy. Further, by sharing the reflection point detection processing between the radar device and the output device, it is possible to reduce the signal processing load in the radar device. Furthermore, it is possible to reduce the size and cost of the radar device. Further, by extracting the radar signal by the radar device and transmitting it to the output device, it is possible to relax the necessary throughput of the network between the radar device and the output device.

<Other embodiments>
The present technology is not limited to the embodiments described above, and various other embodiments can be implemented.

In the above embodiment, the detection processing switching unit 25 determines whether the direction estimation processing unit 26 of the radar device 10 or the integrated detection processing unit 41 of the output device 40 executes the reflection point direction estimation processing. Not limited to this, processing other than direction estimation processing, for example, calculation of distance spectrum, calculation of distance and speed spectrum, or other elements constituting reflection point detection processing such as peak detection, radar device 10 or output It may be switched by device 40 .

In the above embodiment, the direction estimation process of the reflection point is switched based on the distance to the object. Without being limited to this, the direction estimation process may be switched based on other conditions. For example, a radar device alternately transmits a transmission wave for short-range measurement and a transmission wave for long-range measurement, and the radar device executes reflection point detection processing of a digital radar signal corresponding to the transmission wave for short-range measurement. and a reflection point detection process of a digital radar signal corresponding to a transmission wave for long-distance measurement may be performed by the output device.

In the above embodiment, a plurality of radar devices 10 are arranged at fixed locations such as moving bodies and intersections. FIG. 8 shows an example of a block diagram when a plurality of radar devices 10 are mounted on a vehicle. That is, a case where the information processing system 100 is applied to a vehicle 50 and a vehicle control system 51 is shown.

[Configuration example of vehicle control system]
FIG. 8 is a block diagram showing a configuration example of a vehicle control system 51, which is an example of an information processing system to which the present technology is applied.

The vehicle control system 51 is provided in the vehicle 50 and performs processing related to driving support and automatic driving of the vehicle 50 .

The vehicle control system 51 includes a vehicle control ECU (Electronic Control Unit) 52, a communication unit 53, a map information accumulation unit 54, a position information acquisition unit 55, an external recognition sensor 56, an in-vehicle sensor 57, a vehicle sensor 58, a storage unit 59, a travel Assistance/automatic driving control unit 60 , DMS (Driver Monitoring System) 61 , HMI (Human Machine Interface) 62 , and vehicle control unit 63 .

Vehicle control ECU 52, communication unit 53, map information accumulation unit 54, position information acquisition unit 55, external recognition sensor 56, vehicle interior sensor 57, vehicle sensor 58, storage unit 59, driving support/automatic driving control unit 60, driver monitoring system ( DMS) 61 , human machine interface (HMI) 62 , and vehicle control unit 63 are connected via a communication network 64 so as to be able to communicate with each other.
The communication network 64 is, for example, a CAN (Controller Area Network), a LIN (Local Interconnect Network), a LAN (Local Area Network), a FlexRay (registered trademark), or an in-vehicle communication network conforming to a digital two-way communication standard such as Ethernet (registered trademark). It is composed of a communication network, a bus, and the like. The communication network 64 may be used properly depending on the type of data to be transmitted. For example, CAN may be applied to data related to vehicle control, and Ethernet may be applied to large-capacity data. Note that each unit of the vehicle control system 51 performs wireless communication assuming relatively short-range communication such as near field communication (NFC) or Bluetooth (registered trademark), for example, without going through the communication network 64. may be connected directly using

In addition, hereinafter, when each part of the vehicle control system 51 communicates via the communication network 64, the description of the communication network 64 is omitted. For example, when the vehicle control ECU 52 and the communication unit 53 communicate via the communication network 64, it is simply described that the vehicle control ECU 52 and the communication unit 53 communicate.

The vehicle control ECU 52 is composed of various processors such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The vehicle control ECU 52 controls the functions of the entire vehicle control system 51 or a part thereof.

The communication unit 53 communicates with various devices inside and outside the vehicle, other vehicles, servers, base stations, etc., and transmits and receives various data. At this time, the communication unit 53 can perform communication using a plurality of communication methods.

The communication with the outside of the vehicle that can be performed by the communication unit 53 will be described schematically. The communication unit 53 is, for example, 5G (5th generation mobile communication system), LTE (Long Term Evolution), DSRC (Dedicated Short Range Communications), etc., by a wireless communication method, via a base station or an access point, on an external network communicates with a server (hereinafter referred to as an external server) located in the The external network with which the communication unit 53 communicates is, for example, the Internet, a cloud network, or a provider's own network. The communication method performed by the communication unit 53 with respect to the external network is not particularly limited as long as it is a wireless communication method that enables digital two-way communication at a communication speed of a predetermined value or more and a distance of a predetermined value or more.

Also, for example, the communication unit 53 can communicate with terminals existing in the vicinity of the own vehicle using P2P (Peer To Peer) technology. Terminals in the vicinity of one's own vehicle are, for example, terminals worn by pedestrians, bicycles, and other moving objects that move at relatively low speeds, terminals installed at fixed locations in stores, etc., or MTC (Machine Type Communication) terminal. Furthermore, the communication unit 53 can also perform V2X communication. V2X communication includes, for example, vehicle-to-vehicle communication with other vehicles, vehicle-to-infrastructure communication with roadside equipment, etc., and vehicle-to-home communication , and communication between the vehicle and others, such as vehicle-to-pedestrian communication with a terminal or the like possessed by a pedestrian.

For example, the communication unit 53 can receive from the outside a program for updating the software that controls the operation of the vehicle control system 51 (Over The Air). The communication unit 53 can also receive map information, traffic information, information around the vehicle 50, and the like from the outside. Further, for example, the communication unit 53 can transmit information about the vehicle 50, information about the surroundings of the vehicle 50, and the like to the outside. The information about the vehicle 50 that the communication unit 53 transmits to the outside includes, for example, data indicating the state of the vehicle 50, recognition results by the recognition unit 74, and the like. Further, for example, the communication unit 53 performs communication corresponding to a vehicle emergency notification system such as e-call.

For example, the communication unit 53 receives electromagnetic waves transmitted by a road traffic information communication system (VICS (Vehicle Information and Communication System) (registered trademark)) such as radio wave beacons, optical beacons, and FM multiplex broadcasting.

Communication with the inside of the vehicle that can be performed by the communication unit 53 will be described schematically. The communication unit 53 can communicate with each device in the vehicle using, for example, wireless communication. The communication unit 53 performs wireless communication with devices in the vehicle by a communication method such as wireless LAN, Bluetooth, NFC, and WUSB (Wireless USB), which enables digital two-way communication at a communication speed higher than a predetermined value. can be done. Not limited to this, the communication unit 53 can also communicate with each device in the vehicle using wired communication. For example, the communication unit 53 can communicate with each device in the vehicle by wired communication via a cable connected to a connection terminal (not shown).
The communication unit 53 performs digital two-way communication at a predetermined communication speed or higher by wired communication such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), and MHL (Mobile High-definition Link). can communicate with each device in the vehicle.

Here, equipment in the vehicle refers to equipment not connected to the communication network 64 in the vehicle, for example. Examples of in-vehicle devices include mobile devices and wearable devices possessed by passengers such as drivers, information devices that are brought into the vehicle and temporarily installed, and the like.

The map information accumulation unit 54 accumulates one or both of a map obtained from the outside and a map created by the vehicle 50. For example, the map information accumulation unit 54 accumulates a three-dimensional high-precision map, a global map covering a wide area, and the like, which is lower in accuracy than the high-precision map.

High-precision maps are, for example, dynamic maps, point cloud maps, vector maps, etc. The dynamic map is, for example, a map consisting of four layers of dynamic information, semi-dynamic information, semi-static information, and static information, and is provided to the vehicle 50 from an external server or the like. A point cloud map is a map composed of a point cloud (point cloud data). A vector map is, for example, a map adapted to ADAS (Advanced Driver Assistance System) and AD (Autonomous Driving) by associating traffic information such as lane and traffic signal positions with a point cloud map.

The point cloud map and vector map, for example, may be provided from an external server or the like, or a map for matching with a local map to be described later based on sensing results from the camera 65, radar device 66, LiDAR 67, etc. may be created in the vehicle 50 and stored in the map information storage unit 54 . Further, when a high-precision map is provided from an external server or the like, in order to reduce the communication capacity, map data of, for example, several hundred meters square regarding the planned route on which the vehicle 50 will travel from now on is acquired from the external server or the like. .

The position information acquisition unit 55 receives GNSS signals from GNSS (Global Navigation Satellite System) satellites and acquires position information of the vehicle 50 . The acquired position information is supplied to the driving support/automatic driving control unit 60 . In addition, the position information acquisition unit 55 is not limited to the method using the GNSS signal, and may acquire the position information using a beacon, for example.

The external recognition sensor 56 includes various sensors used for recognizing situations outside the vehicle 50 , and supplies sensor data from each sensor to each part of the vehicle control system 51 . The type and number of sensors included in the external recognition sensor 56 are arbitrary.

For example, the external recognition sensor 56 includes a camera 65, a radar device 66, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 67, and an ultrasonic sensor 68. Not limited to this, the external recognition sensor 56 may be configured to include one or more types of sensors among the camera 65 , the radar device 66 , the LiDAR 67 , and the ultrasonic sensor 68 . The number of cameras 65 , radar devices 66 , LiDARs 67 , and ultrasonic sensors 68 is not particularly limited as long as it is a number that can be realistically installed on the vehicle 50 . Further, the type of sensor provided by the external recognition sensor 56 is not limited to this example, and the external recognition sensor 56 may be provided with other types of sensors. An example of the sensing area of each sensor included in the external recognition sensor 56 will be described later.

The imaging method of the camera 65 is not particularly limited. For example, cameras of various types such as a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, and an infrared camera, which are capable of distance measurement, can be applied to the camera 65 as necessary. The camera 65 is not limited to this, and may simply acquire a photographed image regardless of distance measurement.

Also, for example, the external recognition sensor 56 can include an environment sensor for detecting the environment with respect to the vehicle 50. The environment sensor is a sensor for detecting the environment such as weather, climate, brightness, etc., and can include various sensors such as raindrop sensors, fog sensors, sunshine sensors, snow sensors, and illuminance sensors.

Furthermore, for example, the external recognition sensor 56 includes a microphone used for detecting sounds around the vehicle 50 and the position of the sound source.

The in-vehicle sensor 57 includes various sensors for detecting information inside the vehicle, and supplies sensor data from each sensor to each part of the vehicle control system 51 . The types and number of various sensors included in the in-vehicle sensor 57 are not particularly limited as long as they are the types and number that can be installed in the vehicle 50 in practice.

For example, the in-vehicle sensor 57 can include one or more sensors among cameras, radar devices, seating sensors, steering wheel sensors, microphones, and biosensors. As the camera provided in the in-vehicle sensor 57, for example, cameras of various shooting methods capable of distance measurement, such as a ToF camera, a stereo camera, a monocular camera, and an infrared camera, can be used. Not limited to this, the camera provided in the vehicle interior sensor 57 may simply acquire a captured image regardless of distance measurement. The biosensors included in the in-vehicle sensor 57 are provided, for example, on a seat, a steering wheel, or the like, and detect various biometric information of a passenger such as a driver.

The vehicle sensor 58 includes various sensors for detecting the state of the vehicle 50, and supplies sensor data from each sensor to each part of the vehicle control system 51. The types and number of various sensors included in the vehicle sensor 58 are not particularly limited as long as the types and number are practically installable in the vehicle 50 .

For example, the vehicle sensor 58 includes a speed sensor, an acceleration sensor, an angular velocity sensor (gyro sensor), and an inertial measurement unit (IMU (Inertial Measurement Unit)) integrating them. For example, the vehicle sensor 58 includes a steering angle sensor that detects the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor that detects the amount of operation of the accelerator pedal, and a brake sensor that detects the amount of operation of the brake pedal. For example, the vehicle sensor 58 includes a rotation sensor that detects the number of revolutions of an engine or a motor, an air pressure sensor that detects tire air pressure, a slip rate sensor that detects a tire slip rate, and a wheel speed sensor that detects the rotational speed of a wheel. A sensor is provided. For example, the vehicle sensor 58 includes a battery sensor that detects the remaining battery level and temperature, and an impact sensor that detects external impact.

The storage unit 59 includes at least one of a nonvolatile storage medium and a volatile storage medium, and stores data and programs. The storage unit 59 is used as, for example, EEPROM (Electrically Erasable Programmable Read Only Memory) and RAM (Random Access Memory), and storage media include magnetic storage devices such as HDD (Hard Disc Drive), semiconductor storage devices, optical storage devices, And a magneto-optical storage device can be applied. The storage unit 59 stores various programs and data used by each unit of the vehicle control system 51 . For example, the storage unit 59 includes an EDR (Event Data Recorder) and a DSSAD (Data Storage System for Automated Driving), and stores information on the vehicle 50 before and after an event such as an accident and information acquired by the in-vehicle sensor 57.

The driving support/automatic driving control unit 60 controls driving support and automatic driving of the vehicle 50 . For example, the driving support/automatic driving control unit 60 includes an analysis unit 69 , an action planning unit 70 and an operation control unit 71 .

The analysis unit 69 analyzes the vehicle 50 and its surroundings. The analysis unit 69 includes a self-position estimation unit 72 , a sensor fusion unit 73 and a recognition unit 74 .

The self-position estimation unit 72 estimates the self-position of the vehicle 50 based on the sensor data from the external recognition sensor 56 and the high-precision map accumulated in the map information accumulation unit 54. For example, the self-position estimation unit 72 generates a local map based on sensor data from the external recognition sensor 56, and estimates the self-position of the vehicle 50 by matching the local map and the high-precision map. The position of the vehicle 50 is based on, for example, the center of the rear wheels versus the axle.

A local map is, for example, a three-dimensional high-precision map created using techniques such as SLAM (Simultaneous Localization and Mapping), an occupancy grid map, or the like. The three-dimensional high-precision map is, for example, the point cloud map described above. The occupancy grid map is a map that divides the three-dimensional or two-dimensional space around the vehicle 50 into grids (lattice) of a predetermined size and shows the occupancy state of objects in grid units. The occupancy state of an object is indicated, for example, by the presence or absence of the object and the existence probability. The local map is also used, for example, by the recognizing unit 74 for detecting and recognizing the situation outside the vehicle 50 .

The self-position estimation unit 72 may estimate the self-position of the vehicle 50 based on the position information acquired by the position information acquisition unit 55 and the sensor data from the vehicle sensor 58.

The sensor fusion unit 73 combines a plurality of different types of sensor data (for example, image data supplied from the camera 65 and sensor data supplied from the radar device 66) to perform sensor fusion processing to obtain new information. conduct. Methods for combining different types of sensor data include integration, fusion, federation, and the like.

The recognition unit 74 executes a detection process for detecting the situation outside the vehicle 50 and a recognition process for recognizing the situation outside the vehicle 50 .

For example, the recognition unit 74 performs detection processing and recognition processing of the situation outside the vehicle 50 based on information from the external recognition sensor 56, information from the self-position estimation unit 72, information from the sensor fusion unit 73, and the like. .

Specifically, for example, the recognition unit 74 performs detection processing and recognition processing of objects around the vehicle 50 . Object detection processing is, for example, processing for detecting the presence or absence, size, shape, position, movement, and the like of an object. Object recognition processing is, for example, processing for recognizing an attribute such as the type of an object or identifying a specific object. However, detection processing and recognition processing are not always clearly separated, and may overlap.

For example, the recognition unit 74 detects objects around the vehicle 50 by clustering the point cloud based on sensor data from the radar device 66 or the LiDAR 67 or the like for each group of point groups. As a result, presence/absence, size, shape, and position of objects around the vehicle 50 are detected.

For example, the recognizing unit 74 detects the movement of objects around the vehicle 50 by performing tracking that follows the movement of the masses of point groups classified by clustering. As a result, the speed and traveling direction (movement vector) of objects around the vehicle 50 are detected.

For example, the recognition unit 74 detects or recognizes vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc. based on image data supplied from the camera 65 . The recognition unit 74 may also recognize types of objects around the vehicle 50 by performing recognition processing such as semantic segmentation.

For example, the recognizing unit 74, based on the map accumulated in the map information accumulating unit 54, the estimation result of the self-position by the self-position estimating unit 72, and the recognition result of the object around the vehicle 50 by the recognizing unit 74, Recognition processing of traffic rules around the vehicle 50 can be performed. Through this processing, the recognizing unit 74 can recognize the position and state of traffic lights, the content of traffic signs and road markings, the content of traffic regulations, the lanes in which the vehicle can travel, and the like.

For example, the recognition unit 74 can perform recognition processing of the environment around the vehicle 50 . Weather, temperature, humidity, brightness, road conditions, and the like are assumed as the surrounding environment to be recognized by the recognition unit 74 .

The action plan unit 70 creates an action plan for the vehicle 50. For example, the action planning unit 70 creates an action plan by performing route planning and route following processing.

It should be noted that global path planning is the process of planning a rough path from the start to the goal. This route planning is referred to as trajectory planning, and in the planned route, trajectory generation (local path planning) that can proceed safely and smoothly in the vicinity of the vehicle 50 in consideration of the motion characteristics of the vehicle 50 is performed. It also includes the processing to be performed.

　Route following is the process of planning actions to safely and accurately travel the route planned by route planning within the planned time. The action planning unit 70 can, for example, calculate the target speed and target angular speed of the vehicle 50 based on the result of this route following processing.

The motion control unit 71 controls the motion of the vehicle 50 in order to implement the action plan created by the action planning unit 70.

For example, the operation control unit 71 controls the steering control unit 81, the brake control unit 82, and the drive control unit 83 included in the vehicle control unit 63, which will be described later, so that the vehicle 50 can control the trajectory calculated by the trajectory plan. Acceleration/deceleration control and direction control are performed so as to advance. For example, the operation control unit 71 performs cooperative control aimed at realizing functions of ADAS such as collision avoidance or shock mitigation, follow-up driving, vehicle speed maintenance driving, collision warning of own vehicle, and lane departure warning of own vehicle. For example, the operation control unit 71 performs cooperative control aimed at automatic driving in which the vehicle autonomously travels without depending on the operation of the driver.

The DMS 61 performs driver authentication processing, driver state recognition processing, etc., based on sensor data from the in-vehicle sensor 57 and input data input to the HMI 62, which will be described later. As the state of the driver to be recognized, for example, physical condition, wakefulness, concentration, fatigue, gaze direction, drunkenness, driving operation, posture, etc. are assumed.

It should be noted that the DMS 61 may perform authentication processing for passengers other than the driver and processing for recognizing the state of the passenger. Further, for example, the DMS 61 may perform the process of recognizing the situation inside the vehicle based on the sensor data from the sensor 57 inside the vehicle. Conditions inside the vehicle to be recognized include temperature, humidity, brightness, smell, and the like, for example.

The HMI 62 inputs various data, instructions, etc., and presents various data to the driver.

Input of data by HMI 62 will be described schematically. The HMI 62 includes input devices for human input of data. The HMI 62 generates an input signal based on data, instructions, etc. input from an input device, and supplies the input signal to each section of the vehicle control system 51 . The HMI 62 includes operators such as touch panels, buttons, switches, and levers as input devices. The HMI 62 is not limited to this, and may further include an input device capable of inputting information by a method other than manual operation using voice, gestures, or the like.
Furthermore, the HMI 62 may use, as an input device, a remote control device using infrared rays or radio waves, or an external connection device such as a mobile device or wearable device corresponding to the operation of the vehicle control system 51 .

The presentation of data by the HMI 62 will be briefly explained. The HMI 62 generates visual, auditory, and tactile information for passengers or outside the vehicle. In addition, the HMI 62 performs output control for controlling the output, output content, output timing, output method, and the like of each generated information. The HMI 62 generates and outputs visual information such as an operation screen, a status display of the vehicle 50, a warning display, a monitor image showing the surroundings of the vehicle 50, and information indicated by light and images. The HMI 62 also generates and outputs information indicated by sounds such as voice guidance, warning sounds, warning messages, etc., as auditory information. Furthermore, the HMI 62 generates and outputs, as tactile information, information given to the passenger's tactile sense by force, vibration, movement, or the like.

As an output device from which the HMI 62 outputs visual information, for example, a display device that presents visual information by displaying an image by itself or a projector device that presents visual information by projecting an image can be applied. . In addition to the display device having a normal display, the display device displays visual information within the passenger's field of view, such as a head-up display, a transmissive display, and a wearable device with an AR (Augmented Reality) function. It may be a device. In addition, the HMI 62 can use a display device provided in the vehicle 50 such as a navigation device, an instrument panel, a CMS (Camera Monitoring System), an electronic mirror, a lamp, etc., as an output device for outputting visual information.

Audio speakers, headphones, and earphones, for example, can be applied as output devices for the HMI 62 to output auditory information.

As an output device for the HMI 62 to output tactile information, for example, a haptic element using haptic technology can be applied. A haptic element is provided at a portion of the vehicle 50 that is in contact with a passenger, such as a steering wheel or a seat.

The vehicle control unit 63 controls each unit of the vehicle 50 . The vehicle control section 63 includes a steering control section 81 , a brake control section 82 , a drive control section 83 , a body system control section 84 , a light control section 85 and a horn control section 86 .

The steering control unit 81 detects and controls the state of the steering system of the vehicle 50 . The steering system includes, for example, a steering mechanism including a steering wheel, an electric power steering, and the like. The steering control unit 81 includes, for example, a steering ECU that controls the steering system, an actuator that drives the steering system, and the like.

The brake control unit 82 detects and controls the state of the brake system of the vehicle 50 . The brake system includes, for example, a brake mechanism including a brake pedal, an ABS (Antilock Brake System), a regenerative brake mechanism, and the like. The brake control unit 82 includes, for example, a brake ECU that controls the brake system, an actuator that drives the brake system, and the like.

The drive control unit 83 detects and controls the state of the drive system of the vehicle 50 . The drive system includes, for example, an accelerator pedal, a driving force generator for generating driving force such as an internal combustion engine or a driving motor, and a driving force transmission mechanism for transmitting the driving force to the wheels. The drive control unit 83 includes, for example, a drive ECU that controls the drive system, an actuator that drives the drive system, and the like.

The body system control unit 84 detects and controls the state of the body system of the vehicle 50 . The body system includes, for example, a keyless entry system, smart key system, power window device, power seat, air conditioner, air bag, seat belt, shift lever, and the like. The body system control unit 84 includes, for example, a body system ECU that controls the body system, an actuator that drives the body system, and the like.

The light control unit 85 detects and controls the states of various lights of the vehicle 50 . Lights to be controlled include, for example, headlights, backlights, fog lights, turn signals, brake lights, projections, bumper displays, and the like. The light control unit 85 includes a light ECU that controls the light, an actuator that drives the light, and the like.

The horn control unit 86 detects and controls the state of the car horn of the vehicle 50 . The horn control unit 86 includes, for example, a horn ECU for controlling the car horn, an actuator for driving the car horn, and the like.

FIG. 9 is a diagram showing an example of sensing areas by the camera 65, the radar device 66, the LiDAR 67, the ultrasonic sensor 68, etc. of the external recognition sensor 56 in FIG. 9 schematically shows the vehicle 50 viewed from above, the left end side being the front end (front) side of the vehicle 50, and the right end side being the rear end (rear) side of the vehicle 50.

A sensing area 101F and a sensing area 101B show examples of sensing areas of the ultrasonic sensor 68. The sensing area 101</b>F covers the periphery of the front end of the vehicle 50 with a plurality of ultrasonic sensors 68 . The sensing area 101B covers the periphery of the rear end of the vehicle 50 with a plurality of ultrasonic sensors 68 .

The sensing results in the sensing area 101F and the sensing area 101B are used, for example, for parking assistance of the vehicle 50, and the like.

Sensing areas 102F to 102B are examples of sensing areas of the radar device 66 for short or medium range. The sensing area 102F covers the front of the vehicle 50 to a position farther than the sensing area 101F. Sensing area 102B covers the rear of vehicle 50 to a position farther than sensing area 101B. The sensing area 102L covers the rear periphery of the left side surface of the vehicle 50 . The sensing area 102R covers the rear periphery of the right side surface of the vehicle 50 .

The sensing result in the sensing area 102F is used, for example, to detect vehicles, pedestrians, etc. existing in front of the vehicle 50. The sensing result in the sensing area 102B is used, for example, for the rear collision prevention function of the vehicle 50 or the like. The sensing results in sensing area 102L and sensing area 102R are used, for example, for detecting an object in a blind spot on the side of vehicle 50, or the like.

Sensing areas 103F to 103B show examples of sensing areas by the camera 65. The sensing area 103F covers the front of the vehicle 50 to a position farther than the sensing area 102F. Sensing area 103B covers the rear of vehicle 50 to a position farther than sensing area 102B. The sensing area 103L covers the periphery of the left side surface of the vehicle 50 . The sensing area 103R covers the periphery of the right side surface of the vehicle 50 .

The sensing results in the sensing area 103F can be used, for example, for recognition of traffic lights and traffic signs, lane departure prevention support systems, and automatic headlight control systems. A sensing result in the sensing area 103B can be used for parking assistance and a surround view system, for example. Sensing results in the sensing area 103L and the sensing area 103R can be used, for example, in a surround view system.

The sensing area 104 shows an example of the sensing area of the LiDAR67. The sensing area 104 covers the front of the vehicle 50 to a position farther than the sensing area 103F. On the other hand, the sensing area 104 has a narrower lateral range than the sensing area 103F.

The sensing results in the sensing area 104 are used, for example, to detect objects such as surrounding vehicles.

A sensing area 105 shows an example of a sensing area of the long-range radar device 66 . Sensing area 105 covers the front of vehicle 50 to a position farther than sensing area 104 . On the other hand, the sensing area 105 has a narrower lateral range than the sensing area 104 .

The sensing results in the sensing area 105 are used, for example, for ACC (Adaptive Cruise Control), emergency braking, and collision avoidance.

The sensing regions of the cameras 65, the radar device 66, the LiDAR 67, and the ultrasonic sensors 68 included in the external recognition sensor 56 may have various configurations other than those shown in FIG. Specifically, the ultrasonic sensor 68 may also sense the sides of the vehicle 50 , and the LiDAR 67 may sense the rear of the vehicle 50 . Moreover, the installation position of each sensor is not limited to each example mentioned above. Also, the number of each sensor may be one or plural.

FIG. 10 is a block diagram showing a hardware configuration example of the detection processing unit 20. As shown in FIG.

The detection processing unit 20 includes a CPU 201, a ROM 202, a RAM 203, an input/output interface 205, and a bus 204 that connects these to each other. A display unit 206, an input unit 207, a storage unit 208, a communication unit 209, a drive unit 210, and the like are connected to the input/output interface 205. FIG.

The display unit 206 is, for example, a display device using liquid crystal, EL, or the like. The input unit 207 is, for example, a keyboard, pointing device, touch panel, or other operating device. When input unit 207 includes a touch panel, the touch panel can be integrated with display unit 206 .

The storage unit 208 is a non-volatile storage device, such as an HDD, flash memory, or other solid-state memory. The drive unit 210 is a device capable of driving a removable recording medium 211 such as an optical recording medium or a magnetic recording tape.

The communication unit 209 is a modem, router, and other communication equipment for communicating with other devices that can be connected to a LAN, WAN, or the like. The communication unit 209 may use either wired or wireless communication. The communication unit 209 is often used separately from the detection processing unit 20 .

Information processing by the detection processing unit 20 having the hardware configuration as described above is realized by cooperation between software stored in the storage unit 208 or the ROM 202 or the like and hardware resources of the detection processing unit 20 . Specifically, the information processing method according to the present technology is realized by loading a program constituting software stored in the ROM 202 or the like into the RAM 203 and executing the program.

The program is installed in the detection processing unit 20 via the recording medium 211, for example. Alternatively, the program may be installed in the detection processing unit 20 via a global network or the like. In addition, any computer-readable non-transitory storage medium may be used.

An information processing method and a program according to the present technology are executed by linking a computer installed in a communication terminal with another computer that can communicate via a network or the like, and a detection processing switching unit according to the present technology is constructed. may

That is, the information processing system, information processing apparatus, and information processing method according to the present technology can be executed not only in a computer system configured by a single computer, but also in a computer system in which a plurality of computers work together. In the present disclosure, a system means a set of multiple components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device housing a plurality of modules within a single housing, are both systems.

Execution of the information processing system, information processing apparatus, and information processing method according to the present technology by a computer system is, for example, a case where calculation of a distance spectrum, detection of a peak, switching of detection processing, and the like are executed by a single computer. , and when each process is executed by a different computer. Execution of each process by a predetermined computer includes causing another computer to execute part or all of the process and obtaining the result.

That is, the information processing system, information processing device, and information processing method according to the present technology can be applied to a configuration of cloud computing in which a single function is shared by a plurality of devices via a network and processed jointly. is.

Each configuration of the distance distribution calculation unit, the detection processing switching unit, the direction estimation processing unit, and the like, the control flow of the communication system, and the like, which have been described with reference to the drawings, are merely one embodiment, and are within the scope of the present technology. , can be arbitrarily transformed. That is, any other configuration, algorithm, or the like for implementing the present technology may be employed.

It should be noted that the effects described in the present disclosure are merely examples and are not limited, and other effects may also occur. The above description of multiple effects does not necessarily mean that those effects are exhibited simultaneously. It means that at least one of the above-described effects can be obtained depending on the conditions, etc., and of course, effects not described in the present disclosure may also be exhibited.

It is also possible to combine at least two of the characteristic portions of each form described above. That is, various characteristic portions described in each embodiment may be combined arbitrarily without distinguishing between each embodiment.

Note that the present technology can also adopt the following configuration.
(1)
a plurality of radar devices;
a distance calculation unit that calculates a distance to an object using at least one of the plurality of radar devices;
When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of a processing unit that performs processing for detecting a common reflection point of the object by synthesizing digital radar signals of the radar device; and an information processing device comprising:
An information processing system comprising: an output device having an output unit that outputs reflection point information about the reflection point based on the detection processing of the processing unit.
(2) The information processing system according to (1),
The information processing device has a first estimating unit that estimates reflection point information of each of the different reflection points based on a radar signal related to the object when the distance to the object is less than the predetermined threshold. death,
The output device has a second estimator that estimates reflection point information of the common reflection point based on the radar signal related to the object when the distance to the object is equal to or greater than the predetermined threshold. system.
(3) The information processing system according to (1),
The information processing system, wherein the reflection point information includes at least one of distance, speed, and direction.
(4) The information processing system according to (1),
The information processing system, wherein the predetermined threshold is determined based on the positional relationship of the plurality of radar devices.
(5) The information processing system according to (2),
The information processing device, based on a first estimation result of the reflection point information by the first estimation unit and a second estimation result of the reflection point information by the second estimation unit, the plurality of radar devices has a switching unit for switching to processing for detecting different reflection points of the object or processing for detecting a common reflection point of the object by synthesizing digital radar signals of the plurality of radar devices.
(6) The information processing system according to (5),
When a plurality of objects are detected at different angles according to the second estimation result, the switching unit detects a common reflection point of the object by synthesizing the digital radar signals of the plurality of radar devices. Switch information processing system.
(7) The information processing system according to (5),
When the number of reflection points detected by the second estimation result is larger than the number of reflection points detected by the first estimation result, the switching unit synthesizes the digital radar signals of the plurality of radar devices. An information processing system that switches to processing for detecting a common reflection point of the object.
(8) The information processing system according to (5),
The switching unit causes each of the plurality of radar devices to detect a different reflection point of the object based on the distance and speed of the reflection point when the reflection point is close according to the first estimation result. processing, or switching to processing for detecting a common reflection point of the object by synthesizing digital radar signals of the plurality of radar devices.
(9) The information processing system according to (2),
When the distance to the object is less than the predetermined threshold, the processing unit outputs the reflection point information estimated by the first estimation unit to the output unit via a first network, An information processing system that outputs a radar signal related to the object to the second estimation unit via a second network when the distance to the object is equal to or greater than the predetermined threshold.
(10)
a distance calculation unit that calculates a distance to an object using at least one of a plurality of radar devices;
When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of and a processing unit that detects a common reflection point of the object by synthesizing digital radar signals of radar devices.
(11)
Using at least one of a plurality of radar devices to calculate the distance to the object,
When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of An information processing method in which a computer system performs a process of detecting a common reflection point of said object by synthesizing digital radar signals of radar equipment.

DESCRIPTION OF SYMBOLS 10... Plurality of radar apparatuses 20... Detection process part 25... Detection process switching part 26... Direction estimation process part 40... Output device 41... Integrated detection process part 100... Information processing system

Claims (11)

1. a plurality of radar devices;
   a distance calculation unit that calculates a distance to an object using at least one of the plurality of radar devices;
   When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of a processing unit that performs processing for detecting a common reflection point of the object by synthesizing digital radar signals of the radar device; and an information processing device comprising:
   An information processing system comprising: an output device having an output unit that outputs reflection point information about the reflection point based on the detection processing of the processing unit.
2. The information processing system according to claim 1,
   The information processing device has a first estimating unit that estimates reflection point information of each of the different reflection points based on a radar signal related to the object when the distance to the object is less than the predetermined threshold. death,
   The output device has a second estimator that estimates reflection point information of the common reflection point based on the radar signal related to the object when the distance to the object is equal to or greater than the predetermined threshold. system.
3. The information processing system according to claim 1,
   The information processing system, wherein the reflection point information includes at least one of distance, speed, and direction.
4. The information processing system according to claim 1,
   The information processing system, wherein the predetermined threshold is determined based on the positional relationship of the plurality of radar devices.
5. The information processing system according to claim 2,
   The information processing device, based on a first estimation result of the reflection point information by the first estimation unit and a second estimation result of the reflection point information by the second estimation unit, the plurality of radar devices has a switching unit for switching to processing for detecting different reflection points of the object or processing for detecting a common reflection point of the object by synthesizing digital radar signals of the plurality of radar devices.
6. The information processing system according to claim 5,
   When a plurality of objects are detected at different angles according to the second estimation result, the switching unit detects a common reflection point of the object by synthesizing the digital radar signals of the plurality of radar devices. Switch information processing system.
7. The information processing system according to claim 5,
   When the number of reflection points detected by the second estimation result is larger than the number of reflection points detected by the first estimation result, the switching unit synthesizes the digital radar signals of the plurality of radar devices. An information processing system that switches to processing for detecting a common reflection point of the object.
8. The information processing system according to claim 5,
   The switching unit causes each of the plurality of radar devices to detect a different reflection point of the object based on the distance and speed of the reflection point when the reflection point is close according to the first estimation result. processing, or switching to processing for detecting a common reflection point of the object by synthesizing digital radar signals of the plurality of radar devices.
9. The information processing system according to claim 2,
   When the distance to the object is less than the predetermined threshold, the processing unit outputs the reflection point information estimated by the first estimation unit to the output unit via a first network, An information processing system that outputs a radar signal related to the object to the second estimation unit via a second network when the distance to the object is equal to or greater than the predetermined threshold.
10. a distance calculation unit that calculates a distance to an object using at least one of a plurality of radar devices;
    When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of and a processing unit that detects a common reflection point of the object by synthesizing digital radar signals of radar devices.
11. Using at least one of a plurality of radar devices to calculate the distance to the object,
    When the distance to the object is less than a predetermined threshold, each of the plurality of radar devices detects a different reflection point of the object, and when the distance to the object is greater than or equal to the predetermined threshold, the plurality of An information processing method in which a computer system performs a process of detecting a common reflection point of said object by synthesizing digital radar signals of radar equipment.

Images