WO2025062547A1 - 見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム - Google Patents

見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム Download PDF

Info

Publication number
WO2025062547A1
WO2025062547A1 PCT/JP2023/034175 JP2023034175W WO2025062547A1 WO 2025062547 A1 WO2025062547 A1 WO 2025062547A1 JP 2023034175 W JP2023034175 W JP 2023034175W WO 2025062547 A1 WO2025062547 A1 WO 2025062547A1
Authority
WO
WIPO (PCT)
Prior art keywords
subject
physical property
surrounding structure
information
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2023/034175
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
貴士 太田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to PCT/JP2023/034175 priority Critical patent/WO2025062547A1/ja
Priority to JP2025524814A priority patent/JPWO2025062547A1/ja
Publication of WO2025062547A1 publication Critical patent/WO2025062547A1/ja
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • This disclosure relates to technology for detecting objects in blind spots.
  • Patent Document 1 discloses the following notification device.
  • the notification device performs image analysis on the captured image to generate structure information about the current surrounding environment of the vehicle.
  • the structure information is information that indicates the structure of various objects in the surrounding environment.
  • the structure information includes distances to various structures.
  • the notification device can obtain structural information and detect blind spots based on the monitoring results of the navigation device.
  • Patent Document 1 does not mention how to generate structural information or how to assign physical property information. Patent Document 1 also does not mention how to integrate structural information.
  • the purpose of this disclosure is to enable accurate detection of objects in blind spots.
  • object detection can be performed using surrounding structure information to which physical property information has been added together with radar data. This makes it possible to accurately detect objects in blind spots (objects outside the line of sight).
  • FIG. 1 is a configuration diagram of a non-line-of-sight object detection system 200 according to a first embodiment.
  • FIG. 1 is a configuration diagram of an outside-of-line-of-sight object detection device 100 according to a first embodiment.
  • FIG. 2 is a functional configuration diagram of a non-line-of-sight object detection system 200 according to the first embodiment.
  • 4 is a flowchart of a method for detecting an object outside the line of sight in the first embodiment. 4 is a flowchart showing an example of a procedure of a contour extraction unit 111 according to the first embodiment. 4 is a flowchart showing an example of a procedure of a mesh generating unit 112 in the first embodiment.
  • FIG. 1 is a configuration diagram of a non-line-of-sight object detection system 200 according to a first embodiment.
  • FIG. 1 is a configuration diagram of an outside-of-line-of-sight object detection device 100 according to a first embodiment.
  • FIG. 2 is a functional configuration
  • the communication device 104 is a receiver and a transmitter.
  • the communication device 104 is a communication chip or a NIC.
  • the communication of the non-line-of-sight object detection device 100 is performed using the communication device 104.
  • NIC is an abbreviation for Network Interface Card.
  • the over-the-line-of-sight object detection program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or flash memory.
  • FIG. 3 shows the functional configuration of the non-line-of-sight object detection system 200 .
  • the surrounding structure detection unit 110 includes elements such as a contour extraction unit 111 and a mesh generation unit 112 .
  • the non-line-of-sight object detection unit 140 includes elements such as a mixer 141 , a converter 142 , a signal processing unit 143 , and an object detection unit 144 .
  • the map database 211 and the physical property database 212 are stored in, for example, the storage unit 190 .
  • the operation procedure of the non-line-of-sight object detection system 200 corresponds to a non-line-of-sight object detection method. Also, the operation procedure of the non-line-of-sight object detection device 100 corresponds to a processing procedure by a non-line-of-sight object detection program.
  • the surrounding structure information indicates, for example, the three-dimensional shapes of objects such as roads, roadside fences, vehicles on the road, and objects installed on the road.
  • step S110 the contour extraction unit 111 acquires distance measurement data from the distance measurement sensor 202 .
  • the contour extraction unit 111 generates surrounding structure information based on the distance measurement data.
  • the mesh generating unit 112 generates a mesh (three-dimensional model).
  • the surrounding structure information is generated, for example, by using SLAM, which stands for Simultaneous Localization and Mapping.
  • the surrounding structure detection unit 110 estimates the amount of movement using SLAM and combines the distance measurement data. As a result, the feature points or the 3D point clouds acquired by the distance measurement sensor 202 are combined.
  • the surrounding structure detection unit 110 may estimate the amount of movement based on motion information obtained from a wheel encoder and inertial information obtained from an IMU.
  • the non-line-of-sight object detection system 200 includes a wheel encoder and an IMU. IMU is an abbreviation for Inertial Measurement Unit.
  • FIG. 6 shows an example of a procedure of the mesh generating unit 112.
  • the mesh generation unit 112 determines whether or not there is pre-structure information in the map database 211.
  • the pre-structure information is surrounding structure information that is generated in advance and registered in the map database 211. If there is pre-structure information in the map database 211, the process proceeds to step S1122. If there is no prior structure information in the map database 211, processing proceeds to step S1123.
  • the mesh generating unit 112 merges the ranging structure information with the preliminary structure information.
  • the ranging structure information is the surrounding structure information generated by the contour extracting unit 111.
  • the mesh generating unit 112 generates a mesh based on the ranging structure information or the advance structure information into which the ranging structure information has been merged.
  • the mesh is an example of a representation of surrounding structural information.
  • Point cloud information or other shape information may be used instead of the mesh.
  • the region classification unit 120 detects one or more subject regions from the captured image. Then, the region classification section 120 identifies the subject type and the subject range for each subject region.
  • the region classification unit 120 performs semantic segmentation on the captured image.
  • FIG. 7 shows an example of semantic segmentation for a captured image. Using semantic segmentation, captured images are classified into regions based on subjects such as roads, sidewalks, medians, cars, poles, and forests.
  • semantic segmentation uses deep learning, although rule-based methods may also be used.
  • a CNN model is used in deep learning.
  • CNN is an abbreviation for Convolutional Neural Network.
  • a Transformer model or other models may also be used.
  • step S130 the physical property information adding unit 130 adds, for each subject region, physical property information of an object of the same type as the subject type to the surrounding structure information in association with the subject range.
  • the correspondence between the physical property information and the subject range is performed as follows: The mounting positions of the camera 203 and the distance measuring sensor 202 are adjusted in advance. Then, the physical property information and the subject range are associated by projecting the surrounding structure information onto the image based on the mounting positions of the camera 203 and the distance measuring sensor 202. Specifically, when the surrounding structure information is projected onto the image, the physical property information assigning unit 130 assigns the physical property information to pixels corresponding to the surrounding structure information.
  • the subject range is a range that corresponds to the subject area within the distance measurement range.
  • the physical property information indicates the properties that affect the reflection of the radio waves emitted from the radar 201.
  • the physical property database 212 is used.
  • the physical property database 212 is a database in which physical property information is registered for each object type.
  • the physical property information adding unit 130 acquires, for each subject region, physical property information of the object type that is the same as the subject type from the physical property database 212 .
  • the physical property information adding section 130 adds the acquired physical property information for each subject region to the surrounding structure information in association with the subject range.
  • step S140 the non-line-of-sight object detection unit 140 detects non-line-of-sight objects using the radar data and surrounding structure information.
  • Radar data is data obtained from radar 201 by making measurements using radar 201 .
  • the surrounding structure information is provided with physical property information for each subject region in step S130.
  • FIG. 11 shows the procedure of step S140.
  • the mixer 141 performs a mixing process on the radar data.
  • the mixing process combines the RX and TX signals to generate an IF signal, which has a lower frequency than the RX and TX signals.
  • RX stands for receive
  • TX stands for transmit
  • IF stands for intermediate frequency.
  • millimeter-wave radar uses a band of several tens of gigahertz, but to make signal processing easier, a lower-frequency signal is generated based on the difference between the two signals.
  • the frequency of the IF signal is equal to the instantaneous frequency difference between the RX signal and the TX signal.
  • the phase of the IF signal is equal to the phase difference between the RX signal and the TX signal.
  • the frequency band of the IF signal is reduced from the tens of gigahertz band of the RX signal and the TX signal to a few kilohertz band.
  • FIG 12 shows a sample RF circuit.
  • RF is an abbreviation for Radio Frequency.
  • FIG. 13 shows the procedure of the mixing process.
  • the mixer 141 receives the reflected signal.
  • the mixer 141 multiplies the chirp signal by the received signal.
  • the mixer 141 generates an IF signal.
  • the mixer 141 outputs the IF signal.
  • step S142 the converter 142 performs analog-to-digital conversion processing on the IF signal.
  • Analog-to-digital conversion converts an analog signal into a digital signal by performing certain processing on the analog signal.
  • FIG. 14 shows the procedure of the analog-to-digital conversion process.
  • the converter 142 receives the IF signal.
  • the converter 142 performs a sampling process on the IF signal. The sampling process converts the IF signal to be time-converted into discretized data by dividing the IF signal into unit times and extracting values at each time.
  • the converter 142 performs a quantization process on the discretized data, which converts the discretized data into quantized data by rounding the intensity of the discretized data at a constant interval.
  • the converter 142 performs an encoding process on the quantized data. The encoding process converts the quantized data into binary data.
  • the converter 142 outputs a binary data signal (digital signal).
  • step S143 the signal processing unit 143 performs signal processing on the digital signal.
  • This signal processing is typical radar signal processing. This signal processing performs Doppler conversion on the digital signal to detect the distance to an object, the relative speed of the object, and the relative angle of the object. This signal processing may be performed by a Digital Signal Processor (DSP).
  • DSP Digital Signal Processor
  • FFT is an abbreviation for Fast Fourier Transform.
  • the signal processing unit 143 receives a digital signal.
  • the signal processor 143 performs a distance FFT on the digital signal. Specifically, the signal processor 143 performs an FFT on the digital signal in units of chirps. This allows the distance to the object to be calculated. It is not possible to distinguish between multiple targets at the same distance.
  • the signal processor 143 performs a velocity FFT on the digital signal.
  • the velocity FFT is also called a Doppler FFT.
  • the signal processor 143 performs an FFT on the digital signal on a frame-by-frame basis based on the result of the distance FFT.
  • the signal processor 143 performs an angle FFT on the digital signal.
  • the angle FFT is also called Direction of Arrival (DoA).
  • DoA Direction of Arrival
  • the signal processor 143 performs an FFT on the digital signal across all receiving antennas based on the result of the speed FFT. This allows the relative angle of the object to be calculated.
  • the positions of multiple targets can be determined even if the multiple targets are at the same distance and have the same speed.
  • the signal processing unit 143 outputs the object information.
  • the object information includes position information and speed information.
  • the position information indicates the relative distance and the relative angle of the object.
  • the speed information indicates the relative speed of the object.
  • the object detection process uses object information and surrounding structure information to determine whether a detected object is a virtual image.
  • the detected object is the object detected in step S143.
  • the detected object is the object for which object information has been calculated in step S143.
  • the object detection unit 144 calculates the reflection source of the radio wave by using the distance information, the angle information, and the surrounding structure information. Then, the object detection unit 144 compares the position of the reflection source with the surrounding structure information to determine whether the detected object is a virtual image or a real image.
  • FIG. 16 shows the procedure of the object detection process.
  • object detection section 144 determines, for each detected object, based on the relative distance and relative angle, whether the detected object is indicated in the surrounding structure information.
  • the object detection section 144 determines whether there is a detected object that is not indicated in the surrounding structure information. If there is a detected object not indicated in the surrounding structure information, processing proceeds to step S1443. If there is no detected object not indicated in the surrounding structure information, the process ends.
  • object detection section 144 outputs object information of detected objects not indicated in the surrounding structure information. ***Advantages of First Embodiment*** According to the first embodiment, the physical property information can be added to the surrounding structure information.
  • the non-line-of-sight object detection device 100 may include multiple processing circuits that replace the processing circuit 109.
  • processing circuit 109 some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.
  • each element of the non-line-of-sight object detection device 100 may be interpreted as a "process,” “step,” “circuit,” or “circuitry.”
  • Non-line-of-sight object detection device 101: processor, 102: memory, 103: auxiliary storage device, 104: communication device, 105: input/output interface, 109: processing circuit, 110: surrounding structure detection unit, 111: contour extraction unit, 112: mesh generation unit, 120: area classification unit, 130: physical property information assignment unit, 140: non-line-of-sight object detection unit, 141: mixer, 142: converter, 143: signal processing unit, 144: object detection unit, 190: memory unit, 200: non-line-of-sight object detection system, 201: radar, 202: distance measurement sensor, 203: camera, 211: map database, 212: physical property database.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Electromagnetism (AREA)
  • Radar Systems Or Details Thereof (AREA)
PCT/JP2023/034175 2023-09-21 2023-09-21 見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム Pending WO2025062547A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2023/034175 WO2025062547A1 (ja) 2023-09-21 2023-09-21 見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム
JP2025524814A JPWO2025062547A1 (https=) 2023-09-21 2023-09-21

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/034175 WO2025062547A1 (ja) 2023-09-21 2023-09-21 見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム

Publications (1)

Publication Number Publication Date
WO2025062547A1 true WO2025062547A1 (ja) 2025-03-27

Family

ID=95072435

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/034175 Pending WO2025062547A1 (ja) 2023-09-21 2023-09-21 見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム

Country Status (2)

Country Link
JP (1) JPWO2025062547A1 (https=)
WO (1) WO2025062547A1 (https=)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005128603A (ja) * 2003-10-21 2005-05-19 Alpine Electronics Inc 物体認知装置および物体認知方法
JP2015230566A (ja) * 2014-06-04 2015-12-21 トヨタ自動車株式会社 運転支援装置
JP2018195293A (ja) * 2017-05-18 2018-12-06 三菱電機株式会社 画像処理システム、画像においてマルチラベル意味エッジ検出を行う方法、および、非一時的コンピューター可読記憶媒体
JP2021076422A (ja) * 2019-11-06 2021-05-20 日産自動車株式会社 物体認識方法及び物体認識装置
WO2023053718A1 (ja) * 2021-10-01 2023-04-06 ソニーセミコンダクタソリューションズ株式会社 情報処理装置及び情報処理方法、学習装置及び学習方法、並びにコンピュータプログラム
WO2023079881A1 (ja) * 2021-11-05 2023-05-11 ソニーグループ株式会社 情報処理装置、情報処理方法およびプログラム

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005128603A (ja) * 2003-10-21 2005-05-19 Alpine Electronics Inc 物体認知装置および物体認知方法
JP2015230566A (ja) * 2014-06-04 2015-12-21 トヨタ自動車株式会社 運転支援装置
JP2018195293A (ja) * 2017-05-18 2018-12-06 三菱電機株式会社 画像処理システム、画像においてマルチラベル意味エッジ検出を行う方法、および、非一時的コンピューター可読記憶媒体
JP2021076422A (ja) * 2019-11-06 2021-05-20 日産自動車株式会社 物体認識方法及び物体認識装置
WO2023053718A1 (ja) * 2021-10-01 2023-04-06 ソニーセミコンダクタソリューションズ株式会社 情報処理装置及び情報処理方法、学習装置及び学習方法、並びにコンピュータプログラム
WO2023079881A1 (ja) * 2021-11-05 2023-05-11 ソニーグループ株式会社 情報処理装置、情報処理方法およびプログラム

Also Published As

Publication number Publication date
JPWO2025062547A1 (https=) 2025-03-27

Similar Documents

Publication Publication Date Title
US11506512B2 (en) Method and system using tightly coupled radar positioning to improve map performance
CN113015922B (zh) 一种检测方法、检测装置以及存储介质
CN107144839B (zh) 通过传感器融合检测长对象
US11036237B2 (en) Radar-based system and method for real-time simultaneous localization and mapping
CN109215083B (zh) 车载传感器的外部参数标定的方法和设备
WO2020107038A1 (en) Method and system for positioning using radar and motion sensors
JP2020507137A (ja) 車両周辺の物体を識別して測位するためのシステムおよび方法
Cui et al. 3D detection and tracking for on-road vehicles with a monovision camera and dual low-cost 4D mmWave radars
US11954918B2 (en) Object detection device, object detection method, and storage medium
US11204610B2 (en) Information processing apparatus, vehicle, and information processing method using correlation between attributes
CN116507940A (zh) 用于雷达信号的信号处理的方法、雷达系统和车辆
CN115777072A (zh) Radar对象检测中的射程相关的虚假警报减少
EP3642645A2 (en) Methods and apparatus for distributed, multi-node, low-frequency radar systems for degraded visual environments
US20250305851A1 (en) Method and system for map building using radar and motion sensors
Iqbal et al. Imaging radar for automated driving functions
WO2020191978A1 (zh) Sar成像方法及其成像系统
US8723718B2 (en) Method and device for determining aspect angle progression
JP2023178490A (ja) 物体認識システム
JP2023068009A (ja) 地図情報作成方法
CN110023781A (zh) 用于根据车辆周围环境的雷达签名确定车辆的准确位置的方法和设备
WO2025062547A1 (ja) 見通し外物体検出装置、見通し外物体検出方法、見通し外物体検出プログラムおよび見通し外物体検出システム
WO2018227612A1 (en) Methods and apparatus for selecting a map for a moving object, system, and vehicle/robot
WO2019151109A1 (ja) 路面情報取得方法
CN112050830B (zh) 一种运动状态估计方法及装置
TWI918667B (zh) 用於無線電探測和測距的方法、裝置、非暫時性電腦可讀取媒體及系統

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2025524814

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025524814

Country of ref document: JP

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23953034

Country of ref document: EP

Kind code of ref document: A1