KR102618924B1 - System and method for detecting object based fmcw radar - Google Patents

Abstract

The method performed by the FMCW radar-based object detection system according to the present invention includes the steps of receiving radar sensing information about an object approaching within a predetermined distance (hereinafter referred to as a proximity object) through the FMCW radar; Generating 3D feature map information based on the radar sensing information; Receiving stereo image information captured through a stereo camera targeting the nearby object; generating 3D depth map information based on the stereo image information; generating a combined 3D map by combining the 3D feature map information and the 3D depth map information; and displaying distance and posture information of the object based on the combined 3D map.

Description

FMCW radar based object detection system and method {SYSTEM AND METHOD FOR DETECTING OBJECT BASED FMCW RADAR}

The present invention relates to an FMCW radar-based object detection system and method, and particularly to a system and method for detecting a target (object) such as a threat based on information acquired through an FMCW radar and a stereo camera.

Radar systems according to the prior art detect objects using a point method and generally consist of a radome, an antenna, a transceiver, a signal processing device, and a housing. Radar antennas emit and receive radio waves and collect reflected signals. The transceiver manages the transmission and reception of radio waves and calculates the Doppler frequency. The signal processing device can analyze the received data and extract the distance, speed, and angle of the object. The housing can protect the radar system and maintain its stability.

In general, radar detects distance, angle, and speed as a single point. In particular, Continuous Wave Frequency Modulated Continuous Wave (FMCW) radar can detect the distance and speed of multiple objects, and can detect the distance and speed of multiple objects. An appropriate compromise is possible between data, complexity, and cost characteristics. And by installing multiple radars and extracting object information in the surround environment based on radar images, accurate object information can be generated.

For reference, Figure 1 is a diagram showing an example of distance-Doppler for extracting the distance of an object by radar. Figure 2 is a diagram showing an example of distance-angle for angle extraction of an object by radar.

Meanwhile, in the prior art, radar sensors are relatively robust to external environments such as rain or fog, detect accurate distances using the round-trip time of radio waves, and directly obtain speed information using Doppler frequencies reflected from objects. At this time, because there is a limit to the expansion of the number of receiving antennas due to the size of the radar system, the angle detection performance is low and all objects are recognized as one point, which makes it very difficult to classify and recognize the type of object, but the Doppler spectrum A technique is used to recognize/classify objects by extracting feature vectors of objects.

In order to detect objects with high precision with a radar sensor system, multiple radars must be installed to generate radar images in a surround environment. Although it is possible to extract object information based on these radar images, each radar sensor simply points out the object. , and the distance and angle detection performance must be advanced to high resolution. At this time, there is a time-division MIMO (Multiple Input Multiple Output) technique to detect the angle of an object at high resolution, and this technique can detect not only a 2D (distance, angle) map but also 3D object information consisting of distance-horizontal coordinates-vertical coordinates. However, compared to existing visible light sensors or infrared sensors, there is a problem of low resolution, especially at close range.

In addition, unlike visible light sensors or infrared sensors, the radar sensor system can extract Doppler information for each point detected by the radar, so 4D images can be obtained when using not only simple radio images but also the Doppler spectrum that exists at each scattering point. Although it is possible to improve the recognition rate of objects based on point-cloud data, there is still a problem with low resolution at close range.

The problem that the present invention aims to solve is to detect objects approaching by FMCW radar by combining the point cloud 3D image and stereo image of the FMCW radar in the absence of a radar with advanced, high-resolution distance and angle detection performance. It relates to an FMCW radar-based object detection system and method that can generate a precise 3D map.

However, the problem to be solved by the present invention is not limited to the problems described above, and other problems may exist.

The method performed by the FMCW radar-based object detection system according to the first aspect of the present invention to solve the above-described problem involves radar sensing of an object approaching within a predetermined distance (hereinafter referred to as a nearby object) through the FMCW radar. receiving information; Generating 3D feature map information based on the radar sensing information; Receiving stereo image information captured through a stereo camera targeting the nearby object; generating 3D depth map information based on the stereo image information; generating a combined 3D map by combining the 3D feature map information and the 3D depth map information; and displaying distance and posture information of the object based on the combined 3D map.

In some embodiments of the present invention, generating 3D feature map information based on the radar sensing information includes extracting feature points from the radar sensing information; estimating distance information and angle information to an object from the radar sensing information; And it may include generating 3D feature map information based on the feature point, distance information, and angle information.

In some embodiments of the present invention, estimating distance information and angle information to an object from the radar sensing information includes calculating a difference between a transmitted signal frequency and a received signal frequency of the radar sensing signal; calculating a slope reflecting the difference value and delay time; And it may include estimating the distance information based on the difference value and the slope value.

In some embodiments of the present invention, estimating distance information and angle information to an object from the radar sensing information includes obtaining distance information between an operating frequency of the radar and a plurality of receiving antennas; Transmitting and receiving radar signals to the object; calculating a frequency difference between the received radar signals; And it may include calculating the angle information based on the operating frequency of the radar, the frequency difference between radar signals, and distance information between receiving antennas.

In some embodiments of the present invention, the step of calculating the frequency difference between the received radar signals may be calculated based on distance change information between the radar antenna and the object and the operating frequency of the radar.

In some embodiments of the present invention, generating 3D depth map information based on the stereo image information includes extracting feature points from the stereo image information; generating 3D depth map information based on the feature points; And it may include estimating distance information and angle information to an object based on the 3D depth map information.

In some embodiments of the present invention, generating a combined 3D map by combining the 3D feature map information and the 3D depth map information includes extracting a first object distortion change amount from the 3D feature map information; extracting a second object distortion change amount from the 3D depth map information; and combining the 3D feature map information and the 3D depth map information based on the first and second object distortion changes.

In addition, the FMCW radar-based object detection system according to the second aspect of the present invention receives radar sensing information about an object approaching within a predetermined distance (hereinafter referred to as a proximity object) through the FMCW radar, and targets the proximity object. A communication module that receives stereo image information captured through a stereo camera, a memory storing a program for detecting an object based on the radar sensing information and the stereo image information, and executing the program stored in the memory, the radar Generating 3D feature map information based on sensing information, generating 3D depth map information based on the stereo image information, and then combining the 3D feature map information and the 3D depth map information to generate a combined 3D map, and a processor that controls display of distance and posture information of the object based on the combined 3D map.

In some embodiments of the present invention, the processor extracts a feature point from the radar sensing information, estimates distance information and angle information to an object from the radar sensing information, and then calculates the feature point based on the distance information and angle information. You can generate 3D feature map information.

In some embodiments of the present invention, the processor calculates a difference between the transmitted signal frequency and the received signal frequency of the radar sensing signal, calculates a slope reflecting the difference and delay time, and then calculates the difference and slope. The distance information can be estimated based on the value.

In some embodiments of the present invention, the processor obtains distance information between the operating frequency of the radar and a plurality of receiving antennas, and calculates the frequency difference of the received radar signal as the radar signal is transmitted and received from the object. And, the angle information can be calculated based on the operating frequency of the radar, the frequency difference between radar signals, and distance information between receiving antennas.

In some embodiments of the present invention, the processor may calculate the distance change information between the radar antenna and the object and the operating frequency of the radar.

In some embodiments of the present invention, the processor extracts feature points from the stereo image information, generates 3D depth map information based on the feature points, and then generates distance information and angle with the object based on the 3D depth map. Information can be estimated.

In some embodiments of the present invention, the processor extracts a first object distortion change amount from the 3D feature map information, extracts a second object distortion change amount from the 3D depth map information, and determines the first and second object distortion changes. Based on the amount of change, the 3D feature map information and the 3D depth map information can be combined.

A computer program according to another aspect of the present invention for solving the above-described problem is combined with a computer as hardware to execute the FMCW radar-based object detection method, and is stored in a computer-readable recording medium.

Other specific details of the invention are included in the detailed description and drawings.

According to an embodiment of the present invention described above, even in an environment that does not have an existing high-resolution radar, the distance and angle of an object approaching within 0.2 m can be precisely detected with high resolution using an FMCW radar.

Additionally, data collected through the FMCW radar is used to express the object's location as a 3D point cloud, images are taken using a stereo camera, and depth information is generated based on the displacement between the two images, and then the FMCW radar By combining the 3D point cloud obtained from the 3D map and the 3D map obtained from the stereo camera to create a precise 3D map of the object, the distance, direction, and depth information of the object can be provided in high resolution. By using these precise 3D maps, precise distance and posture information of an object can be generated and provided, and the location and direction of the object can be determined more precisely.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

The drawings attached below are intended to aid understanding of the present embodiment and provide examples along with a detailed description. However, the technical features of this embodiment are not limited to specific drawings, and the features disclosed in each drawing may be combined to form a new embodiment.
Figure 1 is a diagram illustrating a range-Doppler example for distance extraction of an object by radar.
Figure 2 is a diagram showing an example of distance-angle for angle extraction of an object by radar.
Figure 3 is a flowchart of an FMCW radar-based object detection method according to an embodiment of the present invention.
is a diagram for explaining the FMCW radar-based object detection method in terms of functions according to an embodiment of the present invention.
Figure 5 is a diagram showing an example of 3D feature map information according to an embodiment of the present invention.
Figure 6 is a diagram for explaining the FMCW radar-based distance calculation process in one embodiment of the present invention.
Figures 7a and 7b are diagrams for explaining the FMCW radar-based angle calculation process in one embodiment of the present invention.
Figure 8 is a diagram showing feature point information extracted from the FMCW radar.
Figure 9 is a diagram illustrating an example of feature point information through a stereo camera according to an embodiment of the present invention.
Figure 10 is a diagram for explaining the process of extracting depth and distance changes when generating a stereo image according to an embodiment of the present invention.
Figure 11 is a diagram showing an example of 3D feature map information according to an embodiment of the present invention.
Figure 12 is a diagram showing an example of a combined 3D map generated in an embodiment of the present invention.
Figure 13 is a block diagram of an FMCW radar-based object detection system according to an embodiment of the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, a method performed by the FMCW radar-based object detection system 100 according to an embodiment of the present invention will be described with reference to FIGS. 3 to 12.

Figure 3 is a flowchart of an FMCW radar-based object detection method according to an embodiment of the present invention. Figure 4 is a diagram for explaining the FMCW radar-based object detection method in terms of functionality according to an embodiment of the present invention. Figure 5 is a diagram showing an example of 3D feature map information according to an embodiment of the present invention.

The FMCW radar-based object detection method according to an embodiment of the present invention first receives radar sensing information about an object approaching within a predetermined distance (hereinafter referred to as a proximity object) through the FMCW radar (S110).

One embodiment of the present invention is aimed at detecting an object approaching with an FMCW radar. For example, the predetermined distance between the FMCW radar and the object may be set to within 0.2 m, but is not necessarily limited thereto.

Next, 3D feature map information is generated based on the radar sensing information (S120).

In one embodiment, in order to generate 3D feature map information through radar sensing information, first, feature points are extracted from the radar sensing information (S121), and then distance information and angle information to the object are estimated from the radar sensing information (S122) ), 3D feature map information can be generated based on feature points, distance information, and angle information (S133). An example of the 3D feature map information generated in this way is shown in FIG. 5.

Figure 6 is a diagram for explaining the FMCW radar-based distance calculation process in one embodiment of the present invention.

In one embodiment, in order to calculate distance information to an object, the difference between the transmitted signal frequency and the received signal frequency of the radar sensing signal is calculated. Next, calculate the slope reflecting the difference value and delay time (τ). When the difference between the transmitted signal frequency and the received signal frequency is △f _FMCW , the slope can be calculated through the following equation 1.

[Equation 1]

Next, the distance information (R) can be estimated based on the difference value and the slope value according to Equation 2 below. At this time, c in Equation 2 refers to the electromagnetic wave speed.

[Equation 2]

Meanwhile, an embodiment of the present invention uses an FMCW radar to determine the relative speed of an object ( ) can be calculated, which can be calculated through the following equation 3 based on the change in the received signal frequency.

[Equation 3]

or

At this time, in Equation 3, f ₀ is the average frequency, B is the bandwidth, △f _a is the frequency difference of the rising ramp, △f _b is the frequency difference of the falling ramp, and △f _D is the change value of the received frequency.

Figures 7a and 7b are diagrams for explaining the FMCW radar-based angle calculation process in one embodiment of the present invention. At this time, Figure 7a shows a transmitting antenna (Tx) and two receiving antennas (Rx), and Figure 7b is for explaining the process of estimating the angle from the two receiving antennas (a, b).

In one embodiment, in order to calculate angle information with an object, first the operating frequency of the radar ( ) and obtain distance information (d) between the plurality of receiving antennas. Then, the radar signal is transmitted to the object and received, and the frequency difference of the received radar signal is Calculate based on Equation 4. At this time, in equation 4 As the distance between the radar antenna and the object changes, am.

[Equation 4]

Then, based on the operating frequency of the radar signal, the frequency difference between the radar signals, and the distance information between the receiving antennas, the angle information with the object using the FMCW radar ( ) can be calculated based on the following equation 5.

[Equation 5]

At this time, in the case of equation 4 It can also be expressed as

Figure 8 shows feature point information (F(x, y, z)) extracted from the FMCW radar, and the previously calculated distance information ( ) and angle information (y, z) can be reflected in the feature points to generate 3D feature map information.

Referring again to FIG. 1, next, stereo image information captured through a stereo camera targeting a nearby object is received (S130), and 3D depth map information is generated based on the stereo image information (S140).

Figure 9 is a diagram illustrating an example of feature point information through a stereo camera according to an embodiment of the present invention. Figure 10 is a diagram for explaining the process of extracting depth and distance changes when generating a stereo image according to an embodiment of the present invention. Figure 11 is a diagram showing an example of 3D feature map information according to an embodiment of the present invention.

In one embodiment, in order to generate a 3D depth map based on stereo image information, feature points are first extracted from the stereo image information (S141). Next, 3D depth map information is generated based on the feature points (S142), and distance information and angle information to the object are estimated based on the 3D depth map information (S143).

At this time, the feature point I (x, y, z) extracted through the stereo image contains the extracted distance ( ), information about angle (y, z) values may be included.

In one embodiment, in the case of calculating distance information to an object from stereo image information, the received signal frequencies (①, ②, ③, ④, ⑤) expressed in the distance information calculation method of the FMCW radar described in FIG. 6 When creating a stereo depth map (①', ②', ③', ④', ⑤'), distance information for each corresponding location must be calculated. This is to combine 3D feature map information and 3D depth map information in a later step.

Referring again to FIG. 3, next, the 3D feature map information and the 3D depth map information are combined to generate a combined 3D map (S150), and the distance and posture information of the object are displayed based on the combined 3D map (S160).

Figure 12 is a diagram showing an example of a combined 3D map generated in an embodiment of the present invention.

In one embodiment, the process of generating a combined 3D map first extracts the first object distortion change amount from 3D feature map information (S151) and extracts the second object distortion change amount from 3D depth map information (S152). Next, 3D feature map information and 3D depth map information can be combined based on the first and second object distortion changes (S153).

At this time, the data combination 3D map reflecting the feature points in the FMCW radar-based 3D feature map in the stereo 3D depth map information is as shown in FIG. 13, where the expression of data combination reflecting the feature points can be expressed as Equation 6 below. .

[Equation 6]

Meanwhile, in the above description, steps S110 to S160 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Additionally, some steps may be omitted or the order between steps may be changed as needed. In addition, even if other omitted content, the content described in FIGS. 3 to 12 and the content described in FIG. 13 can be mutually applied.

Figure 13 is a block diagram of the FMCW radar-based object detection system 100 according to an embodiment of the present invention.

The object detection system 100 according to an embodiment of the present invention includes a communication module 110, a memory 120, and a processor 130.

The communication module 110 receives radar sensing information about a nearby object within a predetermined distance through the FMCW radar, and receives stereo image information captured through a stereo camera targeting the nearby object. This communication module 110 may include both a wired communication module and a wireless communication module. The wired communication module can be implemented as a power line communication device, telephone line communication device, home cable (MoCA), Ethernet, IEEE1294, integrated wired home network, and RS-485 control device. In addition, wireless communication modules include WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology, as well as 5G (5th generation communication) and LTE-A. It may be composed of modules to implement functions such as (long term evolution-advanced), LTE (long term evolution), and Wi-Fi (wireless fidelity).

A program for detecting objects based on radar sensing information and stereo image information is stored in the memory 120. Here, the memory 120 is a general term for non-volatile storage devices and volatile storage devices that continue to retain stored information even when power is not supplied. For example, memory 120 may include compact flash (CF) cards, secure digital (SD) cards, memory sticks, solid-state drives (SSD), and micro SD. This includes NAND flash memory such as cards, magnetic computer storage devices such as hard disk drives (HDD), and optical disc drives such as CD-ROM, DVD-ROM, etc. You can.

As the processor 130 executes the program stored in the memory 120, it generates 3D feature map information based on radar sensing information and 3D depth map information based on stereo image information. Then, the processor 130 generates a combined 3D map by combining the 3D feature map information and the 3D depth map information, and controls the display of the distance and posture information of the object based on the combined 3D map.

Meanwhile, in one embodiment of the present invention, the processor 130 may use at least one of machine learning, neural network, or deep learning algorithms as an artificial intelligence algorithm in the combined 3D map generation process. there is. For example, as an artificial intelligence algorithm, at least one of machine learning, neural network, or deep learning algorithm may be used. Examples of neural network networks include Convolutional Neural Network (CNN) and Deep Neural Network (DNN). Network) and RNN (Recurrent Neural Network).

The FMCW radar-based object detection system 100 and method according to an embodiment of the present invention described above may be implemented as a program (or application) and stored in a medium to be executed in combination with a computer, which is hardware.

The above-mentioned program is C, C++, JAVA, Ruby, and It may include code encoded in a computer language such as machine language. These codes may include functional codes related to functions that define the necessary functions for executing the methods, and include control codes related to execution procedures necessary for the computer's processor to execute the functions according to predetermined procedures. can do. In addition, these codes may further include memory reference-related codes that indicate at which location (address address) in the computer's internal or external memory additional information or media required for the computer's processor to execute the above functions should be referenced. there is. In addition, if the computer's processor needs to communicate with any other remote computer or server in order to execute the above functions, the code uses the computer's communication module to determine how to communicate with any other remote computer or server. It may further include communication-related codes regarding whether communication should be performed and what information or media should be transmitted and received during communication.

The storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers that the computer can access or on various recording media on the user's computer. Additionally, the medium may be distributed to computer systems connected to a network, and computer-readable code may be stored in a distributed manner.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as unitary may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (14)

In a method performed by an FMCW radar-based object detection system,
Receiving radar sensing information about an object approaching within a predetermined distance (hereinafter referred to as a proximity object) through the FMCW radar;
Generating 3D feature map information based on the radar sensing information;
Receiving stereo image information captured through a stereo camera targeting the nearby object;
generating 3D depth map information based on the stereo image information;
generating a combined 3D map by combining the 3D feature map information and the 3D depth map information; and
FMCW radar-based object detection method, comprising displaying distance and pose information of the object based on the combined 3D map.
According to paragraph 1,
The step of generating 3D feature map information based on the radar sensing information includes:
extracting feature points from the radar sensing information;
estimating distance information and angle information to an object from the radar sensing information; and
An FMCW radar-based object detection method comprising generating 3D feature map information based on the feature points, distance information, and angle information.
According to paragraph 2,
The step of estimating distance information and angle information to an object from the radar sensing information,
calculating a difference between the transmitted signal frequency and the received signal frequency of the radar sensing signal;
calculating a slope reflecting the difference value and delay time; and
FMCW radar-based object detection method, including the step of estimating the distance information based on the difference value and the slope value.
According to paragraph 2,
The step of estimating distance information and angle information to an object from the radar sensing information,
Obtaining information on the operating frequency of the radar and the distance between the plurality of receiving antennas;
Transmitting and receiving radar signals to the object;
calculating a frequency difference between the received radar signals; and
An FMCW radar-based object detection method comprising calculating the angle information based on the operating frequency of the radar, the frequency difference between radar signals, and distance information between receiving antennas.
According to paragraph 4,
The step of calculating the frequency difference of the received radar signal is,
An FMCW radar-based object detection method that is calculated based on distance change information between the radar antenna and the object and the operating frequency of the radar.
According to paragraph 1,
The step of generating 3D depth map information based on the stereo image information includes:
extracting feature points from the stereo image information;
generating 3D depth map information based on the feature points; and
FMCW radar-based object detection method, including the step of estimating distance information and angle information to an object based on the 3D depth map information.
According to paragraph 1,
The step of generating a combined 3D map by combining the 3D feature map information and the 3D depth map information,
extracting a first object distortion change amount from the 3D feature map information;
extracting a second object distortion change amount from the 3D depth map information; and
An FMCW radar-based object detection method comprising combining the 3D feature map information and the 3D depth map information based on the first and second object distortion changes.
In the FMCW radar-based object detection system,
A communication module that receives radar sensing information about a nearby object (hereinafter referred to as a close object) within a predetermined distance through the FMCW radar and receives stereo image information captured through a stereo camera targeting the close object,
A memory storing a program for detecting an object based on the radar sensing information and stereo image information, and
By executing the program stored in the memory, 3D feature map information is generated based on the radar sensing information, 3D depth map information is generated based on the stereo image information, and then the 3D feature map information and the 3D An FMCW radar-based object detection system comprising a processor that combines depth map information to generate a combined 3D map, and controls to display distance and posture information of an object based on the combined 3D map.
According to clause 8,
The processor extracts feature points from the radar sensing information, estimates distance information and angle information to an object from the radar sensing information, and then generates 3D feature map information based on the feature points, distance information, and angle information. In,FMCW radar-based object detection system.
According to clause 9,
The processor calculates a difference between the transmitted signal frequency and the received signal frequency of the radar sensing signal, calculates a slope reflecting the difference and delay time, and then estimates the distance information based on the difference and slope. FMCW radar-based object detection system.
According to clause 9,
The processor obtains distance information between the operating frequency of the radar and a plurality of receiving antennas, calculates the frequency difference of the received radar signal as the radar signal is transmitted and received to the object, and determines the operating frequency of the radar and the radar. An FMCW radar-based object detection system that calculates the angle information based on the frequency difference between signals and distance information between receiving antennas.
According to clause 11,
The FMCW radar-based object detection system wherein the processor calculates the calculation based on distance change information between the radar antenna and the object and the operating frequency of the radar.
According to clause 8,
The processor extracts feature points from the stereo image information, generates 3D depth map information based on the feature points, and then estimates distance information and angle information to the object based on the 3D depth map. based object detection system.
According to clause 8,
The processor extracts a first object distortion change amount from the 3D feature map information, extracts a second object distortion change amount from the 3D depth map information, and extracts the 3D feature map information based on the first and second object distortion change amounts. An FMCW radar-based object detection system that combines the 3D depth map information.