KR102613536B1 - Proximity Object Distance/Pose Precision Calculation Apparatus and Method by Radar 3D map and Stereo Depth map - Google Patents

Abstract

The present invention relates to a device and method for precisely calculating the distance/posture of a close-up object using a radar 3D map and a stereo depth map. In particular, the present invention relates to a point A device and method for precise calculation of close object distance/posture using a radar 3D map and stereo depth map that allows calculating the precise distance/posture of an object through a cloud-based radar 3D map and a stereo depth 2.5D map of reference values will be.

Description

Proximity Object Distance/Pose Precision Calculation Apparatus and Method by Radar 3D map and Stereo Depth map}

The present invention relates to a device and method for precisely calculating the distance/posture of a close-up object using a radar 3D map and a stereo depth map. In particular, the present invention relates to a point A device and method for precise calculation of close object distance/posture using a radar 3D map and stereo depth map that allows calculating the precise distance/posture of an object through a cloud-based radar 3D map and a stereo depth 2.5D map of reference values will be.

Conventional technology uses point-type radar and 2D images from visible light sensors (CCD, CMOS) or infrared sensors (thermal imaging, thermal) to easily measure distance and attitude, but only point-type radar without a visible light sensor or infrared sensor is used. In that case, there is a way to roughly estimate the relative distance.

In this conventional technology, the method of calculating the distance and posture of an object using a point-type radar and 2D images of a visible light sensor or an infrared sensor has the disadvantage that the calculation of the relative distance is inaccurate because the results using the point are not precise. there is.

In addition, the amount of horizontal and vertical change of an object can be calculated from the 2D image of a visible light sensor or an infrared sensor, but if prior information about the object is not known, it is difficult to predict the degree of tilting of the object and the calculation may be inaccurate. There is an inevitable problem.

Therefore, the present invention is to solve the problems caused by the above-described prior art, and the purpose of the present invention is to propose a method of calculating the distance and posture by reducing the error of the target object using a point cloud-type radar 3D map. The aim is to provide a device and method for precisely calculating close object distance/position using a radar 3D map and stereo depth map that enable accurate distance and attitude measurement without separately using 2D images from visible light sensors and infrared sensors.

In other words, the present invention measures the precise distance/posture of an object through a point cloud-type radar 3D map and a stereo depth 2.5D map of reference values without having a device for measuring the precise distance/position of the object at a close distance of 0.2 m or more. The purpose is to provide a device and method for precise calculation of close object distance/posture using a radar 3D map and stereo depth map that can calculate .

However, the problem to be solved by the present invention is not limited to the problems described above, and other problems may exist.

In order to achieve the above object, a device for precisely calculating close object distance/posture using a radar 3D map and a stereo depth map according to an embodiment of the present invention uses a Point Cloud type radar 3D map and measurement to measure the distance and attitude in front. A data processing unit that processes the processed data and calculates precise distance and posture as a stereo depth 2.5D map; and an exhibition unit that displays the calculated distance and posture.

The data processing unit may include an object data acquisition unit that acquires object data for a detected object in front; a radar 2D map generator that generates a radar 2D map using the acquired object data; A radar 3D map generator that generates a radar 3D map by adding depth data to the generated radar 2D map; a comparison unit that estimates an object distance by comparing the generated radar 3D map with a previously stored stereo 2.5D map; a distance and angle detection unit that calculates the distance and angle of the object using the object distance information estimated by the comparison unit and a previously stored stereo depth 2.5D map; and a distortion change detection unit that detects the distortion change of the object using the calculated distance and angle data, and displays precise distance/posture information on the display unit based on the detected distortion change amount of the object.

The radar 2D map generator transmits the electromagnetic wave beam of the radar and records the values reflected from the object during a specific time from the upper left to the right of the object, and also records the data received from the left to the right of the object in the next line. It can be created by performing repetitive tasks so that the data received until the end of the characteristic time is recorded.

The depth data can extract information about the pixel difference between the two cameras.

The distance of the object can be extracted by calculating the number of pixels between the left and right cameras, which are stereo cameras.

The distance and angle detector may calculate the distance and angle values of the object by finding a stereo depth 2.5D map with a threshold of 95% or more similarity, including the distance of the detected object and object frame information, from the memory.

The distortion change detection unit may calculate the distortion of the object by calculating △d _x and △d _y for the depth information (d _x , d _y ) in the image coordinates (x, y) of the stereo depth map stored in the memory. .

Meanwhile, the method for precisely calculating close object distance/posture using a radar 3D map and a stereo depth map according to another embodiment of the present invention processes the point cloud-type radar 3D map and measured data to measure the distance and posture in front. A data processing step to calculate precise distance and pose with stereo depth 2.5D maps; And it may include a display step of displaying the calculated distance and posture on a display device.

The data processing step includes obtaining object data for a detected object in front; Generating a radar 2D map using the acquired object data; Generating a radar 3D map by adding depth data to the generated radar 2D map; estimating an object distance by comparing the generated radar 3D map with a previously stored stereo 2.5D map; calculating the distance and angle of the object using the estimated object distance information and a previously stored stereo depth 2.5D map; And it may include detecting a change in the distortion of the object using the calculated distance and angle data, and displaying precise distance/posture information based on the amount of change in the detected distortion of the object on the display device.

In the step of generating the radar 2D map, the electromagnetic wave beam of the radar is transmitted and the received value reflected from the object during a specific time is recorded from the top left of the object to the right, and in the next line from the left to the right of the object. Received data is recorded and can be created by performing a repetitive task so that the received data is recorded until the end of the characteristic time.

The depth data can extract information about the pixel difference between the two cameras.

The distance of the object can be extracted by calculating the number of pixels between the left and right cameras, which are stereo cameras.

In the step of calculating the distance and angle, the distance and angle values of the object can be calculated by finding a stereo depth 2.5D map with a threshold of 95% or more similarity including the distance of the detected object and object frame information from the memory. there is.

To detect a change in the object's distortion, the distortion of the object is calculated by calculating _{△d x and} △ _{d y} for the depth information (d You can.

A computer program according to another aspect of the present invention for solving the above-described problem is combined with a computer as hardware to execute a PRF selection method in the aircraft radar system, and is stored in a computer-readable recording medium.

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, the precise distance/posture of an object can be measured through a point cloud-type radar 3D map and a stereo depth 2.5D map of reference values without having a device for measuring the precise distance/position of the object at a close distance of 0.2 m or more. By presenting a method to calculate , it is possible to measure accurate distance and posture without separately using 2D images from visible light sensors and infrared sensors.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

The drawings attached below are intended to aid understanding of the present embodiment and provide examples along with a detailed description. However, the technical features of this embodiment are not limited to specific drawings, and the features disclosed in each drawing may be combined to form a new embodiment.
Figure 1 is a diagram showing the block configuration of a close object distance/position precision calculation device using a radar 3D map and a stereo depth map according to the present invention.
FIG. 2 is a diagram illustrating the operation of generating a point cloud-type radar 3D map using object data obtained in the present invention shown in FIG. 1.
Figure 3 is a diagram showing an example of a point cloud type radar 2D map generated using object data obtained in the present invention shown in Figure 1.
Figure 4 is a screen example of a stereo depth 2.5D map preset in the storage unit shown in Figure 1.
Figure 5 is a diagram for explaining the operation of estimating an object distance in a stereo camera in the present invention.
Figure 6 is a diagram for explaining the operation of estimating the angle between an object frame and a stereo depth 2.5D mapper in the present invention.
Figure 7 is an example screen diagram for explaining the operation of extracting the amount of tilting change using a stereo depth 2.5D map.
Figure 8 is a diagram illustrating an operation flowchart for a method for calculating close object distance/pose precisely using a radar 3D map and a stereo depth map according to the present invention.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, a method for precisely calculating a close object distance/posture using a radar 3D map and a stereo depth map according to a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a diagram showing the block configuration of a device for precision calculating close object distance/posture using a radar 3D map and a stereo depth map according to the present invention, and Figure 2 is a block diagram using object data obtained from the present invention shown in Figure 1. A diagram for explaining the operation of generating a point cloud-type radar 3D map. FIG. 3 is a diagram showing an example of a point cloud-type radar 2D map generated using object data obtained in the present invention shown in FIG. 1. Figure 4 is a screen example of a stereo depth 2.5D map preset in the storage unit shown in Figure 1, Figure 5 is a diagram for explaining the operation of estimating an object distance in a stereo camera in the present invention, and Figure 6 is a diagram in the present invention. , a diagram to explain the operation of estimating the angle between the object frame and the stereo depth 2.5D map, and FIG. 7 is an example screen diagram to explain the operation of extracting the amount of tilting change using the stereo depth 2.5D map.

This invention processes the point cloud type radar 3D map that measures the distance and posture in front and the measured data to calculate precise distance and posture as a stereo depth 2.5D map and displays the calculated distance and posture data.

Referring to FIG. 1, the device for precisely calculating a close object distance/position using a radar 3D map and a stereo depth map according to the present invention includes an object detection unit 100, an object data acquisition unit 110, and a radar 2D map generation unit 120. ), radar 3D map generator 130, comparison unit 140, stereo depth 2.5D map storage unit 150, distance angle estimation unit 160, distortion change detection unit 170, and precise distance/position display unit 180 ) may include.

First, the object detection unit 100 detects an object in front using radar and provides the detected information to the object data acquisition unit 110.

The object data acquisition unit 110 acquires object data from the object detected by the object detection unit 100. Here, the acquired object data is data that can estimate the distance to a specific location (point), and the data that can estimate the posture is in a certain area in terms of space and time within the field of view. The corresponding radar 2D map (see Figure 2) must be created.

That is, the radar 2D map generator 120 generates a radar 2D map as shown in FIG. 2 using the acquired object data (distance and posture data), and then generates the generated radar 2D map into the radar 3D map generator 130. Provided as.

The radar 2D map generation in the radar 2D map generator 120, for example, if the size of the object is 5 m, transmits the electromagnetic wave beam of the radar and reflects and receives the value during a specific time from the upper left to the right of the object. It records the received data from left to right of the object in the next line, and repeats the operation so that the received data is recorded until the end of the characteristic time. The data generated in this way is a radar 2D map.

The radar 3D map generator 130 adds depth data to the radar 2D map provided from the radar 2D map generator 120 to generate a radar 3D map as shown in FIG. 1, and then creates the radar 3D map. is provided to the comparison unit 140.

The comparison unit 140 compares the radar 3D map generated by the radar 3D map generator 130 with the stereo depth map stored in the stereo depth 2.5D map storage unit 150 and stores the comparison result in the distance and angle estimation unit ( 160).

The distance and angle estimation unit 160 estimates the distance and angle using the comparison result between the radar 3D map and the stereo depth 2.5D map provided by the comparison unit 140, and then distorts the estimated distance and angle values. It is provided to the change detection unit 170.

The distortion change detection unit 170 detects changes in the distortion of the detected object using the distance and angle estimate values provided by the distance and angle estimation unit 160 and then displays the distortion in the precision distance/posture display unit 180.

We will now explain the specific operation of the device for precision calculating close object distance/position using a radar 3D map and a stereo depth map according to the present invention, which has the above configuration.

First, object data is acquired through the object data acquisition unit 110 shown in FIG. 1. The data obtained here is data that can estimate the distance to a specific location (point), and the data that can estimate posture corresponds to a certain area in terms of space and time within the field of view. A radar 2D map (see FIG. 3) is generated through the radar 2D map generator 120.

Here, the creation of a radar 2D map involves, for example, transmitting a radar electromagnetic wave beam and recording the reflected and received values during a certain time from the top left of the object to the right, and then from the left to the right of the object in the next line. Received data is recorded, and repetitive tasks are performed so that the received data is recorded until the end of the characteristic time.

The radar 3D map generator 130 adds depth data to the radar 2D map generated by the radar 2D map generator 120 as described above to generate a radar 3D map as shown in FIG. 2 and provides it to the comparison unit 140. .

Since the accurate depth, or distance, cannot be estimated only with the data of the radar 3D map generated in this way, the comparison unit 140 uses a stereo depth 2.5D map ( By comparing the generated radar 3D map (see FIG. 4), the depth and distance are extracted and provided to the distance and angle estimation unit 160.

Here, the comparison unit 140 extracts depth information based on the pixel difference between the two cameras, and as shown in FIG. 5, the distance to the object can be extracted by calculating the number of pixels between the left and right cameras, which are stereo cameras.

The distortion change detection unit 170 compares the extracted depth/length information with the object frame information as shown in FIG. 6, finds a stereo depth 2.5D map with a threshold of 95% or more similarity, and determines the distance and angle values of the object. can be calculated. The value calculated here can use the value of a stereo depth map that has been precisely measured in advance and stored in the stereo depth 2.5D map storage unit 150.

That is, as shown in FIG. 7, the tilt change detection unit 170 uses the image coordinates (x, y) of the stereo depth map stored in the stereo depth 2.5D map storage unit 150 to calculate the amount of change in object tilting. ), the distortion of the object can be calculated by calculating △d _x , △d _y for the depth information (d _x , d _y ).

In this way, the exact distance/posture information of the object is displayed on the precise distance/posture display unit 180 according to the amount of change in the object's distortion detected by the distortion change detection unit 170.

Let us explain step by step the method of calculating close object distance/position accurately using the radar 3D map and stereo depth map according to the present invention with reference to FIG. 8.

Figure 8 is a diagram illustrating an operation flowchart for a method for calculating close object distance/pose precisely using a radar 3D map and a stereo depth map according to the present invention.

Referring to FIG. 8, first, when an object is detected through radar (S801), data on the detected object is acquired (S802). The data obtained here is data that can estimate the distance to a specific location (point), and the data that can estimate posture corresponds to a certain area in terms of space and time within the field of view. A radar 2D map (see FIG. 3) is created (S803).

Here, the creation of a radar 2D map is, for example, if the object size is 5m, the electromagnetic wave beam of the radar is transmitted and the reflected and received values during a specific time are recorded from the upper left of the object to the right, and then the object is recorded in the next line. The received data is recorded from left to right, and a repetitive operation is performed so that the received data is recorded until the end of the characteristic time.

Next, depth data is added to the radar 2D map generated through step S803 to generate a radar 3D map as shown in FIG. 2 (S804).

Since the accurate depth, or distance, cannot be estimated only with the data of the radar 3D map generated in this way, the depth is estimated by comparing the generated radar 3D map with the stereo depth 2.5D map (see FIG. 4) worked in advance and stored in memory. and extract the distance (S805).

Here, in step S805, depth information is extracted from the pixel difference between the two cameras, and the distance of the object is extracted by calculating the number of pixels between the left and right cameras, which are stereo cameras, as shown in FIG. 5.

Next, by comparing the depth/length information extracted through step S805 with the object frame information as shown in FIG. 6, the distance and angle values of the object having a threshold similarity of 95% or more can be calculated (S806). . The value calculated here can use the value of the stereo depth map that has been precisely measured in advance and stored in memory.

Then, as shown in FIG. 7, △d x and △d _y are calculated for the depth information (d _x , d _y ) in the image coordinates ( _x , y) of the stereo depth map stored in the memory, thereby The distortion can be calculated (S807).

In this way, the exact distance/posture information of the object is displayed on the display device according to the amount of change in distortion of the detected object (S808).

In the end, the present invention measures the precise distance/posture of an object through a point cloud-type radar 3D map and a stereo depth 2.5D map of reference values without possessing a device for measuring the precise distance/posture of the object at a close distance of 0.2 m or more. By presenting a method to calculate , it is possible to measure accurate distance and posture without separately using 2D images from visible light sensors and infrared sensors.

The method for calculating close object distance/position precisely using a radar 3D map and a stereo depth map according to an embodiment of the present invention described above is implemented as a program (or application) to be executed in combination with a computer, which is hardware, and is transmitted to the media. It can be saved.

The above-mentioned program is C, C++, JAVA, Ruby, and It may include code encoded in a computer language such as machine language. These codes may include functional codes related to functions that define the necessary functions for executing the methods, and include control codes related to execution procedures necessary for the computer's processor to execute the functions according to predetermined procedures. can do. In addition, these codes may further include memory reference-related codes that indicate at which location (address address) in the computer's internal or external memory additional information or media required for the computer's processor to execute the above functions should be referenced. there is. In addition, if the computer's processor needs to communicate with any other remote computer or server in order to execute the above functions, the code uses the computer's communication module to determine how to communicate with any other remote computer or server. It may further include communication-related codes regarding whether communication should be performed and what information or media should be transmitted and received during communication.

The storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., but are not limited thereto. That is, the program may be stored in various recording media on various servers that the computer can access or on various recording media on the user's computer. Additionally, the medium may be distributed to computer systems connected to a network, and computer-readable code may be stored in a distributed manner.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as unitary may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

100: object detection unit
110: object data acquisition unit
120: Radar 2D map generation unit
130: Radar 3D map generation unit
140: comparison unit
150: Stereo depth 2.5D map storage unit
160: Distance and angle estimation unit
170: Twisting change detection unit
180: Precision distance/posture display section

Claims (14)

In a device for precise calculation of close object distance/posture using radar 3D map and stereo depth map,
A data processing unit that processes the point cloud-type radar 3D map that measures the distance and posture in front and the measured data to calculate precise distance and posture using a stereo depth 2.5D map; and
Includes an exhibition unit that displays the calculated distance and posture,
The data processing unit,
an object data acquisition unit that acquires object data for the detected object in front;
a radar 2D map generator that generates a radar 2D map using the acquired object data;
A radar 3D map generator that generates a radar 3D map by adding depth data to the generated radar 2D map;
a comparison unit that estimates an object distance by comparing the generated radar 3D map with a previously stored stereo 2.5D map;
a distance and angle detection unit that calculates the distance and angle of the object using the object distance information estimated by the comparison unit and a previously stored stereo depth 2.5D map; and
A distortion change detection unit that detects a distortion change of an object using the calculated distance and angle data and displays precise distance/posture information on the display unit based on the detected distortion change amount of the object,
The radar 2D map generator,
By transmitting the radar's electromagnetic wave beam, the values reflected from the object and received during a specific time are recorded from the top left of the object to the right. In addition, the data received from the left to the right of the object are recorded in the next line, and the characteristic time It is created by performing repetitive tasks so that the data received until the end is recorded,
The depth data extracts information about the pixel difference between the two cameras,
The distance of the object is extracted by calculating the number of pixels between the left and right cameras, which are stereo cameras,
The distance and angle detector,
Calculate the distance and angle values of the object by finding a stereo depth 2.5D map with a threshold of 95% or more similarity, including the distance and object frame information of the detected object, from memory,
The distortion change detection unit,
Radar 3D map and stereo, which calculates the distortion of an object by calculating △d _x , △d _y for the depth information (d _x , d _y ) in the image coordinates (x, y) of the stereo depth map stored in memory Close object distance/pose precision calculation device using depth map.
delete delete delete delete delete delete In the method of calculating close object distance/pose precisely using radar 3D map and stereo depth map,
A data processing step that processes the point cloud-type radar 3D map that measures the distance and posture in front and the measured data to calculate precise distance and posture using a stereo depth 2.5D map; and
Comprising the step of displaying the calculated distance and posture on a display device,
The data processing step is,
Obtaining object data for the detected object in front;
Generating a radar 2D map using the acquired object data;
Generating a radar 3D map by adding depth data to the generated radar 2D map;
estimating an object distance by comparing the generated radar 3D map with a previously stored stereo 2.5D map;
calculating the distance and angle of the object using the estimated object distance information and a previously stored stereo depth 2.5D map; and
Detecting a change in distortion of an object using the calculated distance and angle data, and displaying precise distance/posture information based on the amount of change in distortion of the detected object on the display device,
The step of generating the radar 2D map is,
By transmitting the radar's electromagnetic wave beam, the values reflected from the object and received during a specific time are recorded from the top left of the object to the right. In addition, the data received from the left to the right of the object are recorded in the next line, and the characteristic time It is created by performing repetitive tasks so that the data received until the end is recorded,
The depth data extracts information about the pixel difference between the two cameras,
The distance of the object is extracted by calculating the number of pixels between the left and right cameras, which are stereo cameras,
The step of calculating the distance and angle is,
Calculate the distance and angle values of the object by finding a stereo depth 2.5D map with a threshold of 95% or more similarity, including the distance and object frame information of the detected object, from memory,
Detection of changes in distortion of the object,
Radar 3D map and stereo, which calculates the distortion of an object by calculating △d _x , △d _y for the depth information (d _x , d _y ) in the image coordinates (x, y) of the stereo depth map stored in memory Precise calculation method of close object distance/pose using depth map.
delete delete delete delete delete delete

Images