KR20200099887A - Apparatus and method for visualizing sensor data - Google Patents

Abstract

Provided are a sensor data visualization device to integrally display and visualize recognition information of a radar sensor and a Lidar sensor in the same coordinate system, and a method thereof. The presented device includes: a first coordinate calculation unit which converts collected radar information into a value of a distance coordinate system; a correction unit which corrects data from the first coordinate calculation unit; a first coordinate conversion unit which converts data from the correction unit pixel-unit coordinate values; a first visualization unit which outputs visualization information corresponding to information from the first coordinate conversion unit; a second coordinate calculation unit which changes the collected Lidar information into a value of the distance coordinate system; a second coordinate conversion unit which converts data from the second coordinate calculation unit into the pixel-unit coordinate value; a second visualization unit which outputs visualization information corresponding to information from the second coordinate conversion unit; and a combining unit which synthesizes and displays the visualization information of the first visualization unit and the second visualization unit.

Description

Apparatus and method for visualizing sensor data}

The present invention relates to an apparatus and method for visualizing sensor data, and more particularly, to an apparatus and method for visualizing recognition information of a radar sensor and a lidar sensor.

A radar sensor is a device that measures the distance, direction, and speed of the subject by measuring the reflected wave returned by the radio wave transmitted from the sensor hitting the subject. Typically, commercial radar sensors include information on a subject (whether it is a newly recognized subject or an already recognized subject), the distance between the sensor and the subject, the angle between the sensor and the subject, and whether the subject is moving (still or moving Whether or not it is a target), and information such as a moving speed of a subject may be provided.

The lidar sensor is a device that measures the distance to the subject by measuring the time until the laser beam emitted from the sensor is reflected on the subject and returned to the sensor. Typically, commercial LiDAR sensors include information about the LiDAR channel (in the case of a sensor having multiple channels, information about the vertical angle can be obtained using the channel information), the horizontal angle, and the distance to the subject. Can provide.

However, the commercial radar sensor and the commercial radar sensor described above provide useful information about the surrounding situation in the vehicle environment, but the visualization is not specifically considered from the user's point of view.

Prior Art 1: Republic of Korea Patent Registration No. 10-1427364 (Scanning system for generating 3D indoor maps using a lidar device) Prior Art 2: Korean Patent Laid-Open Patent No. 10-2018-0068600 (motion sensor-based moving object detection and speed measurement system) Prior Art 3: Republic of Korea Patent Registration No. 10-1852945 (Target pointing and tracking method using a lidar system and lidar system having a target pointing and tracking function)

The present invention has been proposed to solve the above-described conventional problem, and an object of the present invention is to provide a sensor data visualization apparatus and method for integrally displaying and visualizing recognition information of a radar sensor and a lidar sensor in the same coordinate system.

In order to achieve the above object, an apparatus for visualizing sensor data according to a preferred embodiment of the present invention includes: a first coordinate calculator for changing collected radar information to a value of a distance coordinate system; A correction unit correcting data from the first coordinate calculation unit; A first coordinate conversion unit that converts the data from the correction unit into pixel-unit coordinate values; A first visualization unit for outputting visualization information corresponding to information from the first coordinate conversion unit; A second coordinate calculation unit that changes the collected LiDAR information into a value of the distance coordinate system; A second coordinate conversion unit that converts data from the second coordinate calculation unit into a pixel-based coordinate value; A second visualization unit for outputting visualization information corresponding to information from the second coordinate conversion unit; And a synthesis unit that synthesizes and displays the visualization information of the first visualization unit and the second visualization unit.

As the first coordinate calculation unit receives information about the distance and direction of the detected subject, the following equation,

Y _radar = γ×sinθ, X _radar = √(γ ² -Y _radar ² )

(Where, X _radar is the X-axis distance between the radar sensor and the subject, Y _radar is the Y-axis distance between the radar sensor and the subject, γ is the distance between the radar sensor and the subject, and θ is the distance between the front of the radar sensor and the subject. Angle)

Using, the X-axis distance between the radar sensor and the subject and the Y-axis distance between the radar sensor and the subject can be obtained.

The correction unit, the following equation,

_X'radar = X _radar + ΔX, _Y'radar = Y _radar + ΔY

(Wherein, X _'radar is the x-axis calibration position of the radar sensor, Y' _radar is the y-axis calibration position of the radar sensor, ΔX is the x-axis distance between the centers of the central and La of the radar sensors, ΔY is the radar The y-axis distance between the center of the sensor and the center of the lidar sensor)

By using, the x-axis correction position of the radar sensor and the y-axis correction position of the radar sensor can be obtained.

As the first coordinate conversion unit receives the x-axis correction position of the radar sensor and the y-axis correction position of the radar sensor, the following equation,

X'= h + _X'radar × (h/distance _max ), Y'= h- _Y'radar × (h/distance _max )

(Where X'is the number of pixels on the x-axis between the radar sensor and the subject, h is the number of vertical pixels in the screen display area, distance _max is the maximum visible distance of the radar sensor, and Y'is the y-axis screen pixels)

Through this, the number of pixels on the x-axis screen between the radar sensor and the subject and the number of pixels on the y-axis screen between the radar sensor and the subject can be calculated.

The first visualization unit may differentiate and visualize a new subject and an already recognized subject, and the new subject may be displayed as a box of a first color, and the previously recognized subject may be displayed as a box of a second color.

The first visualization unit divides and visualizes the presence or absence of speed information of the subject, but when information on the moving speed of the subject is provided, the corresponding speed may be displayed as a number in a box.

The first visualization unit may separate and visualize a still subject and a moving subject, and may display the moving subject using an arrow or a straight line to distinguish it from the still subject.

The first visualization unit separates and visualizes the presence or absence of information on the movement direction of the subject, but when there is information on the movement direction of the subject, the movement direction is displayed using an arrow, and when there is no information on the movement direction The moving direction from the immediate previous position of the subject to the current position can be calculated and displayed in a straight line.

The first visualization unit may visualize the moving direction of the subject, and may schematically display information on the location of the subject in chronological order.

When there is information on the moving direction of the subject, the first visualization unit may display the moving direction using an arrow, and when there is no information on the moving direction of the subject, the first visualization unit may display the moving direction using a straight line.

The collected lidar information is a vertical angle from the xy plane of the lidar channel recognizing the subject, the horizontal angle from the x-axis of the lidar channel recognizing the subject, and the distance between the lidar sensor and the subject. 2 The coordinate calculation unit is the following equation,

Z _lidar = γ × sinπ

X _lidar = sinθ × √(γ ² -Z _lidar ² )

Y _lidar = √(γ ² -X _lidar ² -Z _lidar ² )

(Where, Z _lidar is the z-axis distance between the _lidar sensor and the subject, γ is the distance between the _lidar sensor and the subject, π is the vertical angle with the xy plane of the _lidar channel that recognizes the subject, and X _lidar is The x-axis distance between the _lidar sensor and the subject, θ is the horizontal angle with the x-axis of the _lidar channel that recognizes the subject, and Y _lidar is the y-axis distance between the _lidar sensor and the subject)

Using, the z-axis distance between the lidar sensor and the subject, the x-axis distance between the lidar sensor and the subject, and the y-axis distance between the lidar sensor and the subject can be obtained.

The second coordinate conversion unit calculates the x-axis distance between the lidar sensor and the subject and the y-axis distance between the lidar sensor and the subject by the following equations,

X'= h + X _lidar × (h/distance _max ), Y'= h-Y _lidar × (h/distance _max )

(Where, X'is the number of pixels on the x-axis between the lidar sensor and the subject, h is the number of vertical pixels in the screen display area, distance _max is the maximum visible distance of the radar sensor, and Y'is the _lidar sensor and the subject. Is the number of pixels on the y-axis of the screen)

By substituting in, the number of pixels on the x-axis screen between the lidar sensor and the subject and the number of pixels on the y-axis screen between the lidar sensor and the subject can be calculated.

The second visualization unit may visualize the information from the second coordinate conversion unit on a two-dimensional distance coordinate system, and may differentiate color values according to the number of overlapping data.

Meanwhile, in the sensor data visualization method according to a preferred embodiment of the present invention, the radar information collected by the first coordinate calculator is changed to the value of the distance coordinate system, and the radar information collected by the second coordinate calculator is converted to the value of the distance coordinate system. Changing; A step of correcting, by a correction unit, data from the first coordinate calculation unit; Converting, by a first coordinate conversion unit, data from the correction unit to a value of coordinates in pixel units, and by a second coordinate conversion unit, converting data from the second coordinate calculation unit to a value of coordinates in pixels; Outputting visualization information corresponding to the information from the first coordinate conversion unit, and outputting visualization information corresponding to the information from the second coordinate conversion unit by a first visualization unit; And a synthesizing unit, synthesizing and displaying the visualization information of the first visualization unit and the second visualization unit.

According to the present invention having such a configuration, recognition information of the radar sensor and the lidar sensor can be integratedly displayed and visualized in the same coordinate system to facilitate identification of the recognition information.

1 is a block diagram of a sensor data visualization apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram used for explaining the operation of the first coordinate calculation unit shown in FIG. 1.
3 is a diagram used to explain the operation of the correction unit illustrated in FIG. 1.
4 and 5 are diagrams used to describe the operation of the first coordinate conversion unit shown in FIG. 1.
6 to 12 are diagrams used to describe the operation of the first visualization unit shown in FIG. 1.
13 and 14 are diagrams used to explain the operation of the second coordinate calculator shown in FIG. 1.
15 is a diagram used to explain the operation of the second coordinate conversion unit shown in FIG. 1.
16 and 17 are diagrams used to explain the operation of the second visualization unit shown in FIG. 1.
18 is a diagram used to explain the operation of the synthesis unit shown in FIG. 1.
19 is a flowchart illustrating a sensor data visualization method according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In describing the present invention, in order to facilitate an overall understanding, the same reference numerals are used for the same elements in the drawings, and duplicate descriptions for the same elements are omitted.

1 is a block diagram of a sensor data visualization apparatus according to an embodiment of the present invention, FIG. 2 is a diagram employed to explain the operation of a first coordinate calculation unit shown in FIG. 1, and FIG. Figures used to explain the operation of the government, Figures 4 and 5 are diagrams used to explain the operation of the first coordinate conversion unit shown in Figure 1, Figures 6 to 12 are the operation of the first visualization unit shown in Figure 1 13 and 14 are diagrams used to describe the operation of the second coordinate calculation unit shown in Fig. 1, and Fig. 15 is a view used to explain the operation of the second coordinate conversion unit shown in Fig. 1 16 and 17 are diagrams used to describe the operation of the second visualization unit shown in FIG. 1, and FIG. 18 is a diagram used to explain the operation of the synthesis unit shown in FIG. 1.

The sensor data visualization apparatus according to an embodiment of the present invention includes a radar sensor information collection unit 10, a first coordinate calculation unit 12, a correction unit 14, a first coordinate conversion unit 16, and a first visualization unit. (18), a lidar sensor information collection unit 20, a second coordinate calculation unit 22, a second coordinate conversion unit 24, a second visualization unit 26, and a synthesis unit 30.

The radar sensor may provide information on a distance and a direction to a detected subject.

The radar sensor information collection unit 10 includes information measured by a radar sensor (not shown) (e.g., information about a subject, distance between the sensor and the subject, angle between the sensor and the subject, whether the subject is moving, the moving speed of the subject, etc.) ) To collect.

The first coordinate calculation unit 12 changes the radar information collected by the radar sensor information collection unit 10 into a value of the distance coordinate system through a predetermined conversion equation.

For example, the first coordinate calculation unit 12 is the distance to the subject detected from the radar sensor information collecting unit 10 (ie, the distance between the radar sensor and the subject (γ)) and direction (that is, the radar sensor When information about the angle (θ) between the front and the subject is received, the X-axis distance between the radar sensor and the subject (X _radar ), and the Y-axis distance between the radar sensor and the subject ( Y _radar ).

(Equation 1)

Y _radar = γ×sinθ, X _radar = √(γ ² -Y _radar ² )

In other words, as shown in FIG. 2, when there is a distance γ between the radar sensor and the subject P, and an angle θ between the front of the radar sensor and the subject P, the first coordinate calculation unit 12 Information of the distance coordinate system (ie, X _radar and Y _radar ) can be obtained using the transformation equation of Equation 1.

In FIG. 1, the correction unit 14 receives and corrects data output from the first coordinate calculation unit 12. That is, when the data of the radar sensor and the data of the radar sensor are to be visualized and displayed in the same coordinate system, correction for the position where each sensor is mounted is required. Accordingly, in an embodiment of the present invention, the correction unit 14 corrects the position of the radar sensor for data integration visualization. In this case, the correction unit 14 corrects using the correction equation of Equation 2 below. .

(Equation 2)

_X'radar = X _radar + ΔX, _Y'radar = Y _radar + ΔY

Here, X _radar is the X-axis position of the radar sensor (ie, it corresponds to the X-axis distance between the radar sensor and the subject), and Y _radar is the Y-axis position of the radar sensor (ie, it corresponds to the Y-axis distance between the radar sensor and the subject). to be.

For example, if it is assumed that the lidar sensor 32 and the radar sensor 34 are separated from each other as in the figure above of FIG. 3, the center of the radar sensor 34 and the The x-axis distance between the centers of the lidar sensor 32 may be referred to as ΔX, and the y-axis distance between the center of the radar sensor 34 and the center of the lidar sensor 32 may be referred to as ΔY.

Thus, the correction section 14 can obtain the 'y-axis calibration position (Y in _(radar, radar sensor _radar) x-axis calibration position X)' of the radar sensor with the correction formula of the above-described equation (2).

In FIG. 1, the first coordinate conversion unit 16 converts data (ie, distance unit coordinates) from the correction unit 14 into pixel unit coordinate values. That is, since the screen coordinate system for visualization and the distance coordinate system of the sensor information are different, the coordinate system needs to be corrected, and the distance unit coordinates need to be converted into pixel unit coordinates.

To this end, the length of the screen display area (refer to FIG. 4) is designated as twice the height of the screen display area (refer to FIG. 4) so that distortion according to the ratio does not occur because the front view and the side distance of the sensor to be expressed are the same. That is, w = 2h. Here, w is the number of horizontal pixels in the screen display area, and h is the number of vertical pixels in the screen display area.

In addition, the first coordinate conversion unit 16 obtains a distance ratio according to a location for screen display and a screen size using the conversion equation of Equation 3 below.

(Equation 3)

X'= h + _X'radar × (h/distance _max ), Y'= h- _Y'radar × (h/distance _max )

Here, X'is the number of pixels on the x-axis screen between the radar sensor and the subject, h is the number of vertical pixels in the screen display area, _X'radar is the x-axis correction position of the radar sensor, and distance _max is the maximum visibility of the radar sensor. It is the distance, _{Y'is the} number of pixels on the y-axis screen between the radar sensor and the subject, and _Y'radar is the y-axis correction position of the radar sensor.

In other words, when the first coordinate conversion unit 16 receives the x-axis correction position (X' _radar ) of the radar sensor and the y-axis correction position (Y' _radar ) of the radar sensor from the correction unit 14, the above As shown in FIG. 5, through Equation 3, the number of pixels on the x-axis screen between the radar sensor and the subject (X') and the number of pixels on the y-axis screen between the radar sensor and the subject (P) (Y') are calculated. In FIG. 5, distance _display is the maximum display distance for visualization, and distance _display ≤ distance _max .

In FIG. 1, the first visualization unit 18 outputs visualization information corresponding to information from the first coordinate conversion unit 16.

As an example, the first visualization unit 18 may distinguish and visualize a new subject and an already recognized subject. For example, as shown in FIG. 6, the new subject new is displayed as a blue box, and has already been recognized. Existing subjects (updated) may be marked with a black box so that they can be distinguished by the color of the box. In FIG. 6, the box of the new wave body (new) will be shown as displayed in black, but it can be understood that it is actually displayed in blue.

As another example, the first visualization unit 18 may distinguish and visualize the presence or absence of speed information of the subject. When information on the moving speed of the subject is provided as shown in FIG. 7, the corresponding speed is displayed as a number in a box. . For example, if the moving speed of the subject is "0 (zero)" (speed none), no speed is displayed, and if the moving speed of the subject is "34" (speed), 34 is placed in the box of the subject. Can be displayed.

As another example, the first visualization unit 18 may separate and visualize a still subject and a moving subject. For example, as shown in FIG. 8, a moving subject is displayed using an arrow or a straight line to distinguish it from a still subject. Make this possible.

As another example, the first visualization unit 18 may distinguish and visualize the presence or absence of information on the movement direction of the subject. For example, as shown in FIG. 9, information on the movement direction of the subject is provided from the radar sensor. In this case, the direction of movement is indicated by using an arrow, and when information on the direction of movement is not separately provided, the movement direction from the immediately preceding position of the subject to the current position may be calculated and displayed in a straight line.

As another example, the first visualization unit 18 may visualize the moving direction of the subject, and information on the position of the subject provided from the radar sensor is obtained in the order of time _t-2 , time _t-1 , and time _t . In the case of a schematic diagram, it can be expressed as in FIG. 10. More specifically, when information on the moving direction of the subject is provided from the radar sensor, such as information on the location of the subject, the moving direction may be displayed using an arrow as shown in FIG. 11. Conversely, when information on the moving direction of the subject is not separately provided from the radar sensor, the direction information from the subject position at time _{t-1 to} the subject position at time _{t is} displayed using a straight line as shown in FIG. I can.

In FIG. 1, the lidar sensor information collection unit 20 collects information (eg, information about a lidar channel, information about a horizontal angle, distance information from a subject, etc.) measured by a lidar sensor (not shown). do.

In FIG. 1, the second coordinate calculation unit 22 changes the LiDAR information collected by the LiDAR sensor information collection unit 20 into a value of the distance coordinate system through a predetermined transformation equation.

For example, if the data provided from the lidar sensor (not shown) is schematically illustrated, it can be illustrated as in FIG. 13. In FIG. 13, π is a vertical angle with the xy plane of the lidar channel recognizing the subject, θ is the horizontal angle with the x-axis of the lidar channel recognizing the subject, and γ is the distance between the lidar sensor and the subject. to be.

Accordingly, the second coordinate calculation unit 22 calculates the data (π, θ, γ) provided as described above using Equation 4 below, and the z-axis distance between the _lidar sensor and the subject (Z _lidar ), the _lidar sensor The x-axis distance between the and the subject (X _lidar ), and the y-axis distance between the _lidar sensor and the subject (Y _lidar ) are obtained.

(Equation 4)

Z _lidar = γ × sinπ

X _lidar = sinθ × √(γ ² -Z _lidar ² )

Y _lidar = √(γ ² -X _lidar ² -Z _lidar ² )

In FIG. 1, the second coordinate conversion unit 24 converts data from the second coordinate calculation unit 22 (that is, coordinates in units of distance) into values of coordinates in units of pixels.

That is, since the screen coordinate system for visualization and the distance coordinate system of the sensor information are different, the coordinate system needs to be corrected, and the distance unit coordinates need to be converted into pixel unit coordinates.

To this end, the length of the screen display area (refer to FIG. 4) is designated as twice the height of the screen display area (refer to FIG. 4) so that distortion according to the ratio does not occur because the front view and the side distance of the sensor to be expressed are the same. That is, w = 2h. Here, w is the number of horizontal pixels in the screen display area, and h is the number of vertical pixels in the screen display area.

In addition, the second coordinate conversion unit 24 obtains a position for screen display and a distance ratio according to the screen size using the conversion equation of Equation 5 below.

(Equation 5)

X'= h + X _lidar × (h/distance _max ), Y'= h-Y _lidar × (h/distance _max )

Here, X'is the number of pixels on the x-axis between the _lidar sensor and the subject, h is the number of vertical pixels in the screen display area, X _lidar is the x-axis distance between the _lidar sensor and the subject, and distance _max is the radar sensor. Is the maximum visible distance of, _{Y'is the} number of pixels on the y-axis screen between the _lidar sensor and the subject, and Y _lidar is the y-axis distance between the _lidar sensor and the subject.

In other words, the second coordinate conversion unit 24 receives the x-axis distance between the _lidar sensor and the subject (X _lidar ) and the y-axis distance between the _lidar sensor and the subject (Y _lidar ) from the second coordinate calculation unit 22 Then, as shown in FIG. 15, the number of pixels on the x-axis screen between the lidar sensor and the subject (X') and the number of pixels on the y-axis between the lidar sensor and the subject (Y') are calculated through Equation 5 above. Serve. In FIG. 15, distance _display is the maximum display distance for visualization, and distance _display ≤ distance _max .

In FIG. 1, the second visualization unit 26 outputs visualization information corresponding to information from the second coordinate conversion unit 24. When the data provided from the lidar sensor is visualized, it can be projected onto a 3D distance coordinate system, but in an embodiment of the present invention, for visibility, a top view is visualized using a 2D distance coordinate system as shown in FIG. 16. In FIG. 16, the figure on the left is a 3D distance coordinate system, and the figure on the right is a 2D distance coordinate system. Also, the size of a pixel in which data is displayed is approximately 3×3 pixels.

When the lidar sensor information is visualized in a two-dimensional distance coordinate system, data corresponding to the number of channels of the sensor can be displayed in the same coordinate, and in this case, it is difficult to distinguish visually. Accordingly, as shown in FIG. 16, color values are differentiated according to the number of overlapping data to enable visual identification. Here, the color classification is calculated by using R, G, and B color values and giving a weight equal to the number of data overlapping the G color value. For example, R = 255, G = 255-int(255/(LIDAR total number of channels -1)) × (weight -1), B = 0.

The weight is the following calculation method (calculation algorithm), for example,

channel ₀ = center channel of the lidar sensor,

channel _i = channel _{i of the} lidar sensor,

Xdisp[channeli] = position of pixels on the x-axis screen of the object of channel i of the lidar sensor,

Ydisp[channeli] = y-axis screen pixel position of the object of channel i of the lidar sensor,

weight = 0,

i=1 .. (Number of channels in the lidar sensor -1)

if

|X _disp [channel ₀ ]-X _disp [channel _i ]| <3

&&

|Y _disp [channel ₀ ]-Y _disp [channel _i ]| <3

then

weight+ = 1

If the display position of the data is within 3 pixels horizontally and vertically through the same method (algorithm), it is divided into overlapping data and calculated. In the above calculation method, X _disp [channel ₀ ] is the position of the x-axis screen pixel of the subject of the central channel of the lidar sensor, and Y _disp [channel ₀ ] is the position of the y-axis screen of the subject of the central channel of the lidar sensor.

In addition, when the lidar sensor is 3 channels or 16 channels, a color table according to the number of overlapping data can be visualized as shown in FIG. 17.

In FIG. 1, the synthesis unit 30 synthesizes the outputs of the first visualization unit 18 and the second visualization unit 26 and displays them in the same coordinate system. That is, when the result of visualizing the radar coordinate conversion value by the first coordinate conversion unit 16 and the visualization result of the radar coordinate conversion value by the second coordinate conversion unit 24 are combined, as illustrated in FIG. The same synthesis result 40 can be displayed in the same coordinate system.

19 is a flowchart illustrating a sensor data visualization method according to an embodiment of the present invention.

First, the radar sensor information collection unit 10 includes information measured by a radar sensor (not shown) (e.g., information about a subject, distance between the sensor and the subject, angle between the sensor and the subject, whether the subject is moving, or the subject is moved). Speed, etc.), and the lidar sensor information collection unit 20 includes information measured by a lidar sensor (not shown) (e.g., information about a lidar channel, information about a horizontal angle, distance information from a subject, etc.) ) Is collected (S10).

Then, the first coordinate calculation unit 12 changes the radar information collected by the radar sensor information collection unit 10 to a value of the distance coordinate system through a predetermined conversion equation, and the second coordinate calculation unit 22 The lidar information collected by the IDA sensor information collection unit 20 is changed to a value of the distance coordinate system through a predetermined conversion equation. That is, the first coordinate calculation unit 12 is the distance to the subject detected by the radar sensor information collecting unit 10 (ie, the distance between the radar sensor 34 and the subject (γ)) and direction (ie, the radar sensor (34) Y-axis distance between the front and receive information on the angle (θ) to the subject) to the X-axis distance between the radar sensor and the object through the following equation 1 (X _radar), and the radar sensor and the subject of (Y _radar ). Meanwhile, the second coordinate calculation unit 22 includes information from the lidar sensor information collection unit 20 (that is, a vertical angle (π) with respect to the xy plane of the lidar channel that recognizes the object, and the lidar that recognizes the object. By receiving the horizontal angle between the x-axis of the channel (θ) and the distance between the _lidar sensor and the subject (γ)), through Equation 4, the z-axis distance between the _lidar sensor and the subject (Z _lidar ), and the _lidar sensor. The x-axis distance (X _lidar ) between the subjects and the y-axis distance (Y _lidar ) between the _lidar sensor and the subject are obtained (S20).

Then, the correction unit 14 receives the data output from the first coordinate calculation unit 12 and corrects it. That is, the correction unit 14 corrects the position of the radar sensor for data integration visualization. In this case, the correction unit 14 corrects the radar data by using the correction equation of Equation 2. Accordingly, the correction unit 14 is the x-axis calibration position of the radar sensor by using the equation 2 (X, _radar), y-axis calibration position (Y of the radar sensor, obtains the _radar) (S30).

Thereafter, the first coordinate conversion unit 16 converts the data (ie, distance unit coordinates) from the correction unit 14 into pixel unit coordinate values, and the second coordinate conversion unit 24 calculates the second coordinate. The data from the unit 22 (that is, coordinates in units of distance) are converted into values of coordinates in units of pixels (S40). Here, a more detailed description of the transformation into pixel unit coordinates has already been described with reference to FIGS. 4, 5, and 15.

In addition, the first visualization unit 18 visualizes the information from the first coordinate conversion unit 16 in various forms, and the second visualization unit 26 converts the information from the second coordinate conversion unit 24 to two-dimensional. Visualize in the distance coordinate system (S50). Here, a more detailed description of the visualization has already been described with reference to FIGS. 6 to 12, 16 and 17.

Finally, the synthesis unit 30 synthesizes the outputs of the first visualization unit 18 and the second visualization unit 26 and displays the synthesized result 40 in the same coordinate system as shown in FIG. 18 (S60). Here, a more detailed description of the synthesis has already been described with reference to FIG. 18.

In addition, the sensor data visualization method of the present invention described above may be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer-readable recording medium is distributed over a computer system connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes and code segments for implementing the method can be easily deduced by programmers in the art to which the present invention belongs.

As described above, an optimal embodiment has been disclosed in the drawings and specifications. Although specific terms have been used herein, these are only used for the purpose of describing the present invention and are not used to limit the meaning or the scope of the present invention described in the claims. Therefore, those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical scope of the present invention should be determined by the technical spirit of the appended claims.

10: radar sensor information collection unit 12: first coordinate calculation unit
14: correction unit 16: first coordinate conversion unit
18: first visualization unit 20: lidar sensor information collection unit
22: second coordinate calculation unit 24: second coordinate conversion unit
26: second visualization unit 30: synthesis unit

Claims (14)

A first coordinate calculation unit that changes the collected radar information into a value of a distance coordinate system;
A correction unit correcting data from the first coordinate calculation unit;
A first coordinate conversion unit that converts the data from the correction unit into pixel-unit coordinate values;
A first visualization unit for outputting visualization information corresponding to information from the first coordinate conversion unit;
A second coordinate calculation unit that changes the collected LiDAR information into a value of the distance coordinate system;
A second coordinate conversion unit that converts data from the second coordinate calculation unit into a pixel-based coordinate value;
A second visualization unit for outputting visualization information corresponding to information from the second coordinate conversion unit; And
And a synthesis unit for synthesizing and displaying visualization information of the first visualization unit and the second visualization unit. The method according to claim 1,
The first coordinate calculation unit,
As the information on the distance and direction of the detected subject is received, the following equation,
Y _radar = γ×sinθ, X _radar = √(γ ² -Y _radar ² )
(Where, X _radar is the X-axis distance between the radar sensor and the subject, Y _radar is the Y-axis distance between the radar sensor and the subject, γ is the distance between the radar sensor and the subject, and θ is the distance between the front of the radar sensor and the subject. Angle)
A sensor data visualization device, characterized in that the X-axis distance between the radar sensor and the subject and the Y-axis distance between the radar sensor and the subject are obtained by using. The method according to claim 2,
The correction unit,
The following equation,
_X'radar = X _radar + ΔX, _Y'radar = Y _radar + ΔY
(Wherein, X _'radar is the x-axis calibration position of the radar sensor, Y' _radar is the y-axis calibration position of the radar sensor, ΔX is the x-axis distance between the centers of the central and La of the radar sensors, ΔY is the radar The y-axis distance between the center of the sensor and the center of the lidar sensor)
A sensor data visualization device, characterized in that to obtain the x-axis correction position of the radar sensor and the y-axis correction position of the radar sensor. The method of claim 3,
The first coordinate conversion unit,
Upon receiving the x-axis correction position of the radar sensor and the y-axis correction position of the radar sensor, the following equation,
X'= h + _X'radar × (h/distance _max ), Y'= h- _Y'radar × (h/distance _max )
(Where X'is the number of pixels on the x-axis between the radar sensor and the subject, h is the number of vertical pixels in the screen display area, distance _max is the maximum visible distance of the radar sensor, and Y'is the y-axis screen pixels)
A sensor data visualization device, characterized in that calculating the number of pixels on the x-axis screen between the radar sensor and the subject and the number of pixels on the y-axis screen between the radar sensor and the subject. The method according to claim 1,
The first visualization unit,
A sensor data visualization device, characterized in that a new subject and an already recognized subject are distinguished and visualized, wherein the new subject is displayed as a box of a first color, and the already recognized subject is displayed as a box of a second color. The method according to claim 1,
The first visualization unit,
A sensor data visualization device, characterized in that the presence/absence of speed information of the subject is classified and visualized, and when information about the moving speed of the subject is provided, the corresponding speed is displayed in a number in a box. The method according to claim 1,
The first visualization unit,
A sensor data visualization device, characterized in that the still subject and the moving subject are separated and visualized, and the moving subject is displayed using an arrow or a straight line to distinguish it from the still subject. The method according to claim 1,
The first visualization unit,
Visualize the presence or absence of information on the moving direction of the subject separately, but if there is information on the moving direction of the subject, use an arrow to indicate the moving direction, and if there is no information on the moving direction, from the position immediately preceding the subject A sensor data visualization device, characterized in that the movement direction to the current position is calculated and displayed in a straight line. The method according to claim 1,
The first visualization unit,
A sensor data visualization apparatus, characterized in that for visualizing a moving direction of a subject, and plotting information on the location of the subject in chronological order. The method of claim 9,
The first visualization unit,
A sensor data visualization device, characterized in that when there is information on the moving direction of the subject, the moving direction is indicated by using an arrow, and when there is no information on the moving direction of the subject, it is displayed using a straight line. The method according to claim 1,
The collected lidar information is a vertical angle from the xy plane of the lidar channel recognizing the subject, the horizontal angle from the x-axis of the lidar channel recognizing the subject, and the distance between the lidar sensor and the subject,
The second coordinate calculation unit is the following equation,
Z _lidar = γ × sinπ
X _lidar = sinθ × √(γ ² -Z _lidar ² )
Y _lidar = √(γ ² -X _lidar ² -Z _lidar ² )
(Where, Z _lidar is the z-axis distance between the _lidar sensor and the subject, γ is the distance between the _lidar sensor and the subject, π is the vertical angle with the xy plane of the _lidar channel that recognizes the subject, and X _lidar is The x-axis distance between the _lidar sensor and the subject, θ is the horizontal angle with the x-axis of the _lidar channel that recognizes the subject, and Y _lidar is the y-axis distance between the _lidar sensor and the subject)
A sensor data visualization device, characterized in that to obtain the z-axis distance between the lidar sensor and the subject, the x-axis distance between the lidar sensor and the subject, and the y-axis distance between the lidar sensor and the subject. The method of claim 11,
The second coordinate conversion unit,
The following equations for the x-axis distance between the lidar sensor and the subject and the y-axis distance between the lidar sensor and the subject,
X'= h + X _lidar × (h/distance _max ), Y'= h-Y _lidar × (h/distance _max )
(Where, X'is the number of pixels on the x-axis between the lidar sensor and the subject, h is the number of vertical pixels in the screen display area, distance _max is the maximum visible distance of the radar sensor, and Y'is the _lidar sensor and the subject. Is the number of pixels on the y-axis of the screen)
A sensor data visualization device, characterized in that the number of pixels of the x-axis screen between the lidar sensor and the subject and the number of pixels of the y-axis screen between the lidar sensor and the subject are calculated by substituting in. The method according to claim 1,
The second visualization unit,
Visualizing the information from the second coordinate conversion unit in a two-dimensional distance coordinate system, the sensor data visualization device, characterized in that the color values are differentiated according to the number of overlapping data. Changing the radar information collected by the first coordinate calculator to a value of the distance coordinate system, and changing the radar information collected by the second coordinate calculator to a value of the distance coordinate system;
A step of correcting, by a correction unit, data from the first coordinate calculation unit;
Converting, by a first coordinate conversion unit, data from the correction unit to a value of coordinates in pixel units, and by a second coordinate conversion unit, converting data from the second coordinate calculation unit to a value of coordinates in pixels;
Outputting visualization information corresponding to the information from the first coordinate conversion unit, and outputting visualization information corresponding to the information from the second coordinate conversion unit by a first visualization unit; And
And synthesizing and displaying the visualization information of the first visualization unit and the second visualization unit by a synthesis unit.

Images