KR102115004B1 - Apparatus and method for generating three dimensional map using aerial images - Google Patents

Abstract

Disclosed is a method for generating a 3D map according to an embodiment, including the steps of: generating a first graph including constraint factors related to sensing data for nodes and a moving object; generating a first 3D map based on the generated first graph; determining a correction value such that coordinates of at least one point in the first 3D map can correspond to coordinates of a point in an aerial map matching the point; determining a second graph to which constraint factors related to the determined correction value are added to the first graph; and generating a second 3D map based on the second graph.

Description

Apparatus and method for generating 3D map using aerial photographs {APPARATUS AND METHOD FOR GENERATING THREE DIMENSIONAL MAP USING AERIAL IMAGES}

This disclosure relates to the field of map generation. More specifically, the present disclosure relates to an apparatus and method for generating a three-dimensional map.

A general 3D map is produced based on ground surveying using expensive mobile mapping system (MMS) equipment. These MMS devices create a 3D map by combining light detection and ranging (LIDAR) or 3D coordinates of points obtained from a camera in a 3D driving route based on a global positioning system (GPS) / inertial navigation system (INS). .

In addition, in order to increase the precision of the 3D map, the 3D map is optimized by utilizing a ground correction point. Therefore, if the GPS / INS measurement result is not good, the reliability of the map also decreases.

Meanwhile, as a method of quickly producing a large area map at a low cost, a method of mapping based on aerial photography has been proposed. This method is produced by drawing lane and road sign information from high-precision aerial photographs, and is an efficient method for quickly producing maps with errors within about 10 cm. However, an aerial photograph-based map can be efficiently and quickly produced, but there is a problem that it is impossible to visualize the obscured area on the aerial photograph, and it is impossible to obtain 3D information around the road.

An apparatus for generating a map according to an embodiment and a method for generating a 3D map using the same is a technical task of rapidly generating a high-precision 3D map at low cost.

A map generating apparatus according to an embodiment and a method of generating a 3D map using the same are technical tasks of generating a 3D map that provides 3D information that cannot be identified by aerial photography alone.

The apparatus for generating a map and a method for generating a 3D map according to an embodiment generate a 3D map by compensating for a lack of each of data obtained at the aviation level and data acquired at the ground level, thereby achieving precision at low cost. The technical task is to create a high three-dimensional map.

A method for generating a 3D map according to an embodiment is

Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object; Generating a first 3D map based on the generated first graph; Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point; Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And generating a second 3D map based on the second graph.

Map generating apparatus according to another embodiment,

Processor; And a memory for storing at least one instruction, wherein the processor generates a first graph including constraints associated with sensing data for the moving object, nodes, and according to the at least one instruction, the generated A first 3D map is generated based on a first graph, and a correction value is determined such that coordinates of at least one point in the first 3D map correspond to coordinates of points in the aerial map matching the points. A second graph to which the constraint factors related to the determined correction value are added to the first graph may be determined, and a second three-dimensional map may be generated based on the second graph.

The apparatus for generating a map and a method for generating a 3D map according to an embodiment may quickly generate a high-precision 3D map at low cost.

A map generating apparatus and a method for generating a 3D map according to an embodiment may generate a 3D map that provides 3D information of a portion that cannot be identified by aerial photography.

The apparatus for generating a map and a method for generating a 3D map according to an embodiment generate a 3D map by compensating for a lack of each of data obtained at the aviation level and data acquired at the ground level, thereby achieving precision at low cost. It can generate high 3D maps.

However, effects that can be achieved by the map generating apparatus and the method for generating a 3D map according to an embodiment are not limited to those mentioned above, and other effects that are not mentioned are disclosed in the following description. It will be clearly understood by those skilled in the art.

A brief description of each drawing is provided to better understand the drawings cited herein.
1 is a view showing a moving object and a map generating apparatus according to an embodiment.
2 is a flowchart illustrating a method for generating a 3D map according to an embodiment.
3 is a diagram illustrating a first graph according to an embodiment.
4 is an exemplary diagram illustrating a three-dimensional sub-map corresponding to each node included in the first graph.
5 is a diagram illustrating a second graph according to an embodiment.
6 is a diagram for explaining a method of determining a correction value through point-to-point matching according to an embodiment.
7 and 8 are diagrams illustrating a method of determining a correction value through point-to-line matching according to an embodiment.
9 and 10 are diagrams illustrating a method of determining a correction value through line-to-line matching according to an embodiment.
11 and 12 are diagrams illustrating a method of determining a correction value through marker matching according to an embodiment.
13 is a diagram for explaining a method of determining a correction value through template matching according to an embodiment.
14 is a block diagram showing the configuration of a map generating apparatus according to an embodiment.

Since the present disclosure can be modified in various ways and have various embodiments, specific embodiments will be illustrated in the drawings and described through detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that all modifications, equivalents, and substitutes included in the spirit and scope of the present disclosure are included.

In describing the embodiments, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter, detailed descriptions thereof will be omitted. In addition, the numbers (for example, first, second, etc.) used in the description of the embodiment are merely identification numbers for distinguishing one component from other components.

In addition, when one component is referred to as “connected” or “connected” with another component in the present specification, the one component may be directly connected to or directly connected to the other component, but is specifically opposed It should be understood that as long as there is no description to be made, it may be connected or connected through another element in the middle.

In addition, in this specification, two or more components represented by '~ unit (unit)', 'module', etc. are combined into one component, or two or more components are divided into more detailed functions. It may be differentiated into. In addition, each of the components to be described below may additionally perform some or all of the functions of other components in addition to the main functions in charge of them, and some of the main functions of each component are different. Needless to say, it can also be carried out by components.

In addition, in this specification, 'global coordinates' refers to the location of a predetermined object based on the constant origin, and may be referred to as 'absolute coordinates'. The global coordinates of a given object are always constant.

In addition, in this specification, 'local coordinates' refers to the location of a predetermined object based on a variable origin, and may be referred to as 'relative coordinates'. Local coordinates of a given object may vary depending on where the origin is determined.

Hereinafter, embodiments according to the technical spirit of the present disclosure will be described in detail.

1 is a diagram illustrating a moving object 100 and a map generating device 200 according to an embodiment.

The moving object 100 may be a vehicle, a robot, or a drone. Referring to FIG. 1, the movable body 100 may include at least one sensor. Sensing data according to the movement of the moving object 100 may be generated using at least one sensor. For example, the moving object 100 is at least one sensor, an IMU (Inertial Measurement Unit) sensor 101, a global positioning system (GPS) sensor 102, a wheel encoder sensor 103, a light detection and ranging LIDAR ) Sensor 104 and the like. However, the sensors included in the movable body 100 are not limited thereto, and various types of sensors may be included in the movable body 100, or some of the sensors illustrated in FIG. 1 may not be included in the movable body 100. . The at least one sensor may be located inside or outside the mobile body 100.

The IMU sensor 101 may include an acceleration sensor and a gyroscope sensor (a gyroscope) as a measure of inertia. For example, the IMU sensor 101 is a rotation angle of the moving body 100, and the angular velocity increments of the forward direction (Roll axis direction), the right direction of the forward direction (pitch axis direction), and the gravity direction (yaw axis direction), respectively. Sensing data such as an incremental speed of the moving object 100 may be generated. In addition, the IMU sensor 101 may generate IMU Odometry using sensing data. That is, it is also possible to measure the amount of change in acceleration and angular velocity from the initial conditions, and provide the moving direction and moving distance of the moving body 100 as an IMU odometry.

The GPS sensor 102 may communicate with a satellite to provide coordinates on the surface of the mobile body 100 as sensing data. For example, the GPS sensor 102 is a geodetic coordinate of the WGS84 coordinate system, and may provide latitude, longitude, ellipse height, and the like.

The wheel encoder sensor 103 measures the number of revolutions of a wheel provided in the moving object 100. When the wheel encoder sensor 103 is used, it is possible to measure a moving distance and a moving speed of the moving object 100.

The LIDAR sensor 104 may generate depth information on points around the moving object 100 by emitting a laser pulse or the like and measuring the time to return after being reflected. Here, the rotation amount of the moving object 100 may be measured using the LIDAR sensor 104 or a Lidar Odometry may be generated. Therefore, when the LIDAR sensor 104 is used, a moving direction, a moving distance, and the like of the moving object 100 can be measured. As the LIDAR sensor 104, MLS-511 of SICK, VLP-16 of Velodyne, and the like may be used, but are not limited thereto.

Additionally, the moving object 100 may further include a camera or the like, and it is also possible to take an image corresponding to the movement of the moving object 100 using the camera. In this case, a visual odometry of the moving object 100 may be generated from the camera.

The map generating apparatus 200 receives the sensing data generated by the sensor included in the mobile body 100 and generates a first graph based on the sensing data. Here, the first graph means a graph before being updated according to an aerial map described later. Using the graph, it is possible to accurately predict the six degrees of freedom value of the moving object 100, that is, the coordinate value and the attitude value of the moving object 100.

The coordinate values of the moving object 100 indicate positions on three-dimensional coordinates, and may include, for example, coordinate values on the x-axis, y-axis, and z-axis. In addition, the posture value of the movable body 100 may include a pitch value, a roll value, and a yaw value of the movable body 100.

The map generating apparatus 200 may be coupled to the mobile body 100 and move together with the mobile body 100, or may be separated from the mobile body 100 and receive sensing data from the mobile body 100 through a wired or wireless communication method.

2 is a flowchart illustrating a method for generating a 3D map according to an embodiment.

In operation S210, the map generating apparatus 200 generates a first graph including nodes and constraint factors related to sensing data for the moving object 100. According to the first graph, the six degrees of freedom value of the moving object 100 corresponding to each of the nodes can be predicted. First, the first graph will be described with reference to FIG. 3.

3 is a diagram illustrating a first graph 300 according to an embodiment.

The map generating apparatus 200 may generate a node according to a reference period, and may generate a graph by setting a constraint factor for nodes based on sensing data synchronized with the reference period. Here, the map generating apparatus 200 may set any one of the sensing data as reference data, and may set the generation cycle of the reference data as the reference period. That is, according to the generation cycle of the reference data, a node may be generated and included in the graph. In addition. The map generating apparatus 200 may include sensing data synchronized with a reference period as a constraint factor of the node in the graph. In one embodiment, the IMU odometry of the IMU sensor 101 may be determined as reference data.

In the above example, the case where the reference data has a constant generation period has been described, but according to another example, the reference data may be collected according to a certain distance interval. In this case, the map generating apparatus 200 may generate a node corresponding to each of the reference data collected for each reference distance interval and include it in the graph. The map generating apparatus 200 may include sensing data synchronized at a sensing time of a node generated for each reference distance interval as a constraint factor of the node in the graph.

The constraint factor associated with sensing data may be divided into a unary factor and a binary factor. Here, the unitary factor represents the measurement result at the time of generating the sensing data, and includes coordinate values measured by the GPS sensor 102. For example, the map generating apparatus 200 may add the latitude and longitude coordinate values of the moving object 100 at a specific time point generated by the GPS sensor 102 as a unitary factor of the node.

The binary factor may be a factor that constrains the relationship between different nodes. For example, the binary factor indicates relationship information between a measurement start point and a measurement end point of sensing data as a measurement result, and the IMU odometry of the IMU sensor 101 corresponds to this. That is, information such as a moving direction and a moving distance using an amount of change in acceleration between a measurement start point and a measurement end point may correspond to a binary factor, not an acceleration value at a specific time point. Such information may be expressed as relative pose (position and posture) information.

As shown in FIG. 3, the map generation device 200 may first generate the first node 310 at a time t = 0, and the initial condition of the first node 310 is the first node 310. It can be set by the unitary factor 311 of. Thereafter, the second node 320 may be generated at a time t = 5, and a binary factor 315 that connects between the first node 310 and the second node 320 may be set. Here, the reference period may be 5, and an IMU odometry between t = 0 and t = 5 may be set as a battery factor. In this way, a node corresponding to the moving object 100 is newly generated for each reference period, and a constraint factor corresponding to the sensing data is set at each node to generate the first graph 300 for the moving object 100. have.

In one embodiment, after generating each of the nodes 310 and 320 corresponding to t = 0 and t = 5, asynchronous sensing data corresponding to a time point of t = 4 is input to the map generating apparatus 200 There may be. Here, since the first graph 300 is already generated up to t = 5, it is necessary to modify the graph to reflect asynchronous sensing data corresponding to t = 4. As illustrated in FIG. 3, the map generating apparatus 200 may newly generate a third node 330 corresponding to t = 4 where asynchronous sensing data is generated, and a third node corresponding to t = 4 ( The unitary factor 331 corresponding to the asynchronous sensing data may be set in 330). Also, the binary factor 325 may be set between the second node 320 and the third node 330. For example, when the asynchronous sensing data generated at t = 4 is GPS coordinates, after comparing the GPS coordinates at t = 4 with the GPS coordinates at t = 5, t = 5 from the GPS coordinates at t = 4 The binary factor 325 may be set as a wheel encoder value for progressing to GPS coordinates.

Referring back to FIG. 2, in step S220, the map generating apparatus 200 generates a first 3D map based on the first graph. The first 3D map may include coordinates of points identified by the LIDAR sensor 104 as the moving object 100 moves. Here, the coordinates may be three-dimensional global coordinates. That is, the map generating apparatus 200 calculates the six degrees of freedom value of the moving object 100 corresponding to each of the nodes based on the first graph, and moves the local coordinates of the points sensed around the moving object 100 ( The first three-dimensional map can be generated by changing the global coordinates according to the six degrees of freedom value of 100).

The map generating apparatus 200 may optimize the first graph to accurately predict the six degrees of freedom of the moving object 100 based on the first graph.

로 나타낼 수 있다. 이 노드들은 특정한 구속 팩터들에 의해 연결 관계를 갖고, 이들 구속 팩터들로 구성된 그래프의 에러를 최소화하는 노드들의 6자유도 값이 산출될 수 있다. 노드 간의 연결 관계는 그래프의 구성요소 중 에지(edge)에 해당할 수 있다.Specifically, a graph with n nodes is

Can be represented as These nodes have a connection relationship by specific constraint factors, and 6 degrees of freedom values of nodes that minimize errors in a graph composed of these constraint factors can be calculated. The connection relationship between nodes may correspond to an edge among components of the graph.

Equation 1 below may be used for graph optimization.

[Equation 1]

및

각각은 시간 i, j에서의 노드를 의미하고, 센서들로부터 측정한 노드의 6자유도 값과 그래프 상에서의 노드의 6자유도 값 사이의 오차가

로 표현된다. 그래프의 최적화는 오차

를 최소화하는 그래프를 산출하는 것이다. 여기서,

는 센싱 데이터의 정보 매트릭스(information matrix)이다. 그래프의 최적화는 Gauss-Newton Method 또는 Levenberg-Marquardt Method 등의 비선형 최적화 방법을 통해 이루어질 수 있다.In Equation 1 above

And

Each represents a node at time i, j, and the error between the 6 degrees of freedom value of the node measured from the sensors and the 6 degrees of freedom value of the node on the graph

It is expressed as The optimization of the graph is an error

Is to produce a graph that minimizes. here,

Is an information matrix of sensing data. The optimization of the graph can be achieved through a nonlinear optimization method such as Gauss-Newton Method or Levenberg-Marquardt Method.

FIG. 4 is an exemplary diagram showing a 3D sub-map corresponding to each node included in the first graph, and based on sensing data obtained from the LIDAR sensor 104 or the like at a time corresponding to each node. A 3D sub-map corresponding to the node may be generated. At this time, the six degrees of freedom value of the moving object 100 for a specific time between the node and the node may be calculated through interpolation. The global map may be calculated by accumulating the sub-maps illustrated in FIG. 4.

Referring back to FIG. 2, in step S230, the map generating apparatus 200 may determine a correction value such that coordinates of at least one point in the first 3D map correspond to coordinates of at least one point in the aerial map. have. For example, the map generating apparatus 200 may determine a correction value for matching the coordinates of the points in the first 3D map to the coordinates of the points in the aerial map. The aerial map may include data obtained by vehicles such as drones, satellites, airplanes, etc., for example, maps generated from photographs.

The first 3D map and the aerial map may consist of several points, each point having corresponding coordinates, specifically global coordinates. As described above, the first 3D map is generated based on the sensing data sensed by the mobile body 100 at the ground level. At the ground level, satellite-based positioning data (eg GPS) is generated due to the influence of surrounding obstacles. Because of the low reliability, the coordinates of the points in the first 3D map may be inaccurate. However, at the aviation level higher than the ground level, the reliability of the satellite-based positioning data is higher than the ground level. Accordingly, in one embodiment, a correction value for generating a high-precision 3D map is determined based on an aerial map with high coordinate reliability.

The correction value is for correcting the six degrees of freedom value of the nodes calculated based on the first graph, and the correction value may also include a six degree of freedom value. The correction value may be determined based on an error between coordinates of matching points that match each other in the first 3D map and the aerial map. Certain possible objects in the real space, such as markers drawn on roads (lanes, stops, pedestrian crossings, traffic signs), etc., can be manually or automatically designated as registration points.

The correction values include point-to-point matching, point-to-line matching, line-to-line matching, marker matching, or the like between the first three-dimensional map and the aerial map. It may be calculated through template matching, etc., which will be described later with reference to FIGS. 6 to 12. In step S240, the map generating apparatus 200 adds constraint factors related to the correction value to the first graph. , Determine the second graph. Here, the second graph refers to a graph determined by newly including constraint factors in the first graph.

In one embodiment, the map generating apparatus 200 may add the correction value to the first graph as a unity factor of at least one node included in the first graph. The map generating apparatus 200 may select a node to be connected to a unitary factor among nodes of the first graph in consideration of which submap the correction value is determined. For example, if the correction value is determined in the M _t submap of FIG. 4, the map generating apparatus 200 may add the correction value to the first graph as a unitary factor of the X _t node.

In addition, in one embodiment, the map generating apparatus 200 may include a value in which the correction value is reflected in the six degrees of freedom value of at least one node included in the first graph, as a unity factor of the at least one node. Can be added to The map generating apparatus 200 may select a node to be connected to a unitary factor among nodes of the first graph in consideration of which submap the correction value is determined. For example, if the determined correction value in M _t sub-map of FIG. 4, the map creation device 200 includes a first value reflecting the correction value to the value 6 freedom X _t node as oil Nourishing factor of X _t node You can add it to the graph.

In addition, in one embodiment, the map generating apparatus 200 calculates and calculates a distance between at least one node and a matching point based on the coordinates of at least one node and the matching points in the first 3D map. The reliability inversely proportional to the distance may be added to the first graph as an element of the unity factor of at least one node. The matching point in the first 3D map is determined through a matching process with an aerial map.

In one embodiment, the map generating apparatus 200 may add a unitary factor to the Xt node based on the coordinates of the matching point when the matching point is determined in the M _t submap illustrated in FIG. 4. The unitary factor may include correction values and reliability. For example, the unity factor may be expressed as follows.

Unary_factor (symbol_id (X_t), pose, reliability)

Here, symbol_id (x_t) corresponds to information of a node to which the unitary factor is connected, pose is a correction value, and reliability is reliability.

The correction value is a value for correcting the coordinates of the Xt node. For example, it is assumed that the first matching point in the first three-dimensional map and the second matching point in the aerial map are actually expected to be the same object (ex. Lanes) and match each other. At this time, the correction value may be a value set to add B-A to the coordinates of the Xt node so that the coordinate A of the first matching point is the same as the coordinate B of the second matching point. Since the global coordinates of the points in the first 3D map are calculated by the global coordinates of the node and the local coordinates of each point based on the node, a method of correcting the global coordinates of the node to correct the global coordinates of each point You can choose.

If the graph is optimized to be 100% satisfied by the correction value (generally, it is unlikely that the graph will be completely optimized by one factor because the number of constraint factors is large), the global coordinates of the Xt node If is moved by BA, the coordinates (global coordinates) of the first matching point calculated based on the Xt node become A + (BA) = B, which coincides with the global coordinates (B) of the second matching point in the matched aerial map. You can do it.

Reliability may be a factor indicating how strongly the binding force of the corresponding unit factor will be optimized when optimizing the graph. According to an example, the reliability may be set to be inversely proportional to the distance between the matching point on which the correction value is calculated and the node to which the corresponding unitary factor is connected. According to this, the map generating apparatus 200 may include a correction value based on the difference between the coordinates of the first registration point in the first 3D map and the coordinates of the second registration point in the aerial map, and the first registration point and the X _t node A reliability factor inversely proportional to the distance of is determined, and a unitary factor including such a correction value and a reliability can be added to the first graph as the unitary factor of the X _t node. Reliability may have a smaller value as the above-described distance is larger, in order to lower the influence on the node when optimizing the graph, considering that the correction value calculated using the matching point located farther from the node will be lower in accuracy. A method of calculating reliability according to an example may be expressed by Equation 2 below.

[Equation 2]

In Equation 2, Pi is the coordinate of the first registration point in the first three-dimensional map that is the basis for determining the correction value of the node Xt, and Pi 'is the second matching point in the aerial map that matches the first matching point. The coordinates of, ms () is the matching score function between Pi and Pi ', d () is the Euclidean distance between the node Xt and the matching point Pi, n is the number of matching points.

The matching score can be an indicator of how accurately the two matching points matched. The matching score may be a value that serves as a residual in statistics. For example, the larger the distance difference between the first matching point and the second matching point, the lower the matching score can be calculated. According to another example, the higher the degree of matching between the patterns of the first matching point and the second matching point, the higher the matching score can be calculated. The method of calculating the matching score is not limited to this.

는 해당 신뢰도 값의 스케일을 조정하기 위한 값이다.According to Equation 2, reliability is inversely proportional to the average of the distances between the node Xt and the first matching points, and is calculated to be proportional to the matching score. Therefore, the larger the average distance, the lower the reliability, and the larger the average matching score, the higher the reliability.

Is a value for adjusting the scale of the corresponding reliability value.

According to an example, on the assumption that the difference between the coordinates of the first matching point (Pi) and the coordinates of the second matching point (Pi) is not large, d (Xt-Pi) in Equation 2 above is d (Xt-Pi ' ) Or d (Xt-Pi "). Pi" may be another value calculated using Pi and Pi ', such as an average value.

As described later, when a plurality of matching points are determined in the first three-dimensional map, the reliability of the unity factor of the at least one node is inversely proportional to the average value of the distances between the at least one node and each matching point. Can be set as an element. In addition, when a matching line is determined in the first three-dimensional map, the reliability inversely proportional to the average value of the distances between each of the points included in the matching line and at least one node is one of the unitary factor of at least one node. Can be set as an element. In addition, when a plurality of matching markers are determined in the first three-dimensional map, the reliability of the at least one node is inversely proportional to the average value of distances between each of the points included in the plurality of matching markers and at least one node. It can be set as an element of the unity factor.

In addition, in one embodiment, the map generating apparatus 200 may add the coordinates of the matching points in the aerial map to the first graph as landmarks of a plurality of nodes. Landmark is a kind of binary factor, and can function as a constraint factor for multiple nodes. The landmark according to an example may be added to the first graph as a bearing-range factor.

For example, assuming that a specific “lane” is designated as a matching point, global coordinates of the corresponding lane in the first 3D map may be calculated based on the first node or may be calculated based on the second node. . However, if the global coordinates of the first node and the second node are incorrect, the global coordinates of the corresponding lane may be calculated differently depending on which node is used. The landmark according to an example is to give a constraint that the corresponding lane corresponds to the same object, that is, a constraint that global coordinates are the same. For example, when the first node and the second node refer to the same "lane as a landmark," the global coordinates of the lane calculated based on the first node and the global coordinates of the lane calculated based on the second node are the same. " If the landmark includes the coordinate values (assuming that the coordinates of the aerial map are more accurate) on the aerial map of the lane, the "lane calculated based on the first node" In addition to the condition that the coordinates of and the coordinates of the lane calculated based on the second node are the same ", it is possible to constrain what value these coordinates should be.

The map generating apparatus 200 may select a plurality of nodes to be connected to the landmark from among the nodes of the first graph. For example, the map generating apparatus 200 may select a predetermined number of nodes in an order close to the matching point, and add a landmark commonly viewed by the selected number of nodes to the first graph as a constraint factor. The landmark may include coordinates in the aerial map of the registration point.

Adding a landmark to the graph as above means that the specific landmark viewed from the first node and the specific landmark viewed from the second node are the same object, that is, the constraint that the global coordinates of the landmark are the same is displayed in the graph. Is to add For example, when optimizing a graph in which the information of the first landmark is added as a binary factor between the first node and the second node, the graph calculates the global coordinates of the first landmark and the second node based on the first node. The global coordinates of the first landmark calculated based on the reference are optimized in the same direction.

5 is a diagram illustrating a second graph 500 according to an embodiment.

Referring to FIG. 5, compared to the first graph 300 illustrated in FIG. 3, the unitary factor 521 of the second node 320 when t = 5 and the third node 330 when t = 4 It can be seen that the unitary factor 532 and the landmarks 525 for the second node 320 and the third node 330 are added.

That is, the second graph 500 may be determined by adding constraint factors determined based on the aerial photograph to the first graph 300.

Referring back to FIG. 2, in step S250, the map generating apparatus 200 generates a second 3D map based on the second graph.

Specifically, the map generating apparatus 200 may generate a second three-dimensional map by determining the updated six degrees of freedom of each of the nodes through optimization of the second graph. Specifically, the map generating apparatus 200 may generate a second 3D map by calculating updated coordinate information of points sensed by the moving object 100 based on the updated 6 degrees of freedom of each of the nodes. . For example, the map generating apparatus 200 uses the updated six degrees of freedom value of each of the nodes and local coordinates of each point based on the node to update updated global coordinate information of points sensed by the moving object 100. It is possible to calculate and generate a second 3D map including points having the calculated global coordinates.

Hereinafter, a method of determining a correction value for generating a second graph will be described in detail with reference to FIGS. 6 to 12.

6 is a diagram for explaining a method of determining a correction value through point-to-point matching according to an embodiment.

The map generating device 200 includes first matching points P1, P2, P3, and P4 and first matching points P1 'and P2 that match each other in the first 3D map 610 and the aerial map 630. ', P3', P4 '). The first 3D map 610 illustrated in FIG. 6 may be any one sub-map illustrated in FIG. 4. For example, in the first 3D map 610 of FIG. 6, only lanes and road signs extracted based on reflectivity from any one submap shown in FIG. 4 may be displayed.

6, the first matching point (P1, P2, P3, P4) and the second matching point (P1 ', P2', P3 ', P4') is shown as a plurality, but the first matching point (P1, P2) , P3, P4) and each of the second matching points P1 ', P2', P3 ', and P4'.

The first matching points P1, P2, P3, and P4 and the second matching points P1 ', P2', P3 ', and P4' may be determined manually or automatically. As an example, the map generating device 200 may be configured to provide first matching points P1, P2, P3, and P4 that match each other among points in the first 3D map 610 and the aerial map 630 from the manager. 2 Matching points (P1 ', P2', P3 ', P4') can be input. As another example, the map generating apparatus 200 may be configured using template matching or local descriptors using information about each of points in the first 3D map 610 and the aerial map 630 and surrounding information. One matching point (P1, P2, P3, P4) and a second matching point (P1 ', P2', P3 ', P4') matching it may be determined.

The map generating device 200 coordinates the first matching points P1, P2, P3, and P4 in the first three-dimensional map 610 and the second matching points P1 ', P2' in the aerial map 630, The difference value between the coordinates of P3 'and P4') can be determined as a correction value. As described above, the determined correction value or a value in which the correction value is reflected in the coordinates of the node may be added to the first graph as a unitary factor. The map generating device 200 is a unity factor of the node closest to the first matching points P1, P2, P3, and P4, and adds the correction value or a value reflecting the correction value to the coordinates of the node to the first graph. Can be. Also, coordinates of the second matching points P1 ', P2', P3 ', and P4' may be added to the first graph as landmarks for a plurality of nodes.

In addition, the map generation device 200 determines a node corresponding to the first matching point (P1, P2, P3, P4), the distance between the determined node and the first matching point (P1, P2, P3, P4) An inversely proportional reliability may be added to the first graph as an element of the node's unity factor.

6, in the first 3D map 610 and the aerial map 630, a plurality of first matching points P1, P2, P3, and P4 and a plurality of second matching points P1 If ', P2', P3 ', P4') is determined, the coordinates of the plurality of first matching points P1, P2, P3, P4 and the plurality of second matching points P1 ', P2', P3 ', The average of the difference values of each of the coordinates of P4 ') may be determined as a correction value.

When the second graph is determined based on the correction value, the updated six degrees of freedom value of each node is determined based on the second graph, and the points in the second three-dimensional map determined based on the updated six degrees of freedom value. The coordinates of the will correspond to the coordinates of the points on the aerial map. In other words, the coordinates of the first matching point P1, P2, P3, or P4 calculated based on any one node are (1, 1, 1), and the second matching point P1 ', P2', P3 ' Or, if the coordinates of P4 ') are (2, 2, 2), by adjusting the position of the node through the addition of a constraint factor, the first matching point (P1, P2, P3 or P4) calculated based on the node The coordinates are (2, 2, 2) (or coordinates close to them).

7 and 8 are diagrams illustrating a method of determining a correction value through point-to-line matching according to an embodiment.

On the other hand, identifiable markers (lanes, road signs, etc.) on the road can be specified not only as points, but also as lines. In this case, the map generating apparatus 200 may determine matching points P1, P2, and P3 and matching lines L 'that match each other in the first 3D map 610 and the aerial map 630. .

The matching points P1, P2, P3 and the matching line L 'can be determined manually or automatically. As an example, the map generating apparatus 200 inputs matching points P1, P2, and P3 and matching lines L 'that match each other in the first 3D map 610 and the aerial map 630 from the manager. Can receive As another example, the map generating apparatus 200 analyzes the first 3D map 610 and the aerial map 630 to determine matching points P1, P2, and P3 that match each other and the matching line L '. It might be.

When the matching points P1, P2, and P3 in the first 3D map 610 are transformed (or updated) according to the correction value, the coordinates of the transformed matching points are the aerial map 630. It should be located on my matching line (L '). Accordingly, as illustrated in FIG. 8, the map generating apparatus 200 determines a correction value at which the sum of the distances d between the matching points P1, P2, and P3 and the matching line L ′ is minimum. Can be. Here, the correction value may have a 6 degree of freedom value.

When the coordinates of the matching points P1, P2, and P3 calculated based on any one node are not on the matching line L ', any one of the above nodes is added through the addition of a constraint factor corresponding to the correction value. The 6 degree of freedom value of can be changed, and accordingly, the coordinates of the matching points P1, P2, and P3 are matched or similar to those of the points included in the matching line L 'of the aerial map.

9 and 10 are diagrams illustrating a method of determining a correction value through line-to-line matching according to an embodiment.

The map generating apparatus 200 includes first matching lines L1, L2, L3, L4, L5 and second matching lines L1 'that match each other in the first 3D map 610 and the aerial map 630. , L2 ', L3', L4 ', L5'). 9 illustrates a plurality of first matching lines L1, L2, L3, L4, and L5 and a plurality of second matching lines L1 ', L2', L3 ', L4', L5 ', but one The first matching line and one second matching line may be determined.

The first matching lines L1, L2, L3, L4, L5 and the second matching lines L1 ', L2', L3 ', L4', L5 'can be determined manually or automatically. As an example, the map generating apparatus 200 may include first matching lines L1, L2, L3, L4, and L5 that match each other in the first 3D map 610 and the aerial map 630 from the manager. Matching lines (L1 ', L2', L3 ', L4', L5 ') can be input. As another example, the map generating apparatus 200 analyzes the first 3D map 610 and the aerial map 630 to match the first matching lines L1, L2, L3, L4, and L5 that match each other. Matching lines L1 ', L2', L3 ', L4', L5 'may also be determined.

When the first matching line (L1, L2, L3, L4, L5) in the first three-dimensional map 610 is transformed (or updated) according to the correction value, the transformed first matching line ( L1, L2, L3, L4, L5) must coincide with the second matching lines L1 ', L2', L3 ', L4', L5 '. Accordingly, as illustrated in FIG. 10, the map generating apparatus 200 includes first matching lines L1, L2, L3, L4, and L5, specifically, points and second matching lines included in the first matching line The correction value at which the sum of the distances (d) between (L1 ', L2', L3 ', L4', L5 ') is minimum can be determined. Here, the correction value may have a 6 degree of freedom value. In one embodiment, a plurality of first matching lines (L1, L2, L3, L4, L5) and a plurality of second matching lines (L1 ', L2', L3 ', L4', L5 ') are determined, respectively The average of the correction values determined from the first matching line and the second matching line of may be determined as the final correction value.

The six degrees of freedom value of any one node may be changed through the addition of a constraint factor corresponding to the correction value, and accordingly, the points included in the first matching lines L1, L2, L3, L4, L5 The coordinates are matched or similar to the coordinates of points included in the second registration lines L1 ', L2', L3 ', L4', and L5 'of the aerial map 630.

11 and 12 are diagrams illustrating a method of determining a correction value through marker matching according to an embodiment.

There are various markers on the road, such as traffic signs. The map generating apparatus 200 includes first markers M1, M2, M3, and M4 and second markers M1 'and M2' that match each other in the first 3D map 610 and the aerial map 630. , M3 ', M4'). The first markers M1, M2, M3, M4 and the second markers M1 ', M2', M3 ', M4' can be determined manually or automatically. As an example, the map generating apparatus 200 may include first markers M1, M2, M3, and M4 and second markers that match each other in the first 3D map 610 and the aerial map 630 from the manager. (M1 ', M2', M3 ', M4') can be input. As another example, the map generating apparatus 200 analyzes the first 3D map 610 and the aerial map 630 to match the first markers M1, M2, M3, and M4 and the second markers. (M1 ', M2', M3 ', M4') can also be determined.

The coordinates of the points included in each of the first markers M1, M2, M3, and M4 and the coordinates of the points included in each of the second markers M1 ', M2', M3 ', M4' must match. , The map generating apparatus 200 includes coordinates of points included in each of the first markers M1, M2, M3, and M4 included in each of the second markers M1 ', M2', M3 ', and M4'. It is possible to determine a correction value to match or be similar to the coordinates of the points. The points included in the first markers M1, M2, M3, and M4 are blocks representing each of the first markers M1, M2, M3, and M4, for example, the square block B1 shown in FIG. B2, B3, B4) may be a midpoint point (C1, C2, C3, C4), likewise, points included in the second markers M1 ', M2', M3 ', M4' are second markers Each block may be, for example, a midpoint point C1 ', C2', C3 ', C4' of the square blocks B1 ', B2', B3 ', and B4' shown in FIG. 12. 12, the coordinates of the points C1, C2, C3, and C4 included in the first markers M1, M2, M3, and M4 are the second markers M1 ', M2', and M3. The 3D coordinate value (t) and the attitude value (R) for matching the coordinates of the points (C1 ', C2', C3 ', C4') included in ', M4') may be determined as correction values. The center point can mean the center of gravity of the block.

In addition, when determining the correction value, the map generating apparatus 200 has coordinates of points included in each of the first markers M1, M2, M3, and M4, and the second markers M1 ', M2', and M3 '. , M4 ') while matching or similar to the coordinates of the points included in each, connecting the points C1, C2, C3, C4 included in each of the first markers M1, M2, M3, M4 Points C1 ', C2', C3 in which the cabinets θ1, θ2, θ3, and θ4 of the first figure F are included in each of the second markers M1 ', M2', M3 ', M4' It is possible to determine a correction value to be matched or similar to the internal angles θ1 ', θ2', θ3 ', and θ4' of the second figure F 'connecting the', C4 '). The correction value may be a 6 degree of freedom value including a movement amount and a rotation amount.

13 is a diagram for explaining a method of determining a correction value through template matching according to an embodiment.

In one embodiment, the map generating apparatus 200 may determine a correction value through template matching. The positions of the points included in the first three-dimensional map 610 in the first three-dimensional map 610 are the same as the positions in the aerial map 630 of the points included in the aerial map 630. Since it should be, the difference between the pixel values corresponding to the points of the first 3D map 610 and the pixel values corresponding to the points of the aerial map 630 should be minimized. Accordingly, the map generating apparatus 200 corrects such that the sum of the differences between the pixel values corresponding to the points of the first 3D map 610 and the pixel values corresponding to the points of the aerial map 630 is minimized. The value can be determined and a constraint factor can be added to the first graph based on the determined correction value. For example, as illustrated in FIG. 13, the map generating device 200 may display a first 3D map 610 through an image 650 in which the first 3D map 610 and the aerial map 630 are overlapped. ) May determine a correction value in which a difference between pixel values corresponding to points of the pixel and pixel values corresponding to points of the aerial map 630 is minimized.

Depending on the implementation, the aforementioned point-to-point matching, point-to-line matching, line-to-line matching, marker matching, or template matching may be performed based on deep learning.

14 is a block diagram showing the configuration of a map generating apparatus 200 according to an embodiment.

Referring to FIG. 14, the map generating apparatus 200 includes a memory 1410, a communication unit 1430, and a control unit 1450. The memory 1410, the communication unit 1430, and the control unit 1450 may be implemented by at least one processor, and may operate according to instructions stored in the memory 1410.

The communication unit 1430 receives sensing data from the mobile body 100. The communication unit 1430 may receive the sensing data through a wired network and / or a wireless network.

The control unit 1450 generates a 3D map based on the sensing data. Specifically, the control unit 1450 generates a first graph including nodes and constraint factors, and generates a first three-dimensional map based on the first graph. Since the coordinates of the points included in the first 3D map may be different from the actual coordinates, the controller 1450 is a correction value such that the coordinates of the points in the first 3D map correspond to the coordinates of the points in the aerial map. And a second graph to which constraint factors related to the correction value are added. The control unit 1450 generates a second 3D map based on the second graph. The aerial map may be received by the communication unit 1430 from another device through a wired network and / or a wireless network, or stored in the memory 1410.

Since the method for generating the graph, the method for determining the correction value, and the method for generating the 3D map are described above, a detailed description is omitted.

Meanwhile, in the above-described embodiments of the present disclosure, the aerial map may be a 2D, 2.5D, or 3D map. According to an example, the aerial map may be 2.5-dimensional, where 2.5-dimensional means data having one (x) height (z) for each (x, y) coordinate of the 2D. The map generating apparatus 200 is an n-dimensional correction value when the point coordinates of the ground level map (for example, the above-described three-dimensional map) and the aerial map generated based on the moving object 100 are the same n-dimensional. Can be calculated. On the other hand, when the point coordinate of the ground level map is n-dimensional and the point coordinate of the aerial map is m-dimensional, the map generating apparatus 200 may calculate the correction value using only the lower-dimensional coordinate values. However, in the case of 2.5-dimensional, since it has all the x, y, and z-coordinate values, it can be handled according to the 3-dimensional.

Meanwhile, the above-described embodiments of the present disclosure can be written as a program that can be executed on a computer, and the created program can be stored in a medium.

The medium may be a computer that continuously stores executable programs or may be temporarily stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a combination of single or several hardware, and is not limited to a medium that is directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks, And program instructions including ROM, RAM, flash memory, and the like. In addition, examples of other media include an application store for distributing applications, a site for distributing or distributing various software, and a recording medium or storage medium managed by a server.

As described above, the technical idea of the present disclosure has been described in detail with reference to preferred embodiments, but the technical idea of the present disclosure is not limited to the above embodiments and has ordinary knowledge in the art within the scope of the technical idea of the present disclosure. Various modifications and changes are possible by the ruler.

100: moving object
200: map generating device
1410: memory
1430: Communication Department
1450: control

Claims (20)

Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
Determining the second graph,
Adding the correction value or a value reflecting the correction value to the six degrees of freedom value of at least one node included in the first graph as the unity factor of the at least one node to the first graph Method of generating a three-dimensional map, characterized in that it comprises a.
delete According to claim 1,
Determining the correction value,
And determining the difference between the coordinates of the first registration point in the first three-dimensional map and the coordinates of the second registration point in the aerial map matching the first registration point as the correction value. How to create a 3D map.
According to claim 3,
Determining the second graph,
Calculating a distance between the at least one node and the first matching point based on the coordinates of the at least one node and the coordinates of the first matching point; And
And adding a reliability inversely proportional to the calculated distance as an element of the unity factor.
The method of claim 4,
Determining the second graph,
And adding coordinates of a matching point in the aerial map to the first graph as landmarks of a plurality of nodes.
According to claim 1,
Determining the correction value,
And determining a correction value that minimizes the distance between the matching points in the first three-dimensional map and the matching line in the aerial map.
According to claim 1,
Determining the correction value,
And determining a correction value that minimizes the distance between the points included in the registration line in the first three-dimensional map and the matching line in the aerial map.
According to claim 1,
Determining the correction value,
And determining a correction value for matching coordinates of points included in registration markers in the first three-dimensional map to coordinates of points included in registration markers in the aerial map. 3D map creation method.
According to claim 1,
Determining the correction value,
And determining a correction value that minimizes a difference between pixel values corresponding to points of the first 3D map and pixel values corresponding to points of the aerial map. Creation method.
According to claim 1,
Generating the first three-dimensional map,
Determining a 6-degree-of-freedom value of each of the nodes through optimization of the first graph; And
And generating coordinates of points sensed by the moving object based on the six degrees of freedom of each of the nodes to generate the first three-dimensional map.
The method of claim 10,
Generating the second three-dimensional map,
Determining an updated six degrees of freedom value of each of the nodes through optimization of the second graph; And
And generating updated second coordinates of the points sensed by the moving object based on the updated six degrees of freedom value of each of the nodes, thereby generating the second three-dimensional map. Way.
According to claim 1,
The generating step,
The second 3D map is generated by optimizing a second graph in a direction that satisfies both the constraint factors already included in the first graph and the added constraint factors. Creation method.
A program stored in a medium for executing a method of generating a 3D map of any one of claims 1, 3 to 12 in combination with hardware.
Processor; And
Includes memory to store at least one instruction,
The processor according to the at least one instruction,
Create a first graph including constraint factors associated with the sensing data for the nodes and the moving object,
A first 3D map is generated based on the generated first graph,
Determine a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point,
Determine a second graph in which constraint factors related to the determined correction value are added to the first graph,
A second 3D map is generated based on the second graph,
When determining the second graph, the correction value or a value reflecting the correction value to the six degrees of freedom value of at least one node included in the first graph is used as a unitary factor of the at least one node. Map generating apparatus characterized in that it is added to the first graph.
Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
The determining of the correction value includes determining a correction value that minimizes the distance between the matching points in the first 3D map and the matching line in the aerial map. Creation method.
Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
The determining of the correction value may include determining a correction value that minimizes the distance between the points included in the registration line in the first 3D map and the matching line in the aerial map. How to create a 3D map.
Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
Determining the correction value,
And determining a correction value for matching coordinates of points included in registration markers in the first three-dimensional map to coordinates of points included in registration markers in the aerial map. 3D map creation method.
Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
Determining the correction value,
And determining a correction value that minimizes a difference between pixel values corresponding to points of the first 3D map and pixel values corresponding to points of the aerial map. Creation method.
Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
Generating the first three-dimensional map,
Determining a 6-degree-of-freedom value of each of the nodes through optimization of the first graph; And
And generating coordinates of points sensed by the moving object based on the six degrees of freedom of each of the nodes to generate the first three-dimensional map.
Generating a first graph comprising nodes and constraint factors associated with sensing data for the moving object;
Generating a first 3D map based on the generated first graph;
Determining a correction value such that coordinates of at least one point in the first three-dimensional map correspond to coordinates of a point in the aerial map matching the point;
Determining a second graph in which constraint factors related to the determined correction value are added to the first graph; And
And generating a second 3D map based on the second graph,
The generating step,
The second 3D map is generated by optimizing a second graph in a direction that satisfies both the constraint factors already included in the first graph and the added constraint factors. Creation method.

Images