KR20210010309A - Apparatus and method for generating three dimensional map using aerial images - Google Patents

Abstract

Disclosed according to an embodiment is a method for generating a three-dimensional map, the method comprising the steps of: generating a first graph including nodes and constraint factors related to sensing data for a moving object; generating a first three-dimensional map on the basis of the generated first graph; determining a correction value such that a coordinate of at least one point in the first three-dimensional map can correspond to a coordinate of a point in an aerial map matching the point; determining a second graph in which constraint factors related to the determined correction value are added to the first graph; and generating a second three-dimensional map on the basis of the second graph.

Description

Apparatus and method for generating 3D maps using aerial photographs {APPARATUS AND METHOD FOR GENERATING THREE DIMENSIONAL MAP USING AERIAL IMAGES}

The present disclosure relates to the field of map generation. More specifically, the present disclosure relates to an apparatus and method for generating a three-dimensional map.

A general three-dimensional map is being produced based on ground surveying using expensive mobile mapping system (MMS) equipment. These MMS devices create a three-dimensional map by fusion of light detection and ranging (LIDAR) or three-dimensional coordinates of points obtained from a camera in a three-dimensional driving path based on a global positioning system (GPS)/inertial navigation system (INS). .

In addition, in order to increase the precision of the 3D map, the 3D map is optimized by utilizing the correction points on the ground. Therefore, if the GPS/INS measurement result is not good, the reliability of the map is also degraded.

On the other hand, recently, a map production method based on aerial photographs has been proposed as a method of producing a map of a large area quickly at low cost. This method is produced by drawing lane and road sign information from a high-precision aerial photograph, and is an efficient method that can quickly produce a map with an error within about 10 cm. However, although an aerial photo-based map can be efficiently and quickly produced, there is a problem in that it is impossible to draw the area covered by the aerial photo and it is not possible to obtain 3D information around the road.

A map generating apparatus according to an exemplary embodiment and a method of generating a three-dimensional map by the same is a technical task of rapidly generating a high-precision three-dimensional map at low cost.

A map generating apparatus according to an embodiment and a method of generating a three-dimensional map by the same is a technical task of generating a three-dimensional map that provides three-dimensional information that cannot be identified using only aerial photographs.

The map generating apparatus according to an embodiment and a method of generating a 3D map by the method of generating a 3D map by compensating for each insufficient portion of the data acquired at the aerial level and the data acquired at the ground level are mutually complementary to generate a 3D map. Creating a high 3D map is a technical challenge.

A method of generating a 3D map according to an embodiment,

Generating a first graph including nodes and constraint factors related to sensing data for the moving object; Generating a first 3D map based on the generated first graph; Determining a correction value such that the coordinates of the at least one point in the first 3D map correspond to the coordinates of the point in the aerial map matching the point; Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And generating a second 3D map based on the second graph.

A map generating device according to another embodiment,

Processor; And a memory for storing at least one instruction, wherein the processor generates, according to the at least one instruction, a first graph including nodes and constraint factors related to sensing data for a moving object, and the generated A first 3D map is generated based on the first graph, and a correction value is determined so that the coordinates of at least one point in the first 3D map correspond to the coordinates of a point in the aerial map matching the point. In addition, a second graph to which constraint factors related to the determined correction value are added to the first graph may be determined, and a second 3D map may be generated based on the second graph.

The apparatus for generating a map and a method for generating a 3D map according to the exemplary embodiment can quickly generate a 3D map with high precision at low cost.

The map generating apparatus and the method for generating a 3D map according to the exemplary embodiment may generate a 3D map providing 3D information of a portion that cannot be identified by an aerial photograph.

The map generating apparatus according to an embodiment and a method of generating a 3D map by the method of generating a 3D map by compensating for each insufficient portion of the data acquired at the aerial level and the data acquired at the ground level are mutually complementary to generate a 3D map. High 3D maps can be created.

However, the effects that can be achieved by the map generating apparatus and the method for generating a 3D map according to an embodiment are not limited to those mentioned above, and other effects not mentioned are disclosed in the following description. It will be clearly understood by those of ordinary skill in the relevant technical field.

In order to more fully understand the drawings cited in the present specification, a brief description of each drawing is provided.
1 is a diagram illustrating a moving object and a map generating apparatus according to an exemplary embodiment.
2 is a flowchart illustrating a method of generating a 3D map according to an exemplary embodiment.
3 is a diagram illustrating a first graph according to an exemplary embodiment.
4 is an exemplary diagram illustrating a 3D submap corresponding to each node included in a first graph.
5 is a diagram illustrating a second graph according to an exemplary embodiment.
6 is a diagram for describing a method of determining a correction value through point-to-point matching according to an exemplary embodiment.
7 and 8 are diagrams for describing a method of determining a correction value through point-to-line matching according to an exemplary embodiment.
9 and 10 are diagrams for describing a method of determining a correction value through line-to-line matching according to an exemplary embodiment.
11 and 12 are diagrams for explaining a method of determining a correction value through marker matching, according to an exemplary embodiment.
13 is a diagram for describing a method of determining a correction value through template matching according to an exemplary embodiment.
14 is a block diagram illustrating a configuration of a map generating apparatus according to an exemplary embodiment.

In the present disclosure, various changes may be made and various embodiments may be provided, and specific embodiments are illustrated in the drawings, and will be described through detailed description. However, this is not intended to limit the present disclosure to a specific embodiment, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present disclosure.

In describing the embodiments, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter, the detailed description thereof will be omitted. In addition, numbers (eg, first, second, etc.) used in the description process of the embodiment are merely identification symbols for distinguishing one component from another component.

In addition, in the present specification, when one component is referred to as "connected" or "connected" to another component, the one component may be directly connected to the other component or may be directly connected, It should be understood that as long as there is no substrate to be used, it may be connected or may be connected via another component in the middle.

In addition, in the components expressed as'~ unit (unit)','module', etc. in the present specification, two or more components are combined into one component, or two or more components are divided into more subdivided functions. It can also be differentiated into. In addition, each of the components to be described below may additionally perform some or all of the functions that other components are responsible for in addition to its own main function, and some of the main functions that each component is responsible for are different. It goes without saying that it may be performed exclusively by components.

In addition, in the present specification,'global coordinates' indicates a location of a predetermined object based on an invariant origin, and may be referred to as'absolute coordinates'. The global coordinates of a given object are always constant.

In addition, in the present specification,'local coordinates' denotes a location of a predetermined object based on a variable origin, and may also be referred to as'relative coordinates'. The local coordinates of the predetermined object may vary depending on where the origin is determined.

Hereinafter, embodiments according to the technical idea of the present disclosure will be described in detail in order.

1 is a diagram illustrating a moving object 100 and a map generating apparatus 200 according to an exemplary embodiment.

The moving body 100 may be a vehicle, a robot, or a drone. Referring to FIG. 1, the moving body 100 may include at least one sensor. Sensing data according to the movement of the moving object 100 may be generated by using at least one sensor. For example, the moving object 100 is at least one sensor, and includes an IMU (Inertial Measurement Unit) sensor 101, a global positioning system (GPS) sensor 102, a wheel encoder sensor 103, and a light detection and ranging (LIDAR) sensor. ) May include a sensor 104 and the like. However, the sensor included in the moving body 100 is not limited thereto, and various types of sensors may be included in the moving body 100, or some of the sensors shown in FIG. 1 may not be included in the moving body 100. . At least one of the sensors may be located inside or outside the moving body 100.

The IMU sensor 101 may include an acceleration sensor and a gyro sensor (gyroscope) as an inertial measurement means. For example, as the rotation angle of the moving body 100, the IMU sensor 101 includes increments of angular velocity in each of the forward direction (Roll axis direction), the right direction of the forward direction (pitch axis direction), and the gravity direction (yaw axis direction). Sensing data such as a speed increment value of the moving object 100 may be generated. Also, the IMU sensor 101 may generate IMU odometry by using the sensing data. That is, it is possible to measure the amount of change in acceleration and angular velocity from the initial condition, and provide the moving direction and the moving distance of the moving object 100 as IMU odometry.

The GPS sensor 102 may communicate with a satellite to provide coordinates on the ground of the mobile body 100 as sensing data. For example, the GPS sensor 102 may provide latitude, longitude, and elliptic height as a geodetic coordinate of the WGS84 coordinate system.

The wheel encoder sensor 103 measures the number of revolutions of a wheel provided in the moving body 100. When the wheel encoder sensor 103 is used, the moving distance and the moving speed of the moving object 100 can be measured.

The LIDAR sensor 104 radiates a laser pulse or the like to the periphery, and then measures a time to be reflected and returned to generate depth information about points around the moving object 100. Here, the LIDAR sensor 104 may be used to measure the amount of rotation of the moving body 100 or to generate Lidar Odometry. Therefore, using the LIDAR sensor 104, it is possible to measure the moving direction and the moving distance of the moving object 100. As the LIDAR sensor 104, SICK's MLS-511, Velodyne's VLP-16, etc. may be used, but the present invention is not limited thereto.

Additionally, the moving object 100 may further include a camera, and an image corresponding to the movement of the moving object 100 may be captured using a camera. In this case, a visual odometry of the moving object 100 may be generated from the camera.

The map generating apparatus 200 receives sensing data generated by a sensor included in the moving object 100 and generates a first graph based thereon. Here, the first graph means a graph before being updated according to an aerial map to be described later. Using the graph, it is possible to accurately predict the 6 degrees of freedom value of the moving object 100, that is, the coordinate value and the posture value of the moving object 100.

The coordinate value of the moving object 100 indicates a position on the 3D coordinate, and may include, for example, coordinate values on the x-axis, y-axis, and z-axis. In addition, the posture value of the moving body 100 may include a pitch value, a roll value, and a yaw value of the moving body 100.

The map generating apparatus 200 may be coupled to the mobile body 100 to move together with the mobile body 100, or may be separated from the mobile body 100 to receive sensing data from the mobile body 100 through a wired or wireless communication method.

2 is a flowchart illustrating a method of generating a 3D map according to an exemplary embodiment.

In step S210, the map generating apparatus 200 generates a first graph including nodes and constraint factors related to sensing data for the moving object 100. According to the first graph, it is possible to predict 6 degrees of freedom of the mobile 100 corresponding to each of the nodes. First, a first graph will be described with reference to FIG. 3.

3 is a diagram illustrating a first graph 300 according to an exemplary embodiment.

The map generating apparatus 200 may generate nodes according to the reference period, and may generate a graph by setting constraint factors for nodes based on sensing data synchronized with the reference period. Here, the map generating apparatus 200 may set any one of the sensing data as reference data, and may set the generation period of the reference data as the reference period. That is, nodes may be generated and included in the graph according to the generation period of the reference data. Also. The map generating apparatus 200 may include sensing data synchronized with the reference period as a constraint factor of a node in the graph. In one embodiment, the IMU odometry of the IMU sensor 101 may be determined as reference data.

In the above example, a case where the reference data has a constant generation period is described, but according to another example, the reference data may be collected according to a predetermined distance interval. In this case, the map generating apparatus 200 may generate a node corresponding to each of the reference data collected at each reference distance interval and include it in the graph. The map generating apparatus 200 may include sensing data synchronized with the sensing time point of the node generated at each reference distance interval as a constraint factor of the node in the graph.

Constraint factors related to sensing data can be classified into a unary factor and a binary factor. Here, the uni factor represents a measurement result at the time when sensing data is generated, and includes a coordinate value measured by the GPS sensor 102 and the like. For example, the map generating apparatus 200 may add the latitude and longitude coordinate values of the moving object 100 at a specific point in time generated by the GPS sensor 102 as a unique factor of a node.

The binary factor may be a factor binding relationships between different nodes. For example, the binary factor represents the relationship information between the measurement start point and the measurement end point of the sensing data as a measurement result, and the IMU odometry of the IMU sensor 101 corresponds to this. That is, information such as a moving direction and a moving distance using an acceleration change amount between a measurement start point and a measurement end point, not an acceleration value at a specific point in time, may correspond to a binary factor. This information may be expressed as relative pose (position and posture) information.

As shown in FIG. 3, the map generating apparatus 200 may first generate the first node 310 at a time point t=0, and the initial condition of the first node 310 is set to the first node 310 It can be set to the unary factor 311 of. Thereafter, the second node 320 may be created at a time point t=5, and a binary factor 315 connecting the first node 310 and the second node 320 may be set. Here, the reference period may be 5, and the IMU odometry between t=0 and t=5 may be set as a binary factor. In this way, a node corresponding to the moving object 100 is newly created for each reference period, and a constraint factor corresponding to the sensing data is set in each node, so that the first graph 300 for the moving object 100 can be generated. have.

In one embodiment, after generating each of the nodes 310 and 320 corresponding to t=0 and t=5, when asynchronous sensing data corresponding to the time point t=4 is input to the map generating apparatus 200 There may be. Here, since the first graph 300 has already been generated up to the time point t=5, it is necessary to modify the graph in order to reflect the asynchronous sensing data corresponding to the time point t=4. As shown in FIG. 3, the map generating apparatus 200 may newly generate a third node 330 corresponding to t=4 in which asynchronous sensing data is generated, and a third node corresponding to t=4 ( In 330, a unitary factor 331 corresponding to asynchronous sensing data may be set. In addition, a binary factor 325 may be set between the second node 320 and the third node 330. For example, if the asynchronous sensing data generated at t=4 is GPS coordinates, after comparing the GPS coordinates at t=4 with the GPS coordinates at t=5, t=5 from the GPS coordinates at t=4 The binary factor 325 may be set as a wheel encoder value for proceeding to GPS coordinates.

Referring back to FIG. 2, in step S220, the map generating apparatus 200 generates a first 3D map based on the first graph. The first 3D map may include coordinates of points identified by the LIDAR sensor 104 or the like while the moving object 100 moves. The coordinates here may be three-dimensional global coordinates. That is, the map generating apparatus 200 calculates 6 degrees of freedom value of the moving object 100 corresponding to each of the nodes based on the first graph, and calculates the local coordinates of points sensed around the moving object 100 as the moving object ( The first 3D map can be generated by changing to global coordinates according to the 6 degrees of freedom value of 100).

The map generating apparatus 200 may optimize the first graph in order to accurately predict the 6 degrees of freedom value of the moving object 100 based on the first graph.

로 나타낼 수 있다. 이 노드들은 특정한 구속 팩터들에 의해 연결 관계를 갖고, 이들 구속 팩터들로 구성된 그래프의 에러를 최소화하는 노드들의 6자유도 값이 산출될 수 있다. 노드 간의 연결 관계는 그래프의 구성요소 중 에지(edge)에 해당할 수 있다.To explain this in detail, a graph with n nodes

It can be expressed as These nodes have a connection relationship by specific constraint factors, and a value of 6 degrees of freedom of nodes that minimizes the error of the graph composed of these constraint factors can be calculated. The connection relationship between nodes may correspond to an edge among the elements of the graph.

Equation 1 below may be used to optimize the graph.

[Equation 1]

및

각각은 시간 i, j에서의 노드를 의미하고, 센서들로부터 측정한 노드의 6자유도 값과 그래프 상에서의 노드의 6자유도 값 사이의 오차가

로 표현된다. 그래프의 최적화는 오차

를 최소화하는 그래프를 산출하는 것이다. 여기서,

는 센싱 데이터의 정보 매트릭스(information matrix)이다. 그래프의 최적화는 Gauss-Newton Method 또는 Levenberg-Marquardt Method 등의 비선형 최적화 방법을 통해 이루어질 수 있다.In Equation 1 above

And

Each represents a node at times i and j, and the error between the node's 6 degrees of freedom value measured from the sensors and the node's 6 degrees of freedom value on the graph is

Is expressed as Graph optimization error

It is to produce a graph that minimizes. here,

Is an information matrix of sensing data. The graph can be optimized through a nonlinear optimization method such as the Gauss-Newton Method or the Levenberg-Marquardt Method.

4 is an exemplary diagram showing a 3D submap corresponding to each node included in the first graph, based on sensing data obtained from the LIDAR sensor 104 at a time corresponding to each node. A 3D submap corresponding to the node may be generated. In this case, a value of 6 degrees of freedom of the moving object 100 for a specific viewpoint between the node and the node may be calculated through interpolation. A global map may be calculated by accumulating the sub-maps shown in FIG. 4.

Referring back to FIG. 2, in step S230, the map generating apparatus 200 may determine a correction value such that the coordinates of at least one point in the first 3D map correspond to the coordinates of at least one point in the aerial map. have. For example, the map generating apparatus 200 may determine a correction value for matching coordinates of points in the first 3D map with coordinates of points in the aerial map. The aerial map may include a map generated from data acquired by a vehicle such as a drone, an artificial satellite, or an airplane, for example, a photograph.

The first 3D map and the aerial map may consist of several points, and each point has a corresponding coordinate, specifically a global coordinate. As described above, the first 3D map is generated based on the sensing data sensed by the moving object 100 at the ground level. At the ground level, the satellite-based positioning data (eg GPS) is Since the reliability is low, coordinates of points in the first 3D map may be incorrect. However, at an aerial level higher than the ground level, the reliability of satellite-based positioning data is higher than the ground level. Accordingly, in an embodiment, a correction value for generating a high-precision 3D map is determined based on an aerial map with high reliability of coordinates.

The correction value is for correcting 6 degrees of freedom values of nodes calculated based on the first graph, and the correction value may also include a value of 6 degrees of freedom. This correction value may be determined based on an error between coordinates of matching points in the first 3D map and the aerial map. Targets that can be specified in real space, for example, markers drawn on a road (lanes, stop lines, crosswalks, traffic signs), etc., may be manually or automatically designated as matching points.

The correction values are point to point matching, point to line matching, line to line matching, marker matching or between the first 3D map and the aerial map. It may be calculated through template matching, etc., which will be described later with reference to FIGS. 6 to 12. In step S240, the map generating apparatus 200 adds constraints related to the correction value to the first graph. , Determine the second graph. Here, the second graph refers to a graph determined by newly including constraint factors in the first graph.

In an embodiment, the map generating apparatus 200 may add the correction value to the first graph as a uni factor of at least one node included in the first graph. The map generating apparatus 200 may select a node to be connected to the unary factor among nodes of the first graph in consideration of which submap the correction value is determined from. For example, if the correction value is determined from the M _t submap of FIG. 4, the map generating apparatus 200 may add the correction value to the first graph as a unary factor of the X _t node.

In addition, in an embodiment, the map generating apparatus 200 applies a value reflecting the correction value to a value of 6 degrees of freedom of at least one node included in the first graph, as a unary factor of at least one node. Can be added to The map generating apparatus 200 may select a node to be connected to the unary factor among nodes of the first graph in consideration of which submap the correction value is determined from. For example, if the determined correction value in M _t sub-map of FIG. 4, the map creation device 200 includes a first value reflecting the correction value to the value 6 freedom X _t node as oil Nourishing factor of X _t node Can be added to the graph.

In addition, in one embodiment, the map generating apparatus 200 calculates and calculates the distance between at least one node and the registration point based on the coordinates of the at least one node and the coordinates of the registration point in the first 3D map. A reliability that is inversely proportional to the set distance may be added to the first graph as an element of the unequal factor of at least one node. The matching point in the first 3D map is determined through a matching process with an aerial map.

In an embodiment, when a matching point is determined in the M _t submap shown in FIG. 4, the map generating apparatus 200 may add a uni factor to the Xt node based on the coordinates of the matching point. The unique factor may include a correction value and a reliability. For example, the unary factor can be expressed as follows.

Unary_factor (symbol_id(X_t), pose, reliability)

Here, symbol_id (x_t) corresponds to information of the node to which the unary factor is connected, pose corresponds to a correction value, and reliability corresponds to reliability.

The correction value is a value for correcting the coordinates of the Xt node. For example, it is assumed that the first matching point in the first 3D map and the second matching point in the aerial map are expected to be actually the same target (ex. lane) and match each other. In this case, the correction value may be a value set to add B-A to the coordinates of the Xt node so that the coordinates (A) of the first registration point are the same as the coordinates (B) of the second registration point. Since the global coordinates of the points in the first 3D map are calculated by the global coordinates of the node and the local coordinates of each point based on the node, a method of correcting the global coordinates of the node to correct the global coordinates of each point. You can choose.

If the graph is optimized to be 100% satisfied by the corresponding correction value (generally, since the number of constraint factors is large, it is unlikely that the graph will be completely optimized by one factor), the global coordinate of the Xt node. If is moved by BA, the coordinates (global coordinates) of the first registration point calculated based on the Xt node become A+(BA)=B, and are identical to the global coordinates (B) of the second registration point in the matched aerial map. Can be done.

Reliability can be a factor that indicates how strongly the binding force of the corresponding unary factor will be reduced when optimizing the graph. According to an example, the reliability may be set to be inversely proportional to a distance between a matching point, which is a basis for calculating a correction value, and a node to which a corresponding unary factor is connected. Accordingly, the map generating apparatus 200 includes a correction value based on the difference between the coordinates of the first registration point in the first 3D map and the coordinates of the second registration point in the aerial map, the first registration point and the X _t node. A reliability that is inversely proportional to the distance of is determined, and a uni-factor including such a correction value and the reliability may be added to the first graph as a uni-factor of the node X _t . Reliability may have a smaller value as the above-described distance increases. This is to reduce the influence on the node when optimizing the graph by considering that the accuracy will be lower as the correction value calculated using the matching point located farther from the node. A method of calculating the reliability according to an example may be expressed by Equation 2 below.

[Equation 2]

In Equation 2, Pi is the coordinate of the first matching point in the first three-dimensional map, which is the basis for determining the correction value of the node Xt, and Pi' is the second matching point in the aerial map matching the first matching point. The coordinates of, ms() are the matching score function between Pi and Pi', d() is the Euclidean distance between the node Xt and the matching point Pi, and n is the number of matching points.

The matching score may be an index indicating how accurately two matching points are matched. The matching score may be a value that acts like a residual in statistics. For example, as the distance difference between the first matching point and the second matching point increases, the matching score may be calculated lower. According to another example, as the degree to which the patterns of the first matching point and the second matching point coincide is higher, the matching score may be calculated higher. The method of calculating the matching score is not limited thereto.

는 해당 신뢰도 값의 스케일을 조정하기 위한 값이다.According to Equation 2, reliability is inversely proportional to the average of the distances between the node Xt and the first matching points, and is calculated to be proportional to the matching score. Therefore, the larger the average distance, the lower the reliability, and the larger the average matching score, the higher the reliability.

Is a value for adjusting the scale of the reliability value.

According to an example, under the assumption that the difference between the coordinates Pi of the first matching point and the coordinates Pi of the second matching point is not large, d(Xt-Pi) in Equation 2 above is d(Xt-Pi' ) Or d(Xt-Pi"). Pi" may be another value calculated using Pi and Pi', for example an average value.

As will be described later, when a plurality of matching points are determined in the first 3D map, the reliability of at least one node is inversely proportional to the average value of the distances between each matching point. It can be set as one element. In addition, when a matching line is determined in the first 3D map, the reliability inversely proportional to the average value of the distances between each of the points included in the matching line and at least one node is one of the unity factor of at least one node. Can be set as an element. In addition, when a plurality of registration markers are determined in the first 3D map, the reliability of at least one node is inversely proportional to the average value of the distances between each of the points included in the plurality of registration markers and at least one node. It can be set as an element of the unary factor.

In addition, in an embodiment, the map generating apparatus 200 may add coordinates of matching points in the aerial map to the first graph as landmarks of a plurality of nodes. A landmark is a kind of binary factor and can function as a constraint factor for a plurality of nodes. A landmark according to an example may be added to the first graph as a bearing-range factor.

For example, assuming that a specific "lane" is designated as a matching point, the global coordinates of the corresponding lane in the first 3D map may be calculated based on the first node or the second node. . However, if the global coordinates of the first node and the second node are incorrect, the global coordinates of the corresponding lane can be calculated differently depending on which node is the reference. A landmark according to an example imparts a constraint that a corresponding lane corresponds to the same object, that is, a constraint that the global coordinates are the same. For example, when the first node and the second node refer to the same "lane as a landmark, the global coordinate of the lane calculated based on the first node and the global coordinate of the lane calculated based on the second node are the same." This is the same as the constraint given to the graph. If the landmark value of the coordinates on the aerial map of the corresponding lane (assuming that the coordinates of the aerial map are more accurate) are included in the landmark, "the lane calculated based on the first node In addition to the condition that the coordinates of and the coordinates of the lane calculated based on the second node are the same", it is possible to constrain what value such coordinates should be.

The map generating apparatus 200 may select a plurality of nodes to be connected to the landmark among nodes of the first graph. For example, the map generating apparatus 200 may select a predetermined number of nodes in an order close to the matching point, and add a landmark commonly viewed by the selected predetermined number of nodes to the first graph as a constraint factor. The landmark may include coordinates of the registration point in the aerial map.

Adding a landmark to the graph as above means that the specific landmark viewed from the first node and the specific landmark viewed from the second node are the same object, that is, the constraint that the global coordinates of the corresponding landmark are the same in the graph. Is to add. For example, when optimizing a graph in which information on the first landmark is added as a binary factor between a first node and a second node, the graph is based on the global coordinates of the first landmark and the second node calculated based on the first node. It is optimized in a direction in which the global coordinates of the first landmark calculated as the reference are the same.

5 is a diagram illustrating a second graph 500 according to an exemplary embodiment.

Referring to FIG. 5, compared to the first graph 300 illustrated in FIG. 3, the unequal factor 521 of the second node 320 when t=5, and the third node 330 when t=4. ), and a landmark 525 for the second node 320 and the third node 330 have been added.

That is, the second graph 500 may be determined by adding constraint factors determined based on the aerial photograph to the first graph 300.

Referring back to FIG. 2, in step S250, the map generating apparatus 200 generates a second 3D map based on the second graph.

Specifically, the map generating apparatus 200 may generate a second 3D map by determining an updated 6 DOF value of each of the nodes through optimization of the second graph. Specifically, the map generating apparatus 200 may generate a second 3D map by calculating updated coordinate information of points sensed by the mobile body 100 based on the updated 6 degrees of freedom value of each of the nodes. . For example, the map generating apparatus 200 uses the updated 6 degrees of freedom value of each of the nodes and the local coordinates of each point based on the node, and stores updated global coordinate information of points sensed by the moving object 100. A second 3D map including points having calculated and calculated global coordinates may be generated.

Hereinafter, a method of determining a correction value for generating a second graph will be described in detail with reference to FIGS. 6 to 12.

6 is a diagram for describing a method of determining a correction value through point-to-point matching according to an exemplary embodiment.

The map generating device 200 includes first matching points P1, P2, P3, P4 and second matching points P1' and P2 that match each other in the first 3D map 610 and the aerial map 630. ', P3', P4') can be determined. The first 3D map 610 shown in FIG. 6 may be any one submap shown in FIG. 4. For example, only lanes and road signs extracted based on reflectivity from any one sub-map shown in FIG. 4 may be displayed on the first 3D map 610 of FIG. 6.

6 illustrates that there are a plurality of first matching points P1, P2, P3, and P4 and second matching points P1', P2', P3', P4', but the first matching points P1 and P2. , P3, P4) and the second matching point (P1', P2', P3', P4') may each have one number.

The first matching points P1, P2, P3, and P4 and the second matching points P1', P2', P3' and P4' may be determined manually or automatically. As an example, the map generating device 200 includes a first matching point (P1, P2, P3, P4) and a first matching point (P1, P2, P3, P4) matching each other among points in the first 3D map 610 and the aerial map 630 from the administrator. 2 Matching points P1', P2', P3', and P4' can be input. As another example, the map generating apparatus 200 may match each of the points in the first 3D map 610 and the aerial map 630 and the surrounding information using template matching or a local descriptor. One matching point P1, P2, P3, and P4 and second matching points P1', P2', P3', P4' matching thereto may be determined.

The map generating device 200 includes coordinates of the first registration points P1, P2, P3, and P4 in the first three-dimensional map 610 and the second registration points P1 ′ and P2 ′ in the aerial map 630, A difference value between coordinates of P3' and P4') may be determined as a correction value. As described above, a determined correction value or a value in which the correction value is reflected in the coordinates of the node may be added to the first graph as a unique factor. The map generating apparatus 200 may add the correction value or a value reflecting the correction value to the coordinates of the corresponding node as the unity factor of the node closest to the first matching point (P1, P2, P3, P4) to the first graph. I can. Also, the coordinates of the second matching points P1 ′, P2 ′, P3 ′, and P4 ′ may be added to the first graph as landmarks for a plurality of nodes.

In addition, the map generating apparatus 200 determines a node corresponding to the first matching point P1, P2, P3, P4, and determines the distance between the determined node and the first matching point P1, P2, P3, P4. An inversely proportional reliability may be added to the first graph as an element of the unitary factor of the corresponding node.

6, in the first 3D map 610 and the aerial map 630, a plurality of first registration points P1, P2, P3, P4 and a plurality of second registration points P1 ', P2', P3', P4') is determined, the coordinates of the plurality of first matching points P1, P2, P3, P4 and the plurality of second matching points P1', P2', P3', The average of the difference values of each of the coordinates of P4') may be determined as a correction value.

When the second graph is determined based on the correction value, the updated 6 degrees of freedom value of each node is determined based on the second graph, and a point in the second 3D map determined based on the updated 6 degrees of freedom value The coordinates of these correspond to the coordinates of the points on the aerial map. In other words, the coordinates of the first matching point P1, P2, P3 or P4 calculated based on any one node are (1, 1, 1), and the second matching point P1', P2', P3' Or, if the coordinates of P4') are (2, 2, 2), the position of the node is adjusted through the addition of a constraint factor, so that the first matching point (P1, P2, P3, or P4) calculated based on the node is This is to make the coordinates (2, 2, 2) (or a coordinate close to it).

7 and 8 are diagrams for describing a method of determining a correction value through point-to-line matching according to an exemplary embodiment.

On the other hand, identifiable markers (lanes, road signs, etc.) on the road can not only be specified as points, but are also expressed as lines. In this case, the map generating apparatus 200 may determine a matching point P1, P2, P3 and a matching line L'that match each other in the first 3D map 610 and the aerial map 630. .

The matching points P1, P2, P3 and the matching line L'may be determined manually or automatically. As an example, the map generating apparatus 200 inputs a matching point P1, P2, P3 and a matching line L'that match each other in the first 3D map 610 and the aerial map 630 from the manager. I can receive it. As another example, the map generating apparatus 200 analyzes the first 3D map 610 and the aerial map 630 to determine matching points P1, P2, P3 and matching lines L'. May be.

When the registration points P1, P2, P3 in the first 3D map 610 are transformed (or updated) according to the correction value, the coordinates of the transformed registration points are the aerial map 630 It should be located on my matching line (L'). Accordingly, as shown in FIG. 8, the map generating apparatus 200 determines a correction value at which the sum of the distance d between the registration points P1, P2, and P3 and the registration line L'becomes minimum. I can. Here, the correction value may have 6 degrees of freedom.

When the coordinates of the registration points (P1, P2, P3) calculated based on any one node are not on the registration line (L'), any one node through the addition of a constraint factor corresponding to the correction value It is possible to change the 6 degrees of freedom value of, and accordingly, the coordinates of the registration points P1, P2, and P3 are identical or similar to the coordinates of the points included in the registration line L'of the aerial map.

9 and 10 are diagrams for describing a method of determining a correction value through line-to-line matching according to an exemplary embodiment.

The map generating apparatus 200 includes first matching lines L1, L2, L3, L4, L5 and second matching lines L1' that match each other in the first 3D map 610 and the aerial map 630. , L2', L3', L4', L5') can be determined. 9 illustrates a plurality of first matching lines L1, L2, L3, L4, and L5 and a plurality of second matching lines L1', L2', L3', L4', and L5', but one The first matching line and one second matching line may be determined.

The first matching lines L1, L2, L3, L4, and L5 and the second matching lines L1', L2', L3', L4', L5' may be determined manually or automatically. As an example, the map generating device 200 includes first matching lines L1, L2, L3, L4, and L5 matching each other in the first 3D map 610 and the aerial map 630 from the administrator. Matching lines L1', L2', L3', L4', and L5' may be input. As another example, the map generating apparatus 200 analyzes the first 3D map 610 and the aerial map 630 to match the first matching lines L1, L2, L3, L4, L5 and the second Matching lines L1', L2', L3', L4', and L5' may be determined.

When the first matching line (L1, L2, L3, L4, L5) in the first 3D map 610 is transformed (or updated) according to the correction value, the transformed first matching line ( L1, L2, L3, L4, L5) must coincide with the second matching lines L1', L2', L3', L4' and L5'. Accordingly, as shown in FIG. 10, the map generating apparatus 200 includes first matching lines L1, L2, L3, L4, and L5, specifically, points included in the first matching line and the second matching line. A correction value at which the sum of the distances d between (L1', L2', L3', L4', L5') becomes minimum may be determined. Here, the correction value may have 6 degrees of freedom. In one embodiment, when a plurality of first matching lines (L1, L2, L3, L4, L5) and a plurality of second matching lines (L1', L2', L3', L4', L5') are determined, respectively An average of the correction values determined from the first matching line and the second matching line of may be determined as a final correction value.

The six degrees of freedom value of any one node may be changed through the addition of a constraint factor corresponding to the correction value, and accordingly, the points included in the first matching lines L1, L2, L3, L4, and L5 The coordinates coincide or become similar to coordinates of points included in the second matching lines L1', L2', L3', L4', and L5' of the aerial map 630.

11 and 12 are diagrams for explaining a method of determining a correction value through marker matching, according to an exemplary embodiment.

There are various markers such as traffic signs on the road. The map generating apparatus 200 includes first markers M1, M2, M3, M4 and second markers M1' and M2' that match each other in the first 3D map 610 and the aerial map 630. , M3', M4') can be determined. The first markers M1, M2, M3, and M4 and the second markers M1', M2', M3', M4' may be determined manually or automatically. As an example, the map generating device 200 includes first markers M1, M2, M3, M4 and second markers that match each other in the first 3D map 610 and the aerial map 630 from the administrator. (M1', M2', M3', M4') can be input. As another example, the map generating apparatus 200 analyzes the first 3D map 610 and the aerial map 630 to match the first markers M1, M2, M3, M4 and second markers. (M1', M2', M3', M4') can also be determined.

Since the coordinates of the points included in each of the first markers M1, M2, M3, M4 and the coordinates of the points included in each of the second markers M1', M2', M3', M4' must match, , The map generating device 200 includes coordinates of points included in each of the first markers M1, M2, M3, and M4, respectively, in the second markers M1', M2', M3', and M4'. It is possible to determine a correction value to match or be similar to the coordinates of the points. Points included in the first markers M1, M2, M3, and M4 are blocks representing each of the first markers M1, M2, M3, and M4, for example, the square blocks B1 and M4 shown in FIG. B2, B3, B4) may be the central point (C1, C2, C3, C4), and likewise, the points included in the second markers M1', M2', M3', M4' are second markers. It may be a block representing each, for example, midpoint points C1 ′, C2 ′, C3 ′, and C4 ′ of the rectangular blocks B1 ′, B2 ′, B3 ′, and B4 ′ shown in FIG. 12. As shown in FIG. 12, coordinates of points C1, C2, C3, and C4 included in the first markers M1, M2, M3, and M4 are referred to as second markers M1', M2', and M3. A three-dimensional coordinate value t and a posture value R for matching the coordinates of the points C1', C2', C3', and C4' included in', M4') may be determined as correction values. The central point may mean the center of gravity of the block.

In addition, when determining the correction value, the map generating apparatus 200 determines the coordinates of points included in each of the first markers M1, M2, M3, and M4 as the second markers M1', M2', and M3'. , M4') connecting the points (C1, C2, C3, C4) included in each of the first markers (M1, M2, M3, M4) while matching or similar to the coordinates of the points included in each The inner angles θ1, θ2, θ3, θ4 of the first figure F are the points C1', C2', and C3 included in each of the second markers M1', M2', M3', and M4'. A correction value that matches or becomes similar to the inner angles θ1 ′, θ2 ′, θ3 ′, and θ4 ′ of the second figure F′ connecting the “, C4 ′) may be determined. The correction value may be a value of 6 degrees of freedom including a movement amount and a rotation amount.

13 is a diagram for describing a method of determining a correction value through template matching according to an exemplary embodiment.

In an embodiment, the map generating apparatus 200 may determine a correction value through template matching. The positions of the points included in the first 3D map 610 in the first 3D map 610 are the same as the positions in the aerial map 630 of the points included in the aerial map 630 Therefore, the difference between pixel values corresponding to the points of the first 3D map 610 and the pixel values corresponding to the points of the aerial map 630 should be minimized. Accordingly, the map generating apparatus 200 corrects the sum of differences between pixel values corresponding to points of the first 3D map 610 and pixel values corresponding to the points of the aerial map 630 to be minimized. A value may be determined, and a constraint factor may be added to the first graph based on the determined correction value. As an example, as shown in FIG. 13, the map generating apparatus 200 uses a first 3D map 610 through an image 650 in which the first 3D map 610 and the aerial map 630 are overlapped. A correction value that minimizes a difference between pixel values corresponding to points of) and pixel values corresponding to points of the aerial map 630 may be determined.

Depending on the implementation, the aforementioned point-to-point matching, point-to-line matching, line-to-line matching, marker matching, or template matching may be performed based on deep learning.

14 is a block diagram illustrating a configuration of a map generating apparatus 200 according to an exemplary embodiment.

Referring to FIG. 14, the map generating apparatus 200 includes a memory 1410, a communication unit 1430, and a control unit 1450. The memory 1410, the communication unit 1430, and the control unit 1450 may be implemented with at least one processor and may operate according to instructions stored in the memory 1410.

The communication unit 1430 receives sensing data from the mobile body 100. The communication unit 1430 may receive sensing data through a wired network and/or a wireless network.

The controller 1450 generates a 3D map based on the sensing data. Specifically, the controller 1450 generates a first graph including nodes and constraint factors, and generates a first 3D map based on the first graph. Since the coordinates of the points included in the first 3D map may be different from the actual coordinates, the controller 1450 is a correction value such that the coordinates of the points in the first 3D map correspond to the coordinates of the points in the aerial map. And determine a second graph to which constraint factors related to the correction value are added. The controller 1450 generates a second 3D map based on the second graph. The aerial map may be received by the communication unit 1430 from another device through a wired network and/or a wireless network, or may be stored in the memory 1410.

Since a method of generating a graph, a method of determining a correction value, and a method of generating a 3D map have been described above, detailed descriptions are omitted.

Meanwhile, in the above-described embodiments of the present disclosure, the aerial map may be a two-dimensional, 2.5-dimensional, or three-dimensional map. According to an example, the aerial map may be 2.5 dimensional, where 2.5 dimensional data means data in which each (x, y) coordinate of two dimensions has one height value (z). The map generating device 200 is an n-dimensional correction value when the point coordinates of the ground level map (for example, the above-described 3D map) and the aerial map generated based on the moving object 100 are the same n-dimensional Can be calculated. On the other hand, when the point coordinates of the ground level map are n-dimensional and the point coordinates of the aerial map are m-dimensional, the map generating device 200 may calculate the correction value using only the coordinate values of the lower dimension. However, since 2.5D has all x, y, and z coordinate values, it can be handled according to 3D.

Meanwhile, the above-described embodiments of the present disclosure can be written as programs that can be executed on a computer, and the written programs can be stored in a medium.

The medium may be one that continuously stores a program executable by a computer, or temporarily stores a program for execution or download. In addition, the medium may be a variety of recording means or storage means in a form in which a single piece of hardware or several pieces of hardware are combined. The medium is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks, and And a ROM, RAM, flash memory, and the like, and may be configured to store program instructions. In addition, examples of other media include an app store that distributes applications, a site that supplies or distributes various software, and a recording medium or storage medium managed by a server.

Above, the technical idea of the present disclosure has been described in detail with reference to a preferred embodiment, but the technical idea of the present disclosure is not limited to the above embodiments, and those having ordinary knowledge in the art within the scope of the technical idea of the present disclosure Various modifications and changes are possible by the user.

100: moving object
200: map generating device
1410: memory
1430: Ministry of Communications
1450: control unit

Claims (19)

Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which new constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The step of determining the second graph,
Adding the correction value or a value reflecting the correction value to the degree of freedom value of at least one node included in the first graph as a new constraint factor of at least one node included in the first graph A method of generating a map comprising the step of.
The method of claim 1,
The step of determining the correction value,
And determining a difference between the coordinates of the first registration point in the first multidimensional map and the coordinates of the second registration point in the aerial map matched with the first registration point as the correction value. How to create a map.
The method of claim 2,
The step of determining the second graph,
Calculating a distance between the at least one node and the first matching point based on the coordinates of the at least one node and the coordinates of the first matching point; And
And adding a reliability inversely proportional to the calculated distance as an element of the new constraint factor.
The method of claim 2,
The step of determining the second graph,
And adding the coordinates of the second matching point in the aerial map as landmarks of a plurality of nodes to the first graph.
The method of claim 1,
The step of determining the correction value,
And determining a correction value that minimizes the distance between the registration points in the first multidimensional map and the registration line in the aerial map.
The method of claim 1,
The step of determining the correction value,
And determining a correction value for minimizing the distance between points included in the registration line in the first multidimensional map and the registration line in the aerial map.
The method of claim 1,
The step of determining the correction value,
And determining a correction value for matching coordinates of points included in the registration markers in the first multidimensional map to coordinates of points included in the registration markers in the aerial map. How to create a map.
The method of claim 1,
The step of determining the correction value,
And determining a correction value that minimizes a difference between pixel values corresponding to points of the first multidimensional map and pixel values corresponding to points of the aerial map.
The method of claim 1,
The step of generating the first multidimensional map,
Determining a degree of freedom value of each of the nodes through optimization of the first graph; And
And generating the first multidimensional map by calculating coordinates of points sensed by the moving object based on the values of the degrees of freedom of each of the nodes.
The method of claim 9,
Generating the second multi-dimensional map,
Determining an updated degree of freedom value of each of the nodes through optimization of the second graph; And
And generating the second multidimensional map by calculating updated coordinates of points sensed by the moving object based on the updated degrees of freedom value of each of the nodes.
The method of claim 1,
The generating step,
And generating the second multidimensional map by optimizing a second graph in a direction that satisfies both the constraint factors already included in the first graph and the new constraint factor.
A program stored in a medium for executing the method of generating a map according to any one of claims 1 to 11 in combination with hardware.
Processor; And
Including a memory for storing at least one instruction,
The processor according to the at least one instruction,
Create a first graph including nodes and constraint factors related to sensing data for a moving object,
Generate a first multidimensional map based on the generated first graph,
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point,
Determine a second graph to which new constraint factors related to the determined correction value are added to the first graph,
Generate a second multidimensional map based on the second graph,
When determining the second graph, the correction value or a value reflecting the correction value to the degree of freedom value of at least one node included in the first graph is used as a new constraint factor of the at least one node. Map generating device, characterized in that to add to.
Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The determining of the correction value includes determining a correction value that minimizes the distance between the registration points in the first multi-dimensional map and the registration line in the aerial map.
Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The determining of the correction value includes determining a correction value for minimizing the distance between points included in the registration line in the first multidimensional map and the registration line in the aerial map. Method of creation.
Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The step of determining the correction value,
And determining a correction value for matching coordinates of points included in the registration markers in the first multidimensional map to coordinates of points included in the registration markers in the aerial map. How to create a map.
Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The step of determining the correction value,
And determining a correction value that minimizes a difference between pixel values corresponding to points of the first multidimensional map and pixel values corresponding to points of the aerial map.
Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The step of generating the first multidimensional map,
Determining a degree of freedom value of each of the nodes through optimization of the first graph; And
And generating the first multidimensional map by calculating coordinates of points sensed by the moving object based on the values of the degrees of freedom of each of the nodes.
Generating a first graph including nodes and constraint factors related to sensing data for the moving object;
Generating a first multidimensional map based on the generated first graph;
Determining a correction value such that the coordinates of at least one point in the first multidimensional map correspond to the coordinates of the point in the aerial map matched with the point;
Determining a second graph to which constraint factors related to the determined correction value are added to the first graph; And
And generating a second multidimensional map based on the second graph,
The generating step,
And generating the second multidimensional map by optimizing a second graph in a direction that satisfies both the constraint factors already included in the first graph and the added constraint factor.

Images