JP7448945B2 - 3D map generation method and 3D map generation device - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act On May 23, 2020, the abstract of the lecture will be posted on the website of The 30th IEEE Intelligent Vehicles Symposium (https://iv2019.org/program/). Published by distribution at The 30th IEEE Intelligent Vehicles Symposium on June 8th Presented at The 30th IEEE Intelligent Vehicles Symposium on June 10th, 2020

The present invention relates to a three-dimensional map generation method and a three-dimensional map generation device, and particularly relates to a technique for generating a three-dimensional map for an autonomous vehicle.

Highly accurate three-dimensional maps are necessary to enable safe autonomous driving of vehicles. Highly accurate positioning is possible using GNSS (Global Navigation Satellite System), but not only is it expensive, but it can also be used in long tunnels and densely packed trees. In areas where there are many tall buildings, satellite signals may be deflected or obstructed, making it difficult to generate highly accurate maps. If these issues are not resolved, a significant amount of time, data and money will be lost and dangerous maps will be generated that could cause autonomous vehicles to drift and cause accidents. obtain.

Therefore, conventionally, SLAM (Simultaneous Localization And Mapping) has been applied to point clouds representing 3D distance images obtained from LiDAR (Light Detection and Ranging) mounted on a moving vehicle. A technique has been proposed that generates a map by executing it (see Patent Document 1).

JP2019-207220A

However, the technology of Patent Document 1 generates a map with a three-dimensional image using a point cloud, and it is difficult to determine the accurate global position (position on the XY plane) and accurate elevation (position on the Z plane). It does not generate a three-dimensional map with high positional accuracy.

More specifically, the technique of Patent Document 1 uses a point cloud to generate a map based on SLAM, but it is very complicated to implement. This is because a point cloud is used to represent the three-dimensional shape of the environment. Point clouds have extremely large data sizes, which creates a heavy load when processing long courses. Additionally, point clouds map vehicles and other dynamic objects and are therefore particularly affected by traffic flow in the Z direction. Moreover, the most important solution to obtain good results in SLAM is to use environmental features to correct the position error, but the point cloud is very sparse, and it is difficult to match two point clouds. It is unlikely that the error can be corrected by Furthermore, due to the sparsity of point clouds, it is necessary to use iterative matching techniques, which becomes a very expensive process in terms of time consumption.

Therefore, an object of the present invention is to provide a three-dimensional map generation method and a three-dimensional map generation device that generate a three-dimensional map with high positional accuracy.

In order to achieve the above object, a three-dimensional map generation method according to one embodiment of the present invention uses a point cloud obtained from LiDAR that is mounted on a vehicle and detects an environment including roads. a three-dimensional data generation step of generating three-dimensional data including an XY-plane map composed of image frames and a Z-plane map composed of elevation image frames indicating the elevation of pixels constituting the intensity image; a map generation step of generating a three-dimensional map by reducing positional errors in the three-dimensional data, and the map generation step includes performing graph SLAM (Simultaneous Localization and Mapping) on the XY plane map. By doing so, a graph SLAM is applied to the XY plane error reduction step for reducing the positional error in the XY plane and the Z plane map corresponding to the XY plane map whose positional error has been reduced by the XY plane error reduction step. and a Z-plane error reduction step of reducing a positional error in the Z-plane by executing the method.

In addition, in order to achieve the above object, a three-dimensional map generation device according to one embodiment of the present invention generates a ground surface including roads from a point cloud obtained from LiDAR that is mounted on a vehicle and detects an environment including roads. a three-dimensional data generation unit that generates three-dimensional data including an XY-plane map composed of a frame of an intensity image representing the intensity image, and a Z-plane map composed of a frame of an elevation image indicating the elevation of pixels constituting the intensity image; a map generation unit that generates a three-dimensional map by reducing positional errors in the generated three-dimensional data, and the map generation unit performs graph SLAM (Simultaneous Localization and Mapping) on the XY plane map. By performing It has a Z-plane error reduction unit that reduces positional errors on the Z-plane by executing SLAM.

The present invention provides a three-dimensional map generation method and a three-dimensional map generation device that generate a three-dimensional map with high positional accuracy.

FIG. 1 is a block diagram showing the configuration of a three-dimensional map generation device according to an embodiment. FIG. 2 is a flowchart showing the operation procedure of the three-dimensional map generation device according to the embodiment. FIG. 3 is a diagram showing an example of the appearance of a vehicle used to generate a three-dimensional map and a point cloud obtained from LiDAR installed in the vehicle. FIG. 4 is a diagram illustrating an example of an intensity image frame and an elevation image frame generated by the three-dimensional data generation unit. FIG. 5 is a diagram illustrating processing by a three-dimensional data generation unit that accumulates frames by dead reckoning. FIG. 6 is a diagram showing an example of an XY plane map generated by the three-dimensional data generation section. FIG. 7 is a diagram showing an example of a Z-plane map generated by the three-dimensional data generation section. FIG. 8 is a diagram showing an example of three-dimensional data generated by the three-dimensional data generation section. FIG. 9 is a diagram illustrating loop closure of graph SLAM performed by the three-dimensional data generation unit. FIG. 10 is a diagram illustrating phase correlation processing by the XY plane error reduction section. FIG. 11 is a diagram illustrating consistency evaluated using a cost function. FIG. 12 is a diagram illustrating coherency evaluated using a cost function. FIG. 13 is a diagram illustrating accuracy evaluated using a cost function. FIG. 14 is a diagram summarizing the processing of steps S12c and S13b in FIG. 2. FIG. 15A is a diagram illustrating the merits of the processing procedure shown in FIG. 14. FIG. 15B is another diagram illustrating the advantages of the processing procedure shown in FIG. 14. FIG. 16 is a diagram showing the traveling trajectory of the vehicle used in the experiment. FIG. 17 is a diagram showing the high positional accuracy of the XY plane map generated by the three-dimensional map generation device. FIG. 18 is a diagram showing a comparison between an XY plane map generated by a three-dimensional map generation device and an XY plane map generated using GNSS in a place where trees are densely populated. FIG. 19 is a diagram showing the high positional accuracy of the Z-plane map generated by the three-dimensional map generation device. FIG. 20 is a diagram showing a comparison between the Z-plane map generated by the three-dimensional map generation device and the Z-plane map generated using GNSS in the area where trees are densely arranged as shown in FIG. 18.

Hereinafter, embodiments of the present invention will be described in detail using the drawings. Note that each of the embodiments described below represents a specific example of the present invention. Numerical values, shapes, methods, components, connection forms of components, steps, order of steps, etc. shown in the following embodiments are merely examples, and do not limit the present invention. Further, in each figure, substantially the same configurations are denoted by the same reference numerals, and overlapping explanations may be omitted or simplified. Furthermore, in this specification, GNSS and GPS (Global Positioning System) mean GNSS/INS-RTK (GNSS-aided Inertial Navigation System-Real Time Kinematic).

FIG. 1 is a block diagram showing the configuration of a three-dimensional map generation device 10 according to an embodiment. The three-dimensional map generation device 10 generates three-dimensional data including an XY-plane map and a Z-plane map by acquiring a point cloud and positioning data from a LiDAR 11 and a GPS 12 mounted on a vehicle running on a road, respectively. It includes a three-dimensional data generation section 13 and a map generation section 14 that generates a three-dimensional map 15 by reducing positional errors in the generated three-dimensional data.

The three-dimensional data generation unit 13 generates an XY plane map consisting of frames of intensity images (that is, brightness images) representing the ground surface including roads and the elevation of pixels forming the intensity images from the point cloud obtained from LiDAR. Three-dimensional data including a Z-plane map made up of frames of elevation images shown is generated.

The map generation unit 14 executes graph-based SLAM (hereinafter referred to as graph SLAM) on the XY plane map of the three-dimensional data generated by the three-dimensional data generation unit 13, thereby generating a map in the XY plane. By executing graph SLAM on the XY plane error reduction unit 14a that reduces positional errors and the Z plane map corresponding to the XY plane map whose positional error has been reduced by the XY plane error reduction unit 14a, and a Z-plane error reduction unit 14b that reduces positional errors.

Note that the three-dimensional data generation unit 13 and the map generation unit 14 are functional blocks that execute data processing, and include a nonvolatile memory that stores a program, a volatile memory that temporarily stores data, a processor that executes the program, This is realized by a computer device equipped with an input/output interface for inputting/outputting data to/from peripheral devices.

FIG. 2 is a flowchart showing the operation procedure (that is, the three-dimensional map generation method) of the three-dimensional map generation device 10 according to the embodiment.

[Step S10 in FIG. 2]
First, the three-dimensional data generation unit 13 acquires a point cloud and positioning data from the LiDAR 11 and GPS 12 mounted on a vehicle traveling on a road, respectively (S10).

FIG. 3 shows an external appearance of the vehicle 20 used to generate the three-dimensional map ((a) in FIG. 3) and an example of a point cloud obtained from the LiDAR 11 mounted on the vehicle 20 ((b) in FIG. 3). ).

[Step S11 in FIG. 2]
Next, the three-dimensional data generation unit 13 generates three-dimensional data consisting of a connection of nodes from the point cloud and positioning data acquired from the LiDAR 11 and the GPS 12 (S11 in the figure). More specifically, the three-dimensional data generation unit 13 generates, from the point cloud obtained from the LiDAR 11, an intensity image frame representing the ground surface including roads and an elevation image frame indicating the altitude of pixels forming the intensity image. The three-dimensional data generation unit 13 then creates an XY-plane map by connecting frames of intensity images in a predetermined rectangular area, a Z-plane map by connecting frames of elevation images corresponding to the intensity images, and generates a Z-plane map by connecting frames of intensity images corresponding to the intensity images. A connection of a plurality of nodes each having a set of a map and a Z-plane map as a node is generated as three-dimensional data. At this time, the three-dimensional data generation unit 13 assigns the global position of a specific corner of the rectangular area indicated by the corresponding XY plane map to each of the plurality of nodes as an identifier.

More specifically, the three-dimensional data generation unit 13 acquires the initial position of each of the plurality of nodes from the GPS 12 mounted on the vehicle 20, and performs dead reckoning (dead reckoning) in which frames are accumulated based on the acquired initial position. (Dead-Reckoning) to generate an XY plane map and a Z plane map of the node.

In this way, instead of using a three-dimensional point distribution pattern, a brightness image of the road is used to represent the environment. Intensity images are useful because road landmarks are stationary and form highly visible patterns that can be used to correct position errors without being influenced by dynamic objects such as vehicle representations in the Z direction. be. Then, the application of graph SLAM is divided into two phases. One is the XY plane and the other is the Z plane. In the XY plane, road surfaces and landmarks are encoded very densely (at 30 cm) using luminance values, since LIDAR frames in large areas are accumulated based on a node strategy. This is a very important point to leverage environmental features instead of using one sparse point cloud. The brightness image of the nodes allows for significant correction of positional errors between nodes in the XY plane. Therefore, graph SLAM is applied on the XY plane to optimize the position of the brightness image of the node in the global coordinate system. As a result, the optimized node elevation image is moved in the XY plane. Therefore, nodes that share some area should be at the same Z level to represent the same environment as the real one, so a graph SLAM is applied to the elevation image to compensate for the position error in the Z plane. Applicable.

FIG. 4 is a diagram showing an example of an intensity image frame (FIG. 4(a)) and an elevation image frame (FIG. 4(b)) generated by the three-dimensional data generation unit 13. The three-dimensional data generation unit 13 generates an intensity image frame representing the ground surface including roads by cutting the point cloud obtained from the LiDAR 11 at a height of 30 cm from the ground surface.

FIG. 5 is a diagram illustrating processing by the three-dimensional data generation unit 13 that accumulates frames by dead reckoning. Here, an example of the position of the vehicle 20 ("Vehicle Position"), the road indicated by the point cloud obtained from LiDAR 11, the edge of the road ("Road edge"), the curb (Curbs), the frame ("One LIDAR Frame", An example of "1 ^st LIDAR Frame in Node" is shown.

As shown in equation 1 below, the three-dimensional data generation unit 13 takes the position obtained from the GPS 12 as the initial position, and accumulates (that is, concatenates) frames as differences with respect to the initial position. For example, an XY plane map and a Z plane map are generated that constitute nodes of a rectangular area including a road of about 200 m.

Here, XY ^DR _t indicates the position of the vehicle 20 on the XY plane map at time t, g (v _t , xy ^DR _t-1 ) indicates a function for calculating the position, and v _t indicates the position of the vehicle 20 on the XY plane map at time t. 20, xy ^DR _t-1 indicates the position of the vehicle 20 on the XY plane map at time (t-1), and Δt indicates the elapsed time from the previous position. Further, Z ^DR _t indicates the position of the vehicle 20 on the Z-plane map at time t, g (v _t , z ^DR _t-1 ) indicates a function for calculating the position, and z ^DR _t-1 is , indicates the position of the vehicle 20 on the Z-plane map at time (t-1).

The generation of such rectangular area nodes (XY plane map and Z plane map) is repeated for a plurality of nodes.

Thus, instead of using GNSS to estimate the vehicle's position, dead reckoning is used to accumulate LIDAR frames within each node to avoid local jumps that can occur due to low accuracy of satellite signals. do. Dead reckoning provides a very smooth vehicle trajectory within each node. Therefore, LIDAR frames are accumulated smoothly and road conditions within each node are maintained.

FIG. 6 is a diagram showing an example of an XY plane map generated by the three-dimensional data generation unit 13. The XY plane maps of the two nodes shown in FIGS. 6(a) and 6(b) are based on the position of the top left corner in the XY global coordinate system on the travel trajectory of the vehicle 20 ("Xmin" and " Ymax") is used as an identifier and combined to generate an overall XY plane map as shown in FIG. 6(c).

FIG. 7 is a diagram showing an example of a Z-plane map (here, an example of a Z-plane map of two nodes ((a) and (b) in FIG. 7) generated by the three-dimensional data generation unit 13. As shown in this figure, in this embodiment, the Z-plane map of each node is based on the average value of non-zero elevations in the Z global coordinate system in the traveling trajectory of the vehicle 20 (153 in (a) of FIG. 7). .22m in FIG. 7 (154.06m in b9) is used as the identifier.

FIG. 8 is a diagram showing an example of three-dimensional data generated by the three-dimensional data generation unit 13. As shown in this figure, the three-dimensional data is created by connecting the XY plane map and Z plane map (here, Node 5) of each node identified by an identifier (here, Node 1, 2,..., 9). This is a LiDAR map composed of More specifically, this figure includes two images: an intensity map (that is, an XY plane map) and an elevation map (that is, a Z plane map). That is, each node contains two types of images (elevation and intensity).

[Step S12 in FIG. 2]
Next, the XY plane error reduction unit 14a of the map generation unit 14 reduces the position error in the XY plane by executing graph SLAM on the XY plane map generated by the three-dimensional data generation unit 13 (Fig. 2 S12). Specifically, the following steps S12a to S12c are performed.

[Step S12a in FIG. 2]
The XY-plane error reduction unit 14a reduces the cumulative error in the XY-plane map by driving the vehicle 20 around a loop in the graph showing the connection of a plurality of nodes generated by the three-dimensional data generation unit 13 and observing the same points. The position error in the XY plane is reduced by closing the graph SLAM, which is a process for reducing the position error (S12a in FIG. 2).

FIG. 9 is a diagram illustrating loop closure of graph SLAM performed by the three-dimensional data generation unit 13. Suppose now that the vehicle 20 starts from position P0 and moves to positions P1, P2, . . . , P11 in order. In such a case, the start position P0 measured using the GPS 12 is accurate, but since frames as differences are subsequently accumulated, the positions close to the start position (for example, P1 and P2) are high. Although positional accuracy is maintained, the cumulative error increases as the vehicle 20 moves further. At position P9, the vehicle has returned to the area at position P3. This is called a loop closure event (or simply loop closure) in the XY plane because the positions between positions P3 and P9 form a loop according to the trajectory of the vehicle. However, due to the cumulative error after position P6, there is a relative position error between position P3 and P position 9. By matching the environmental characteristics detected at positions P3 and P9, the relative position error is corrected and the accuracy of the entire position in the loop (positions P4...P8) is improved. This is the reduction of position error in the XY plane using loop closure in graph SLAM.

In this embodiment, instead of using the position of the vehicle, the position of the top left corner of the global coordinate system is optimized. This is because it uses a node strategy that ensures smooth accumulation of LIDAR frames based on dead reckoning.

[Step S12b in FIG. 2]
When matching the intensity images of two nodes, such as the case of positions P3 and P9 in the loop closure, or a common area between two adjacent nodes, the XY plane error reduction unit 14a uses phase correlation to Position errors are reduced by performing a process (hereinafter referred to as "phase correlation process") that ensures the consistency of the texture and shape of the ground surface, including roads, indicated by the two XY plane maps (see Figure 2). S12b). Phase correlation is a method also called a phase-only correlation method, and is a method of performing image matching using a phase spectrum after Fourier transformation.

Each node encodes a large area of the actual road surface. Therefore, instead of using a 3D sparse point cloud, we can take advantage of the characteristics of the environment by representing landmarks very densely. With such a dense representation, wide road surface, and only 30 cm landmark representation, high-level image processing techniques can be used to match intensity images instead of traditional point cloud matching methods. Therefore, phase correlation can be used to estimate relative positions between nodes with respect to the intensity image. Furthermore, traditional point cloud matching techniques typically align two point clouds through an iterative search. This is due to the rarity of three-dimensional patterns and point clouds. Therefore, matching techniques on point clouds become a very expensive process in terms of time consumption, especially if the course of travel is long and contains many loop closures. On the other hand, the phase correlation in this embodiment is a non-iterative method, and the processing time is significantly reduced.

FIG. 10 is a diagram illustrating phase correlation processing by the XY plane error reduction unit 14a. Here, phase correlation processing is performed on the intensity images of two adjacent nodes A and B ((a) and (b) in FIG. 10) and the common area (area surrounded by a white frame) of these two intensity images. An example of an intensity image obtained using GNSS (FIG. 10(c)) and an accurate image obtained using GNSS (FIG. 10(d)) are shown. Phase correlation is robust and reliable for estimating relative positions at multiple nodes, and can reduce relative position errors.

More specifically, FIGS. 10(a) and 10(b) show two nodes A and B, respectively. FIG. 10(c) shows the matching result of two nodes based on phase correlation. Figure 10(d) shows a simple combination of Node A and Node B based on GNSS using the top left corner. Node A and Node B belong to an area where the GNSS can receive high quality signals from satellites, ie a highly accurate open sky area. Therefore, the combination of two nodes in FIG. 10(d) by GNSS can be considered as a reference (ground of truth). With phase correlation, the same map quality as GNSS is achieved without information about the upper left corners of nodes A and B. This results in a GNSS estimation comparable to the relative position between nodes in an open sky environment due to phase correlation. In addition, phase correlation helps provide a complete estimate of relative position in harsh environments where satellite signals may be deflected or jammed. This is because the phase correlation depends on the environmental features that are broadly/densely represented by the brightness images of the nodes. and because the environmental characteristics are static, fixed, and related to the quality of the satellite signal or the distance traveled.

[Step S12c in FIG. 2]
In addition, the XY plane error reduction unit 14a performs optimization to reduce position errors (relative position error and global position error) in the XY plane using a cost function that evaluates the following three things (S12c in FIG. 2). ). (1) Consistency, that is, the consistency of common areas in the XY plane maps of two different nodes. (2) Coherency, that is, continuity in the XY plane map of two different nodes. (3) Accuracy, that is, consistency between the position on the XY plane map and the position obtained by GPS.

FIG. 11 is a diagram illustrating consistency evaluated using a cost function. As shown in this figure, the intensity images of two nodes (here, nodes N ₁ and _N ) is evaluated as consistency using a cost function. Specifically, the consistency evaluation is expressed by the image edge E ^Img _i,j expressed by Equation 2 below. Note that an edge is a mathematical expression of evaluation.

Here, E ^Img _i,j denotes the image edge that evaluates the common area of two nodes N _i and N _j that compose the loop closure by compensating the relative position, and f() calculates the relative position where X _Ni denotes the X position at node N _i , X _Nj denotes the X position at node N _j , and X ^Img _Ni,j is the relative position of nodes N _i and N _j in the intensity image. Ω() is the covariance error representing the reliability of the phase correlation between nodes N _i and N _j . This covariance error is obtained from the correlation matrix that calculates the phase correlation. Such consistency evaluation contributes to reducing relative position errors in the XY plane map.

FIG. 12 is a diagram illustrating coherency evaluated using a cost function. As shown in this figure, the intensity images of the common area of two adjacent nodes (nodes N ₀ and N ₁ in this case) are determined by the continuity between the intensity images of those two nodes as coherency. Evaluated by a function. Continuity is, for example, a road context. Specifically, the coherency evaluation is expressed by a continuity edge E ^Img _i,i−1 expressed by the following equation 3.

Here, E ^DR _i,i-1 indicates a continuity edge that evaluates the common area of two adjacent nodes N _i and N _i-1 , f() is a function that calculates the relative position, X _Ni indicates the X position at node N _i , X _Ni-1 indicates the X position at node N _i-1 , and X ^Img _Ni,i-1 indicates the X position at node N i and _{N i in} the intensity image. ₋₁ , and Σ() denotes the covariance error that can be estimated from the variance of the velocity within the nodes N _i and N _i−1 of the vehicle 20.

Such coherency evaluation plays an important role in maintaining the continuity of the road context in two nodes, and contributes to reducing the relative position error in the XY plane map.

FIG. 13 is a diagram illustrating accuracy evaluated using a cost function. As shown in this figure, the consistency between the position on the XY plane map and the position obtained by GPS (here, the correspondence between the position on the XY plane map and the position obtained by GPS 12 for each of nodes N ₁ and N _{i + 2)} consistency) is evaluated as accuracy using a cost function. Specifically, the evaluation of accuracy is expressed by the anchoring edge XY ^GPS _i expressed by Equation 4 below.

Here, XY ^GPS _i indicates the anchoring edge for evaluating the accuracy of _node N _i , X _Ni indicates the X position of node N _i in the XY plane map, and X ^GPS _Ni Γ() indicates the covariance error obtained using GPS12. Such accuracy evaluation contributes to reducing global position errors in the XY plane map. A three-dimensional map needs to be generated and expressed in a global coordinate system (absolute coordinate system). For that purpose, GNSS-based anchoring edges are used.

[Step S13 in FIG. 2]
Next, the Z-plane error reduction unit 14b of the map generation unit 14 executes graph SLAM on the Z-plane map corresponding to the XY-plane map whose positional error has been reduced by the XY-plane error reduction unit 14a. The position error on the Z plane is reduced (S13 in FIG. 2). Specifically, the following steps S13a to S13b are performed.

[Step S13a in FIG. 2]
The Z-plane error reduction unit 14b performs a graph SLAM loop on the Z-plane map corresponding to the XY-plane map whose position error has been reduced by the XY-plane error reduction unit 14a using the same method as the XY-plane error reduction unit 14a. By performing the closure, the position error on the Z plane is reduced (S13a in FIG. 2).

[Step S13b in FIG. 2]
Further, the Z-plane error reduction unit 14b uses the same method as the XY-plane error reduction unit 14a to perform the following calculation on the Z-plane map corresponding to the XY-plane map whose position error has been reduced by the XY-plane error reduction unit 14a. Optimization is performed to reduce position errors (relative position error and global position error) on the Z plane using a cost function that evaluates at least one of the three (S13b in FIG. 2). (1) Consistency, that is, the consistency of common areas in the Z-plane maps of two different nodes. (2) Coherency, that is, continuity in the Z-plane map of two different nodes. (3) Accuracy, that is, consistency between the position on the Z-plane map and the position obtained by GPS.

Note that the consistency evaluation on the Z plane is indicated by the image edge Z ^Img _i,j shown by the following formula 5, and the optimization process for this purpose is based on the consistency evaluation for the XY plane map by the XY plane error reduction unit 14a. It is simultaneously realized through sexual processing. This is because the process of ensuring the consistency of common areas in the XY plane maps of two different nodes simultaneously ensures the consistency of common areas in the Z plane maps of those two nodes.

Each notation in this formula 5 corresponds to the notation in formula 2 in which X is replaced with Z. Such consistency evaluation contributes to reducing relative position errors in the Z-plane map.

Furthermore, the evaluation of coherency is expressed by a continuity edge Z ^DR _i,i−1 expressed by the following equation 6.

Each notation in this formula 6 corresponds to the notation in formula 3 in which X is replaced with Z. Such coherency evaluation plays an important role in maintaining the continuity of elevations at two nodes, and contributes to reducing relative position errors in the Z-plane map.

Furthermore, the accuracy evaluation is expressed by the anchoring edge Z ^GPS _i expressed by the following equation 7.

Each notation in this formula 7 corresponds to the notation in formula 4 in which X is replaced with Z. Such accuracy evaluation contributes to reducing global position errors in the Z-plane map.

[Step S14 in FIG. 2]
Finally, the map generation unit 14 uses the three-dimensional data whose positional errors have been reduced by the XY-plane error reduction unit 14a and the Z-plane error reduction unit 14b as a final three-dimensional map 15 on an external storage device, display, etc. output to the output device.

FIG. 14 is a diagram summarizing the processing of steps S12c and S13b in FIG. 2. As shown in this figure, the three-dimensional data (XY-plane map and Z-plane map) generated by the three-dimensional data generation section 13 is first processed by the XY-plane error reduction section 14a of the map generation section 14. After reducing the position error in the XY plane by performing optimization using a cost function on the surface map (S12c in FIG. 2), the The Z-plane error reduction unit 14b performs optimization using a cost function on the Z-plane map (projected) to reduce the positional error on the Z-plane (S13b in FIG. 2), and finally It is output as a three-dimensional map 15.

FIG. 15A is a diagram illustrating the merits of the processing procedure shown in FIG. 14. Here, a simple example of one node having relationships with a plurality of other nodes in the map of FIG. 14 is shown. These relationships affect the positions of nodes by applying graph SLAM to the XY plane and then applying graph SLAM to the Z plane. As shown in this figure, according to the three-dimensional map generation device 10 according to the present embodiment, optimization using a cost function that performs three types of evaluations on the XY plane map (as described above) is first performed. By minimizing the sum of the edges in the three types of XY planes), the XY offset for reducing the position error in the XY plane can be obtained (1st and 2nd rows in (a) of 15A(b)). Each node has two offsets in the x and y directions that are added to the top left corner coordinates. Therefore, it means moving the road surface in an absolute coordinate system based on such an XY graph SLAM offset. A map is then generated by combining the entire nodes.

Next, optimization is performed using a cost function that performs three types of evaluations on the Z-plane map corresponding to (that is, projected) the XY-plane map with the positional error reduced (the three types described above By minimizing the sum of edges in the Z plane), a Z offset for reducing the position error in the Z plane can be obtained (lines 3 and 4 in (a) of Fig. 15A, (c) of Fig. 15A). )). The procedure for modifying the map in the Z plane based on the XY graph SLAM offset is performed by applying graph SLAM to the elevation image. Therefore, a set of Z offsets is obtained, one elevation offset for each node, which is added to the entire pixel of the elevation image. This means moving the elevation image in the Z direction based on the Z graph SLAM offset. Next, all the nodes are combined to generate a Z-plane map. These steps result in a complete map.

This prevents the problem that errors in the XY plane are accumulated into errors in the Z plane, and improves positional accuracy that indicates the accurate global position (position in the XY plane) and accurate altitude (position in the Z plane). A three-dimensional map with a high degree of accuracy is generated.

FIG. 15B is another diagram illustrating the advantages of the processing procedure shown in FIG. 14. Here, the method shown in FIG. 14 is explained, focusing on the relationship between two nodes. FIG. 15B (a) shows an example of an intensity image and an elevation image of two nodes A and B. FIG. 15B (b) shows the problem of three-dimensional deviation between nodes in the XY plane and the Z plane. There is a relative position error in the x and y directions (XY plane) between the two nodes, as shown by the duplication of the road landmark in the XY plane map. Furthermore, the relative position error in the z-direction is also shown, just as the road surface at different z-levels is shown in the Z-plane map. FIG. 15B(c) shows the problem after applying graph SLAM to the XY plane of the intensity image. Relative position errors in the x and y directions have been corrected and ghost effects have been removed from the map. As a result, as shown in FIG. 15B (d), by applying graph SLAM to the elevation image, the elevation error (relative position error in the z direction) between the two roads can be made as small as possible. In the real world, two roads represent the same environment and are therefore at the same Z level.

Next, an experiment was conducted to generate a three-dimensional map using the three-dimensional map generating device 10 according to the present embodiment, which will be explained using figures.

FIG. 16 is a diagram showing the travel trajectory of the vehicle 20 used in the experiment. (a) of FIG. 16 shows a running trajectory and characteristic locations on the running trajectory, and (b) of FIG. 16 shows a map depicting the running trajectory. In this experiment, a three-dimensional map containing 305 nodes and 286 loop closure edges was generated from a point cloud obtained by driving a vehicle 20 over a total distance of 25 km in Korinbo-cho, Kanazawa City. As shown in FIG. 16(a), the travel trajectory includes areas lined with high-rise buildings and areas with dense trees.

FIG. 17 is a diagram showing the high positional accuracy of the XY plane map generated by the three-dimensional map generation device 10. (a) of FIG. 17 shows the standard deviation (m) in the X direction at each node of the XY plane map generated using GNSS (in other words, it shows the prediction accuracy of the X coordinate of the top left corner of each node), (b) of FIG. 17 shows the offset (m) in the X direction at each node of the XY plane map generated by the three-dimensional map generation device 10 (that is, the graph added to the X coordinate of the top left corner of each node Figure 17(c) shows the standard deviation (m) in the Y direction at each node of the XY plane map generated using GNSS (that is, the Y coordinate of the top left corner of each node). (d) of FIG. 17 shows the offset (m) in the Y direction at each node of the XY plane map generated by the three-dimensional map generation device 10 (that is, the (showing the graph SLAM offset added to the coordinates).

As can be seen by comparing (a) and (b) of FIG. 17, according to the three-dimensional map generation device 10, nodes with low accuracy in the X direction are detected by adding the graph SLAM offset, and these nodes are On the other hand, the global map accuracy in the absolute coordinate system is improved. In the Y direction as well, as can be seen by comparing FIGS. 17(c) and 17(d), the global map accuracy in the absolute coordinate system is improved.

FIG. 18 is a diagram showing a comparison between an XY plane map generated by the three-dimensional map generation device 10 and an XY plane map generated using GNSS in a place where trees are densely populated. Figure 18 (a) shows an image taken of a place where trees are densely populated, and Figure 18 (b) is an XY plane map generated using GNSS for the place shown in Figure 18 (a). FIG. 18C shows an example of the XY plane map generated by the three-dimensional map generation device 10 for the location shown in FIG. 18A.

As can be seen from Figure 18(b), in the XY plane map generated using GNSS, satellite signals are obstructed by dense trees, resulting in relative position errors such as overlapping road lanes and ghosts. ing. Additionally, a global position error occurs, as shown in two locations where the road differs from reality. On the other hand, as can be seen from FIG. 18(c), in the XY plane map generated by the three-dimensional map generating device 10, the relative position error and global position error in FIG. 18(b) are significantly and completely reduced. reduced.

FIG. 19 is a diagram showing the high positional accuracy of the Z-plane map generated by the three-dimensional map generation device 10. (a) of FIG. 19 shows the standard deviation (m) in the Z direction at each node of the XY plane map generated using GNSS (in other words, it shows the prediction accuracy of the Z coordinate of the top left corner of each node), (b) of FIG. 19 shows the offset (m) in the Z direction at each node of the Z plane map generated by the three-dimensional map generation device 10 (that is, the graph added to the Z coordinate of the top left corner of each node). SLAM offset). As can be seen by comparing FIGS. 19(a) and 19(b), according to the three-dimensional map generation device 10, low-accuracy nodes in the Z direction are detected by adding graph SLAM offsets, and these nodes are On the other hand, the precision of the global map in the absolute coordinate system is improved.

FIG. 20 is a diagram illustrating a comparison between the Z-plane map generated by the three-dimensional map generation device 10 and the Z-plane map generated using GNSS in the area where trees are densely populated as shown in FIG. 18. FIG. 20(a) shows intensity images of two nodes 69 and 239 to be compared. The dotted line indicates the trajectory of the vehicle. The coordinates of the rectangularly framed area at node 69 are projected to node 239 based on the graph SLAM offset in the XY plane. Therefore, the trajectory of the vehicle at node 69 is projected to node 239 based on the projected area coordinates.

FIG. 20(b) shows the elevation profile of the Z-plane map generated by the three-dimensional map generation device 10 for two different nodes (node 69 and node 239) at the same location (change in elevation due to movement of the vehicle 20). and the elevation profile of the Z-plane map generated using GNSS. In this figure, a broken line (NodeSlam69) and a dotted line (NodeSlam239) indicate the elevation profiles of the Z-plane map of node 69 and node 239, respectively, generated by the three-dimensional map generation device 10. The solid line (NodeGPS69) and the dashed-dotted line (NodeGPS69) indicate the elevation profiles of the Z-plane map of node 69 and node 239, respectively, generated using GNSS.

(c) of FIG. 20 shows the elevation at node 69 and the elevation at node 239 at each location of the movement trajectory for the Z-plane map generated by the three-dimensional map generation device 10 and the Z-plane map generated using GNSS. shows the difference between In this figure, the broken line (DiffSlam) indicates the difference between the elevation at node 69 and the elevation at node 239 at each location of the movement trajectory on the Z-plane map generated by the three-dimensional map generation device 10, and the solid line (DiffSlam) ) indicates the difference between the elevation at node 69 and the elevation at node 239 at each location of the movement trajectory on the Z-plane map generated using GNSS.

As mentioned above, since node 69 and node 239 are in the same location, the difference in elevation should be zero, but in the Z-plane map generated using GNSS, it is a large value, whereas in 3D The Z-plane map generated by the map generation device 10 has an extremely small value. Therefore, according to the three-dimensional map generation device 10 according to the present embodiment, optimization using a cost function is performed after applying graph SLAM, so an accurate Z-plane map with small positional error on the Z-plane is generated. It can be seen that is generated.

Note that the error in the Z plane map generated using GNSS in Figure 20 is after applying graph SLAM to the XY plane map, and the error in the XY plane map generated using GNSS is ignored. There is. Therefore, if the error in the XY plane map generated using GNSS is taken into account, the error in the Z plane map generated using GNSS will be a larger value.

As described above, the three-dimensional map generation method according to the present embodiment generates intensity image frames representing the ground surface including roads from a point cloud obtained from the LiDAR 11 mounted on the vehicle 20 and detecting the environment including roads. a three-dimensional data generation step (S10, S11) for generating three-dimensional data including an XY-plane map composed of A map generation step (S12, S13) that generates a three-dimensional map by reducing positional errors in the three-dimensional data obtained, and the map generation step executes graph SLAM on the XY plane map, Graph SLAM is performed on the Z-plane map corresponding to the XY-plane map whose positional error has been reduced by the XY-plane error reduction step (S12) that reduces positional errors in the XY-plane and the XY-plane error reduction step (S12). This includes a Z-plane error reduction step (S13) for reducing positional errors on the Z-plane.

As a result, after the position error in the XY plane is reduced, the process of reducing the position error is performed on the Z plane map corresponding to the XY plane map with the position error reduced, so the error in the XY plane is reduced. The problem of accumulated errors on the Z plane is suppressed, and a 3D map with high positional accuracy is generated that shows the accurate global position (position on the XY plane) and accurate elevation (position on the Z plane). .

In addition, in the three-dimensional data generation step (S11), the intensity image in a predetermined rectangular area is set as an XY plane map, the elevation image corresponding to the intensity image is set as a Z plane map, and the set of the XY plane map and the Z plane map is set as a node. A connection of a plurality of nodes is generated as three-dimensional data, and the global position of a specific corner of a rectangular area indicated by a corresponding XY plane map is assigned to each of the plurality of nodes as an identifier. As a result, a wide-area three-dimensional map is expressed as a graph made up of connected nodes, making it possible to reduce position errors by applying graph SLAM.

In addition, in the three-dimensional data generation step (S11), the initial position of each of the plurality of nodes is acquired from the GPS mounted on the vehicle 20, and frames are accumulated based on the acquired initial position using dead reckoning navigation. , generates an XY plane map and a Z plane map of the node. Compared to map generation using GNSS, which always requires satellite signals, this allows you to locate locations where satellite signals are difficult to reach, such as in long tunnels, areas with dense trees, or areas with many tall buildings. It becomes possible to generate highly accurate three-dimensional maps.

In addition, in the XY plane error reduction step (S12) and the Z plane error reduction step (S13), the vehicle 20 moves around a loop in the graph showing the connection of a plurality of nodes for the XY plane map and the Z plane map, respectively. Position errors in the XY and Z planes are reduced by closing a loop that reduces cumulative errors by observing the same point. This reduces cumulative positional errors in the XY plane and the Z plane, making it possible to generate a three-dimensional map with high positional accuracy.

In addition, in the XY plane error reduction step (S12), the position error is reduced by ensuring the consistency of the texture and shape of the ground surface including roads indicated by the two XY plane maps through phase correlation. As a result, a robust and highly reliable phase correlation is used to estimate the relative positions of a plurality of nodes, making it possible to generate a three-dimensional map with high position accuracy and reduced relative position errors.

In addition, in the XY plane error reduction step (S12) and the Z plane error reduction step (S13), consistency, which is the consistency of a common area in the XY plane map and Z plane map of two different nodes, and consistency, which is the consistency of a common area in the At least one of coherency, which is the continuity of edges in the XY plane map and Z plane map of a node, and accuracy, which is the consistency of the position in the XY plane map and Z plane map, and the position obtained by GPS. The cost function to be evaluated is used to reduce the position error. As a result, optimization is performed to reduce relative position errors and global position errors in the XY plane and Z plane, making it possible to generate a three-dimensional map with high position accuracy.

Moreover, the three-dimensional map generation device 10 according to the present embodiment is configured with frames of intensity images representing the ground surface including roads from a point cloud obtained from the LiDAR 11 that is mounted on the vehicle 20 and detects the environment including roads. A 3D data generation unit 13 that generates 3D data including an XY plane map to be generated and a Z plane map made up of a frame of an elevation image showing the elevation of pixels constituting the intensity image; The map generation unit 14 generates a three-dimensional map by reducing position errors, and the map generation unit 14 reduces position errors in the XY plane by executing graph SLAM on the XY plane map. The position error on the Z plane is reduced by executing graph SLAM on the XY plane error reduction unit 14a and the Z plane map corresponding to the XY plane map whose position error has been reduced by the XY plane error reduction unit 14a. and a Z-plane error reduction section 14b.

As a result, after the position error in the XY plane is reduced, the process of reducing the position error is performed on the Z plane map corresponding to the XY plane map with the position error reduced, so the error in the XY plane is reduced. The problem of accumulated errors on the Z plane is suppressed, and a 3D map with high positional accuracy is generated that shows the accurate global position (position on the XY plane) and accurate elevation (position on the Z plane). .

As described above, according to the three-dimensional map generation device 10 and its method according to the present embodiment, data association such as loop closure detection, edge calculation, and data storage is realized by a method of dividing a road into nodes. processing time is reduced. Furthermore, in this embodiment, the top left corner of the node is processed in the optimization process, so the three-dimensional map can be efficiently regenerated. Therefore, the image of the node is only shifted in the global coordinate system based on the obtained offset, rather than re-accumulating a huge number of point clouds. The image of the node encodes a large area of the road and the surrounding environment relative to each other. The reflectance values provide a matching pattern in which the drawn landmarks stand out. This makes it possible to apply the phase correlation method and reduces false detection of common areas between nodes.

Regarding the reduction of processing time, as explained using Fig. 16, a course consisting of 305 nodes representing 25 km consists of a point cloud with the number of collected LIDAR frames being 61755, but the loop The number of closed edges is 286, the processing time for detecting loop closure between nodes is 44 seconds, the calculation of edges is about 57 minutes, and the execution time for optimization processing is 30 seconds. The time to generate the three-dimensional map was 17 seconds. According to the traditional method, it is difficult to detect the correct correspondence between 61755 point clouds, and we apply an iterative scan matching technique between two point clouds (which takes about 17 seconds for each two point clouds). Performing the optimization process and re-accumulating the point cloud to generate the map requires significant processing time.

The three-dimensional map generation method and three-dimensional map generation device 10 of the present invention have been described above based on the embodiments, but the present invention is not limited to these embodiments. Without departing from the spirit of the present invention, the scope of the present invention includes various modifications that can be thought of by those skilled in the art to the present embodiment, and other forms constructed by combining some of the constituent elements of the embodiment. contained within.

For example, the vehicle 20 is equipped with a camera that photographs the environment including the road, and the three-dimensional map generation device 10 uses images obtained from the camera in a complementary manner to the point cloud obtained from the LiDAR 11 to increase the intensity. Frames of images may also be generated.

The present invention is a 3D map generation device that generates a 3D map with high positional accuracy from data obtained by a vehicle equipped with LiDAR, for example, as a device that generates a 3D map used in an autonomous vehicle. Available.

10 3D map generation device 11 LiDAR
12 GPS
13 3D data generation section 14 Map generation section 14a XY plane error reduction section 14b Z plane error reduction section 15 3D map 20 Vehicle

Claims (7)

The intensity image is constructed from a point cloud obtained from LiDAR (Light Detection and Ranging), which is mounted on a vehicle and detects the environment including roads, and an XY plane map consisting of frames of intensity images representing the ground surface including roads. a three-dimensional data generation step of generating three-dimensional data including a Z-plane map composed of frames of elevation images indicating the elevation of pixels;
a map generation step of generating a three-dimensional map by reducing positional errors in the generated three-dimensional data,
The map generation step includes:
an XY plane error reduction step of reducing a position error in the XY plane by executing graph SLAM (Simultaneous Localization and Mapping) on the XY plane map;
a Z-plane error reduction step of reducing a positional error on the Z-plane by executing graph SLAM on the Z-plane map corresponding to the XY-plane map whose positional error has been reduced in the XY-plane error reduction step; 3D map generation method. In the three-dimensional data generation step, the intensity image in a predetermined rectangular area is set as the XY plane map, the elevation image corresponding to the intensity image is set as the Z plane map, and the XY plane map and the Z plane map are combined. Generating a connection of a plurality of nodes with a set as a node as the three-dimensional data,
2. The three-dimensional map generation method according to claim 1, wherein a global position of a specific corner of a rectangular area indicated by the corresponding XY plane map is assigned as an identifier to each of the plurality of nodes. The three-dimensional data generation step includes acquiring an initial position of each of the plurality of nodes from a GPS (Global Positioning System) mounted on the vehicle, and accumulating the frames based on the acquired initial position. 3. The three-dimensional map generation method according to claim 2, wherein the XY plane map and the Z plane map of the node are generated by dead reckoning. In the XY plane error reduction step and the Z plane error reduction step, the vehicle goes around a loop in the graph showing the connection of the plurality of nodes to the same point on the XY plane map and the Z plane map, respectively. 4. The three-dimensional map generation method according to claim 2, wherein position errors in the XY plane and the Z plane are reduced by closing a loop that reduces accumulated errors by observing. Any one of claims 2 to 4, wherein in the XY plane error reduction step, the position error is reduced by ensuring consistency of texture and shape of the ground surface including roads indicated by the two XY plane maps by phase correlation. The three-dimensional map generation method according to item 1. In the XY plane error reduction step and the Z plane error reduction step, respectively,
Consistency, which is the consistency of common areas in the XY plane map and the Z plane map of two different nodes;
coherency, which is the continuity of edges in the XY plane map and the Z plane map of two different nodes;
The positional error is reduced using a cost function that evaluates at least one of: accuracy, which is the consistency between the position on the XY plane map and the Z plane map, and the position obtained by GPS. 5. The three-dimensional map generation method according to any one of 5. The intensity image is constructed from a point cloud obtained from LiDAR (Light Detection and Ranging), which is mounted on a vehicle and detects the environment including roads, and an XY plane map consisting of frames of intensity images representing the ground surface including roads. a 3-dimensional data generation unit that generates 3-dimensional data including a Z-plane map made up of frames of elevation images indicating the elevation of pixels;
a map generation unit that generates a three-dimensional map by reducing positional errors in the generated three-dimensional data,
The map generation unit includes:
an XY plane error reduction unit that reduces positional errors in the XY plane by executing graph SLAM (Simultaneous Localization and Mapping) on the XY plane map;
a Z-plane error reduction unit that reduces positional errors on the Z-plane by executing graph SLAM on the Z-plane map corresponding to the XY-plane map whose positional error has been reduced by the XY-plane error reduction unit; A three-dimensional map generation device.

Images