KR102325119B1 - Vehicles updating point cloud maps using lidar coordinates - Google Patents

Abstract

The present invention provides a vehicle capable of efficiently and precisely updating map data by using a relation between LiDAR coordinates and the existing map data. In accordance with one embodiment of the present invention, the vehicle can comprise: a memory storing the reference map data; the LiDAR sensor acquiring a LiDAR coordinate around the vehicle; and a processor which extracts the coordinate of the reference map data, determines the map data coordinate on the reference map data corresponding to a random point of the LiDAR coordinate, determines the distance between the random point and the map data coordinate, and, if the distance exceeds a predetermined reference value, determines that the random point is the newly generated map data, or if the distance is under the predetermined reference value, determines the random point as the existing map data, thereby updating the reference map data.

Description

Vehicle that performs point cloud map update using lidar coordinates {VEHICLES UPDATING POINT CLOUD MAPS USING LIDAR COORDINATES}

The present invention relates to a vehicle that generates map data necessary for autonomous driving, and more particularly, relates to a vehicle that performs point cloud map update using lidar coordinates.

Self-driving cars are receiving a lot of spotlight as a future industry, and one of the fields in which R&D is being actively conducted. Autonomous driving may be implemented with a cognitive system that recognizes the surrounding environment, a judgment system that plans and corrects a route from the recognized environment, and a control system that controls the vehicle along the planned route.

Meanwhile, various data recognized by the vehicle may be used to form map data provided by autonomous driving.

A vehicle may recognize a surrounding environment and an object by using sensors such as a camera, LiDAR, and RADAR. In order for this system to work, it is very important to calculate the distance between the current vehicle and the object.

LiDAR emits a pulsed laser to the target and until the light returns, while the vehicle needs to update the generated map data depending on the point of time. Therefore, there is a problem in that it is difficult to provide a vehicle that performs autonomous driving.

Therefore, research to solve these problems is being actively conducted.

Laid-open Patent Publication KR 10-2019-0043035 (2019.04.25)

The present invention provides a vehicle that efficiently and accurately performs map data update using a relationship between lidar coordinates and existing map data.

A vehicle according to an embodiment of the present invention includes a memory for storing reference map data; a lidar sensor that acquires lidar coordinates around the vehicle; and extracting the coordinates of the reference map data,

determining map data coordinates on the reference map data corresponding to any point among lidar coordinates;

determining the distance between the arbitrary point and the map data coordinates;

If the distance exceeds a predetermined reference value, the arbitrary point is determined as newly created map data, or

and a processor configured to update the reference map data by determining the arbitrary point as existing map data when the distance is less than a predetermined reference value.

The processor may perform a flip operation on each of the height data included in the reference map data, and form the ground map data using a set of a plurality of planes on the height data on which the flip operation is performed.

The processor may perform a predetermined interpolation algorithm (Moving Least Square) on a region corresponding to the plurality of planes to form the ground map data.

When the distance exceeds a first reference value, the processor may determine the arbitrary coordinates as the generated point.

The processor calculates a distance between the arbitrary point and the ground map data, and determines the arbitrary point as a generated point if the distance between the arbitrary point and the ground map data is less than a second predetermined value. can

The processor calculates a distance between the arbitrary point and the ground map data, and determines the arbitrary point as a missing point if the distance between the arbitrary point and the ground map data exceeds a second predetermined value. can

The processor may update the reference map data by adding data corresponding to the generated point to the reference map data and removing the data corresponding to the missing point.

The processor adds data corresponding to the generated point to the reference map data,

performing an update of the reference map data by removing corresponding to the disappeared point,

A point corresponding to the reference map information is set as a first set, a point corresponding to the lidar coordinate is set as a second set, the generated point is set as a third set, and the disappearing point is set as a fourth set create a set,

If the lidar coordinates are not included in the first set, the lidar coordinates are determined as the generated points,

If the coordinates determined by the generated points are not included in the second set and are not included in the ground map data, it is determined that the generated points are included in the fourth set,

The fifth set may be generated based on a difference between the reference map data and the missing point, and the reference map data may be updated by calculating the union of the third set and the fifth set.

The processor may determine a reference point matching the arbitrary point on the reference map data, and synchronize the lidar coordinates with the reference map data using the arbitrary point and the reference point.

The processor divides the reference map data into a plurality of grids of a predetermined size, determines a grid corresponding to the arbitrary point among the plurality of grids, and a grid corresponding to the arbitrary point and the arbitrary point. included in

The reference map data may be updated based on the distance between the map data coordinates.

The vehicle according to an embodiment of the present invention further comprises a GPS module and an inertial measurement device for obtaining the location data of the vehicle,

The reference map data may be formed by accumulating coordinate values formed by synchronizing the lidar coordinates obtained by the lidar sensor with the position data of the vehicle through the inertial measurement device.

A vehicle according to an embodiment may efficiently and accurately update map data using a relationship between lidar coordinates and existing map data.

The vehicle according to an embodiment may perform safe autonomous driving by using updated map data.

The vehicle according to an embodiment may efficiently manage the vehicle's power through efficient map data update.

1 is a control block diagram of a vehicle according to an embodiment of the present invention.
2 and 3 are flowcharts showing the operation of the present invention according to an embodiment of the present invention.
4A and 4B are diagrams for explaining an operation of determining ground map data included in map data according to an embodiment of the present invention.
5A and 5B are diagrams illustrating reference map data and data obtained by lidar points.
6A, 6B, and 6C are diagrams for explaining points generated from reference map data and points that have disappeared.
7A is a diagram illustrating map data before updating, and FIG. 7B is a diagram illustrating map data after updating.

Like reference numerals refer to like elements throughout. This specification does not describe all elements of the embodiments, and general content in the technical field to which the present invention pertains or content that overlaps among the embodiments is omitted. The term 'part, module, member, block' used in the specification may be implemented in software or hardware, and according to embodiments, a plurality of 'part, module, member, block' may be implemented as one component, It is also possible for one 'part, module, member, block' to include a plurality of components.

Throughout the specification, when a part is "connected" with another part, it includes not only direct connection but also indirect connection, and indirect connection includes connection through a wireless communication network. do.

In addition, when a part "includes" a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated.

Throughout the specification, when a member is said to be located "on" another member, this includes not only a case in which a member is in contact with another member but also a case in which another member exists between the two members.

Terms such as 1st, 2nd, etc. are used to distinguish one component from another component, and the component is not limited by the above-mentioned terms.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In each step, the identification code is used for convenience of description, and the identification code does not describe the order of each step, and each step may be performed differently from the specified order unless the specific order is clearly stated in the context. have.

Hereinafter, the working principle and embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a control block diagram of a vehicle 1 according to an embodiment of the present invention.

Referring to FIG. 1 , a vehicle 1 according to an embodiment may include a memory 300 , a GPS module 100 , an inertial measurement sensor 500 , a lidar sensor 400 , and a processor 200 .

The GPS module 100 may refer to a communication module capable of receiving a signal from a GPS satellite and calculating a current location of the vehicle.

The inertial measurement sensor (IMU) 500 may refer to a device for measuring the speed and direction, gravity, and acceleration of the vehicle. The basic components of the inertial measurement device may mean a gyroscope, an accelerometer, and a geomagnetic sensor that measure free movement in a three-dimensional space.

The gyroscope detects a predetermined reference direction, and the accelerometer measures the speed change to detect roll, yaw, and pitch of the vehicle.

The lidar sensor 400 may refer to a sensor capable of sensing a distance from an object and various physical properties by illuminating a laser on a target.

The memory 300 is a non-volatile memory device or RAM such as a cache, read only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), and flash memory. It may be implemented as at least one of a volatile memory device such as (Random Access Memory), a hard disk drive (HDD), or a storage medium such as a CD-ROM, but is not limited thereto. The memory 300 may be a memory implemented as a chip separate from the processor in relation to the processor 200 , or may be implemented as a single chip with the processor.

The memory 300 may store reference map data.

The reference map data may mean a three-dimensional precision point cloud map used for autonomous driving of a vehicle.

The processor 200 may extract coordinates of the reference map data. That is, the reference map data may be matched with data of a Cartesian coordinate system, and the processor may extract data of such coordinates.

The processor 200 may determine the map data coordinates on the reference map data corresponding to any point among the lidar coordinates obtained by the lidar sensor 400 .

That is, the processor 200 may determine the area of the reference map data of the area corresponding to the position obtained by the lidar sensor.

Also, the processor 200 may determine a distance between any point of the lidar coordinates and the map data coordinates.

When the distance exceeds a predetermined reference value, the processor 200 may determine an arbitrary point as newly generated map data.

The newly generated map data may refer to data that is determined as surrounding data changed from the existing reference map data and updates the map data.

Also, if the distance is less than a predetermined reference value, the processor 200 may determine an arbitrary point as existing map data.

Existing map data may mean determining as data that does not need to be updated.

The processor 200 may perform a flip operation on each of the height data included in the reference map data, and form the ground map data using a set of a plurality of planes on the height data on which the flip operation is performed.

Also, the processor 200 may form ground map data by performing a predetermined interpolation algorithm (Moving Least Square) on a region corresponding to a plurality of planes.

That is, the processor 200 extracts the ground from the precision map by using the Cloth Simulation Filter (CSF) algorithm, and if the point cloud estimated to disappear is the ground extracted by the CSF algorithm, it may be updated in a manner that does not erase it.

When the distance exceeds the first reference value, the processor 200 may determine an arbitrary coordinate as the generated point.

That is, the processor 200 may determine the new coordinates when the coordinates obtained by the LiDAR points are greater than a specific reference value than the distance of the existing coordinates.

In addition, the processor 200 may calculate a distance between an arbitrary point and the ground map data, and if the distance between the arbitrary point and the ground map data is less than a predetermined second value, the arbitrary point may be determined as a generated point. have.

That is, the processor 200 may calculate a distance from the existing ground map data, and if the distance is less than a specific value, determine that the point is a reliable point, and determine the generated point.

On the other hand, the processor 200 calculates the distance between any point and the ground map data,

When the distance between any point and the ground map data exceeds a second predetermined value, the arbitrary point may be determined as a missing point.

That is, the processor 200 may update the reference map data by adding data corresponding to the generated point to the reference map data and removing the data corresponding to the missing point.

The processor 200 may perform an operation of updating the reference precision map by removing points estimated to have disappeared from reference map data that has become a reference, and adding point clouds estimated to be newly generated.

Meanwhile, the processor 200 may determine a reference point matching the arbitrary point on the reference map data, and synchronize the LIDAR coordinates with the reference map data using the arbitrary point and the reference point.

Meanwhile, as described above, the processor 200 may divide the reference map data into a plurality of grids having a predetermined size.

The processor 200 may determine a grid corresponding to an arbitrary point among a plurality of grids.

Also, the processor 200 may update the reference map data based on a distance between an arbitrary point and the map data coordinates included in the grid corresponding to the arbitrary point.

The reference map data may be formed by accumulating coordinate values formed by synchronizing the lidar coordinates obtained by the lidar sensor with the position data of the vehicle through the inertial measurement device.

At least one component may be added or deleted according to the performance of the components of the vehicle 1 illustrated in FIG. 1 . In addition, it will be readily understood by those skilled in the art that the mutual positions of the components may be changed corresponding to the performance or structure of the system.

Meanwhile, each component illustrated in FIG. 1 means a hardware component such as software and/or a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

2 and 3 are flowcharts illustrating the operation of the present invention according to an embodiment.

Referring to FIG. 2 , the vehicle may form reference map data ( S201 ).

In addition, the vehicle may partition the reference map data into a predetermined grid (S202).

Thereafter, the vehicle may acquire lidar coordinates using the lidar sensor (S203).

Thereafter, when the distance from the reference map data corresponding to the newly acquired lidar coordinate distance by comparing the reference map data and the lidar coordinates exceeds the reference value, the corresponding point may be determined as a newly created point (S205 and S204).

However, if the distance from the reference map data corresponding to the newly acquired lidar coordinate distance by comparing the reference map data and the lidar coordinates is less than the reference value, the corresponding point may be determined as the existing point ( S206 ).

Meanwhile, referring to FIG. 3 , when the vehicle acquires the lidar coordinates through the lidar sensor (S301), the distance between the reference map data and the lidar coordinates is calculated. It can be determined (S302, S306).

Meanwhile, if the reference map data and the lidar coordinate distance are greater than the reference value, and the ground map data and the lidar coordinate distance are greater than the reference value, the corresponding point may be determined as a missing point (S302, S303).

On the other hand, if the distance between the reference map data and the lidar coordinates is greater than the reference value, but the distance between the ground map data and the lidar coordinates is less than the reference value, the corresponding point may be determined as a new point (S305).

4A and 4B are diagrams for explaining an operation of determining ground map data included in map data according to an exemplary embodiment.

Referring to FIGS. 4A and 4B together, when a flip operation is performed on the height data of the PD41, the height inverse operation is performed to convert the height data like the PD42.

The processor may determine a flat set of height data of the ground map data.

That is, the data of the map data converted based on the above-described operation may form a plane.

The processor may up-sample only the road surface data provided on the map data. The algorithm for the processor to perform the above-described operation may be implemented as a Cloth Simulation Filter (CSF).

As shown in FIG. 4B , the processor may use a method of separating the corresponding plane by extracting only the points in contact with the cloth, assuming that the map data is turned over and a cloth is placed thereon.

The processor may perform upsampling using an interpolation algorithm between planes.

Referring again to FIG. 4B , the image processor may form a ground surface in the V4 region where the space is empty by using a plane interpolation method.

Finally, the image processor extracts the area corresponding to S4 as ground map data, and can separate the integrated map data into ground map data and the remaining ground map data.

Hereinafter, an operation of updating map data will be described in detail on the premise that such an operation is performed.

5A and 5B are diagrams for comparing reference map data and lidar points acquired by a lidar sensor.

5A shows reference map data stored in a vehicle, and FIG. 5B shows lidar coordinates obtained by a lidar sensor at the same location.

5A and 5B, specific parts are the same, but other parts are shown in different shapes.

Therefore, in the description of the subsequent operation, the operation of updating the existing reference map data using the points constituting the reference map data and the points obtained by the lidar sensor will be described in detail.

Meanwhile, when performing such an operation, the processor may divide the reference map data into a grid.

According to an embodiment of the present invention, the processor may perform the map data update by dividing the reference map data into a grid for an area at a distance of 460 m along the x-axis and 400 m on the y-axis.

In addition, the z-axis values were removed from all GPS values of the map data in a top-view method, and the values of the x-axis and y-axis were included in the corresponding grid.

Hereinafter, the update operation of the map data will be described in detail on the premise of these conditions.

6A, 6B, and 6C are diagrams for explaining points generated from reference map data and points that have disappeared.

The processor may calculate a distance between an arbitrary point and the ground map data, and if the distance between the arbitrary point and the ground map data is less than a predetermined second value, the arbitrary point may be determined as a generated point.

That is, the processor may calculate a distance from the existing ground map data, and if the distance is less than a specific value, the processor may determine that the point is a reliable point and determine the generated point.

Specifically, the processor may determine one newly collected lidar point.

The processor calculates distances and points on the precision map that exist in the same grid as lidar points. Such an operation can be derived from the relationship presented in Equation 1 below.

Referring to Equation 1, M means existing map data, S means scanned lidar coordinates, and U _S means updated scan data, that is, a new point.

If a point corresponding to the reference map point of the same grid exists within a predetermined distance, it may be determined that the corresponding point is a point existing in the reference map data.

And also the processor is determined and included in the U _S of new update the point to be a point does not exist within the predetermined distance based on the map data, the new point is not.

The processor may perform such an operation on points constituting the reference map data of FIG. 6A . Based on this, the update may be performed to the drawings similar to those of FIG. 6A and FIG. 6B.

That is, in FIG. 6B , points such as z6b provided as blue points may be determined to be new points.

In FIG. 6B , it may be determined that the tree is newly planted outside the building under construction.

Meanwhile, the processor calculates the distance between any point and the ground map data,

When the distance between any point and the ground map data exceeds a second predetermined value, the arbitrary point may be determined as a missing point.

That is, the processor may update the reference map data by adding data corresponding to the generated point to the reference map data and removing the data corresponding to the missing point.

The processor may compare the points of the reference map data with the newly collected lidar data, and compare the points with the ground map data extracted through the CSF algorithm to estimate the missing points.

Such an operation may be derived based on the relation of Equation 2 below.

G may mean a point set of ground map data, and m may mean newly acquired point data. Also, R may mean a set of missing points.

The processor may determine any point in the reference map data.

The processor can calculate the distance to the newly collected LiDAR point cloud that exists in the same grid as any point.

If ground map data within a certain distance exists in the same grid, it can be determined that the corresponding precision map point is a point that still exists.

On the other hand, if the precision map points do not exist within a certain distance, the distances may be compared with points of the same grid of the ground map data generated by the above-described CSF algorithm.

Even at this time, if a point does not exist within a certain distance, the processor determines that it is a point that exists in the reference map data, but does not exist in the newly collected data and is not a ground map data that is obscured by a dynamic object. It can be included in R.

Comparing FIGS. 6A and 6C , it can be determined that the point z6c of the red point does not exist in the lidar data, but exists in the reference map data and is a missing point.

In Figure 6c, it can be determined that the existing cranes and the outer walls have disappeared.

7A is a diagram illustrating map data before update, and FIG. 7C is a diagram illustrating map data after updating.

Meanwhile, after the above-described operation, the processor may calculate updated map data based on Equation 3 below.

Referring to Equation 3, U _m may mean an updated map, M may mean reference map data, and M _exist for data of points that have not disappeared.

The processor may be divided into a step of updating the reference precision map by removing the points estimated to have disappeared from the reference map data, which became the reference, and adding the newly created point clouds.

Based on Equation 3, the processor may determine the updated point data U _{m by using the union of the new point U S} and the existing point data M _exist.

As a result, in Fig. 7b, unlike Fig. 7a, there were no point clouds around the building due to the construction outer wall, indicating that the outer wall disappeared after the construction was completed, and the ground and trees inside were newly updated (z7b).

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing instructions executable by a computer. Instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

The computer-readable recording medium includes any type of recording medium in which instructions readable by the computer are stored. For example, there may be a read only memory (ROM), a random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, and the like.

The disclosed embodiments have been described with reference to the accompanying drawings as described above. Those of ordinary skill in the art to which the present invention pertains will understand that the present invention may be practiced in other forms than the disclosed embodiments without changing the technical spirit or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

1: vehicle
100: GPS module
200 : processor
300 : memory
400: lidar sensor
500: inertial measurement sensor

Claims (10)

a memory for storing reference map data;
a lidar sensor that acquires lidar coordinates around the vehicle; and
extract the coordinates of the reference map data,
determining map data coordinates on the reference map data corresponding to any point among lidar coordinates;
determining the distance between the arbitrary point and the map data coordinates;
If the distance exceeds a predetermined reference value, the arbitrary point is determined as newly created map data, or
If the distance is less than a predetermined reference value, the arbitrary point is determined as existing map data and the reference map data is updated,
the processor
performing a flip operation on each of the height data included in the reference map data,
A vehicle that forms the ground map data by using a set of a plurality of planes on the height data on which the flip operation is performed. a memory for storing reference map data;
a lidar sensor that acquires lidar coordinates around the vehicle; and
extract the coordinates of the reference map data,
determining map data coordinates on the reference map data corresponding to any point among lidar coordinates;
determining the distance between the arbitrary point and the map data coordinates;
If the distance exceeds a predetermined reference value, the arbitrary point is determined as newly created map data, or
If the distance is less than a predetermined reference value, the arbitrary point is determined as existing map data and the reference map data is updated,
The processor is
determining a reference point matching the arbitrary point on the reference map data;
A vehicle that synchronizes the lidar coordinates with the reference map data using the arbitrary point and the reference point. a memory for storing reference map data;
a lidar sensor that acquires lidar coordinates around the vehicle; and
extract the coordinates of the reference map data,
determining map data coordinates on the reference map data corresponding to any point among lidar coordinates;
determining the distance between the arbitrary point and the map data coordinates;
If the distance exceeds a predetermined reference value, the arbitrary point is determined as newly created map data, or
If the distance is less than a predetermined reference value, the arbitrary point is determined as existing map data and the reference map data is updated,
The processor is
dividing the reference map data into a plurality of grids of a predetermined size;
determining a lattice corresponding to the arbitrary point among the plurality of lattices,
A vehicle that updates the reference map data based on a distance between the arbitrary point and the map data coordinates included in a grid corresponding to the arbitrary point. a memory for storing reference map data;
a lidar sensor that acquires lidar coordinates around the vehicle; and
extract the coordinates of the reference map data,
determining map data coordinates on the reference map data corresponding to any point among lidar coordinates;
determining the distance between the arbitrary point and the map data coordinates;
If the distance exceeds a predetermined reference value, the arbitrary point is determined as newly created map data, or
If the distance is less than a predetermined reference value, the arbitrary point is determined as existing map data and the reference map data is updated,
Further comprising a GPS module and an inertial measurement device for obtaining the location data of the vehicle,
The reference map data is
The lidar sensor acquires
A vehicle formed by accumulating coordinate values formed by synchronizing the lidar coordinates with position data of the vehicle through the inertial measurement device. 5. The method according to any one of claims 1 to 4,
The processor is
A vehicle for forming the ground map data by performing a predetermined interpolation algorithm (Moving Least Square) on a region corresponding to the plurality of planes. 5. The method according to any one of claims 1 to 4,
The processor is
When the distance exceeds a first predetermined reference value, the vehicle determines the arbitrary coordinates as a generated point. 7. The method of claim 6,
The processor is
calculating the distance between the arbitrary point and the ground map data,
If the distance between the arbitrary point and the ground map data is less than a predetermined second value, the vehicle determines the arbitrary point as a generated point. 7. The method of claim 6,
The processor is
calculating the distance between the arbitrary point and the ground map data,
When a distance between the arbitrary point and the ground map data exceeds a second predetermined value, the vehicle determines the arbitrary point as a missing point. 9. The method of claim 8,
The processor is
adding data corresponding to the generated point to the reference map data,
performing an update of the reference map data by removing corresponding to the disappeared point,
Set a point corresponding to the reference map information as a first set,
Set the points corresponding to the lidar coordinates as a second set,
Set the generated points as a third set,
generating the disappeared points as a fourth set,
If the lidar coordinates are not included in the first set, the lidar coordinates are determined as the generated points,
If the coordinates determined by the generated points are not included in the second set and are not included in the ground map data, it is determined that the generated points are included in the fourth set,
The vehicle generates the fifth set based on a difference between the reference map data and the missing point, and updates the reference map data by calculating the union of the third set and the fifth set.
delete

Images