WO2023131203A1

WO2023131203A1 - Semantic map updating method, path planning method, and related apparatuses

Info

Publication number: WO2023131203A1
Application number: PCT/CN2023/070501
Authority: WO
Inventors: 何鹏; 周光; 蔡一奇
Original assignee: 深圳元戎启行科技有限公司
Priority date: 2022-01-04
Filing date: 2023-01-04
Publication date: 2023-07-13

Abstract

Disclosed in the present application is a semantic map updating method. The method comprises: acquiring at least one object point cloud which corresponds to the current collection region; constructing at least one first point cloud object according to the at least one object point cloud, and extracting corresponding first semantic information; acquiring from an original semantic map at least one second point cloud object and corresponding second semantic information; and according to the first semantic information and the second semantic information, obtaining a matching result of the at least one first point cloud object and the at least one second point cloud object, and then, according to the matching result, updating the original semantic map by using the at least one first point cloud object in a crowdsourcing semantic map. Further disclosed in the present application are a path planning method and related apparatuses. By means of the present application, new data is collected by means of crowdsourcing, and is matched with an existing semantic map for updating, thereby providing a guarantee for map matching and updating, and thus improving the map updating precision.

Description

Semantic map update method, path planning method, and related devices

【Technical field】

The disclosed embodiments of the present application relate to the field of automatic driving map and positioning technology, and more specifically, relate to a semantic map updating method, a path planning method and related devices.

【Background technique】

With the development of computer technology, autonomous driving technology is the future trend (such as human travel, logistics and other fields). In the autonomous driving solution, in principle, it is the process of making the car intelligent: "perception-positioning-decision-execution". The high-precision map is the core of perception and positioning. Therefore, the collection, generation and update of maps has become one of the core technologies of autonomous driving.

There are two common schemes for map building, professional collection and crowdsourcing collection. Among them, crowdsourcing collection can be understood as the user collects road data through the sensor of the self-driving vehicle or other low-cost sensors and transmits it to the cloud for data fusion, and improves the accuracy of the data through this fusion to complete the crowdsourcing. Production of high-precision maps or semantic maps. At present, most crowdsourcing mapping solutions are based on 2D visual data to generate 3D point cloud maps and then semantically convert them into semantic maps. However, the 3D point clouds obtained by such methods are sparse and have low precision, making it difficult to ensure the accuracy of the generated autonomous driving maps.

【Content of invention】

According to the embodiments of the present application, the present application proposes a semantic map updating method, a path planning method and related devices to solve the above problems.

The first aspect of the present application discloses a semantic map update method, the method comprising: obtaining at least one object point cloud corresponding to the current acquisition area; constructing at least one first point cloud object according to the at least one object point cloud and extracting the corresponding First semantic information, wherein the at least one first point cloud object and the corresponding first semantic information are used to construct a crowdsourced semantic map; in response to the existence of an original semantic map in the current collection area, obtain the original semantic map, and Obtain the at least one second point cloud object and corresponding second semantic information from the original semantic map; obtain the at least one first point cloud object according to the first semantic information and the second semantic information and the matching result of the at least one second point cloud object, and update the original semantic map by using the at least one first point cloud object in the crowdsourced semantic map according to the matching result.

The second aspect of the present application discloses a path planning method, including: obtaining a semantic map; performing path planning according to the original semantic map; wherein, the original semantic map is updated through the semantic map as described in the first aspect obtained by the method.

The third aspect of the present application discloses a device for updating a semantic map, the device comprising: an acquisition module, configured to acquire at least one object point cloud corresponding to the current acquisition area; a first extraction module, configured to The cloud constructs at least one first point cloud object and extracts the corresponding first semantic information, wherein the at least one first point cloud object and the corresponding first semantic information are used to construct a crowdsourced semantic map; the second acquisition module is used to Responding to the presence of an original semantic map in the current collection area, obtain the original semantic map, and obtain at least one second point cloud object and corresponding second semantic information from the original semantic map; an update module, configured to The first semantic information and the second semantic information, obtain the matching result of the at least one first point cloud object and the at least one second point cloud object, and use the crowdsourcing semantics according to the matching result The at least one first point cloud object in the map updates the original semantic map.

The fourth aspect of the present application discloses a path planning device, including: an acquisition module, configured to acquire a semantic map, wherein the original semantic map is obtained through the method for updating the semantic map as described in the first aspect; A path planning module, configured to perform path planning according to the original semantic map.

The fifth aspect of the present application discloses a computer device, including a memory and a processor, the processor is used to execute the program instructions stored in the memory, so as to realize the semantic map update method described in the first aspect, or to realize The path planning method described in the second aspect.

The sixth aspect of the present application discloses a non-volatile computer-readable storage medium, on which program instructions are stored. When the program instructions are executed by a processor, the semantic map update method described in the first aspect is implemented, or in the form of Implement the path planning method described in the second aspect.

The beneficial effects of the present application include: acquiring at least one object point cloud corresponding to the current collection area, constructing at least one first point cloud object and extracting corresponding first semantic information according to at least one object point cloud, wherein at least one first point cloud object and the corresponding first semantic information are used to construct a crowdsourced semantic map; in response to the existence of an original semantic map in the current collection area, the original semantic map is obtained, and at least one second point cloud object and the corresponding second semantic map are obtained from the original semantic map information; according to the first semantic information and the second semantic information, obtain a matching result of at least one first point cloud object and at least one second point cloud object, and then use at least one first point in the crowdsourced semantic map according to the matching result The cloud object updates the original semantic map, that is, collects new data through crowdsourcing, and performs matching update with the existing semantic map, which provides a guarantee for map matching update, thereby improving the accuracy of map update.

【Description of drawings】

The application will be further described below in conjunction with the accompanying drawings and embodiments. In the accompanying drawings:

Fig. 1 is the application environment diagram of the semantic map updating method in an embodiment of the present application;

Fig. 2 is a schematic flow chart of a method for updating a semantic map according to an embodiment of the present application;

Fig. 3 is a schematic diagram of semantic information of a point cloud object in an embodiment of the present application;

Fig. 4 is a schematic diagram of partial semantic information of a point cloud object in another embodiment of the present application;

Fig. 5 is a schematic diagram of a point cloud object matching pair in an embodiment of the present application;

FIG. 6 is a schematic flowchart of bounding box collision processing according to an embodiment of the present application;

Fig. 7 is a schematic diagram of the effect of bounding box collision according to an embodiment of the present application;

Fig. 8 is a schematic diagram of the effect of point cloud object mismatching in the embodiment of the present application;

FIG. 9 is another schematic flowchart of bounding box collision processing according to the embodiment of the present application;

FIG. 10 is a schematic flow chart of lane line update in the embodiment of the present application;

Fig. 11 is a schematic diagram of the effect of sampling points in the embodiment of the present application;

FIG. 12 is a schematic diagram of the application scene of the embodiment of the present application;

FIG. 13 is a schematic flowchart of a path planning method according to an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a device for updating a semantic map according to an embodiment of the present application;

FIG. 15 is a schematic structural diagram of a path planning device according to an embodiment of the present application;

FIG. 16 is a schematic structural diagram of a computer device according to an embodiment of the present application;

FIG. 17 is a schematic structural diagram of a non-volatile computer-readable storage medium according to an embodiment of the present application.

【Detailed ways】

Reference in this application to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

In order to enable those skilled in the art to better understand the technical solution of the present application, the technical solution of the present application will be further described in detail below in conjunction with the drawings and specific embodiments.

The semantic map update method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1 , which is an application environment diagram of the semantic map update method in an embodiment of the present application. Wherein, the collection device 102 communicates with the terminal 104 through the network. The data storage system may store data that needs to be processed by the terminal 104 . The data storage system can be integrated on the terminal 104, or placed on the cloud or other network servers. The terminal 104 acquires the point cloud of the target object extracted from the crowdsourced map corresponding to the current collection area collected by the collection device 102; constructs the first point cloud object according to the point cloud of the target object and extracts the corresponding first semantic information; when the newly collected collection area exists In the semantic map, all the second point cloud objects in the semantic map and the corresponding second semantic information are obtained; according to the first semantic information and the second semantic information, the crowdsourcing map is matched with the semantic map to obtain the matching result; according to the matching The result updates the semantic map. Wherein, the terminal 104 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, or automatic driving computing platforms. The acquisition device can be lidar, millimeter-wave radar or ultrasonic radar, or it can be integrated on the terminal. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be implemented through interaction between the terminal and the server.

In one embodiment, as shown in FIG. 2, FIG. 2 is a schematic flowchart of a method for updating a semantic map according to an embodiment of the present application. The method is applied to the terminal in FIG. 1 as an example for illustration, including the following steps:

S202: Obtain at least one object point cloud corresponding to the current collection area.

Obtain at least one object point cloud corresponding to the current collection area, which can be crowdsourcing collection. The user collects road data through the sensor of the self-driving vehicle or other low-cost sensors and sends it to the cloud for data fusion, and through this fusion Crowdsourced high-precision maps or semantic maps can be completed by improving the accuracy of data in a way; crowdsourced maps or crowdsourced maps refer to those collected by the terminal through the cloud, collected by other vehicles and uploaded to the cloud by lidar There are point clouds in the point cloud data that are not required for building a semantic map. It is necessary to extract the collected point cloud data to obtain the point cloud of the target object.

Object point cloud refers to the point cloud of objects (such as traffic indication objects) required for constructing semantic maps. Objects required for semantic maps include but are not limited to traffic lights, traffic signs, lane lines, sidewalks and other objects that carry necessary traffic information .

S204: Construct at least one first point cloud object and extract corresponding first semantic information according to at least one object point cloud, wherein the at least one first point cloud object and the corresponding first semantic information are used to construct a crowdsourced semantic map.

Construct at least one first point cloud object based on at least one object point cloud and extract corresponding first semantic information, wherein the semantic information is predefined, each point cloud object has corresponding semantic information, and the semantic information of each point cloud object Information is different. Semantic information includes point cloud object identification, point cloud center, point cloud convex hull, point cloud bounding box (Oriented Bounding Box, OBB) (can be understood as the point cloud minimum calibration box OBB), point cloud PCA (Principal Components Analysis) coordinates system direction, point cloud main direction and point cloud histogram, etc. It can be understood that the point cloud of the target object here is not limited to one, but may be multiple; there may also be one or more objects corresponding to the first point cloud.

The point cloud object identification of each object is different, that is, unique; the minimum calibration box OBB of the point cloud can be represented by eight vertices in the world coordinate system; the direction of the point cloud PCA coordinate system is decomposed by the eigenvalue of the point cloud covariance matrix, The direction of the three dimensions is calculated by the eigenvector (as shown in Figure 3), and this coordinate system can be defined as the local coordinate system of the object; the main direction of the point cloud is after the PCA calculation, and the eigenvector corresponding to the smallest eigenvalue is selected as The principal orientation of the point cloud object. For example, the point cloud of a traffic sign is similar to a plane. In fact, the eigenvalue corresponding to the normal direction of the plane is the smallest. We define the normal direction as the main direction, as shown in Figure 3, which is the semantic information of the point cloud object in an embodiment of the present application Schematic diagram; the point cloud convex hull is the smallest convex hull that contains all points of an object; the point cloud histogram stores the three-dimensional histogram of x, y, and z in a one-dimensional vector. As shown in Figure 4, Figure 4 is a schematic diagram of part of the semantic information of point cloud objects in another embodiment of the present application, which is part of the semantic information of some objects in the local three-dimensional crowdsourcing semantic map in one embodiment (including OBB and PCA coordinate systems ).

The first semantic information includes the point cloud object identification of the first point cloud object, the point cloud PCA (Principal Components Analysis) coordinate system direction, the point cloud main direction, the point cloud object center, and the point cloud minimum bounding box (Oriented Bounding Box, OBB) , point cloud convex hull and point cloud histogram. It is understandable that the first point cloud object here is not limited to one, but can be multiple; the compactness of OBB is relatively good, which can greatly reduce the number of bounding boxes participating in the intersection test, and the overall performance is better than AABB. When the geometric object rotates, you only need to perform the same rotation on the OBB.

Further, the point cloud object identification of each object is different, that is, unique. Point cloud PCA is to decompose the eigenvalue of the point cloud covariance matrix, determine the direction of the three dimensions through the eigenvector, and define this coordinate system as the local coordinate system of the object. The main direction of the point cloud is after the PCA calculation, and the eigenvector corresponding to the smallest eigenvalue is selected as the main direction of the point cloud object. For example, for common traffic signs, the point cloud approximates a plane (of course, a threshold will be given in the construction map, and if it is less than the threshold, a minimum width (width) will be given to ensure that the point cloud object is a three-dimensional object), then the normal direction of the plane The corresponding eigenvalue is the smallest, and we define the normal direction as the main direction.

The point cloud object center is determined according to the maximum and minimum coordinates of the points in the point cloud, that is, the maximum and minimum values on different axes are determined in the local coordinate system, and after weighted average processing is performed according to the maximum and minimum values, Convert the obtained coordinates to world coordinates; for example, the maximum and minimum values of the point cloud midpoint of the first point cloud object in the directions of x, y, and z axes are x _min , x _max , y _min , y _max , z _min , z _max , the point cloud object center is C=(C _X C _Y C _Z ) ^T , where C _X ＝(x _min +x _max )/2, C _Y ＝(y _min +y _max ,)/2, C _Z =(z _min +z _max )/2.

The size of the bounding box of the minimum bounding box OBB of the point cloud is width (width), height (height), and depth (depth). Among them, width<height<depth can be represented by eight vertices in the world coordinate system; the convex hull of the point cloud is Contains the minimum convex hull of all points of an object; the point cloud histogram stores the three-dimensional histogram of x, y, and z in a one-dimensional vector.

Specifically, when there is a corresponding semantic map in the updated collection area, it is necessary to judge the matching of point cloud objects between the newly collected crowdsourced map and the existing corresponding semantic map, determine the objects required for constructing the semantic map, and use the crowdsourced map Extract the point cloud of the target object used to represent the traffic indication object, perform denoising and clustering processing on the point cloud of the target object, obtain the first point cloud object in the crowdsourcing map, and extract all the first point cloud objects in the crowdsourcing map The first semantic information of the first point cloud object and the corresponding first semantic information are used to construct the crowdsourcing semantic map.

S206: In response to an original semantic map existing in the current acquisition area, acquire the original semantic map, and acquire at least one second point cloud object and corresponding second semantic information from the original semantic map.

In response to the presence of an original semantic map in the current collection area, the original semantic map is acquired, wherein at least one second point cloud object exists in the semantic map, and each second point cloud object has corresponding second semantic information.

Specifically, it is judged whether the original semantic map already exists in the collection area, and when the original semantic map exists in the newly collected collection area, all second point cloud objects and corresponding second semantic information in the original semantic map are obtained; wherein, the original semantic map There is at least one second point cloud object; the second semantic information of the second point cloud object at least includes the semantic information including point cloud object identification, point cloud center, point cloud convex hull, point cloud bounding box (Oriented Bounding Box, OBB), point cloud PCA (Principal Components Analysis) coordinate system direction, point cloud main direction and point cloud histogram, etc. "First" and "second" are only used to distinguish the semantic information of different point cloud objects. For example, the first semantic information can also be named the second semantic information, and the second semantic information can also be named the first semantic information.

S208: Obtain a matching result of at least one first point cloud object and at least one second point cloud object according to the first semantic information and the second semantic information, and use at least one first point in the crowdsourced semantic map according to the matching result Cloud objects, update the original semantic map.

Obtain a matching result of at least one first point cloud object and at least one second point cloud object, wherein the matching result includes a point cloud object matching pair and an absence of point cloud object matching pair; for example, the first point cloud object A is in semantic If there is a matching second point cloud object B in the map, then the first point cloud object A and the second point cloud object B are a matching pair of point cloud objects; the first point cloud object A does not have a matching second point in the semantic map cloud object, there is no point cloud object matching pair.

The matching method adopts but is not limited to Hungarian matching, and can also be other map matching methods. The matching results include two cases where there is an object matching pair in the crowdsourcing map and the semantic map and there is no object matching pair in the crowdsourcing map and the semantic map. Further, there is an object matching pair in the crowdsourcing map and the semantic map, that is, the first point cloud object in the crowdsourcing map has a matching second point cloud object in the semantic map; there is no object matching pair in the crowdsourcing map and the semantic map includes Two cases: the first case is that the first point cloud object in the crowdsourced map does not find a corresponding matching point cloud object in the semantic map, and the second case is that the second point cloud object in the semantic map is not found in the crowdsourced No corresponding matching point cloud object was found in the map. In this patent, for ease of understanding, the unmatched first point cloud object in the crowdsourcing map in the first case is called a new point cloud object (that is, the first point cloud object is a newly added object), In the second case, the second point cloud object that is not matched in the semantic map is called a disappearing point cloud object (that is, the second point cloud object does not exist in the newly collected crowdsourced map).

According to the matching result, at least one first point cloud object in the crowdsourced semantic map is used to update the original semantic map, specifically, when the matching result is that the first point cloud object of the crowdsourced map has a matching second point cloud object in the semantic map object, the semantic information of the first point cloud object and the second point cloud object is weighted and averaged to obtain the semantic information of the new point cloud object; the semantic map is updated according to the semantic information of the new point cloud object; When the first point cloud object of the map is a new object, add the first point cloud object in the semantic map; when the second point cloud object does not exist in the newly collected crowdsourced map, add the original second point cloud object in the semantic map Point cloud objects are deleted to obtain an updated semantic map.

Specifically, according to the first semantic information and the second semantic information, determine the semantic distance between the first point cloud object and each second point cloud object; that is, according to the first semantic information and the second semantic information, determine the first The center position coordinate distance difference value, the point cloud object direction difference, the calibration frame size difference and the appearance feature difference between the point cloud object and each second point cloud object; according to the center position coordinate distance difference value, the point cloud object direction difference value , the difference value of the calibration frame size and the difference value of the appearance feature determine the semantic distance. Based on the semantic distance, the crowdsourced map is matched with the semantic map to obtain the matching result.

Further, determine the physical geometry information such as the center of the point cloud, the main direction of the point cloud, the size of the minimum calibration frame of the point cloud in the first semantic information of the first point cloud object in the crowdsourced map, and determine the second Physical geometry information such as the point cloud center, point cloud main direction, and point cloud minimum calibration frame size in the second semantic information of the point cloud object; according to the point cloud center, point cloud main Calculate the semantic distance between the first point cloud object in the crowdsourcing map and the second point cloud object in the semantic map according to the direction and the size of the minimum calibration frame of the point cloud; weight the obtained semantic distance with the preset weight, Obtain the final semantic distance; obtain the number n of the first point cloud object in the crowdsourcing map and the number m of the second point cloud object in the semantic map, and obtain an association matrix of n*m; use the semantic distance as an element of the association matrix; according to Predetermining a threshold and an association matrix to obtain the matching result of the first point cloud object in the crowdsourced map and the second point cloud object in the semantic map.

In this embodiment, at least one object point cloud corresponding to the current collection area is obtained, at least one first point cloud object is constructed according to at least one object point cloud and corresponding first semantic information is extracted, wherein at least one first point cloud object and The corresponding first semantic information is used to construct a crowdsourced semantic map; in response to the existence of an original semantic map in the current collection area, the original semantic map is obtained, and at least one second point cloud object and corresponding second semantic information are obtained from the original semantic map ; Obtain a matching result of at least one first point cloud object and at least one second point cloud object according to the first semantic information and the second semantic information, and then use at least one first point cloud in the crowdsourcing semantic map according to the matching result The object updates the original semantic map, that is, collects new data through crowdsourcing, and performs matching update with the existing semantic map, which provides a guarantee for map matching update, thereby improving the accuracy of map update.

In an embodiment, determining the semantic distance of the point cloud object includes: determining a center position coordinate distance difference value according to the point cloud center coordinates of the first point cloud object and the point cloud center coordinates of the second point cloud object.

Among them, the first point cloud object is constructed according to the point cloud of the target object extracted from the newly collected crowdsourcing map, and the first semantic information is the physical and geometric information of the point cloud extracted from the first point cloud object, including point cloud Information such as object identification, point cloud center, point cloud convex hull, point cloud minimum calibration frame OBB, point cloud PCA coordinate system direction, point cloud main direction, and point cloud histogram.

Specifically, extract the point cloud of the target object from the crowdsourcing map and construct the corresponding first point cloud object, and extract the first semantic information of the first point cloud object; determine the second point cloud object from the existing semantic map, and Second semantic information of the second point cloud object. For example, in a local map (crowdsourcing collection), the center coordinates of the first point cloud object are (x1, y1, z1), the main direction of the point cloud is a, the size of the calibration frame is (w1, h1, d1), and the shape feature is 30-dimensional vector s1, the number of original point clouds is n1. On the other hand, the center coordinates of the second point cloud object in the semantic map are (x2, y2, z2), the main direction of the point cloud is b, the size of the calibration frame is (w2, h2, d2), and the shape feature is 30 dimensions (also Can be other digital dimensions) vector s2, the number of original point clouds is n2.

Among them, according to the point cloud center coordinates (x1, y1, z1) of the first point cloud object and the point cloud center coordinates (x2, y2, z2) of the second point cloud object, determine the center position coordinate distance difference value, which can be expressed as :

In one embodiment, determining the semantic distance of the point cloud object further includes: determining the point cloud object direction difference value according to the point cloud main direction of the first point cloud object and the point cloud main direction of the second point cloud object; The calibration frame size of the cloud object and the calibration frame size of the second point cloud object determine the difference value of the calibration frame size; according to the shape element of the point cloud histogram of the first point cloud object and the point cloud histogram of the second point cloud object Shape elements, to determine the difference value of appearance features.

Among them, the point cloud object direction difference value can be expressed as: cos(θ)=a b/(|a||b|); the larger the angle, the greater the difference, the smaller the cos value; the smaller the angle, The smaller the difference, the larger the cos value; distance2=1-cos(θ), the smaller the difference, the smaller the direction distance.

Among them, the size of the calibration frame of the first point cloud object is (w1, h1, d1), and the size of the calibration frame of the second point cloud object is (w2, h2, d2); the difference value of the calibration frame size can be expressed as:

Among them, the appearance feature difference value can be expressed as:

In one embodiment, determining the semantic distance of the point cloud object further includes: by weighting the center position coordinate distance difference value, the point cloud object direction difference value, the calibration frame size difference value and the appearance feature difference value to obtain each first point The semantic distance between the cloud object and the second point cloud object in the semantic map.

Specifically, the weight values w1, w2, w3, w4 corresponding to the distance difference value of the center position coordinates, the point cloud object direction difference value, the calibration frame size difference distance3 and the appearance feature difference value are obtained; The weighted summation is used to obtain the semantic distance score, where the weight value ranges from 0 to 1. That is to say, by adding a weight to each difference value, and then adding it linearly, it becomes the final semantic distance; that is, the semantic distance is w1*distance1+w2*distance2+w3*distance3+w4*distance4. Optionally, in one embodiment, according to the actual situation, among the weight values w1, w2, w3, and w4, one of w1, w2, w3, and w4 may have a weight value of 0, or two weight values may be 0, there is no limit here.

Based on the map matching algorithm, according to the determined semantic distance between each first point cloud object and the second point cloud object in the semantic map, the collected crowdsourcing map is matched with the existing semantic map, and the crowdsourcing map is determined to be consistent with the existing semantic map. The matching situation between the point cloud objects in the semantic map, where the matching situation includes the first point cloud object in the crowdsourcing map has a matching second point cloud object in the semantic map, the first point cloud object in the crowdsourcing map The object does not have a matching second point cloud object in the semantic map (that is, the first point cloud object is a new point cloud object) and the second point cloud object in the semantic map does not have a matching first point cloud object in the crowdsourcing map ( That is, the second point cloud object disappears), etc.

In the above method for determining the semantic distance of point cloud objects, by extracting the semantic information of the first point cloud object in the crowdsourcing map and the semantic information of the second point cloud object in the existing semantic map, the crowdsourcing map is determined according to the geometric information of the object The semantic distance between the point cloud object and the point cloud object in the semantic map provides a guarantee for map matching update.

In an embodiment, the map matching includes: obtaining at least one object point cloud corresponding to the current acquisition area, constructing at least one first point cloud object according to the at least one object point cloud and extracting the corresponding first semantic information, specifically, extracting Perform denoising and clustering on the point cloud of the object, construct at least one first point cloud object, and extract first semantic information of each first point cloud object.

Further, in response to the existence of an original semantic map in the current acquisition area, the original semantic map is obtained, and at least one second point cloud object and corresponding second semantic information are obtained from the original semantic map, according to the first semantic information and the second semantic information , to determine the semantic distance between the first point cloud object and each second point cloud object.

Specifically, according to the point cloud center coordinates of the first point cloud object and the point cloud center coordinates of the second point cloud object, determine the center position coordinate distance difference value; according to the point cloud main direction of the first point cloud object and the second point cloud The main direction of the point cloud of the object, determine the difference value of the direction of the point cloud object; according to the calibration frame size of the first point cloud object and the calibration frame size of the second point cloud object, determine the difference value of the calibration frame size; according to the calibration frame size of the first point cloud object The shape element of the point cloud histogram and the shape element of the point cloud histogram of the second point cloud object determine the difference value of the appearance feature; by scoring the center position coordinate distance difference, the point cloud object direction difference score, the calibration frame size difference score and The appearance feature difference score is weighted to obtain the semantic distance between each first point cloud object and the second point cloud object in the semantic map.

In one embodiment, the map matching further includes: obtaining the number n of the first point cloud object and the number m of the second point cloud object, obtaining an association matrix of n*m, using the semantic distance as an element of the association matrix, according to a predetermined threshold and the correlation matrix to obtain a matching result of at least one first point cloud object and at least one second point cloud object.

It can be understood that each element in the affinity matrix represents the semantic distance between the first point cloud object in the crowdsourced map and the second point cloud object in the semantic map; for example, the first point cloud object in the crowdsourced map and the semantic The number of second point cloud objects in the map is 3, forming a 3*3 association matrix, and each element in the association matrix is the semantic distance between the first point cloud object and the second point cloud object in the semantic map. Wherein, if the semantic distance is less than a predetermined threshold, it indicates that there is a relationship between the corresponding first point cloud object and the second point cloud object.

Specifically, the semantic distance is used as an element of the correlation matrix, and when the first element in the correlation matrix is greater than or equal to a predetermined threshold, it is determined that the first point cloud object and the second point cloud object corresponding to the first element are not matched by the point cloud object Yes; the first point cloud object and the second point cloud object corresponding to the first element are not point cloud object matching pairs include the first point cloud object is a new point cloud object or the second point cloud object is a disappearing point cloud object .

When there is at least one second element in the correlation matrix that is smaller than the predetermined threshold, it is determined that there is at least one matching second point cloud object in the first point cloud object corresponding to the second element; the correlation matrix is segmented based on the second element, and several sub-point cloud objects are obtained. Figure; carry out bipartite graph matching to each subgraph respectively, determine the second point cloud object that matches with the first point cloud object; The matching cost value of each second point cloud object; determine that the second point cloud object corresponding to the matching cost value with the smallest value is the matching point cloud object of the first point cloud object. Among them, the bipartite graph matching is implemented by the Hungarian algorithm, and the Hungarian algorithm is used to perform the bipartite graph matching to obtain the object connection pair (object, crowd_object) with the smallest cost, as shown in Figure 5, which is an example of an object connection pair (object, crowd_object) in an embodiment of the present application. Schematic diagram of a point cloud object matching pair, which may be a point cloud object matching pair; in other words, the degree of association between each second point cloud object in the sub-graph and the corresponding first point cloud object (the degree of association can be understood as a matching cost value) .

For example, the crowdsourcing collection map includes the first point cloud object 1, 2, 3, and the second point cloud object 4, 5, 6 in the existing semantic map, wherein, the first point cloud object 2 and the second point cloud object 4 and the semantic distance between the first point cloud object 2 and the second point cloud object 5 is greater than a predetermined value, which is an unmatched point cloud object, the first point cloud object 1 and the second point cloud object 4, and the first point cloud object If the semantic distance between the first point cloud object 1 and the second point cloud object 5 is less than the predetermined threshold, it is necessary to combine the first point cloud object 1 and the second point cloud object 4, and the semantic distance between the first point cloud object 1 and the second point cloud object 5 The distance is divided into a subgraph, and the existing Hungarian matching algorithm is used for the first point cloud object 1 and the second point cloud object 4, as well as the first point cloud object 1 and the second point cloud object 5, to obtain the matching cost value cost, The one with the smallest cost is determined as the final matching point cloud object of the first point cloud object 1.

In an embodiment, a method for updating a semantic map includes: acquiring at least one object point cloud corresponding to the current collection area, constructing at least one first point cloud object according to the at least one object point cloud and extracting corresponding first semantic information, Determine whether there is an original semantic map in the current collection area,

If so, in response to the existence of an original semantic map in the current collection area, obtain the original semantic map, and obtain at least one second point cloud object and corresponding second semantic information from the original semantic map, otherwise, use at least one first point cloud object and The first semantic information is to create a crowdsourcing semantic map, where the collection area refers to the collection area where the current collection terminal is located.

A method for updating a semantic map includes: obtaining a matching result of at least one first point cloud object and the at least one second point cloud object according to the first semantic information and the second semantic information, and updating the semantic map according to the matching result.

Specifically, according to the first semantic information and the second semantic information, determine the center position coordinate distance difference value between the first point cloud object and each second point cloud object, point cloud object direction difference, calibration frame size difference and appearance features Difference; determine the semantic distance according to the center position coordinate distance difference value, the point cloud object direction difference value, the calibration frame size difference value and the appearance feature difference value; obtain the number n of the first point cloud object in the crowdsourcing map and the second in the semantic map The number m of point cloud objects, get the correlation matrix of n*m;

Using semantic distance as an element of an association matrix; when there is a first element greater than or equal to a predetermined threshold in the association matrix, it is determined that the first point cloud object and the second point cloud object corresponding to the first element are not a point cloud object matching pair; when When at least one second element in the correlation matrix is smaller than the predetermined threshold, it is determined that the first point cloud object corresponding to the second element has at least one matching second point cloud object; the correlation matrix is segmented based on the second element to obtain several subgraphs ; Carry out bipartite graph matching on each sub-graph respectively, determine the matching cost value of the first point cloud object and each second point cloud object in the sub-graph; determine the second point cloud object corresponding to the matching cost value with the smallest value as the first point The cloud object's matching point cloud object.

Specifically, when the matching result is that the first point cloud object is a new point cloud object, add the first point cloud object to the semantic map; when the matching result is that the second point cloud object is a vanishing point cloud object, add the first point cloud object to the semantic map The second point cloud object is deleted from the semantic map; when the matching result is that the first point cloud object has a point cloud object matching pair, the first point cloud object and the matching point cloud object are fused to obtain the fused point cloud object, and determine Fuse the fused semantic information of the point cloud; update the semantic map according to the fused semantic information.

In other words, when there is a matching point cloud object that matches the first point cloud object in the semantic map, algorithms such as ICP and NDT are used to fuse the first point cloud object and the object point cloud corresponding to the matching point cloud object to obtain a fused point cloud , and perform denoising and clustering processing on the fused point cloud, and recalculate and extract semantic information for the fused point cloud after downsampling; recalculating and extracting the semantic information of the fused point cloud can be the sum of the first point cloud object and the first point The semantic information of the second point cloud object matched by the cloud object is obtained by weighted mean processing; for example, the point cloud center, point cloud convex hull, point cloud minimum calibration box OBB, point cloud PCA coordinate system direction in the two semantic information , the main direction of the point cloud and the point cloud histogram to perform mean value processing, the specific method can be realized by the existing method, and will not be repeated here.

In the above semantic map update step, by extracting the target first point cloud object from the newly collected crowdsourced map, and extracting the first semantic information of the first point cloud object, that is, under the premise of retaining useful information, the overall size of the map is greatly increased. Zoom out, and get more map information while extracting semantic information and compressing the data size; determine the matching result of crowdsourcing map and semantic map based on the semantic distance determined by semantic information; realize adding to semantic map according to different matching results Adding and deleting objects and averaging matching objects provide a guarantee for map matching updates, thereby improving the accuracy of map updates.

In another embodiment, at least one object point cloud corresponding to the current collection area is obtained, at least one first point cloud object is constructed according to the at least one object point cloud and the corresponding first semantic information is extracted, and it is judged whether there is an original semantic map in the collection area , if, in response to the existence of the original semantic map in the current collection area, obtain the original semantic map, and obtain at least one second point cloud object and the corresponding second semantic information from the original semantic map, otherwise, when there is no semantic map in the collection area , create a semantic map according to the first semantic information, where the collection area refers to the collection area where the collection terminal is located. Further, according to the first semantic information and the second semantic information, a matching result of the at least one first point cloud object and the at least one second point cloud object is obtained. When the matching result is that the first point cloud object is a new point cloud object, add the first point cloud object to the semantic map; when the matching result is that the second point cloud object is a vanishing point cloud object, add the second point cloud object The object is deleted from the semantic map; when the matching result is that there is a point cloud object matching pair for the first point cloud object, the matching point cloud object matching the first point cloud object is obtained from the second point cloud object; for the first point cloud object The object and the matching point cloud object are fused to obtain the fused point cloud object, and the fused semantic information of the fused point cloud is determined, and then the semantic map is updated according to the fused semantic information.

In one embodiment, a method for updating a semantic map is provided, comprising the following steps:

First, perform semantic averaging processing on the first point cloud object and at least one second point cloud object that successfully matches the first point cloud object to obtain a semantic average point cloud object.

Semantic average processing is performed on the first point cloud object and the matching point cloud object that successfully matches the first point cloud object in the first point cloud object and at least one second point cloud object, that is, the first point cloud object and at least one second point cloud object The semantic information of the matched point cloud object that is successfully matched with the first point cloud object is semantically averaged, wherein the first semantic information and the second semantic information include the direction of the PCA coordinate system, the center of the point cloud, the smallest bounding box of the point cloud, and the point cloud Convex hull and histogram, the first semantic information refers to the semantic information of the newly collected first point cloud object; the second semantic information refers to the semantic information of the second point cloud object in the corresponding semantic map.

Specifically, through the PCA coordinate system direction, point cloud center, point cloud minimum bounding box, point cloud convex hull and histogram in the first semantic information, and the PCA coordinate system direction, point cloud center in the second semantic information , point cloud minimum bounding box, point cloud convex hull and histogram are semantically averaged, and the average semantic information of the updated point cloud object is obtained.

Second, replace the matching point cloud objects in the original semantic map with the semantically averaged point cloud objects, so as to update the original semantic map.

In one embodiment, a semantic information averaging processing method is provided. The method is applied to the terminal in FIG. The direction of the PCA coordinate system is interpolated to obtain the updated direction of the PCA coordinate system; the point cloud center in the first semantic information and the point cloud center in the second semantic information are averaged to obtain the updated point cloud center.

Wherein, the PCA coordinate system direction in the first semantic information and the PCA coordinate system direction in the second semantic information are quaternions, that is, the PCA coordinate system direction in the first semantic information is a quaternion q1, and the second The direction of the PCA coordinate system in the semantic information is the quaternion q2, and the obtained updated direction of the PCA coordinate system is the quaternion q _update .

According to quaternion q1 and quaternion q2, the updated PCA coordinate system direction obtained is quaternion q _update , which is determined by quaternion slerp interpolation calculation, including the following steps:

Calculate the relative rotation Δq between q1 and q2, Δq=q ₁ ^-1 *q2=(Δq _w Δq _X Δq _Y Δq _Z ) ^T ; among them, Δq _w , Δq _X , Δq _Y , Δq _Z are four The four components of an arity.

Calculate the rotation angle θ between the point cloud center of the first point cloud object and the second point cloud object, θ=2*arccos(Δq _w )

The slerp interpolation value is q _update = (q1*sin((1-t)θ/2)+q2*sin(tθ/2))/sin(θ/2)

Among them, t belongs to (0,1). In this example t=0.5.

Specifically, perform interpolation processing on the direction of the PCA coordinate system in the first semantic information and the direction of the PCA coordinate system in the second semantic information to obtain the updated direction of the PCA coordinate system, and update the point cloud master according to the updated direction of the PCA coordinate system Direction, use the same axis (such as x-axis, or y-axis, or z-axis) of the updated PCA coordinate system direction as the update main direction.

Specifically, for the point cloud center (CX ₁ , CY ₁ , CZ ₁ ) in the first semantic information in the local coordinate system of the first point cloud object, and in the second semantic information in the local coordinate system of the second point cloud object The center of the point cloud (CX ₂ , CY ₂ , CZ ₂ ) is subjected to mean value processing and coordinate transformation is performed to obtain the updated point cloud center C _update in world coordinates, which can be expressed in the following way:

C _update ＝[CX _update CY _update CZ _update ] ^T ; Among them, CX _update ＝(CX ₁ +CX ₂ )/2; CY _update ＝(CY ₁ +CY ₂ )/2; CY _update ＝(CY ₁ +CY ₂ )/2;

In an embodiment, the semantic information averaging processing method further includes performing mean value processing on the minimum bounding box of the point cloud in the first semantic information and the minimum bounding box of the point cloud in the second semantic information to obtain an updated minimum bounding box of the point cloud; Based on the updated direction of the PCA coordinate system, the center of the updated point cloud is converted to the updated object coordinate system to obtain the center coordinates of the target point cloud.

Among them, the size of the bounding box of the smallest bounding box of the point cloud is width, height, and depth, and width<height<depth; according to the smallest bounding box of the point cloud (w1, h1, d1) in the first semantic information and the second The smallest bounding box (w2, h2, d2) of the point cloud in the semantic information is averaged, and the size of the smallest bounding box of the updated point cloud is (w _update , h _update , d _update ). Where: w _update = (w1+w2)/2; h _update = (h1+h2)/2; d _update = (d1+d2)/2

Based on the updated PCA coordinate system direction q _update , the coordinate conversion of the updated point cloud center C _update = [CX _update CY _update CZ _update ] ^T is performed, and converted to the updated object coordinate system calculated by PCA to obtain the target point cloud center coordinate C _local = [CX _local CY _local CZ _local ] ^T .

Further, according to the center coordinates of the target point cloud and the minimum bounding box of the updated point cloud, the vertex coordinates of the updated bounding box are obtained.

Specifically, determine the vertex coordinates according to the target point cloud center coordinates and update the size information of the minimum bounding box of the point cloud; wherein the size information includes width, height and depth; specifically include the following steps:

First, the vertex coordinates of the updated bounding box on the x-axis are obtained according to the x-axis coordinates and width of the center of the target point cloud. Secondly, the vertex coordinates of the updated bounding box on the y-axis are obtained according to the y-axis coordinates and the height of the center of the target point cloud. Furthermore, according to the z-axis coordinates and depth of the center of the target point cloud, the vertex coordinates of the updated bounding box on the z-axis are obtained. Finally, the vertex coordinates of the updated bounding box are obtained according to the vertex coordinates of the updated bounding box on the x-axis, the vertex coordinates on the y-axis, and the vertex coordinates on the z-axis.

That is to say, the maximum and minimum values on the x-axis of the local coordinate system are obtained according to the x-axis coordinates and width of the target point cloud center; the maximum value on the y-axis of the local coordinate system is obtained according to the y-axis coordinates and height of the target point cloud center and the minimum value; according to the z-axis coordinates and depth of the target point cloud center, the maximum and minimum values on the z-axis of the local coordinate system are obtained; according to the maximum and minimum values on the x-axis, the maximum and minimum values on the y-axis, And the maximum and minimum values on the z-axis determine the vertex coordinates of the updated bounding box.

Specifically, the maximum and minimum values on the x-axis, the maximum and minimum values on the y-axis, and the maximum and minimum values on the z-axis of the local coordinate system are reversed to the world coordinates, and the corresponding coordinate system is obtained. The maximum and minimum values on the x-axis, the maximum and minimum values on the y-axis, and the maximum and minimum values on the z-axis are obtained to obtain the 8 vertex coordinates of the updated bounding box. That is, according to the updated minimum bounding box of the point cloud (w _update , h _update , d _update ) and Clocal=(CX _local CY _local CZ _local ) ^T in the local coordinate system, determine the x, y, z directions in the local coordinate system The maximum and minimum values are as follows:

Reverse the maximum and minimum values obtained in the above local coordinate system to the world coordinates, and obtain the maximum and minimum values in the x, y, and z directions under the corresponding coordinates x _min , x _max , y _min , y _max , z _min , z _max get the 8 vertex coordinates of the updated bounding box: 1(x _max ,y _max ,z _max ), 2(x _max ,y _max ,z _min ), 3(x _max ,y _min ,z _min ), 4(x _max ,y _min ,z _max ), 5(x _min ,y _max ,z _max ), 6(x _min ,y _max ,z _min ), 6(x _min ,y _min ,z _min ), 8(x _min ,y _min ,z _max ).

Further, the point cloud convex hull in the first semantic information and the point cloud convex hull in the second semantic information are translated, rotated and coordinate transformed to obtain an updated point cloud convex hull; according to the object point cloud of the first point cloud object Perform coordinate system conversion to update the histogram in the second semantic information to obtain the updated histogram.

Specifically, according to the updated PCA coordinate system direction q _update and the updated point cloud center C _update , the point cloud convex hull M1 in the first semantic information and the point cloud convex hull M2 in the second semantic information are translated and rotated to C _Update is the center and q _update is the local coordinate of the rotation direction, and the fusion point cloud convex hull M _local of the two point cloud convex hulls is obtained, and the existing point cloud convex hull calculation method is used to recalculate the fusion point cloud convex hull M _local Convex hull, using the updated minimum bounding box of the point cloud (w _update , h _update , d _update ) to filter the convex hull of the fused point cloud convex hull M _local to obtain the filtered point cloud convex hull M _local+update , and set The point cloud convex hull M _local+update is transformed into the world coordinate system through the coordinate system, and the updated point cloud convex hull M _update is obtained.

Among them, in the semantic map, there is no need to save the object point cloud itself, and the histogram can be re-determined according to the object point cloud data of the newly collected crowdsourced map.

Specifically, according to the relative relationship between the PCA coordinate system direction q _update update point cloud center C _update and the PCA coordinate system direction q1 point cloud center c1 of the first point cloud object, the object point cloud PLC of the first point cloud object ₁ Convert to the PLC _local in the local coordinate system of q _update and C _update , and obtain the PLC _update by converting to the world coordinate system; use the updated minimum bounding box of the point cloud (w _update , h _update , d _update ) to update After filtering the point cloud of the object, after filtering the points outside the minimum bounding box of the point cloud (w _update , h _update , d _update ), recalculate the histogram to obtain the updated histogram of the same object in the crowdsourced map and the semantic map.

At this time, according to the updated PCA coordinate system direction, updated point cloud center, vertex coordinates, updated point cloud convex hull and updated histogram, the average semantic information of the updated first point cloud object is obtained.

In another embodiment, a method for updating a semantic map is provided, which includes: constructing at least one first point cloud object according to at least one object point cloud and extracting corresponding first semantic information; in response to the presence of original Semantic map, obtain the original semantic map, and obtain at least one second point cloud object and the corresponding second semantic information from the original semantic map; The matched point cloud objects with successful object matching are semantically averaged to obtain semantically averaged point cloud objects; the semantically averaged point cloud objects are replaced with matching point cloud objects in the original semantic map to update the original semantic map.

Wherein, the average semantic information includes at least any one of the updated PCA coordinate system orientation, the updated point cloud center, the updated vertex coordinates of the bounding box, the updated convex hull of the point cloud, and the updated histogram.

Specifically, perform semantic averaging processing according to the first semantic information and the second semantic information, update the first semantic information of the first point cloud object, obtain the updated PCA coordinate system direction of the first point cloud object after updating, and update the point cloud Center, update the vertex coordinates of the bounding box, update the convex hull of the point cloud, and update any average semantic information in the histogram.

Optionally, in one embodiment, an interpolation process is performed on the direction of the PCA coordinate system in the first semantic information and the direction of the PCA coordinate system in the second semantic information to obtain an updated direction of the PCA coordinate system; according to the updated PCA Orientation of the coordinate system to obtain the average semantic information of the updated first point cloud object.

Optionally, in one embodiment, after determining the updated PCA coordinate system direction, mean value processing is performed on the point cloud center in the first semantic information and the point cloud center in the second semantic information to obtain an updated point cloud center; According to the updated PCA coordinate system direction and the updated point cloud center, the average semantic information of the updated first point cloud object is obtained.

Optionally, in one embodiment, after determining the updated PCA coordinate system direction and updating the point cloud center, the minimum bounding box of the point cloud in the first semantic information and the minimum bounding box of the point cloud in the second semantic information Perform mean value processing and coordinate conversion processing to obtain the vertex coordinates of the updated bounding box; according to the updated PCA coordinate system direction, the updated point cloud center and the vertex coordinates of the updated bounding box, the average semantic information of the updated first point cloud object is obtained .

Optionally, in one embodiment, after determining the updated PCA coordinate system direction, updating the point cloud center and updating the vertex coordinates of the bounding box, the convex hull of the point cloud in the first semantic information and the second semantic information The convex hull of the point cloud is translated, rotated and coordinate transformed to obtain an updated convex hull of the point cloud; according to the updated PCA coordinate system direction, the updated point cloud center, the vertex coordinates of the updated bounding box and the updated point cloud convex hull, the updated The average semantic information of the first point cloud object of .

Optionally, in one embodiment, determine the updated PCA coordinate system direction, update the point cloud center, update the vertex coordinates of the bounding box and update the point cloud convex hull, and also perform coordinates according to the object point cloud of the first point cloud object System conversion updates the histogram in the second semantic information to obtain an updated histogram; according to the updated PCA coordinate system direction, updated point cloud center, updated vertex coordinates of the bounding box, updated point cloud convex hull and updated histogram, updated After the average semantic information of the first point cloud object.

In another embodiment, a semantic map update method is provided, the method includes: constructing at least one first point cloud object according to at least one object point cloud and extracting the corresponding first semantic information, in response to the existence of original Semantic map, obtaining the original semantic map, and obtaining at least one second point cloud object and the corresponding second semantic information from the original semantic map, for the direction of the PCA coordinate system in the first semantic information and the PCA coordinates in the second semantic information The direction of the PCA coordinate system is interpolated to obtain the updated direction of the PCA coordinate system. Further, the point cloud center in the first semantic information and the point cloud center in the second semantic information are averaged to obtain an updated point cloud center. First, the minimum bounding box of the point cloud in the first semantic information and the second semantic information The minimum bounding box of the point cloud in the information is subjected to mean value processing and coordinate conversion processing to obtain the vertex coordinates of the updated bounding box; secondly, the convex hull of the point cloud in the first semantic information and the convex hull of the point cloud in the second semantic information are translated Rotation and coordinate conversion processing, to obtain the updated convex hull of the point cloud; again, according to the object point cloud of the first point cloud object, the coordinate system conversion is performed to update the histogram in the second semantic information, and the updated histogram is obtained; finally, according to the updated PCA coordinate system direction, update point cloud center, vertex coordinates, update point cloud convex hull and update histogram, get the average semantic information of the updated first point cloud object, update the semantic map according to the average semantic information, and update the semantic map based on The updated semantic map obtained after that is used for localization.

Wherein, the loop closure detection and positioning are performed according to the updated semantic map; wherein, the methods of positioning and loop closure detection can be realized through existing methods, and will not be repeated here. Among them, loopback detection, also known as closed-loop detection, refers to the ability of the device to identify that it has reached a certain scene and make the map closed-loop, that is, it can match the map generated at the moment with the map just generated.

In one embodiment, the matching result of at least one first point cloud object and at least one second point cloud object includes the first unmatched point cloud object set in the crowdsourced semantic map and the second unmatched point in the original semantic map A collection of cloud objects.

Among them, the first set of unmatched point cloud objects refers to the set of point cloud objects in the crowdsourcing map that do not match the corresponding point cloud objects in the semantic map. The point cloud objects in the crowdsourcing map may be newly added point clouds Objects; the second set of unmatched point cloud objects refers to the set of point cloud objects in the semantic map that do not match the corresponding point cloud objects in the crowdsourced map, and the point cloud objects in the semantic map may be point cloud objects that have disappeared.

It is understandable that under non-ideal conditions, such as environmental occlusion, or sensor performance (for example, only half of the object is scanned), it is often mis-matched as two different objects, and the matching point cloud object will not be matched during the matching process; In the first matching process, it will be determined as the newly added point cloud object and the disappearing point cloud object; the newly added point cloud object refers to the point cloud object newly collected in the crowdsourcing map compared with the semantic map, and the disappearing The point cloud object refers to the point cloud object that the newly collected crowdsourced map disappears compared with the semantic map.

The semantic distance is determined according to the semantic information of the newly collected crowdsourced map and the existing semantic map, and the semantic distance is weighted with the preset weight to obtain the final semantic distance; according to the semantic distance The bipartite graph matching is performed between the crowdsourced map and the semantic map to obtain matching pairs of point cloud objects and/or no matching pairs of point cloud objects; the point cloud objects without matching pairs of point cloud objects are determined as unmatched point cloud objects.

Specifically, the semantic distance is determined according to the newly collected crowdsourced map and the semantic information of the point cloud objects in the existing semantic map, and the obtained semantic distance is weighted with the preset weight to obtain the final semantic distance; according to Semantic distance matches the bipartite graph between the crowdsourced map and the semantic map to obtain the matching pairs of point cloud objects and/or the absence of point cloud object matching pairs, and determine the point cloud objects without point cloud object matching pairs as unmatched point clouds Objects; that is to say, the first unmatched point cloud object set that is identified as a new point cloud object in the crowdsourced map during the first matching process, and/or, the second unmatched point cloud object set that disappears in the semantic map collection of objects.

According to the matching result, at least one first point cloud object is used to update the original semantic map, as shown in Figure 6, and the method is applied to the terminal in Figure 1 as an example, including the following steps:

Step 602, judging whether there is a bounding box collision between each first unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within a preset semantic distance range in the original semantic map.

Wherein, the preset semantic distance range is preset, and is used to determine from the semantic map and determine the corresponding first unmatched point cloud object in the first unmatched point cloud object set within the preset semantic distance range Point cloud object (can be understood as the first candidate point cloud object).

Bounding box collision, that is, OBB collision (Oriented Bounding Box, direction bounding box), OBB collision processing uses but is not limited to the separation axis theorem, which can be understood as if an axis can be found, the projections of two convex shapes on the axis do not overlap , then the two shapes do not intersect. If this axis does not exist, and those shapes are convex, you can be sure that two shapes intersect (not applicable for concave shapes, such as crescent shapes, and two crescent shapes may not intersect even if no separating axis can be found).

It can also be understood that if a straight line can be found, so that the bounding box A is completely on one side of the line, and the bounding box B is completely on the other side, then the two bounding boxes do not overlap. And this straight line becomes the separation line (called the separation plane in the three-dimensional world), and must be perpendicular to the separation axis. In this embodiment, the OBB collision processing needs to test 15 separate axes to determine the intersection state of the OBB. Among them, there are 3 coordinate axes for each of the two OBBs, plus 9 axes perpendicular to each axis. The collision judgment is the same as the existing two-dimensional OBB collision, that is, if the projections of two polygons on all axes overlap, it is judged as a collision; otherwise, no collision occurs, so I won’t repeat it here.

Specifically, acquire each first unmatched point cloud object in the first unmatched point cloud object set, and the semantic information of the corresponding point cloud object within the preset semantic distance range, calculate the semantic distance, and determine that within the preset semantic distance range The first candidate point cloud object, determine whether each first unmatched point cloud object in the first unmatched point cloud object set has a bounding box collision with the corresponding first candidate point cloud object, and obtain a bounding box collision result; Bounding box collision results include the presence of bounding box collisions or the absence of bounding box collisions.

Step 604, judging whether there is a bounding box collision between each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the crowdsourcing semantic map.

Wherein, the preset semantic distance range is preset, and is used to determine from the crowdsourced map and determine the corresponding second unmatched point cloud objects in the second unmatched point cloud object set within the preset semantic distance range Point cloud object (can be understood as the second candidate point cloud object).

Specifically, acquire each second unmatched point cloud object in the second unmatched point cloud object set, and the semantic information of the corresponding point cloud object within the preset semantic distance range, calculate the semantic distance, and determine that within the preset semantic distance range The second candidate point cloud object, judge whether there is a bounding box collision between each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding candidate point cloud object, and obtain the bounding box collision result; Collision results include the presence of bounding box collisions or the absence of bounding box collisions.

Step 606: Update the original semantic map according to the bounding box collision result of the first unmatched point cloud object and/or the bounding box collision result of the second unmatched point cloud object.

It can be understood that when matching the newly collected crowdsourced semantic map and the original semantic map in the same collection area, at least one of the first unmatched point cloud object set and the second unmatched point cloud object set is obtained; that is Say, in the first matching process, it is determined that there are new point cloud objects in the crowdsourced semantic map, and/or, there are missing point cloud objects in the original semantic map. Updating the semantic map includes adding new point cloud objects and/or deleting missing point cloud objects.

Specifically, according to the bounding box collision result of the first unmatched point cloud object, there is no bounding box collision for the second unmatched point cloud object, and/or, the bounding box collision result of the second unmatched point cloud object is the first unmatched point cloud object. There is no bounding box collision of the matching point cloud object, and the semantic map is updated; and according to the bounding box collision result of the first unmatched point cloud object, there is a bounding box collision for the second unmatched point cloud object, and/or, the second unmatched point cloud object has a bounding box collision The bounding box collision result of the cloud object is that the first unmatched point cloud object has a bounding box collision, and the semantic map is updated.

Further, when the bounding box collision result of the first unmatched point cloud object is that there is no bounding box collision for the second unmatched point cloud object, the second unmatched point cloud object is deleted from the semantic map; when the second unmatched point cloud object When the bounding box collision result of the point cloud object is that there is no bounding box collision of the first unmatched point cloud object, the first unmatched point cloud object is added to the semantic map.

Match the newly collected crowdsourced map in the same collection area with the existing semantic map, and match the newly collected crowdsourced map in the same collection area with the existing semantic map to obtain the first unmatched point cloud object set and At least one of the second set of unmatched point cloud objects; performing bounding box collision on point cloud objects in the first set of unmatched point cloud objects and point cloud objects in the second set of unmatched point cloud objects, for the first time There is no unmatched point cloud object in the matching, and the matching secondary detection is performed to improve the map update accuracy.

In one embodiment, a method for updating an original semantic map based on a first unmatched point cloud object is provided, and the method is applied to the terminal in FIG. 1 as an example for illustration, including the following steps:

First, when the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, determine the corresponding first colliding point cloud object within a preset distance range from the original semantic map.

Specifically, match the newly collected crowdsourced semantic map and the original semantic map in the same collection area, and when the first unmatched point cloud object set of the crowdsourced semantic map is obtained, judge the first unmatched point cloud object set Whether there is a bounding box collision between each first unmatched point cloud object and the corresponding point cloud object within the preset semantic distance range in the original semantic map; when the bounding box collision result of the first unmatched point cloud object is the first unmatched point cloud When the object has a bounding box collision, determine the corresponding first collision point cloud object within the preset distance range from the original semantic map; when the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object does not exist When the bounding box collides, the first unmatched point cloud object is added to the original semantic map.

As shown in Figure 7, the first unmatched point cloud object set 1 of the crowdsourcing semantic map A includes the first unmatched point cloud object n and the first unmatched point cloud object m, the first unmatched point cloud object n and the preset Assume that there is a bounding box collision of point cloud object d in the original semantic map B (including point cloud object d and point cloud object f) within the distance range, and determine point cloud object d as the first colliding point cloud object.

Secondly, according to the first unmatched point cloud object and the first collision point cloud object, the point cloud proportion is obtained.

Among them, the point cloud proportion refers to the percentage of the number of points in the object point cloud corresponding to the first unmatched point cloud object in the convex hull of the first collision point cloud object to the total number of points in the object point cloud; that is, the first unmatched point cloud The object point cloud corresponding to the object and the object convex hull of the first collision point cloud object are projected or mapped to the same coordinate system, and the first unmatched point cloud object in the same coordinate system is determined to correspond to the object point cloud, and the first collision point cloud object The number of points in the convex hull of the object, and the proportion of the point cloud is determined according to the number of points.

Specifically, determine that the first unmatched point cloud object corresponds to the first object point cloud, and the first point cloud object convex hull corresponding to the first collision point cloud object; determine the points in the first object point cloud in the first point cloud The number of points in the convex hull of the object, and the proportion of the point cloud is obtained according to the number of points and the total number of points in the point cloud of the first object.

Finally, the original semantic map is updated according to the point cloud proportion and the set threshold.

Specifically, when the point cloud proportion is greater than or equal to the set threshold, it is determined that the first unmatched point cloud object and the first collision point cloud object are the same object, and the original point cloud object in the original semantic map is retained; when the point cloud occupies When the ratio is less than the set threshold, it is determined that the first unmatched point cloud object and the first collision point cloud object are not the same object; and the first unmatched point cloud object is added to the original semantic map.

In the above method of updating the original semantic map based on the first unmatched point cloud object, when there is a bounding box collision of the first unmatched point cloud object, the first unmatched point cloud object and the corresponding first collision point cloud object are compared. Cloud detection, update the original semantic map according to the percentage of the number of points in the convex hull of the object point cloud corresponding to the first unmatched point cloud object in the convex hull of the first collision point cloud object to the total points of the object point cloud; Errors, or errors caused by noise interference, improve map update accuracy.

In another embodiment, a method for updating the original semantic map based on the first unmatched point cloud object is provided, and the method is applied to the terminal in FIG. 1 as an example for illustration, including: when the first unmatched point cloud object When the bounding box collision result of the first unmatched point cloud object has a bounding box collision, determine the corresponding first collision point cloud object within the preset distance range from the original semantic map; determine the first unmatched point cloud object and the first Whether the collision point cloud object has a convex hull collision, if so, for the first unmatched point cloud object and the first collision point cloud object, obtain the point cloud proportion, and update the original semantic map according to the point cloud proportion and set the threshold, Otherwise, when there is no convex hull collision, it is determined that the first unmatched point cloud object and the first collided point cloud object are not the same object, and the first unmatched point cloud object is further added to the original semantic map.

Among them, the convex hull collision refers to detecting whether all the convex hull points of a point cloud object are in the convex hull of another point cloud object; for example, to find the equation f(x,y) of each surface of the convex hull of the point cloud object , z)=0, then bring the judgment point into f, and other points of the convex hull into f, if the sign is different, it means that it is not inside, the judgment point here refers to each point cloud convex hull of a point cloud object Point, judge whether each point is in another point cloud convex hull.

It is understandable that there is a collision between the OBB and the convex hull collision processing, and it does not mean that the two point cloud objects are the same object; in the actual data collection process, when the sensor error is considered, or the point cloud clustering constructs the object, there is a relatively large Collisions will occur in large noises; further point cloud judgment is required. As shown in Figure 4, it is two point cloud objects that are successfully matched; as shown in Figure 8, it is two incorrectly matched point cloud objects.

In the above method of updating the original semantic map based on the first unmatched point cloud object, when the first unmatched point cloud object has a bounding box collision, the first unmatched point cloud object and the corresponding first collision point cloud object are convexly Package collision and point cloud detection, update the original semantic map. That is, OBB collision detection, convex hull collision detection, and point cloud judgment are performed in sequence to avoid mis-matching due to occlusion or sensor performance. For unmatched point cloud objects, matching secondary detection is performed to improve map update accuracy.

In an embodiment, a method for updating an original semantic map based on a second unmatched point cloud object is provided, and the method is applied to the terminal in FIG. 1 as an example for illustration, including: when the second unmatched point cloud object When the bounding box collision result of the second unmatched point cloud object has a bounding box collision, determine the corresponding second collision point cloud object within a preset distance range from the crowdsourced semantic map.

Specifically, match the newly collected crowdsourced semantic map with the existing original semantic map in the same collection area, and when the second unmatched point cloud object set of the original semantic map is obtained, judge the second unmatched point cloud object set Whether there is a bounding box collision between each second unmatched point cloud object in the original semantic map and the corresponding point cloud object within the preset semantic distance range; when the bounding box collision result of the second unmatched point cloud object is the second unmatched When there is a bounding box collision of the point cloud object, determine the corresponding second collision point cloud object within the preset distance range from the original semantic map; when the bounding box collision result of the second unmatched point cloud object is the second unmatched point cloud object When there is no bounding box collision, the second unmatched point cloud object is deleted from the original semantic map.

As shown in Figure 7, the second unmatched point cloud object set of the original semantic map A includes the second unmatched point cloud object n and the second unmatched point cloud object m, the second unmatched point cloud object n and the preset distance There is a bounding box collision of the point cloud object d in the crowdsourced semantic map B (including point cloud object d and point cloud object f) within the range, and the point cloud object d is determined as the second colliding point cloud object.

In one embodiment, the method for updating the original semantic map based on the second unmatched point cloud object further includes: judging whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object; When packets collide, determine that the second unmatched point cloud object and the second colliding point cloud object are the same object; otherwise, when there is no convex hull collision, determine that the second unmatched point cloud object and the second colliding point cloud object are not The same object, further, the second unmatched point cloud object is removed from the original semantic map.

In the above-mentioned method of updating the original semantic map based on the second unmatched point cloud object, when it is determined that the bounding box collision of the second unmatched point cloud object exists, that is, the second colliding point cloud object corresponding to the disappearing point cloud object confirmed by the first match Carry out convex hull collision, and update the original semantic map by determining whether there is a convex hull collision to confirm whether it is a disappearing point cloud object or the same point cloud object; avoid sensor errors or errors caused by noise interference during data collection, and improve map update accuracy.

In another embodiment, as shown in FIG. 9 , a method for updating an original semantic map based on 3D point cloud crowdsourcing is provided. The method is applied to the terminal in FIG. 1 as an example for illustration, including the following steps:

Step 902, matching the newly collected crowdsourced semantic map and the existing original semantic map in the same collection area to obtain a first set of unmatched point cloud objects and a second set of unmatched point cloud objects.

Step 904, judging whether the point cloud objects in the first unmatched point cloud object set and the second unmatched point cloud object set have bounding box collisions; if not, execute step 906; if yes, execute step 908.

Specifically, determine whether there is a bounding box collision between each first unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the original semantic map; and/or, determine the first Whether there is a bounding box collision between each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the crowdsourcing semantic map;

Step 906, delete the second unmatched point cloud object from the original semantic map, and/or add the first unmatched point cloud object to the original semantic map.

Step 908, when the bounding box collision result is that the first unmatched point cloud object has a bounding box collision, perform step 910; if the bounding box collision result is that the second unmatched point cloud object has a bounding box collision, perform step 99.

Step 910, determine the corresponding first collision point cloud object within the preset distance range from the original semantic map.

Step 912, judge whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object; if so, go to step 914; otherwise, go to step 918.

Step 914, according to the first unmatched point cloud object and the first colliding point cloud object, obtain the point cloud proportion.

Step 916, update the original semantic map according to the proportion of the point cloud and the set threshold.

Step 918, when there is no convex hull collision, determine that the first unmatched point cloud object and the first colliding point cloud object are not the same object.

Step 920, adding the first unmatched point cloud object to the original semantic map.

Step 99, determine the corresponding second collision point cloud object within the preset distance range from the crowdsourced semantic map.

Step 924 , judging whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object, if yes, go to step 930 , otherwise, go to step 926 .

Step 926, when there is no convex hull collision, determine that the second unmatched point cloud object and the second colliding point cloud object are not the same object.

Step 928, delete the second unmatched point cloud object from the original semantic map.

Step 930, when there is a convex hull collision, determine that the second unmatched point cloud object and the second colliding point cloud object are the same object.

Optionally, in one embodiment, the newly collected crowdsourced semantic map of the same collection area is matched with the existing original semantic map to obtain the first set of unmatched point cloud objects; determine the first unmatched point cloud object Whether there is a bounding box collision between each first unmatched point cloud object in the set and the corresponding point cloud object within the preset semantic distance range in the original semantic map; when the bounding box collision result of the first unmatched point cloud object is the second unmatched When there is no bounding box collision of the matching point cloud object, the second unmatched point cloud object is deleted from the original semantic map.

When the bounding box collision result of the first unmatched point cloud object is that the first unmatched point cloud object has a bounding box collision, determine the corresponding first collision point cloud object within the preset distance range from the original semantic map; according to the first The unmatched point cloud object and the first colliding point cloud object get the point cloud proportion; according to the point cloud proportion and the set threshold, the original semantic map is updated.

Further, when the bounding box collision result of the first unmatched point cloud object is that the bounding box collision of the first unmatched point cloud object exists, it is judged whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object ; When there is a convex hull collision, determine the corresponding first collision point cloud object within the preset distance range from the original semantic map; obtain the point cloud proportion according to the first unmatched point cloud object and the first collision point cloud object; Update the original semantic map according to the point cloud proportion and set the threshold; when there is no convex hull collision, determine that the first unmatched point cloud object and the first collision point cloud object are not the same object; convert the first unmatched point cloud Objects are added to the original semantic map.

Optionally, in one embodiment, the newly collected crowdsourced semantic map of the same collection area is matched with the existing original semantic map to obtain a second set of unmatched point cloud objects; judging the second unmatched point cloud object Whether there is a bounding box collision between each second unmatched point cloud object in the set and the corresponding point cloud object within the preset semantic distance range in the crowdsourced semantic map; when the bounding box collision result of the second unmatched point cloud object is the first When there is no bounding box collision of the unmatched point cloud objects, the first unmatched point cloud object is added to the original semantic map.

When the bounding box collision result of the second unmatched point cloud object is that the second unmatched point cloud object has a bounding box collision, determine the corresponding second collision point cloud object within the preset distance range from the crowdsourced semantic map; determine the second Whether there is a convex hull collision between the unmatched point cloud object and the second collision point cloud object; when there is no convex hull collision, it is determined that the second unmatched point cloud object and the second collision point cloud object are not the same object; the second unmatched point cloud object is not the same object; The matching point cloud object is deleted from the original semantic map; when there is a convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are the same object.

Optionally, in one embodiment, the newly collected crowdsourced semantic map of the same collection area is matched with the existing original semantic map to obtain the first set of unmatched point cloud objects and the second set of unmatched point cloud objects ;

Judging whether there is a bounding box collision between each first unmatched point cloud object in the first unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the original semantic map; judging the second unmatched point cloud object set Whether there is a bounding box collision between each second unmatched point cloud object in the crowdsourcing semantic map and the corresponding point cloud object within the preset semantic distance range; when the bounding box collision result of the first unmatched point cloud object is the second unmatched When there is no bounding box collision of the matching point cloud object, delete the second unmatched point cloud object from the original semantic map; and when the bounding box collision result of the first unmatched point cloud object is that the second unmatched point cloud object does not exist When the bounding box collides, the second unmatched point cloud object is deleted from the original semantic map.

When the bounding box collision result of the first unmatched point cloud object is that there is a bounding box collision for the first unmatched point cloud object, it is judged whether there is a convex hull collision between the first unmatched point cloud object and the first collided point cloud object; When the convex hull collides, determine the corresponding first collision point cloud object within the preset distance range from the original semantic map; get the point cloud proportion according to the first unmatched point cloud object and the first collision point cloud object; according to the point cloud Proportion and set the threshold, update the original semantic map; when there is no convex hull collision, determine that the first unmatched point cloud object and the first collision point cloud object are not the same object; add the first unmatched point cloud object to in the original semantic map.

In one embodiment, as shown in FIG. 10 , a method for processing lane markings is provided, which can be applied to a computer device. The computer device can be a terminal or a server, and the terminal or server can be executed independently by the terminal or server itself, or can be executed by the terminal and the server. The interaction between servers is realized. This embodiment is described by taking the method applied to computer equipment as an example, including the following steps:

Step 1002, acquire multiple frames of images of the target road section and pose points corresponding to each frame of images.

Among them, the pose point refers to a point with position information and attitude information. It can be understood as a point representing vehicle position information and orientation information. Position information can be represented by coordinates, and attitude information can be represented by angles.

Exemplarily, the computer device acquires multiple frames of images collected in the target road section and the pose points corresponding to each frame of images from the vehicle-mounted system or server or the cloud.

Step 1004, determine the road section characteristics and road section speed of the target road section based on the pose points.

Wherein, the road segment feature refers to the shape feature of the road segment. It can be understood as the shape of the road section. The feature of the road section may be a straight road section or a curved road section, and the curved road section may be subdivided into various curved road sections with different degrees of curvature. The road section speed refers to the speed at which the vehicle on which the device for collecting images is located travels on the target road section.

Exemplarily, the computer device performs calculations based on the pose points corresponding to each frame of image, and determines the road section features and road section speed of the target road section according to the calculation results.

In one embodiment, the computer device acquires the current pose point among the plurality of pose points corresponding to the target road section, acquires the next adjacent pose point after the current pose point as the first reference pose point, and acquires the first reference pose point The next adjacent pose point of the pose point is used as the second reference pose point, and the first slope between the current pose point and the first reference pose point is calculated, and the first reference pose point and the second reference pose point The second slope between the attitude points,

Calculate the difference between the second slope and the first slope to obtain the curvature corresponding to the current pose point, and calculate the difference between the curvature corresponding to two adjacent current pose points, if the difference is less than the difference threshold , the target road segment is a straight road segment, and if the difference is greater than the difference degree threshold, the target road segment is a curved road segment. For example, A (x1, y1), B (x2, y2), C (x3, y3) three pose points, get point A as the current pose point, then point B is the first reference pose point, point C is the second reference pose point, the slope between point A and point B is (y2-y1)/(x2-x1), and the slope between point B and point C is (y3-y2)/(x3-x2 ), the curvature of point A is (y3-y2)/(x3-x2)-(y2-y1)/(x2-x1).

In one embodiment, the computer device obtains the reference table of the corresponding relationship between the average curvature and road section features, and then adds the curvature corresponding to each current pose point to obtain the curvature of the curve, and divides the curvature of the curve by the pose The number of points is used to obtain the average curvature of the target, and the road section characteristics corresponding to the average curvature of the target are searched in the reference table to obtain the road section characteristics of the target road section.

In one embodiment, the computer device performs curve fitting on multiple pose points to obtain the fitted curve, solves the curvature expression corresponding to the fitted curve according to the curvature solution formula, and brings the pose points into the curvature expression to obtain the pose point Corresponding curvature, calculate the difference between the curvatures corresponding to two pose points separated by a preset number of pose points, if the difference is less than the difference threshold, then the target road segment is a straight road segment, if the difference is greater than the difference threshold , then the target road segment is a curved road segment.

In one embodiment, the computer device obtains the first pose point and the end pose point corresponding to the target road section, and the first moment corresponding to the first pose point and the second moment corresponding to the end pose point, and calculates the first pose point The physical distance between the pose point and the end pose point, the time interval between the first moment and the second moment, and the speed of the road section is calculated based on the physical distance and the time interval.

In one embodiment, the computer device obtains the moment corresponding to each pose point, calculates the physical distance between two adjacent pose points, calculates the time interval between the corresponding moments of two adjacent pose points, and physically Divide the distance by the time interval to get the speed between two adjacent pose points, add the speeds corresponding to each adjacent pose point to get the total speed, count the number of phase accelerations to get the total number, divide the total speed by the total number , to get the segment speed of the target segment.

In step 1006, multiple frames of images are selected based on the features of the road section and the speed of the road section to obtain a target image. Exemplarily, the computer device determines the selection scheme according to the road section characteristics and the speed of the road section of the target road section, and then selects the target image from the multiple frames of images according to the selection scheme

In one embodiment, the computer device obtains a matching table between road section attributes and selection schemes, road section attributes include multiple feature attributes, and a combination of multiple feature attributes corresponds to a selection scheme. According to the multiple feature attributes corresponding to the target road section in Query the selection scheme corresponding to the target road section in the matching table. For example, the first characteristic attribute in the matching table is the road section feature, which is specifically divided into straight road sections and curved road sections with multiple degrees of curvature. The second feature attribute in the matching table is the road section speed, which is specifically divided into multiple speed intervals. The road section characteristics and road section speed corresponding to the road section determine the selection scheme corresponding to the target road section, and then select the target image from the multiple frames of images according to the selection scheme.

Step 1008, generating a target lane line of the target road segment based on the target image.

Among them, the lane line refers to a line segment in the road that plays a role of restraining and guaranteeing the driving of the vehicle. Lane lines are important traffic signs in road traffic. Lane lines include but are not limited to white dotted lines, white solid lines, yellow dotted lines, and yellow solid lines. For example, pedestrian crossing lines that allow pedestrians to cross the lane, lane dividing lines that separate vehicles traveling in the same direction, and so on.

Exemplarily, the computer device generates the target lane line of the target road section according to the target image.

In the above lane line processing method, the multi-frame images of the target road section and the pose points corresponding to each frame of the image are obtained, the road section characteristics and the road section speed of the target road section are determined according to the pose points, and according to the road section characteristics and the road section speed corresponding to the target road section, from A target image is selected from multiple frames of images, and a target lane line of the target road section is generated based on the target image. Determine the road section characteristics and road section speed corresponding to the target road section through the pose point, select the target image according to the road section characteristics and road section speed, reduce the number of target images involved in generating the target lane line, thereby reducing the error of the target lane line, and improving In order to be representative of the target image involved in generating the target lane line, the target lane line of the target road section is generated according to the target image, and the accuracy of the target lane line is improved.

In one embodiment, selecting multiple frames of images based on road section features and road section speeds, and obtaining the target image includes: if the road section speed of the target road section is zero, then selecting one frame of images from the multiple frames of images as the target image; if the target road section If the road section is a straight line and the speed of the road section is not zero, the number of targets is determined based on the speed of the road section, and the number of target images is selected from the multi-frame images; if the target road section is a curved road section and the speed of the road section is not zero, the multi-frame images are used as the target image.

Exemplarily, the computer device obtains that the speed of the road section corresponding to the target road section is zero, then selects a frame of image from the multi-frame images as the target image, if the target road section is a straight road section, then determines the number of targets according to the speed of the road section, and then selects an image from the multi-frame Select the target image of the target number in the image, if the target road section is a curved road section and the speed of the road section is not zero, then use all the images corresponding to the target road section as the target image.

In one embodiment, the computing computing device selects the target image from the target image as a random selection, for example, if the target road section is a straight road section, then the number of targets is determined according to the speed of the road section, and then the target image of the target number is randomly selected from multiple frames of images .

In one embodiment, the computer device selects the target image from the target graphics as interval sampling. For example, if the target road section is a straight line road section, the number of targets is determined according to the speed of the road section, and then the total number of multiple frames of images corresponding to the target road section is divided by the target. Quantity, get the sampling interval, and then select a target image for each sampling interval from the multi-frame images.

In this embodiment, the target image is selected according to the characteristics of the road section and the speed of the road section, which reduces the number of target images participating in the generation of the target lane line, thereby reducing the error of the target lane line, and improving the accuracy of the target images participating in the generation of the target lane line. representative.

In one embodiment, generating the target lane line of the target road section based on the target image includes: acquiring a set of three-dimensional sampling points corresponding to each frame of the target image; collecting and combining each three-dimensional sampling point into a fusion sampling point set; Curve fitting and sampling to obtain a target sampling point set; generate target lane lines based on the target sampling point set.

Obtain a set of three-dimensional sampling points corresponding to each frame of the target image. Wherein, the set of three-dimensional sampling points refers to a set composed of multiple three-dimensional coordinate points representing lane lines in the image. A three-dimensional coordinate point refers to a point with a certain meaning formed by three independent variables. A three-dimensional coordinate point represents a point in space, and has different expressions in different three-dimensional coordinate systems. For example, a three-dimensional coordinate point (x, y, z) in a three-dimensional Cartesian coordinate system, x, y, and z respectively have The coordinate values of the X-axis, Y-axis, and Z-axis that have a common origin and are orthogonal to each other. Exemplarily, the computer device acquires a set of three-dimensional sampling points representing lane lines in each frame of the target image.

Collect and combine each three-dimensional sampling point into a fusion sampling point set. Exemplarily, the computer device combines multiple sets of three-dimensional sampling points into a set of fused sampling points.

Carry out curve fitting and sampling on the fusion sampling point set to obtain the target sampling point set; among them, curve fitting refers to a method of approaching discrete data with analytical expressions. It can be understood as using a continuous curve to approximately describe or compare a set of discrete points on a plane. Sampling is the process of selecting individuals from a population. Sampling includes random sampling and non-random sampling. Random sampling refers to extracting individuals from the population according to the principle of randomization. Non-random sampling refers to selecting individuals from the population according to set rules. Exemplarily, the computer device performs curve fitting on the fusion sampling point set to obtain a fitting curve, and then samples the fitting curve to obtain a target sampling point set.

Generate the target lane line based on the set of target sampling points. Exemplarily, the computer device generates the target lane line according to the set of target sampling points.

In one embodiment, the computer device connects any two adjacent target sampling points with a line segment, and the target lane line is formed by the target sampling point and the line segment between the adjacent target sampling points.

In one embodiment, the computer device performs smoothing filtering on the set of target sampling points to obtain an optimized set of target sampling points, and generates the target lane line based on the optimized sequence of target sampling points.

In this embodiment, a set of three-dimensional sampling points representing the lane line in each frame of the target image is acquired, multiple three-dimensional sampling point sets are combined into a fusion sampling point set, and curve fitting is performed on the fusion sampling point set to obtain a fitting curve , and then sample the fitting curve to obtain the target sampling point set, and generate the target lane line according to the target sampling point set. The target sampling point set is obtained by curve fitting and sampling the fusion sampling point set. During the curve fitting process, the three-dimensional sampling points that deviate from the whole are filtered out, and the target sampling points with errors in the target sampling point set are reduced, and the improvement is improved. The smoothness and accuracy of the target lane lines are achieved.

In one embodiment, generating the target lane line of the target road segment based on the target image includes: if the speed of the road segment corresponding to the target road segment is not zero and includes a straight road segment and a curved road segment, then selecting the first target image corresponding to the straight road segment and the curved road segment respectively The corresponding second target image; the three-dimensional sampling point set corresponding to the first target image is formed into the first sampling point set, and the three-dimensional sampling point set corresponding to the second target image is formed into the second sampling point set; the first sampling point set is respectively Perform curve fitting and sampling with the second sampling point set to obtain a target sampling point set; generate a target lane line based on the target sampling point set.

If the speed of the road segment corresponding to the target road segment is not zero and includes a straight road segment and a curved road segment, then respectively select the first target image corresponding to the straight road segment and the second target image corresponding to the curved road segment. Exemplarily, the computer device judges that the speed of the target road segment is not zero, and the target road segment contains a straight road segment and a curved road segment, then calculates the speed of the straight road segment according to the corresponding pose points of the straight road segment, and calculates the speed of the straight road segment according to the speed of the straight road segment. The number of targets is determined, and then the first target image of the target number is selected from the multi-frame images corresponding to the straight road section, and all images corresponding to the curved road section are used as the second target image.

The three-dimensional sampling point set corresponding to the first target image is formed into the first sampling point set, and the three-dimensional sampling point set corresponding to the second target image is formed into the second sampling point set. For example, the computer device obtains the three-dimensional sampling point corresponding to the first target image. A set of sampling points, combining the set of three-dimensional sampling points corresponding to the first target image into a first set of sampling points, then obtaining a set of three-dimensional sampling points corresponding to the second target image, and combining the set of three-dimensional sampling points corresponding to the second target image into a second set of sampling points Two sets of sampling points.

Curve fitting and sampling are performed on the first sampling point set and the second sampling point set respectively to obtain a target sampling point set. Exemplarily, the computer device performs curve fitting on the first sampling point set to obtain a first fitting curve, Sampling the first fitting curve to obtain the first target sampling point set, then performing curve fitting on the second sampling point set to obtain the second fitting curve, sampling the second fitting curve to obtain the second target sampling point point set, and finally the first target sampling point set and the second target sampling point set form a target sampling point set.

In this embodiment, curve fitting is performed on the first set of sampling points corresponding to the curved road section and the second set of sampling points corresponding to the straight road section, and the distribution characteristics of the first set of sampling points and the distribution characteristics of the second set of sampling points are retained , improving the accuracy of curve fitting, thereby improving the accuracy of the target sampling point set, generating the target lane line according to the target sampling point set, and improving the accuracy of the target lane line.

In one embodiment, the lane line processing method further includes: if there is a reference lane line in the target road section, obtaining a set of reference sampling points corresponding to the reference lane line; The distance between lane lines.

If there is a reference lane line in the target road segment, a set of reference sampling points corresponding to the reference lane line is obtained. The reference lane line refers to the lane line corresponding to the existing target road segment in the semantic map. Exemplarily, the computer device inquires whether there is a reference lane line of the target road segment in the semantic map, and if yes, obtains a set of reference sampling points corresponding to the reference lane line.

Based on the set of target sampling points and the set of reference sampling points, the degree of separation between the target lane line and the reference lane line is calculated, where the degree of separation refers to the degree of separation between objects. The degree of separation can be expressed by the distance between objects, or by the average distance between objects, and so on. Exemplarily, the computer device calculates the degree of separation between the target lane line and the reference lane line according to the target sampling point set and the reference sampling point set.

Further, the lane line processing method also includes: comparing the degree of separation with a threshold of separation, and if the degree of separation is smaller than the threshold of separation, performing curve fitting and sampling on the set of reference sampling points and the set of target sampling points, An updated sampling point set is obtained, and an updated lane line is generated based on the updated sampling point set; if the distance is equal to or greater than the distance threshold, an updated lane line is generated based on the target sampling point set.

Exemplarily, the computer device compares the degree of separation with the degree of separation threshold, and if the degree of separation is less than the threshold of degree of separation, the set of reference sampling points and the set of target sampling points form a set of fusion sampling points, and the set of fusion sampling points Perform curve fitting and sampling to obtain an updated sampling point set, and use the updated sampling point set to generate an updated lane line; if the separation is equal to or greater than the separation threshold, use the target sampling point set to generate an updated lane line.

In this embodiment, if there is already a reference lane line corresponding to the target lane in the semantic map, then calculate the distance between the reference sampling point set corresponding to the reference lane line and the target sampling point set, if the distance is less than the distance Threshold, indicating that the deviation between the reference sampling point set and the target sampling point set is small, then the reference sampling point set and the target sampling point set are used for curve fitting and sampling, and the updated sampling point set is used to generate an updated lane line, which can be It is understood that the target sampling point set is used to adjust the reference sampling point set to improve the accuracy of updating lane lines. If the degree of separation is greater than or equal to the separation degree threshold, it means that the error of the reference sampling point set is large, and the target sampling point is used directly. The set of points generates updated lane lines and replaces reference lane lines with updated lane lines, which improves the accuracy of lane lines in semantic maps.

In one embodiment, based on the set of target sampling points and the set of reference sampling points, calculating the distance between the target lane line and the reference lane line includes: acquiring target sampling points in the set of target sampling points; calculating the distance between the target sampling point and the reference The interval distance between the reference sampling points in the sampling point set is based on the interval distance to determine the two comparison sampling points corresponding to the target sampling point from the reference sampling point set; calculate the vertical distance between the target sampling point and the straight line where the two comparison sampling points are located; Count each vertical distance to obtain the degree of separation between the target lane line and the reference lane line.

A target sampling point in the target sampling point set is acquired. Exemplarily, the computer device acquires a target sampling point from the target sampling point set.

Calculate the separation distance between the target sampling point and the reference sampling point in the reference sampling point set, and determine two control sampling points corresponding to the target sampling point from the reference sampling point set based on the separation distance. For example, the computer device calculates the target sampling point According to the separation distance between each reference sampling point in the set of reference sampling points, two control sampling points corresponding to the target sampling point are selected.

In one embodiment, the computer device compares multiple separation distances corresponding to the target sampling point, and selects a reference sampling point corresponding to the shortest separation distance and a reference sampling point corresponding to the second shortest separation distance as comparison sampling points.

In one embodiment, the computer equipment adds up the separation distances to obtain the sum of the distances, divides the sum of the distances by the number of separation distances to obtain the average value of the separation distances, compares each separation distance with the average value of the separation distances, and selects the average value of the separation distance. The two reference interval distances with the closest values, and the reference sampling points corresponding to the two reference interval distances are used as control sampling points.

Calculate the vertical distance from the target sampling point to the straight line where the two control sampling points are located. Exemplarily, the computer device calculates the vertical distance from the target sampling point to the straight line where the two comparison points are located according to the calculation method of the vertical distance from the sampling point to the straight line in three-dimensional space. For example, as shown in Figure 28, the target sampling point A and the two control sampling points are respectively B and C, and A, B, and C are all represented by sampling coordinate points, and the target sampling point A is subtracted from the control sampling point B to obtain the vector BA, Control sampling point B minus control sampling point

C obtains the vector BC, cross-multiplies the vector BA and the vector BC to obtain the cross-product result, multiplies the length of the vector BA and the length of the vector BC to obtain the product result, divides the product result by the cross-product result to obtain the vector BA and vector BC The sine value of the included angle α, multiply the length of the vector BA by the sine value of α, and obtain the vertical distance from the target sampling point A to the straight line where the control sampling points B and C are located.

Count each vertical distance to obtain the degree of separation between the target lane line and the reference lane line. Exemplarily, the computer device performs statistics on each vertical distance according to a set calculation rule to obtain the degree of separation between the set of target sampling points and the set of reference sampling points.

In one embodiment, the computer device compares the respective vertical distances, and selects an intermediate value of the vertical distances as the degree of separation between the target sampling point set and the reference sampling point set.

In one embodiment, the computer device adds the vertical distance corresponding to each target sampling point in the target sampling point set, and divides the result obtained by the addition by the total number of target sampling points in the target sampling point set to obtain the target sampling point The distance between the set of points and the set of reference sampling points.

In this embodiment, two control sampling points are selected from the set of reference sampling points, and the vertical distance between the target sampling point and the straight line where the two control sampling points are calculated is calculated. The vertical distance can accurately represent the target sampling point to the reference sampling point set. The distance between each vertical distance is counted to obtain the degree of separation between the target sampling point set and the reference sampling point set, which can accurately represent the distance between the target sampling point set and the reference sampling point set, and improve Accuracy of distance calculation. In one embodiment, the lane line processing method further includes:

If there are multiple target lane lines in the target road section, then obtain the target sampling point set and pose error average value corresponding to the multiple target lane lines; select the target sample point set corresponding to the target lane line whose pose error average value is less than the error threshold, As a set of matching sampling points; each matching sampling point set is formed into a matching fusion sampling point set, and curve fitting and sampling are performed on the matching fusion sampling point set to obtain a matching target sampling point set; a matching target lane line is generated based on the matching target sampling point set .

Among them, the mean value of the pose error refers to the mean value of the pose point errors. It can be understood that the average value of the pose point errors corresponding to the target road segment can measure the accuracy of the pose point of the target road segment. The pose point error can be a relative pose error, an absolute trajectory error, and so on.

Exemplarily, the computer device obtains the target lane lines corresponding to the target road sections provided by multiple vehicles, and then obtains the set of target sampling points corresponding to each target lane line and the average value of the pose error, and combines each average pose error with If the average value of the pose error is less than the error threshold, it is determined that the set of target sampling points corresponding to the average value of the pose error is a set of matching sampling points, and each set of matching sampling points is composed of a set of matching fusion sampling points. The set of sampling points is fused for curve fitting and sampling to obtain a set of matching target sampling points, and the matching lane line is generated based on the set of matching target sampling points, and then the matching lane line is used as the lane line of the target road section in the semantic map.

In this embodiment, by comparing the average value of the pose error with the error threshold, the target sampling point set with the average value of the pose error smaller than the error threshold is selected as the set of matching sampling points. The average value of the pose error is small, indicating that the pose point The accuracy rate is high, and the accuracy rate of the target sampling point set corresponding to the average value of the pose error is high. The target sampling point set with high accuracy rate is used as the matching sampling point set, which improves the accuracy rate of the matching target sampling point set. Based on the matching target The set of sampling points generates the target lane line, which improves the accuracy of matching the target lane line.

The following is the application scenario of the semantic map update method, as shown in Figure 12, including three parts: original data processing, crowdsourcing mapping, positioning and loop detection. The 3D LiDAR point cloud data acquired by LiDAR) and the image (such as 2D camera image) collected by the image acquisition device (such as a camera) are perceived to obtain a newly collected crowdsourcing map, and the target object point is extracted from the newly collected crowdsourcing map. cloud; such as recognizing 2D camera images through the perceptual deep learning model, determining the 2D traffic light calibration frame and 2D lane line calibration frame, projecting the 3D LiDAR point cloud of the lidar into the 2D camera image, and extracting the points in the traffic light calibration frame, Then restore 3D, these points are regarded as the points of traffic lights, and the point cloud corresponding to the traffic lights is obtained; the perception identification points of the traffic signs in the point cloud are determined through the deep learning method of the perception group, and these points marked as traffic signs can be directly taken over As a traffic sign point cloud; through the perceptual deep learning model to perceive the 2D lane marking frame in the 2D camera image, the 3D LiDAR point cloud of the lidar is projected into the 2D camera image, extract the points in the lane marking frame, and then restore the 3D , these points are regarded as the points of the lane line; it also includes the perception of other objects with traffic signs to obtain the corresponding point cloud.

Construct the first point cloud object (such as the object object) according to the target object point cloud and extract the corresponding first semantic information, and perform crowdsourcing and mapping according to the constructed first point cloud object; further, determine whether the newly collected collection area has Figure (that is, whether there is a semantic map), if not, build the map according to the first point cloud object; when there is a semantic map, match the crowdsourced map with the existing semantic map through Hungarian matching, realize change detection, and detect the result Including adding objects (which can be understood as new point cloud objects in existing semantic maps), deleting objects (which can be understood as point cloud objects disappearing in existing semantic maps) and average objects (which can be understood as existing semantic maps and public Matching point cloud object pairs existing in the package map); update the semantic terrain according to the detection results. Wherein, for the specific limitations of the specific detection method, please refer to the limitation of the semantic map update method above, which will not be repeated here.

Further, positioning and loop detection are performed according to the updated semantic map, which improves the detection ability of loop detection, reduces cumulative errors, improves positioning accuracy and speed and avoids obstacles; among them, the positioning and loop detection methods can be used in existing methods implementation, and will not be repeated here. For example, after obtaining the updated semantic map, perform positioning and loop detection on the semantic map, match objects and lane lines, and obtain the positioning result. Among them, loopback detection, also known as closed-loop detection, refers to the ability of the device to identify that it has reached a certain scene and make the map closed-loop, that is, it can match the map generated at the moment with the map just generated.

In the above semantic map update method, by extracting the first point cloud object of the target from the newly collected crowdsourcing map, and extracting the first semantic information of the first point cloud object, that is, under the premise of retaining useful information, the overall size of the map is greatly increased. Zoom out, and get more map information while extracting semantic information and compressing the data size; determine the matching result of crowdsourcing map and semantic map based on the semantic distance determined by semantic information; realize adding to semantic map according to different matching results Adding and deleting objects and averaging matching objects provide a guarantee for map matching updates, thereby improving the accuracy of map updates.

Please refer to FIG. 13 . FIG. 13 is a schematic flowchart of a path planning method according to an embodiment of the present application. The subject of execution of the method may be an electronic device with a computing function, for example, a microcomputer, a server, and mobile devices such as a notebook computer and a tablet computer. It should be noted that the method of the present application is not limited to the flow sequence shown in FIG. 13 if substantially the same result is obtained. In some possible implementation manners, the method may be implemented by a processor calling a computer-readable instruction stored in a memory, as shown in FIG. 13 , the method may include the following steps:

1302: Obtain the original semantic map.

The crowdsourced lane line map is also a semantic map, which can be a partial crowdsourced lane line map generated during vehicle driving. The crowdsourced map is a vehicle with environmental awareness, which collects the surrounding environment and road information data during the driving process. For example, traffic element information such as roads, traffic signs, lane lines, obstacles, etc., the collected information data is uploaded to the cloud, and the cloud builds a highly restored and instantly updated driving map based on the feedback data.

The original semantic map is constructed by receiving information collected by sensor devices. The sensor devices include image sensors and radar sensors. Radar sensors can be used for autonomous driving and meet accuracy requirements, and are used to provide point cloud perception. The image data may be collected by an image sensor, such as a camera. Use radar sensors, such as millimeter-wave radar, lidar, etc., to collect point cloud data. Image sensors and radar sensors can be mounted on a mobile device, such as an autonomous vehicle. Lidar can include mechanical Lidar, semi-solid Lidar, or solid-state Lidar.

In an application scenario, the self-driving vehicle is driving on the road, and the image data used to describe the environment space where the vehicle equipment is located is obtained through the image sensor installed on the self-driving vehicle to obtain an initial data set; the radar sensor is used to obtain It is used to describe the point cloud data of the environment space in which the vehicle equipment is located, and obtain an initial data set. Each sensor perceives and captures an initial data set used to describe the environmental space where the on-board device is located, and each initial data set corresponds to a sensor, and then at least two sensors capture at least two initial data sets, wherein the type of the initial data set Including but not limited to image data and point cloud data.

S1304: Perform path planning according to the original semantic map.

According to the original semantic map, obtain traffic information such as roads, traffic signs, lane lines, obstacles, etc. in the semantic map, and adjust the steering, speed, path planning, lane change, etc. of running vehicles to achieve safe driving on the road. Wherein, the original semantic map is obtained through the above-mentioned updating method of the semantic map, which will not be repeated here.

Please refer to FIG. 14 . FIG. 14 is a schematic structural diagram of an apparatus for updating a semantic map according to an embodiment of the present application. The apparatus 140 for updating a semantic map includes an acquisition module 141 , a first extraction module 142 , a second extraction module 143 and an update module 144 . The acquiring module 141 is configured to acquire at least one object point cloud corresponding to the current acquisition area.

The first extraction module 142 is configured to construct at least one first point cloud object and extract corresponding first semantic information according to at least one object point cloud, wherein at least one first point cloud object and corresponding first semantic information are used to construct a crowd Package Semantic Maps. The second extraction module 143 is configured to obtain the original semantic map in response to the existence of the original semantic map in the current collection area, and obtain at least one second point cloud object and corresponding second semantic information from the original semantic map. The update module 144 is configured to obtain a matching result of at least one first point cloud object and at least one second point cloud object according to the first semantic information and the second semantic information, and use at least one of the crowdsourcing semantic maps according to the matching result A first point cloud object to update the original semantic map.

Please refer to FIG. 15 . FIG. 15 is a schematic structural diagram of a path planning device according to an embodiment of the present application. The path planning device 150 includes an acquisition module 151 and a path planning module 314 . The obtaining module 151 is configured to obtain an original semantic map, wherein the original semantic map is obtained through the above-mentioned semantic map update method.

The path planning module 314 is configured to perform path planning according to the original semantic map. Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.

Please refer to FIG. 16 . FIG. 16 is a schematic structural diagram of a computer device according to an embodiment of the present application. The computer device 160 includes a memory 161 and a processor 162 coupled to each other, and the processor 162 is used to execute the program instructions stored in the memory 161, so as to realize the steps of the above embodiment of the method for updating the semantic map, or to realize the implementation of the above method for path planning. example steps. In a specific implementation scenario, the computer device 160 may include but not limited to: a microcomputer and a server, which are not limited here.

Specifically, the processor 162 is configured to control itself and the memory 161 to implement the steps of the above embodiment of the semantic map updating method, or to implement the steps of the above embodiment of the path planning method. The processor 162 may also be referred to as a CPU (Central Processing Unit, central processing unit), and the processor 162 may be an integrated circuit chip having a signal processing capability. The processor 162 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. In addition, the processor 162 may be jointly realized by an integrated circuit chip.

Please refer to FIG. 17 . FIG. 17 is a schematic structural diagram of a non-volatile computer-readable storage medium according to an embodiment of the present application. The non-volatile computer-readable storage medium 170 is used to store a computer program 1701. When the computer program 1701 is executed by a processor, for example, when executed by the processor 162 in the embodiment of FIG. The steps in the embodiment of the method are updated, or to implement the steps in the embodiment of the path planning method above.

The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments, the same or similar points can be referred to each other, and for the sake of brevity, details are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed methods and related devices may be implemented in other ways. For example, the related device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components may be combined or Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication disconnection shown or discussed may be through some interfaces, and the indirect coupling or communication disconnection of devices or units may be in electrical, mechanical or other forms.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

An integrated unit may be stored in a computer-readable storage medium if it is realized in the form of a software function unit and sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Those skilled in the art can easily understand that many modifications and changes can be made to the devices and methods while maintaining the teaching content of the present application. Accordingly, the above disclosure should be seen as limited only by the scope of the appended claims.

Claims

A method for updating a semantic map, characterized in that the method comprises:

Extract the point cloud of the target object from the crowdsourced map corresponding to the current collection area;

Constructing a first point cloud object according to the target object point cloud and extracting corresponding first semantic information;

When there is a corresponding semantic map in the current acquisition area, acquire all second point cloud objects and corresponding second semantic information in the semantic map;

Matching the crowdsourced map with the semantic map according to the first semantic information and the second semantic information to obtain a matching result;

The semantic map is updated according to the matching result.
The method according to claim 1, further comprising:

In response to the fact that the original semantic map does not exist in the current collection area, the crowdsourced semantic map is created by using the at least one first point cloud object and the first semantic information.
The method according to claim 1, wherein the at least one first point cloud object and the at least one second point cloud object are obtained according to the first semantic information and the second semantic information. Matching results, including:

determining a semantic distance between the first point cloud object and the second point cloud object according to the first semantic information and the second semantic information;

Based on the semantic distance, a matching result of the first point cloud object and the second point cloud object is obtained.
The method according to claim 3, wherein the semantic distance between the first point cloud object and the second point cloud object is determined according to the first semantic information and the second semantic information ,include:

Determine the first semantic information and the second semantic information, calculate the center position coordinate distance difference between the first point cloud object and the second point cloud object, point cloud object direction difference, and calibration frame size differences and differences in appearance characteristics;

Perform weighting processing on the center position coordinate distance difference value, the point cloud object direction difference value, the calibration frame size difference value and the appearance feature difference value to obtain the difference between the first point cloud object and the second point cloud object semantic distance between them.
The method according to claim 4, wherein the first semantic information and the second semantic information both include point cloud center coordinates, point cloud calibration frame size, point cloud main direction and point cloud histogram;

According to the first semantic information and the second semantic information, determine the center position coordinate distance difference value and the point cloud object direction difference value between the first point cloud object and each of the second point cloud objects , the difference value of the calibration frame size and the difference value of the appearance feature, including:

Determine the center position coordinate distance difference value according to the point cloud center coordinates of the first point cloud object and the point cloud center coordinates of the second point cloud object;

Determine the direction difference value of the point cloud object according to the point cloud main direction of the first point cloud object and the point cloud main direction of the second point cloud object;

Determine the difference value of the calibration frame size according to the calibration frame size of the first point cloud object and the calibration frame size of the second point cloud object;

The appearance feature difference value is determined according to the shape element of the point cloud histogram of the first point cloud object and the shape element of the point cloud histogram of the second point cloud object.
The method according to claim 1, wherein the at least one first point cloud object and the at least one second point cloud object are obtained according to the first semantic information and the second semantic information Matching results for , including:

Acquiring the number n of the at least one first point cloud object and the number m of the at least one second point cloud object to obtain an association matrix of n*m;

Using the semantic distance between the first point cloud object and the second point cloud object as an element of the affinity matrix;

According to a predetermined threshold and the correlation matrix, a matching result of the at least one first point cloud object and the at least one second point cloud object is obtained.
The method according to claim 6, wherein said obtaining the matching result of said at least one first point cloud object and said at least one second point cloud object according to a predetermined threshold and said correlation matrix comprises:

When there is a first element in the correlation matrix that is greater than or equal to the predetermined threshold, it is determined that the first point cloud object corresponding to the first element does not match the second point cloud object.
The method according to claim 6, wherein said obtaining the matching result of said at least one first point cloud object and said at least one second point cloud object according to a predetermined threshold and said correlation matrix comprises:

When at least one second element in the correlation matrix is smaller than the predetermined threshold, it is determined that there is at least one matching second point cloud object in the first point cloud object corresponding to the second element;

Segmenting the incidence matrix based on the second element to obtain several subgraphs;

Perform bipartite graph matching on each of the subgraphs to determine a second point cloud object that matches the first point cloud object.
The method according to claim 8, wherein said performing bipartite graph matching on each of said subgraphs to determine a second point cloud object matched with said first point cloud object comprises:

Perform bipartite graph matching on each of the sub-graphs, and determine the matching cost value of the first point cloud object in the sub-graph and each of the second point cloud objects;

It is determined that the second point cloud object corresponding to the matching cost value with the smallest value is the matching point cloud object of the first point cloud object.
The method according to claim 1, wherein the updating of the original semantic map by using the at least one first point cloud object in the crowdsourced semantic map according to the matching result comprises:

When the matching result of the first point cloud object in the crowdsourcing semantic map and the second point cloud object in the original semantic map is that the first point cloud object is a newly added point cloud object, the adding a first point cloud object to said raw semantic map;

When the matching result is that the second point cloud object is a vanishing point cloud object, the second point cloud object is deleted from the original semantic map.
The method according to claim 1, wherein the updating of the original semantic map by using the at least one first point cloud object in the crowdsourced semantic map according to the matching result comprises:

Perform semantic averaging processing on the matching point cloud objects that successfully match the first point cloud object among the first point cloud object and the at least one second point cloud object to obtain a semantic average point cloud object;

replacing the matching point cloud object in the original semantic map with the semantic average point cloud object, so as to update the original semantic map.
The method according to claim 11, wherein the matching point cloud object of the first point cloud object and the at least one second point cloud object that is successfully matched with the first point cloud object is performed Semantic average processing to obtain semantic average point cloud objects, including:

Obtain the matching point cloud object and its second semantic information;

By performing semantic averaging processing on the first semantic information of the first point cloud object and the second semantic information of the matching point cloud object, the average semantic information of the updated point cloud object is obtained, thereby obtaining the semantic average point cloud object .
The method according to claim 12, wherein the updated point cloud is obtained by performing semantic averaging processing on the first semantic information of the first point cloud object and the second semantic information of the matching point cloud object The average semantic information of the object, including:

Perform semantic averaging processing according to the first semantic information and the second semantic information, update the first semantic information of the first point cloud object, and obtain the updated average semantic information of the first point cloud object; The average semantic information includes at least any one of the updated PCA coordinate system direction, the updated point cloud center, the updated vertex coordinates of the bounding box, the updated convex hull of the point cloud, and the updated histogram.
The method according to claim 13, wherein the semantic averaging process is performed according to the first semantic information and the second semantic information, and the first semantic information of the first point cloud object is updated to obtain an updated After the average semantic information of the first point cloud object, including:

Interpolating the direction of the PCA coordinate system in the first semantic information and the direction of the PCA coordinate system in the second semantic information to obtain the updated direction of the PCA coordinate system;

performing mean value processing on the point cloud center in the first semantic information and the point cloud center in the second semantic information to obtain the updated point cloud center;

The updated average semantic information of the first point cloud object is obtained according to the updated PCA coordinate system direction and the updated point cloud center.
The method according to claim 13, wherein the semantic averaging process is performed according to the first semantic information and the second semantic information, and the first semantic information of the first point cloud object is updated to obtain an updated After the average semantic information of the first point cloud object, including:

Perform mean value processing and coordinate conversion processing on the minimum bounding box of the point cloud in the first semantic information and the minimum bounding box of the point cloud in the second semantic information to obtain the vertex coordinates of the updated bounding box;

The updated average semantic information of the first point cloud object is obtained according to the updated PCA coordinate system direction, the updated point cloud center, and the vertex coordinates of the updated bounding box.
The method according to claim 15, wherein the mean value processing and coordinate conversion processing are performed on the point cloud minimum bounding box in the first semantic information and the point cloud minimum bounding box in the second semantic information , get the vertex coordinates of the updated bounding box, including:

Perform mean value processing on the point cloud minimum bounding box in the first semantic information and the point cloud minimum bounding box in the second semantic information to obtain the updated point cloud minimum bounding box;

Based on the updated PCA coordinate system direction, coordinate conversion is performed on the updated point cloud center, and converted to the updated object coordinate system to obtain the center coordinates of the target point cloud;

According to the center coordinates of the target point cloud and the minimum bounding box of the updated point cloud, the vertex coordinates of the updated bounding box are obtained.
The method according to claim 16, wherein the obtaining the vertex coordinates of the updated bounding box according to the center coordinates of the target point cloud and the minimum bounding box of the updated point cloud includes:

Determine the vertex coordinates according to the center coordinates of the target point cloud and the size information of the minimum bounding box of the updated point cloud; the size information includes width, height and depth;

Wherein, the vertex coordinates of the updated bounding box on the x-axis are obtained according to the x-axis coordinates of the center of the target point cloud and the width;

Obtaining the vertex coordinates of the updated bounding box on the y-axis according to the y-axis coordinates of the center of the target point cloud and the height; and

Obtain the vertex coordinates of the updated bounding box on the z-axis according to the z-axis coordinates of the center of the target point cloud and the depth.
The method according to claim 13, wherein the semantic averaging process is performed according to the first semantic information and the second semantic information, and the first semantic information of the first point cloud object is updated to obtain an updated After the average semantic information of the first point cloud object, including:

Perform translation, rotation and coordinate transformation processing on the point cloud convex hull in the first semantic information and the point cloud convex hull in the second semantic information to obtain the updated point cloud convex hull;

According to the updated PCA coordinate system direction, the updated point cloud center, the vertex coordinates of the updated bounding box and the updated point cloud convex hull, obtain the updated average semantic information of the first point cloud object .
The method according to claim 13, wherein the semantic averaging process is performed according to the first semantic information and the second semantic information, and the first semantic information of the first point cloud object is updated to obtain an updated After the average semantic information of the first point cloud object, including:

performing coordinate system transformation according to the object point cloud of the first point cloud object to update the histogram in the second semantic information to obtain the updated histogram;

According to the updated PCA coordinate system direction, updated point cloud center, updated bounding box vertex coordinates, updated point cloud convex hull and updated histogram, the updated average semantic information of the first point cloud object is obtained.
The method according to claim 1, wherein the matching result of the at least one first point cloud object and the at least one second point cloud object comprises the first unmatched point cloud in the crowdsourcing semantic map an object set and the second unmatched point cloud object set in the original semantic map;

The updating of the original semantic map by using the at least one first point cloud object according to the matching result includes:

Judging whether there is a bounding box collision between each first unmatched point cloud object in the first unmatched point cloud object set and a corresponding point cloud object within a preset semantic distance range in the original semantic map;

Judging whether there is a bounding box collision between each second unmatched point cloud object in the second unmatched point cloud object set and the corresponding point cloud object within the preset semantic distance range in the crowdsourcing semantic map;

The original semantic map is updated according to the bounding box collision result of the first unmatched point cloud object and/or the bounding box collision result of the second unmatched point cloud object.
The method according to claim 20, wherein, according to the bounding box collision result of the second unmatched point cloud object, updating the original semantic map includes:

When the bounding box collision result of the second unmatched point cloud object is that there is no bounding box collision for the second unmatched point cloud object, delete the second unmatched point cloud object from the original semantic map .
The method according to claim 20, wherein, according to the bounding box collision result of the first unmatched point cloud object, updating the original semantic map includes:

When the bounding box collision result of the first unmatched point cloud object is that there is no bounding box collision for the first unmatched point cloud object, adding the first unmatched point cloud object to the original semantic map .
The method according to claim 20, wherein, according to the bounding box collision result of the first unmatched point cloud object and the bounding box collision result of the second unmatched point cloud object, comprising:

When the bounding box collision result of the first unmatched point cloud object is that there is no bounding box collision for the first unmatched point cloud object, adding the first unmatched point cloud object to the original semantic map ;as well as

When the bounding box collision result of the second unmatched point cloud object is that there is no bounding box collision for the second unmatched point cloud object, delete the second unmatched point cloud object from the original semantic map .
The method according to claim 20, wherein, according to the bounding box collision result of the first unmatched point cloud object, updating the original semantic map includes:

When the bounding box collision result of the first unmatched point cloud object is a bounding box collision of the first unmatched point cloud object, determine the corresponding first collision point within a preset distance range from the original semantic map cloud objects;

According to the first unmatched point cloud object and the first collision point cloud object, the point cloud proportion is obtained;

Using the point cloud occupancy and setting a threshold, the original semantic map is updated.
The method according to claim 24, wherein said updating said original semantic map using said point cloud occupancy and setting a threshold comprises:

In response to the point cloud proportion being greater than or equal to the set threshold, it is determined that the first unmatched point cloud object and the first collision point cloud object are the same object;

In response to the point cloud proportion being less than the set threshold, it is determined that the first unmatched point cloud object and the first collision point cloud object are not the same object, and then the first unmatched point cloud object is added to the original semantic map.
The method according to claim 24, wherein said obtaining the point cloud proportion according to said first unmatched point cloud object and said first collision point cloud object comprises:

Determining that the first unmatched point cloud object corresponds to the first object point cloud, and the first point cloud object convex hull corresponding to the first collision point cloud object;

Determining the number of points in the point cloud of the first object in the convex hull of the first point cloud object to obtain the proportion of the point cloud.
The method according to claim 24, wherein, before obtaining the point cloud ratio for the first unmatched point cloud object and the first collision point cloud object, the method further comprises:

judging whether there is a convex hull collision between the first unmatched point cloud object and the first colliding point cloud object;

When there is a convex hull collision, perform the step of obtaining the point cloud proportion according to the first unmatched point cloud object and the first collided point cloud object;

When there is no convex hull collision, determine that the first unmatched point cloud object and the first collision point cloud object are not the same object, so as to add the first unmatched point cloud object to the original semantic map middle.
The method according to claim 20, wherein, according to the bounding box collision result of the second unmatched point cloud object, updating the original semantic map includes:

When the bounding box collision result of the second unmatched point cloud object is that there is a bounding box collision for the second unmatched point cloud object, determine the corresponding second collision within a preset distance range from the crowdsourcing semantic map point cloud object;

judging whether there is a convex hull collision between the second unmatched point cloud object and the second colliding point cloud object;

When there is no convex hull collision, it is determined that the second unmatched point cloud object and the second collision point cloud object are not the same object, so as to remove the second unmatched point cloud object from the original semantic map delete;

When there is a convex hull collision, it is determined that the second unmatched point cloud object and the second colliding point cloud object are the same object.
The method according to claim 1, wherein the current collection area includes a target road section;

Said updating said original semantic map also includes:

Obtaining multiple frames of images of the target road section and the pose points corresponding to each frame of the image; determining the road section characteristics and road section speed of the target road section based on the pose points;

The multi-frame images are selected based on the road section features and the road section speed to obtain a target image, and then a target lane line of the target road section is generated based on the target image.
The method according to claim 29, wherein said selecting said multi-frame images based on said road section features and said road section speed, and obtaining a target image comprises:

If the road section speed of the target road section is zero, then select a frame image from the multiple frame images as the target image;

If the target road section is a straight line road section and the speed of the road section is not zero, the number of targets is determined based on the speed of the road section, and the number of target images is selected from the multiple frames of images;

If the target road section is a curved road section and the speed of the road section is not zero, the multi-frame images are used as the target image.
The method according to claim 29, wherein said generating the target lane line of the target section based on the target image comprises:

Obtain a set of three-dimensional sampling points corresponding to the target image in each frame;

Collecting and combining each of the three-dimensional sampling points into a fusion sampling point set;

Curve fitting and sampling are performed on the fused sampling point set to obtain a target sampling point set to generate a target lane line.
The method according to claim 26, wherein said generating the target lane line of the target section based on the target image comprises:

If the speed of the road section corresponding to the target road section is not zero and includes a straight road section and a curved road section, then select the first target image corresponding to the straight road section and the second target image corresponding to the curved road section;

Composing the three-dimensional sampling point set corresponding to the first target image into a first sampling point set, and combining the three-dimensional sampling point set corresponding to the second target image into a second sampling point set;

Curve fitting and sampling are respectively performed on the first sampling point set and the second sampling point set to obtain a target sampling point set to generate a target lane line.
The method according to claim 26, further comprising:

If there is a reference lane line in the target road section, then obtain a set of reference sampling points corresponding to the reference lane line;

calculating the distance between the target lane line and the reference lane line based on the target sampling point set and the reference sampling point set;

Comparing the degree of separation with a degree of separation threshold, if the degree of separation is less than the threshold of degree of separation, performing curve fitting and sampling on the set of reference sampling points and the set of target sampling points to obtain an updated A set of sampling points, generating an updated lane line based on the set of updated sampling points; if the degree of separation is equal to or greater than the threshold of the degree of separation, generating an updated lane line based on the set of target sampling points to update the reference lane line.
The method according to claim 33, wherein the calculating the distance between the target lane line and the reference lane line based on the set of target sampling points and the set of reference sampling points comprises:

Acquiring target sampling points in the set of target sampling points;

calculating the separation distance between the target sampling point and the reference sampling point in the reference sampling point set, and determining two control sampling points corresponding to the target sampling point from the reference sampling point set based on the separation distance;

Calculate the vertical distance from the target sampling point to the straight line where the two comparison sampling points are located;

The respective vertical distances are counted to obtain the degree of separation between the target lane line and the reference lane line.
The method according to claim 26, further comprising:

If there are multiple target lane lines in the target road section, then obtain a plurality of target sampling point sets and pose error averages corresponding to the target lane lines;

Selecting the set of target sampling points corresponding to the target lane line whose mean value of the pose error is less than the error threshold is used as a set of matching sampling points;

Composing each set of matching sampling points into a set of matching and fusion sampling points, performing curve fitting and sampling on the set of matching and fusion sampling points to obtain a set of matching target sampling points; generating a matching target lane line based on the set of matching target sampling points .
A path planning method, characterized in that, comprising:

Get the original semantic map;

Carry out path planning according to the original semantic map;

Wherein, the original semantic map is obtained through the method for updating the semantic map according to any one of claims 1-35.
A device for updating a semantic map, characterized in that the device comprises:

An acquisition module, configured to acquire at least one object point cloud corresponding to the current acquisition area;

The first extraction module is configured to construct at least one first point cloud object and extract corresponding first semantic information according to the at least one object point cloud, wherein the at least one first point cloud object and the corresponding first semantic information are used for building crowdsourced semantic maps;

The second extraction module is configured to acquire the original semantic map in response to the existence of the original semantic map in the current acquisition area, and acquire at least one second point cloud object and corresponding second semantic information from the original semantic map;

An update module, configured to obtain a matching result of the at least one first point cloud object and the at least one second point cloud object according to the first semantic information and the second semantic information, and obtain a matching result according to the matching result , using the at least one first point cloud object in the crowdsourced semantic map to update the original semantic map.
A path planning device, characterized in that it comprises:

An acquisition module, configured to acquire an original semantic map, wherein the original semantic map is obtained through the method for updating a semantic map according to any one of claims 1-35;

A path planning module, configured to perform path planning according to the original semantic map.
A computer device, comprising a memory and a processor, the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 36 when executing the computer program.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 36 are realized.