CN114445644A

CN114445644A - Laser point cloud parameterized semantic feature matching method, device, equipment and storage medium

Info

Publication number: CN114445644A
Application number: CN202011605737.8A
Authority: CN
Inventors: 陈贵宾; 邓欢军; 张硕
Original assignee: Alibaba Group Holding Ltd
Current assignee: Wuzhou Online E Commerce Beijing Co ltd
Priority date: 2020-10-30
Filing date: 2020-12-29
Publication date: 2022-05-06

Abstract

The embodiment of the invention provides a laser point cloud parameterized semantic feature matching method, device, equipment and storage medium. The laser point cloud parameterization semantic feature matching method comprises the following steps: obtaining semantic point cloud for parameterization from a laser point cloud frame; segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; carrying out parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame; combining a plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs; and performing line-to-line and face-to-face feature matching at least based on the current parameterized semantic feature frame and the parameterized semantic feature sub-graph to obtain the pose of the laser radar. The embodiment of the invention improves the accuracy of the laser radar pose obtained based on the laser point cloud geometric feature matching.

Description

Laser point cloud parameterization semantic feature matching method, device, equipment and storage medium

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a laser point cloud parameterized semantic feature matching method, device, equipment and storage medium.

Background

The real-time positioning mapping (SLAM) provides a data basis for control decision-making for an intelligent driving or intelligent robot, and is very important for safe operation of the intelligent driving or intelligent robot, and the laser radar has the characteristics of high measurement precision and no influence of ambient light, so that the positioning mapping based on the laser radar is a mainstream scheme of positioning mapping. In the positioning and mapping scheme based on the laser radar, laser mileage calculation is a basic technology, and the higher the accuracy of the laser mileage calculation is, the more beneficial to positioning and mapping are.

The conventional laser odometer calculation is generally based on the geometrical characteristics of laser point cloud to match a laser point cloud frame (short for point cloud frame) with a laser point cloud sub-map (short for sub-map) so as to obtain the pose of the laser radar, but when the surface of the sub-map has excessive noise, the accuracy of matching the point cloud frame with the sub-map is influenced, and further the accuracy of the determined pose of the laser radar is influenced. Therefore, how to improve the accuracy of the laser radar pose obtained based on the laser point cloud geometric feature matching becomes a problem to be solved by the technical personnel in the field.

Disclosure of Invention

In view of the above, embodiments of the present invention provide a method, an apparatus, a device, and a storage medium for matching parameterized semantic features of laser point cloud to solve or alleviate the above problems.

According to a first aspect of the embodiments of the present invention, there is provided a laser point cloud parameterization semantic feature matching method, including: obtaining semantic point cloud for parameterization from a laser point cloud frame; segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; carrying out parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame; combining a plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs; and performing line-to-line and face-to-face feature matching at least based on the current parameterized semantic feature frame and the parameterized semantic feature sub-image to obtain the pose of the laser radar for collecting the laser point cloud frame.

According to a second aspect of the embodiments of the present invention, there is provided a laser point cloud parameterized semantic feature matching device, including: the acquisition module acquires semantic point cloud for parameterization from the laser point cloud frame; the segmentation module is used for segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; the parameterization processing module is used for carrying out parameterization processing on the semantic point cloud subsection to obtain parameterization semantic features of corresponding types; the first combination module is used for combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame; the second combination module is used for combining the plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs; and the matching module is used for performing line-to-line and surface-to-surface feature matching at least on the basis of the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar for collecting the laser point cloud frame.

According to a third aspect of embodiments of the present invention, there is provided an electronic apparatus, the apparatus including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the method according to the first aspect.

According to a fourth aspect of embodiments of the present invention, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to the first aspect.

In the scheme of the embodiment of the invention, because the semantic point cloud segmentation of the corresponding type comprises reliable local information, the semantic point cloud segmentation is subjected to parameterization, and the local information is effectively extracted, so that line-to-line and surface-to-surface feature matching is carried out based on the parameterized semantic feature frame obtained by the information, and higher-precision feature frame matching can be carried out (for example, on the basis of the traditional geometric feature matching), so that the pose of the laser radar with higher precision is obtained.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present invention, and it is also possible for a person skilled in the art to obtain other drawings based on the drawings.

FIG. 1 is a schematic flow chart diagram of a feature matching method of one embodiment of the present invention;

FIG. 2 is a schematic flow chart diagram of a feature matching method of another embodiment of the present invention;

FIG. 3 is a schematic diagram of a gridding process according to another embodiment of the present invention;

FIG. 4A is a diagram of a parameterized semantic feature frame according to another embodiment of the present invention;

FIG. 4B is a schematic diagram of parametric semantic feature matching for a shaft according to another embodiment of the present invention;

FIG. 5 is a schematic block diagram of a feature matching apparatus of another embodiment of the present invention;

fig. 6 is a hardware configuration of an electronic device according to another embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.

The following further describes specific implementation of the embodiments of the present invention with reference to the drawings. Fig. 1 is a schematic flow chart of a feature matching method of an embodiment of the present invention. Positioning mapping is critical for autonomous driving or other robotic systems. The laser point cloud parameterized semantic feature matching method of fig. 1 performs localization mapping based on a laser point cloud frame acquired by a laser radar, and may be executed by any suitable electronic device with data processing capability, including but not limited to: server, mobile terminal (such as mobile phone, PAD, etc.), PC, etc. The laser radar (Lidar) has the characteristics of high measurement accuracy and no influence of ambient light. A positioning and mapping algorithm based on Lidar is a mainstream scheme in the field of automatic driving. The laser mileage calculation method (LO) is a basic problem in positioning mapping, and the higher the precision of the LO, the more beneficial the positioning mapping is. The embodiment of the invention can be based on the scene of LO. Laser SLAM is a SLAM algorithm that utilizes a cloud of laser points. The laser odometer Lidar-only odometer (LO) addresses the laser SLAM front end problem. That is, the higher the accuracy of the LO, the smaller the trajectory drift problem, and the more beneficial the loop optimization at the back end, the more the overall accuracy and global consistency of the SLAM are improved. The feature matching method of fig. 1 includes:

110: and obtaining semantic point cloud for parameterization from the laser point cloud frame.

It should be understood that a laser point cloud that has undergone motion compensation processing may be input. For example, a point cloud with semantic information may be obtained as a semantic point cloud for parameterization by running a point cloud semantic segmentation algorithm (e.g., RangeNet + +, Pointnet + +, KPConv, etc.). The point cloud semantics in the text is that each point cloud is attached with a semantic label, such as "ground", "building", "rod", "traffic signboard" and the like, by a semantic segmentation algorithm.

120: and segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types.

It should be understood that semantic point cloud segmentation has more local information. Segmentation can be performed through clustering, plane segmentation, meshing, fitting and the like to obtain semantic point cloud segmentation. Fitting herein is to represent the point cloud of a given area as a mathematical equation that describes the shape of the point cloud of that area. Further, types of semantic point clouds include, but are not limited to, ground tags, shaft tags, traffic sign tags, building tags, other road or ground identifiers, and the like. Preferably, the types of semantic point clouds include ground tags, shaft tags, traffic sign tags, and building tags.

130: and carrying out parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types.

It should be understood that the objects of the parameterization process may be parameters indicating local information of the semantic point cloud segmentation. The parameters herein may include at least one of vectorization parameters, confidence, bounding box center point, and semantic label. Preferably, the parameters in the parameterization process include vectorization parameters, confidence, bounding box center points, and semantic labels. Vectorized Semantic Feature (Vectorized Semantic Feature, which is an example of a parameterized Semantic Feature) as a parameterized Semantic Feature is a lightweight point cloud Feature of an embodiment of the present invention, and Vectorized Feature extraction is performed on Semantic point cloud to construct at least one type of parameterized Semantic Feature. For example, a ground vectorization semantic feature, a building vectorization semantic feature, a rod vectorization semantic feature, and a traffic sign vectorization semantic feature are constructed.

140: and combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame.

It is to be understood that the respective types of parameterized semantic features may be combined based on the coordinate system of the lidar. It is also to be understood that at least two types of parameterized semantic features may be combined, and that the respective types of parameterized semantic features may comprise coordinate information based on a lidar coordinate system.

150: and combining the plurality of historical parameterized semantic feature frames into a parameterized semantic feature subgraph.

It should be understood that the current parameterized semantic feature frame may be constructed after four parameterized semantic features (e.g., vectorized semantic features) are extracted from the current frame point cloud. And converting the plurality of historical parameterized semantic feature frames to a global coordinate system (world coordinate system) through the calculated positions to form a parameterized semantic feature subgraph. Parameterization such as vectorization in this context means that a mathematical equation is represented by fitting a given point cloud, and certain parameters (confidence, semantic label, bounding box center point, etc.) are recorded to represent some characteristics of the point cloud.

In one example, the number of the plurality of historical parameterized semantic feature frames may be fixed, that is, the historical parameterized semantic feature frames used for feature matching at different times are all the fixed number. In another example, a parameterized semantic feature subgraph may also be generated based on all historical parameterized semantic feature frames prior to a particular point in time.

160: and performing line-to-line and face-to-face feature matching at least based on the current parameterized semantic feature frame and the parameterized semantic feature sub-image to obtain the pose of the laser radar for collecting the laser point cloud frame.

It should be understood that the pose herein refers to the position (x, y, z) and attitude angle of the lidar or the vehicle on which the lidar is mounted, and may include roll angle, pitch angle, and yaw angle, for example. A trajectory in this context refers to a sequence of sets of vehicle poses. The real-time positioning and mapping algorithm in the text calculates the track of the vehicle for a given image or laser point cloud. The method generally comprises a front-end mileage calculation method and a rear-end loop detection algorithm, wherein the front end is responsible for calculating an initial track and solving the problem of drift (drift) caused by accumulated errors, and the rear end is responsible for eliminating the problem of the accumulated errors of the front end.

It should also be understood that point cloud registration/matching is to solve a transformation matrix of point clouds with different postures under the same coordinate by using two point clouds with partially overlapped or completely overlapped areas. The matrix can be used to achieve accurate registration of the two point clouds. For example, Point cloud registration may be performed using, for example, Iterative Closest Point (IP) registration. ICP registration refers to a point cloud registration algorithm that establishes associations between points by nearest neighbor search (KNN).

It is also understood that the line-to-line match may be a match for a shaft and the face-to-face match may be a match for at least one of the ground, a traffic sign, and a building. For example, the shaft parameterized semantic features perform line-to-line matching, and the matching process of the ground parameterized semantic features, the traffic sign parameterized semantic features, or the building parameterized semantic features is similar to the shaft parameterized semantic features described above, except that face-to-face matching constraints are constructed. Then, the four types of parameterized semantic feature matching constraints are combined with the frame-to-subgraph constraints in the geometric feature matching in a tightly coupled mode to jointly estimate the graph matching pose.

It should also be understood that in addition to line-to-line and face-to-face feature matching based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph, geometric feature matching may also be performed based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph.

It should also be appreciated that the first initial pose may be estimated based on parametric feature matching and geometric feature matching of the current parametric semantic feature frame with the parametric semantic feature subgraph. The second initial pose may be determined based on inter-frame matching between the current parameterized semantic feature frame and a last parameterized semantic feature frame. The pose of the lidar that collects the laser point cloud frame may be determined based on the first initial pose and the second initial pose.

In addition, SuMa + + calculates the vehicle pose by using the laser point cloud, and improves the positioning accuracy by combining the semantic information of the point cloud. But SuMa + + has only a surfel-based point-to-surface point cloud matching constraint, which is limited. The scheme of the embodiment of the invention not only has the traditional point-to-surface and point-to-line constraints, but also has the line-to-line and surface-to-surface constraints based on the parameterized semantic features, thereby improving the accuracy of the laser radar pose obtained based on the laser point cloud geometric feature matching.

In addition, for the LOAM, when performing point cloud registration from a frame to a subgraph, the subgraph has a "ghost" due to a previous positioning error, and the registration based on the KNN nearest neighbor search has a problem of matching to a point cloud surface, which further affects the accuracy of the registration, resulting in an increase in the drift problem of the LO. The algorithm of the embodiment of the invention adds parameterized semantic feature matching, and can alleviate or even eliminate the problem.

In addition, the IMLS-SLAM can solve the problem of matching to the surface of the point cloud by constructing an implicit moving least square surface method, but the method needs to calculate the normal vector of the point cloud, needs to maintain a large subgraph, has low data processing efficiency and cannot realize real-time performance. Compared with the scheme, the scheme of the embodiment of the invention has the advantages that the lightweight vectorization semantic feature parameterization semantic feature can be constructed through the point cloud semantic information, and the problem of surface matching is solved through the matching of the parameterization semantic feature, so that the data processing efficiency is improved, and the real-time performance can be realized on the premise of ensuring the good LO precision due to the use of the two-thread structure of the LOAM.

In another implementation of the present invention, obtaining a semantic point cloud for parameterization from a laser point cloud frame includes: and removing the semantic point cloud with the dynamic semantic information from the laser point cloud frame to obtain the semantic point cloud for parameterization.

Because the semantic point cloud with dynamic semantic information is removed from the semantic point cloud for parameterization processing, the robustness of matching is improved, and the effect of parameterization feature matching is further improved.

As one example, a semantic point cloud of so-called dynamic semantic information includes, but is not limited to, classes of obstacles, such as vehicles, pedestrians, bicycles, motorcycles, and the like. Removing the semantic point cloud with dynamic semantic information from the semantic point cloud for parameterization, i.e. not extracting angular points, surface feature points, etc. therefrom.

As one example, if the semantic point cloud used for the parameterization processing is the ground, lawn, or the like, no corner feature points are extracted.

In another implementation manner of the present invention, segmenting semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types, including: according to the ground semantic labels, carrying out networking segmentation on semantic point clouds to obtain multiple groups of point clouds corresponding to multiple grids respectively, wherein the distance between the center of each group of point clouds and the center of the semantic point clouds and the resolution of the grids corresponding to the group of point clouds have positive correlation, and the semantic point clouds are segmented and subjected to parameterization to obtain parameterized semantic features of corresponding types, and the method comprises the following steps of: determining parameterized semantic features based on at least one of plane fitting parameters, confidence levels, bounding box center point positions, and ground semantic labels of the sets of point clouds, respectively.

Because the distance between the center of each group of point clouds and the semantic point cloud and the resolution of the grids corresponding to the group of point clouds have positive correlation, the uniformity of the number of the point clouds corresponding to each grid can be effectively realized, and therefore, the semantic point clouds are subjected to networked segmentation by adopting a plurality of grids, the robustness of ground parameterization processing can be improved, and the accuracy of parameterization feature matching is further improved. In addition, any one of the plane fitting parameters, the confidence degrees, the bounding box central point positions and the ground semantic labels of the multiple groups of point clouds reliably reflects the local information of the multiple groups of point clouds by adopting a small data volume, so that the parameterization semantic features are determined by at least one of the plane fitting parameters, the confidence degrees, the bounding box central point positions and the ground semantic labels, and the parameterization processing efficiency is improved.

As one example, a location of each of a plurality of meshes in the semantic point cloud may be determined from the ground semantic tags, the plurality of meshes including a first mesh and a second mesh. The distance between each group of point clouds and the center of the semantic point cloud has a positive correlation with the resolution of the mesh corresponding to the group of point clouds, for example, the distance indicated by the position of the first mesh may be greater than the distance indicated by the position of the second mesh. Determining the size of the multiple grids according to the position of each of the multiple grids in the semantic point cloud, so that the size of the first grid is larger than that of the second grid; and performing networking segmentation on the semantic point cloud based on the positions and the sizes of the grids.

Specifically, fig. 3 is a schematic diagram of a gridding process according to another embodiment of the present invention. As shown, in a single frame of laser point cloud, the near is dense and the far is sparse. The distance between each group of point clouds and the center of the semantic point cloud has positive correlation with the resolution of the grids corresponding to the group of point clouds. For example, in a laser radar coordinate system, ground point clouds are divided into a series of grids from small to large according to resolution from near to far according to semantic labels. And determining the plane fitting parameters, the confidence coefficients, the positions of the central points of the bounding boxes and the ground semantic labels of the multiple groups of point clouds as parameterized semantic features. For example, assuming that the ground point clouds in each grid are distributed in a plane, the ground point clouds in each grid are subjected to plane fitting to obtain a vectorization equation, that is, vectorization parameters of the ground semantic features. The confidence of the ground semantic features and the bounding box center point can be calculated from the plane fitting parameters.

In another implementation manner of the present invention, segmenting semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types, including: according to the shaft semantic tag, carrying out clustering processing on semantic point cloud to obtain a plurality of clusters, wherein parameterization processing is carried out on semantic point cloud segmentation to obtain parameterized semantic features of corresponding types, and the method comprises the following steps: determining a parameterized semantic feature based on at least one of a linear fit parameter, a confidence level, a bounding box centroid position, and a shaft semantic label for each of the plurality of clusters.

Because the multiple clusters can effectively reflect the point clouds of different rod-shaped objects, the robustness of the parameterization processing of the rod-shaped objects is improved, and the precision of parameterization feature matching is further improved. In addition, any one of the plane fitting parameters, the confidence degrees, the positions of the central points of the bounding boxes and the semantic labels of the rods of the clusters reliably reflects the local information of the clusters by adopting a small data volume, so that the parameterization semantic features are determined by at least one of the parameters, and the parameterization processing efficiency is improved.

Specifically, according to the semantic tag of the rod-shaped object, clustering processing is carried out on the semantic point cloud to obtain a plurality of clusters. For example, the rod point clouds in the point clouds can be filtered out according to semantic labels, and then the point clouds are clustered. And performing linear fitting on each cluster to obtain linear fitting parameters. The bounding box center can be calculated from the linear fit parameters.

In another implementation manner of the present invention, segmenting semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types, including: according to the semantic labels of the traffic signboards, clustering semantic point clouds to obtain a plurality of clusters; respectively determining a plurality of groups of main plane points based on a plurality of clusters as semantic point cloud segments of corresponding types, wherein the parameterization processing is carried out on the semantic point cloud segments to obtain parameterization semantic features of corresponding types, and the method comprises the following steps: and determining the parameterized semantic features based on the respective linear fitting parameters, confidence degrees, the positions of the center points of the bounding boxes and the semantic labels of the traffic signboards of the multiple groups of main plane points.

Because the clusters can effectively reflect the point clouds of different traffic signboards, a plurality of groups of main plane points are respectively determined based on the clusters, and the point clouds of the main planes of each traffic signboard are improved, so that the robustness of parameterization processing is improved, and the accuracy of parameterization feature matching is improved. In addition, any one of the plane fitting parameters, the confidence degrees, the positions of the center points of the bounding boxes and the semantic labels of the traffic sign of the multiple groups of main plane points reliably reflects the local information of the multiple groups of main plane points by adopting a small data volume, so that the parameterization semantic characteristics are determined by at least one of the parameters, and the parameterization processing efficiency is improved.

In another implementation of the present invention, determining multiple groups of principal plane points based on multiple clusters respectively includes: and performing multiple iteration processing on each cluster, and determining a group of main plane points of the cluster to obtain multiple groups of main plane points, wherein in each iteration processing, plane fitting is performed on the basis of the current group of points which are greater than the threshold value of the target point number, and built-in points are removed from the current group of points until a group of main plane points which are less than the threshold value of the target point number are obtained after multiple iteration processing.

In each iteration process, the built-in points are removed through plane fitting, so that a group of main plane points obtained after iteration can reflect the characteristics of the main plane better, and the analysis efficiency of the main plane data is improved.

Specifically, according to the semantic label of the traffic sign, clustering processing is carried out on semantic point clouds to obtain a plurality of clusters. For example, the traffic nameplate point clouds can be filtered and clustered according to semantic labels. Determining the plurality of groups of principal plane points based on the plurality of clusters, respectively, may employ a multi-plane decimation algorithm. For example, the plane fitting parameters of each of the plurality of sets of principal plane points are obtained by fitted vectorized equations.

In another implementation manner of the present invention, segmenting semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types, including: performing plane segmentation processing on semantic point clouds according to building semantic labels to obtain multiple groups of point clouds; respectively determining multiple groups of main plane points as semantic point cloud segments of corresponding types based on multiple groups of point clouds, wherein the semantic point cloud segments are subjected to parameterization processing to obtain parameterized semantic features of corresponding types, and the method comprises the following steps: and determining the parameterized semantic features based on the respective linear fitting parameters, confidence degrees, the positions of the center points of the bounding boxes and the semantic labels of the traffic signboards of the multiple groups of main plane points.

Because the multiple groups of point clouds can effectively reflect the point clouds of different buildings, the multiple groups of main plane points are respectively determined based on the multiple groups of point clouds, so that the point clouds of the main planes of each building are improved, the robustness of parameterization processing is improved, and the accuracy of parameterization feature matching is improved. In addition, any one of the plane fitting parameters, the confidence degrees, the positions of the central points of the bounding boxes and the building semantic labels of the multiple groups of main plane points reliably reflects the local information of the multiple groups of main plane points by adopting a small data volume, so that the parameterization semantic characteristics are determined by at least one of the parameters, and the parameterization processing efficiency is improved.

In another implementation of the present invention, determining multiple groups of principal plane points based on multiple groups of point clouds respectively includes: and performing repeated iteration processing on each group of point cloud segments, and determining a group of main plane points of the group of point clouds to obtain a plurality of groups of main plane points, wherein in each iteration processing, plane fitting is performed on the basis of the current group of points which are greater than the threshold value of the target point number, and built-in points are removed from the current group of points until a group of main plane points which are less than the threshold value of the target point number are obtained after repeated iteration processing.

Specifically, according to the building semantic tags, semantic point clouds are subjected to plane segmentation processing to obtain multiple groups of point clouds. Since the building point clouds are usually connected into one piece and exist in a large area, performing a plane segmentation process on the semantic point cloud can segment the semantic point cloud into several point cloud segments. The building parameterized semantic features can be obtained by adopting a multi-plane extraction algorithm. For example, determining the sets of principal plane points based on the plurality of clusters, respectively, may employ a multi-plane decimation algorithm. For example, the plane fitting parameters of each of the plurality of sets of principal plane points are obtained by fitted vectorized equations.

FIG. 4A illustrates a parameterized semantic feature frame of another embodiment of the present invention. As shown in fig. 4A, a plurality of historical parameterized semantic feature frames are combined into parameterized semantic feature subgraphs, so as to obtain four types of parameterized semantic feature rendering effects, and numbers in the subgraphs represent confidence degrees of the parameterized semantic features. Specifically, after four kinds of parameterized semantic features are extracted from the current frame point cloud, a current frame of parameterized semantic feature frames is formed. And a plurality of historical parameterized semantic feature frames can be converted to all coordinate systems through the calculated poses to form a historical parameterized semantic feature sub-graph.

In addition, line-to-line and surface-to-surface feature matching is carried out at least on the basis of the current parameterized semantic feature frame and the parameterized semantic feature sub-image, and the pose of the laser radar is obtained. For example, between the parametric semantic feature frame and the parametric semantic feature subgraph, line-to-line (e.g., for rod parametric semantic features) matching and face-to-face (ground parametric semantic features, traffic sign parametric semantic features, or building parametric semantic features) matching are performed based on at least one (e.g., the four above) parametric semantic feature.

In particular, fig. 4B illustrates shaft parametric semantic feature matching of another embodiment of the present invention, where each shaft parametric semantic feature in the current parametric semantic feature frame may be labeled as f0, as shown in the figure, in case of matching based on shaft parametric semantic features. The shaft parameterized semantic features in the parameterized semantic feature subgraph with the center of the search distance F0 being less than a predetermined threshold are determined and labeled F1, F2, F3.

The weighted average F _ avg of F1 to FN can be calculated by the following formula.

In another implementation manner of the present invention, the parameterization processing is performed on the semantic point cloud segments to obtain the corresponding type of parameterized semantic features, which includes: and determining a parameterized semantic feature based on at least one of a linear fitting parameter, a confidence coefficient, a bounding box central point position and a type of the semantic point cloud, wherein the linear fitting parameter of the semantic point cloud segmentation is obtained by performing linear fitting on the semantic point cloud segmentation, the confidence coefficient of the semantic point cloud segmentation is determined based on a ratio between the number of linear fitting built-in points of the semantic point cloud segmentation and the total number of points of the semantic point cloud, and the bounding box central point position of the semantic point cloud segmentation is determined based on the coordinate of the bounding box central point of the semantic point cloud segmentation in a laser sensor coordinate system.

In another implementation manner of the present invention, performing line-to-line and surface-to-surface feature matching based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar acquiring the laser point cloud frame, including: searching the position of the center point of the bounding box through a proximity search algorithm, and determining a line-to-line feature matching point and a plane-to-plane feature matching point between a current parameterized semantic feature frame and a parameterized semantic feature sub-graph, wherein when searching is performed based on the proximity search algorithm, the weight of a residual error function is determined based on the times of searching the point-to-point feature matching point and the plane-to-plane feature matching point, so that the weight of the residual error function is positively correlated with the times; and performing transformation processing based on the point-to-point feature matching points and the surface-to-surface feature matching points to determine the pose of the laser radar for collecting the laser point cloud frame.

Because the times of searching the point-to-point feature matching point and the face-to-face feature matching point reflect the accuracy of the parameterized feature matching, the weight of the residual function is set to be positively correlated with the times, and the accuracy of the parameterized feature matching is improved.

Specifically, the position of the center point of the bounding box is searched through a proximity search algorithm, line-to-line feature matching points and face-to-face feature matching points between the current parameterized semantic feature frame and the parameterized semantic feature subgraph are determined, and the accuracy of establishing association in matching is improved in a mode of designing weights for matched residual error functions. The weight of the residual function is positively correlated with the number. For example, in the KNN search, the larger the number of times the semantic tags coincide, same _ label _ cnt (an example of a point-to-point feature matching point or a surface-to-surface feature matching point), the larger the weight W _ residual of the residual function. In a specific example, W _ residual is 1.0/(1.0+ exp (-2 × same _ label _ cnt + 2)).

In another implementation manner of the present invention, performing line-to-line and surface-to-surface feature matching based on at least a current parameterized semantic feature frame and a parameterized semantic feature sub-graph to obtain a pose of a laser radar for acquiring a laser point cloud frame, includes: determining parameterized semantic feature matching constraints and geometric feature matching constraints between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein the parameterized semantic feature matching constraints indicate matching relationships of the line-to-line and face-to-face feature matches; and determining the pose of the laser radar for collecting the laser point cloud frame based on the parameterized semantic feature matching constraint and the geometric feature matching constraint.

In addition, a double-thread design can be adopted, and a rough laser radar or a vehicle pose inter-frame matching pose with the laser radar can be estimated by using high-frequency frame-to-frame matching. And calculating a relatively accurate laser radar or vehicle pose graph matching pose with the laser radar by using low-frequency frame-to-sub graph matching. Frame-to-frame matching provides a matching initial value for frame-to-sub-image matching, and the initial value for frame-to-frame matching may be determined by a vehicle uniform velocity model. The interframe matching pose and the image matching pose can be integrated to realize a real-time laser mileage calculation method.

In addition, both frame-to-frame matching and frame-to-sub-picture matching may utilize previously extracted corner points and surface geometric features to construct point-to-line constraints and point-to-plane constraints.

As shown in fig. 2, which shows a schematic flow chart of a feature matching method of another embodiment of the present invention. Specifically, line-to-line matching may be performed between F0 and F avg described above, and tight coupling constraints are performed with frame-to-sub-graph matching in geometric feature matching, thereby solving the "face-to-face" matching problem when the latter ghost occurs in the sub-graph.

In another implementation of the invention, the method further comprises: based on the pose of the laser radar, adding the current parameterized semantic feature frame to the historical parameterized semantic feature sub-images through coordinate transformation, and removing the first parameterized semantic feature frame in the historical parameterized semantic feature sub-images to update the historical parameterized semantic feature sub-images.

The current parameterized semantic feature frame is added to the historical parameterized semantic feature sub-image through coordinate transformation, and the first parameterized semantic feature frame in the historical parameterized semantic feature sub-image is removed, so that the dynamic update of the historical parameterized semantic feature sub-image is realized while the number of feature frames in the historical parameterized semantic feature sub-image is ensured, and the accuracy of parameterized feature matching is improved.

In addition, the graph matching pose and the inter-frame matching pose can be integrated, and a real-time laser mileage calculation method (parameterized semantic features LO) is realized. Based on the pose of the laser radar, the current parameterized semantic feature frame is added to the historical parameterized semantic feature sub-graph through coordinate transformation, for example, the parameterized semantic feature sub-graph is updated by using the integrated pose, that is, the current parameterized semantic feature frame is added to the parameterized semantic feature sub-graph after being transformed to the global coordinate system according to the integrated pose.

In addition, the first parameterized semantic feature frame in the historical parameterized semantic feature sub-graph is removed to update the historical parameterized semantic feature sub-graph, for example, the earliest parameterized semantic feature frame is correspondingly removed from the parameterized semantic feature sub-graph. See also fig. 2.

Fig. 5 is a schematic block diagram of a feature matching apparatus according to another embodiment of the present invention. The laser point cloud parameterized semantic feature matching apparatus of fig. 5 may be applied to any suitable electronic device with data processing capability, including but not limited to: server, mobile terminal (such as cell phone, PAD, etc.) and PC, etc., this device includes:

an obtaining module 510, configured to obtain a semantic point cloud for parameterization from a laser point cloud frame;

the segmentation module 520 is used for segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types;

and the parameterization processing module 530 is used for carrying out parameterization processing on the semantic point cloud segmentation to obtain the parameterization semantic features of the corresponding types.

The first combining module 540 combines the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame.

A second combining module 550 combines the plurality of historical parameterized semantic feature frames into a parameterized semantic feature subgraph.

And the matching module 560 performs line-to-line and face-to-face feature matching at least based on the current parameterized semantic feature frame and the parameterized semantic feature sub-image to obtain the pose of the laser radar for collecting the laser point cloud frame.

It should be understood that the pose herein refers to the position (x, y, z) and attitude angle (roll, pitch, yaw) of the lidar or the vehicle on which the lidar is mounted. A trajectory in this context refers to a sequence of sets of vehicle poses. The real-time positioning and mapping algorithm in the text calculates the track of the vehicle for a given image or laser point cloud. The method generally comprises a front-end mileage calculation method and a rear-end loop detection algorithm, wherein the front end is responsible for calculating an initial track and solving the problem of drift (drift) caused by accumulated errors, and the rear end is responsible for eliminating the problem of the accumulated errors of the front end.

It should also be understood that laser SLAM herein refers specifically to SLAM algorithms that utilize only a laser point cloud. In addition, the laser odometer Lidar-only odometer (LO) in the text refers to the aforementioned SLAM front-end problem, and the higher the accuracy of the LO is, the smaller the trajectory drift problem will be, which is more beneficial to the loop optimization of the back end, and thus, the overall accuracy and global consistency of the SLAM are improved. The embodiment of the invention is mainly based on the LO scene.

It should also be understood that point cloud registration/matching in this document is to solve a transformation matrix of point clouds with different postures under the same coordinate by using two point clouds with partially overlapped or completely overlapped areas, and to realize accurate registration of the two point clouds by using the matrix, which is used as a basis of an LO algorithm.

It should also be understood that ICP registration in this context refers to a point cloud registration algorithm that establishes associations between points by nearest neighbor search (KNN). The point cloud semantics in the text is that semantic labels such as 'ground', 'building', 'rod', 'traffic signboard' and the like are attached to each point cloud through a semantic segmentation algorithm. As shown in the following rendering, where blue represents the ground, magenta represents the vehicle, gray represents the building, green represents vegetation, light green represents the lawn, etc.

It should also be understood that the fitting herein is to represent the point cloud of a given area as a mathematical equation that can describe the shape of the point cloud of that area. Vectorization or parameterization in this context means that a mathematical equation is expressed by fitting a given point cloud, and certain parameters (confidence, semantic labels, bounding box center points, etc.) are recorded to express some characteristics of the point cloud. Vectorized Semantic Feature in the text is a lightweight point cloud Feature of the embodiment of the invention, and Vectorized Feature extraction is performed on Semantic point clouds to construct four types of parameterized Semantic features, namely a ground parameterized Semantic Feature, a building parameterized Semantic Feature, a rod parameterized Semantic Feature and a traffic signboard parameterized Semantic Feature.

The sub-map ghost problem in this document is the occurrence of "ghosts" in the accumulated sub-maps due to previous positioning errors.

In another implementation manner of the present invention, the obtaining module is specifically configured to: and removing the semantic point cloud with the dynamic semantic information from the laser point cloud frame to obtain the semantic point cloud for parameterization processing.

In another implementation of the present invention, the cutting module is specifically configured to: according to the ground semantic labels, performing network segmentation on the semantic point cloud to obtain multiple groups of point clouds corresponding to multiple grids respectively, wherein the distance between the center of each group of point cloud and the semantic point cloud has a positive correlation with the resolution of the grids corresponding to the group of point clouds, and the parameterization processing module is specifically used for: determining the parameterized semantic features based on at least one of plane fitting parameters, confidence levels, bounding box center point positions, and the ground semantic tags for each of the plurality of sets of point clouds.

As an example, the slitting module is specifically configured to: determining the positions of a plurality of grids in the semantic point cloud according to the ground semantic labels, wherein the plurality of grids comprise a first grid and a second grid, and the distance indicated by the positions of the first grid is greater than the distance indicated by the positions of the second grid; determining the sizes of the multiple grids according to the positions of the multiple grids in the semantic point cloud respectively, so that the size of the first grid is larger than that of the second grid; and performing networking segmentation on the semantic point cloud based on the positions and the sizes of the grids.

In another implementation of the present invention, the cutting module is specifically configured to: according to the semantic tag of the rod-shaped object, clustering the semantic point cloud to obtain a plurality of clusters, wherein the parameterization processing module is specifically used for: determining a parameterized semantic feature based on at least one of a linear fit parameter, a confidence, a bounding box midpoint location, and a shaft semantic tag for each of the plurality of clusters.

In another implementation of the present invention, the cutting module is specifically configured to: according to the semantic labels of the traffic signboards, clustering the semantic point clouds to obtain a plurality of clusters; respectively determining a plurality of groups of main plane points as semantic point cloud segmentation of corresponding types based on the plurality of clusters, wherein the parameterization processing module is specifically used for: and determining the parameterized semantic features based on the respective linear fitting parameters, confidence degrees, the positions of the center points of the bounding boxes and the semantic labels of the traffic sign of the multiple groups of main plane points.

In another implementation manner of the present invention, a plurality of iterations are performed on a per-cluster basis, and a set of principal plane points of the cluster is determined to obtain a plurality of sets of principal plane points, wherein in each iteration, a plane fitting is performed on the basis of a current set of points greater than a target point number threshold, and a built-in point is removed from the current set of points until the set of principal plane points smaller than the target point number threshold is obtained after the plurality of iterations.

In another implementation of the present invention, the cutting module is specifically configured to: according to the building semantic label, carrying out plane segmentation processing on the semantic point cloud to obtain a plurality of groups of point clouds; respectively determining multiple groups of main plane points as semantic point cloud segments of corresponding types based on the multiple groups of point clouds, wherein the parameterization processing module is specifically used for: and determining the parameterized semantic features based on the respective linear fitting parameters, confidence degrees, the positions of the center points of the bounding boxes and the semantic labels of the traffic sign of the multiple groups of main plane points.

In another implementation of the present invention, the parameterization processing module is specifically configured to: and performing multiple iteration processing on each group of point cloud segments, and determining a group of main plane points of the group of point clouds to obtain multiple groups of main plane points, wherein in each iteration processing, plane fitting is performed on the basis of a current group of points which are greater than a target point threshold value, and built-in points are removed from the current group of points until the group of main plane points which are less than the target point threshold value are obtained after the multiple iteration processing.

In another implementation of the present invention, the parameterization processing module is specifically configured to: and determining a parameterized semantic feature based on at least one of a linear fitting parameter, a confidence coefficient, a bounding box central point position and a type of the semantic point cloud, wherein the linear fitting parameter of the semantic point cloud segmentation is obtained by performing linear fitting on the semantic point cloud segmentation, the confidence coefficient of the semantic point cloud segmentation is determined based on a ratio between the number of linear fitting built-in points of the semantic point cloud segmentation and the total number of points of the semantic point cloud, and the bounding box central point position of the semantic point cloud segmentation is determined based on the coordinate of the bounding box central point of the semantic point cloud segmentation in a laser sensor coordinate system.

In another implementation manner of the present invention, the matching module is specifically configured to: searching the position of the center point of the bounding box through a proximity search algorithm, and determining a line-to-line feature matching point and a face-to-face feature matching point between the current parameterized semantic feature frame and the parameterized semantic feature sub-graph, wherein when searching is performed based on the proximity search algorithm, the weight of a residual function is determined based on the times of searching the point-to-point feature matching point and the face-to-face feature matching point, so that the weight of the residual function is positively correlated with the times; and performing transformation processing based on the point-to-point feature matching points and the surface-to-surface feature matching points, and determining the pose of the laser radar for collecting the laser point cloud frame.

In another implementation manner of the present invention, the matching module is specifically configured to: determining parameterized semantic feature matching constraints and geometric feature matching constraints between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein the parameterized semantic feature matching constraints indicate matching relationships of the line-to-line and face-to-face feature matches; and determining the pose of the laser radar for collecting the laser point cloud frame based on the parameterized semantic feature matching constraint and the geometric feature matching constraint.

In another implementation of the present invention, the apparatus further comprises: and the updating module is used for adding the current parameterized semantic feature frame to the historical parameterized semantic feature sub-images through coordinate transformation based on the pose of the laser radar, and removing the first parameterized semantic feature frame in the historical parameterized semantic feature sub-images so as to update the historical parameterized semantic feature sub-images.

The apparatus of this embodiment is used to implement the corresponding method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again. In addition, the functional implementation of each module in the apparatus of this embodiment can refer to the description of the corresponding part in the foregoing method embodiment, and is not described herein again.

Fig. 6 is a hardware configuration of an electronic device according to another embodiment of the present invention; as shown in fig. 6, the hardware structure of the electronic device may include: a processor 601, a communication interface 602, a storage medium 603, and a communication bus 604;

the processor 601, the communication interface 602 and the storage medium 603 complete communication with each other through the communication bus 604;

alternatively, the communication interface 602 may be an interface of a communication module;

the processor 601 may be specifically configured to: obtaining semantic point cloud for parameterization from a laser point cloud frame; segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; carrying out parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame; combining a plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs; and performing line-to-line and face-to-face feature matching at least based on the current parameterized semantic feature frame and the parameterized semantic feature sub-image to obtain the pose of the laser radar for collecting the laser point cloud frame.

The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage medium may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Read Only Memory (EPROM), an electrically Erasable Read Only Memory (EEPROM), and the like.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a storage medium, the computer program comprising program code configured to perform the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. The computer program performs the above-described functions defined in the method of the present invention when executed by a Central Processing Unit (CPU). It should be noted that the storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access storage media (RAM), a read-only storage media (ROM), an erasable programmable read-only storage media (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage media (CD-ROM), an optical storage media piece, a magnetic storage media piece, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any storage medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code configured to carry out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may operate over any of a variety of networks: including a Local Area Network (LAN) or a Wide Area Network (WAN) -to the user's computer, or alternatively, to an external computer (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions configured to implement the specified logical function(s). In the above embodiments, specific precedence relationships are provided, but these precedence relationships are only exemplary, and in particular implementations, the steps may be fewer, more, or the execution order may be modified. That is, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented by software or hardware. The names of these modules do not in some cases constitute a limitation of the module itself.

As another aspect, the present invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the method as described in the above embodiments.

As another aspect, the present invention also provides a storage medium, which may be contained in the apparatus described in the above embodiments; or may be present separately and not assembled into the device. The storage medium carries one or more programs that, when executed by the apparatus, cause the apparatus to: obtaining semantic point cloud for parameterization processing from the laser point cloud frame; segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types; carrying out parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types; combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame; combining a plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs; and performing line-to-line and surface-to-surface feature matching on the current parameterized semantic feature frame and the parameterized semantic feature sub-image to obtain the pose of the laser radar for collecting the laser point cloud frame.

The expressions "first", "second", "said first" or "said second" used in various embodiments of the present disclosure may modify various components regardless of order and/or importance, but these expressions do not limit the respective components. The above description is only configured for the purpose of distinguishing elements from other elements. For example, the first user equipment and the second user equipment represent different user equipment, although both are user equipment. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the present disclosure.

When an element (e.g., a first element) is referred to as being "operably or communicatively coupled" or "connected" (operably or communicatively) to "another element (e.g., a second element) or" connected "to another element (e.g., a second element), it is understood that the element is directly connected to the other element or the element is indirectly connected to the other element via yet another element (e.g., a third element). In contrast, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (a second element), no element (e.g., a third element) is interposed therebetween.

The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention according to the present invention is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the scope of the invention as defined by the appended claims. For example, the above features and (but not limited to) features having similar functions disclosed in the present invention are mutually replaced to form the technical solution.

Claims

1. A laser point cloud parameterization semantic feature matching method comprises the following steps:

obtaining semantic point cloud for parameterization from a laser point cloud frame;

segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types;

carrying out parameterization processing on the semantic point cloud segments to obtain parameterized semantic features of corresponding types;

combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame;

combining a plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs;

and performing line-to-line and face-to-face feature matching at least based on the current parameterized semantic feature frame and the parameterized semantic feature sub-image to obtain the pose of the laser radar for collecting the laser point cloud frame.

2. The method of claim 1, wherein the obtaining semantic point clouds for parameterization from a laser point cloud frame comprises:

and removing the semantic point cloud with dynamic semantic information from the laser point cloud frame to obtain the semantic point cloud for parameterization processing.

3. The method of claim 1, wherein the segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types comprises:

performing network segmentation on the semantic point cloud according to a ground semantic label to obtain a plurality of groups of point clouds respectively corresponding to a plurality of grids, wherein the distance between the center of each group of point clouds and the semantic point cloud has a positive correlation with the resolution of the grids corresponding to the group of point clouds,

the method for carrying out parameterization processing on the semantic point cloud subsection to obtain the parameterization semantic features of the corresponding types comprises the following steps:

determining the parameterized semantic features based on at least one of plane fitting parameters, confidence levels, bounding box center point positions, and the ground semantic tags for each of the plurality of sets of point clouds.

4. The method of claim 1, wherein the segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types comprises:

according to the semantic label of the rod-shaped object, clustering the semantic point cloud to obtain a plurality of clusters,

determining the parameterized semantic features based on at least one of a linear fit parameter, a confidence level, a bounding box centerline position, and the shaft semantic tags for each of the plurality of clusters.

5. The method of claim 1, wherein the segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types comprises:

according to the semantic labels of the traffic signboards, clustering the semantic point clouds to obtain a plurality of clusters;

determining a plurality of groups of principal plane points, respectively, as semantic point cloud segments of corresponding types, based on the plurality of clusters,

the method for carrying out parameterization processing on the semantic point cloud segmentation to obtain the parameterization semantic features of the corresponding types comprises the following steps:

and determining the parameterized semantic features based on the respective linear fitting parameters, confidence degrees, the positions of the center points of the bounding boxes and the semantic labels of the traffic sign of the multiple groups of main plane points.

6. The method of claim 5, wherein said determining a plurality of sets of principal plane points, respectively, based on said plurality of clusters comprises:

and performing multiple times of iteration processing on each cluster, and determining a group of main plane points of the cluster to obtain multiple groups of main plane points, wherein in each iteration processing, plane fitting is performed on the basis of a current group of points which are greater than a target point threshold value, and built-in points are removed from the current group of points until the group of main plane points which are less than the target point threshold value are obtained after the multiple times of iteration processing.

7. The method of claim 1, wherein the segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types comprises:

according to the building semantic label, carrying out plane segmentation processing on the semantic point cloud to obtain a plurality of groups of point clouds;

respectively determining a plurality of groups of main plane points as semantic point cloud segmentation of corresponding types based on the plurality of groups of point clouds,

8. The method of claim 7, wherein said determining a plurality of sets of principal plane points, respectively, based on the plurality of sets of point clouds comprises:

and performing multiple iteration processing on each group of point cloud segments, and determining a group of main plane points of the group of point clouds to obtain multiple groups of main plane points, wherein in each iteration processing, plane fitting is performed on the basis of the current group of points greater than a target point number threshold, and built-in points are removed from the current group of points until the group of main plane points smaller than the target point number threshold are obtained after the multiple iteration processing.

9. The method of claim 1, wherein the parameterizing the semantic point cloud segments to obtain parameterized semantic features of corresponding types comprises:

determining the parameterized semantic features based on at least one of a linear fit parameter, a confidence level, a bounding box center point location of the semantic point cloud segment, and a type of the semantic point cloud,

the linear fitting parameters of the semantic point cloud segmentation are obtained by performing linear fitting on the semantic point cloud segmentation, the confidence coefficient of the semantic point cloud segmentation is determined based on the ratio of the number of linear fitting built-in points of the semantic point cloud segmentation to the total number of points of the semantic point cloud, and the position of the bounding box center point of the semantic point cloud segmentation is determined based on the coordinate of the bounding box center point of the semantic point cloud segmentation in a laser sensor coordinate system.

10. The method of claim 9, wherein the performing line-to-line and face-to-face feature matching based on the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain a pose of a lidar that acquires the frame of laser point cloud comprises:

searching the position of the center point of the bounding box through a proximity search algorithm, and determining a line-to-line feature matching point and a plane-to-plane feature matching point between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein when searching is performed based on the proximity search algorithm, the weight of a residual function is determined based on the times of searching the point-to-point feature matching point and the plane-to-plane feature matching point, so that the weight of the residual function is positively correlated with the times;

and performing transformation processing based on the point-to-point feature matching points and the surface-to-surface feature matching points, and determining the pose of the laser radar for collecting the laser point cloud frame.

11. The method of claim 1, wherein the deriving a pose of a lidar that acquires the frame of laser point cloud based at least on line-to-line and face-to-face feature matching of the current parameterized semantic feature frame and the parameterized semantic feature sub-graph comprises:

determining parameterized semantic feature matching constraints and geometric feature matching constraints between the current parameterized semantic feature frame and the parameterized semantic feature subgraph, wherein the parameterized semantic feature matching constraints indicate matching relationships of the line-to-line and face-to-face feature matches;

and determining the pose of the laser radar for collecting the laser point cloud frame based on the parameterized semantic feature matching constraint and the geometric feature matching constraint.

12. The method of claim 1, wherein the method further comprises:

based on the pose of the laser radar, adding the current parameterized semantic feature frame to the historical parameterized semantic feature sub-images through coordinate transformation, and removing a first parameterized semantic feature frame in the historical parameterized semantic feature sub-images to update the historical parameterized semantic feature sub-images.

13. A laser point cloud parameterization semantic feature matching device comprises:

the acquisition module acquires semantic point cloud for parameterization from the laser point cloud frame;

the segmentation module is used for segmenting the semantic point cloud according to the type of the semantic point cloud to obtain semantic point cloud segments of corresponding types;

the parameterization processing module is used for carrying out parameterization processing on the semantic point cloud segmentation to obtain parameterization semantic features of corresponding types;

the first combination module is used for combining the parameterized semantic features of the corresponding types into a current parameterized semantic feature frame;

the second combination module is used for combining the plurality of historical parameterized semantic feature frames into parameterized semantic feature subgraphs;

and the matching module is used for performing line-to-line and surface-to-surface feature matching at least on the basis of the current parameterized semantic feature frame and the parameterized semantic feature subgraph to obtain the pose of the laser radar for collecting the laser point cloud frame.

14. An electronic device, the device comprising:

the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the method according to any one of claims 1-12.

15. A storage medium having stored thereon a computer program which, when executed by a processor, carries out the method of any one of claims 1 to 12.