CN115031744A - Cognitive map positioning method and system based on sparse point cloud-texture information - Google Patents
Cognitive map positioning method and system based on sparse point cloud-texture information Download PDFInfo
- Publication number
- CN115031744A CN115031744A CN202210612389.XA CN202210612389A CN115031744A CN 115031744 A CN115031744 A CN 115031744A CN 202210612389 A CN202210612389 A CN 202210612389A CN 115031744 A CN115031744 A CN 115031744A
- Authority
- CN
- China
- Prior art keywords
- data
- point cloud
- traffic identification
- cognitive map
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
- G01C21/30—Map- or contour-matching
- G01C21/32—Structuring or formatting of map data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3804—Creation or updating of map data
- G01C21/3807—Creation or updating of map data characterised by the type of data
- G01C21/3811—Point data, e.g. Point of Interest [POI]
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a cognitive map positioning method and system based on sparse point cloud-texture information, and belongs to the technical field of positioning. The method fully utilizes the accurate depth estimation capability of the laser radar and the good feature expression capability of the camera, obtains traffic identification geometric information by adopting a geometric center estimation algorithm, and generates the cognitive map after point cloud data and picture data are fused by the traffic identification geometric information. Based on the cognitive map, the obtained geometric center information of the traffic identification is utilized, a two-stage search strategy is adopted to match and position the traffic identification at the current position of the automatic driving vehicle and the traffic identification characteristics stored in the semantic layer data of the cognitive map, the corresponding position of the traffic identification of the current vehicle running in the cognitive map is obtained, and the corresponding position is fused with the local positioning result of the vehicle self-carried laser radar odometer, so that the high-precision positioning of the automatic driving vehicle is realized.
Description
Technical Field
The invention relates to the technical field of positioning, in particular to a cognitive map positioning method and system based on sparse point cloud-texture information.
Background
With the technological progress, the automatic driving technology has made important breakthrough and development in recent years. The mapping and positioning technology is used as a core in automatic driving, and the importance of the development of the automatic driving technology is increasingly shown. High-precision maps are gradually paid attention to by the public and are receiving wide attention as important components of future automatic driving development. The high-precision map contains a large amount of static auxiliary information such as road traffic marking lines and surrounding geographic environments, can help the automatic driving vehicle to quickly identify the surrounding environments, and improves the calculation efficiency and precision. Compared with the common map, the high-precision map has higher absolute coordinate precision and contains richer road element attribute information, and completely restores the static object information of the real world. In the static object information, the traffic sign provides important guidance information to the driver as an important sign feature on the road.
The traditional road traffic sign extraction method is that high-line-number laser radar is used for collecting point cloud data with high density, and extraction is carried out through an extraction algorithm. The vehicle-mounted sensor adopted in the extraction process is high in price and huge in computational demand, so that the high-precision map is low in manufacturing efficiency and complex to update. Compared with a high-line-number laser radar, the visual sensor has advantages in the aspects of cost, information acquisition, feature expression capability and the like. But relying on image features alone has the drawback of being susceptible to weather seasons and the like. The cognitive map is an automatic driving map designed based on human cognitive mechanism and has the characteristics of light weight and low demand on computing resources. The visual sensor is used for recording information such as lanes and traffic marks in the road to form a lightweight map, and semantic level positioning can be carried out by referring to the cognitive map in the driving process of the vehicle. Due to the characteristics of small data volume and rich information, the application result in the positioning and navigation of the automatic driving vehicle is remarkable.
In the existing cognitive map-based positioning navigation scheme, the center of a detection frame is often defined as a traffic identification center, however, due to uncertainty of an algorithm, deviation exists between the actual center of the traffic identification and the center of the detection frame, and errors are introduced by using the center of the detection frame as a cognitive map element for storage, so that positioning accuracy based on a cognitive map is influenced. In addition, only one visual sensor of a camera is adopted in the existing cognitive map method, accurate measurement of the traffic identification distance is lacked, and the positioning accuracy is further influenced.
Disclosure of Invention
Aiming at the defects of the existing cognitive map positioning navigation, the invention provides a cognitive map positioning method based on sparse point cloud-texture information, and a cognitive map is generated after point cloud data and picture data are fused based on traffic identification geometric information obtained by the method, so that the accurate depth estimation capability of a laser radar and the good feature expression capability of a camera are fully exerted. The cognitive map generated based on the method is fused with the local positioning result of the laser radar odometer of the vehicle by using the geometric information of the identifier and a two-stage search strategy, so that the high-precision positioning of the automatic driving vehicle is realized.
The technical scheme of the invention is as follows:
a cognitive map positioning method based on sparse point cloud-texture information comprises the following steps:
step 1, collecting current road data, wherein the data comprises image data collected by a camera and point cloud data collected by a laser radar; establishing a global coordinate system by adopting a GNSS;
step 2, extracting a traffic sign from the image data acquired in the step 1, performing binary mask processing on the extracted traffic sign by adopting an image segmentation algorithm, and calculating central point data of the traffic sign by adopting a geometric center estimation algorithm to serve as geometric prior information of the traffic sign;
step 3, carrying out combined calibration on the laser radar and the camera, solving a laser radar-camera coordinate system transformation matrix, and unifying the coordinate systems of the camera and the laser radar by using the matrix; projecting the point cloud data obtained in the step 1 into the image data obtained in the step 1 by a cone projection guiding method to construct image-point cloud associated data;
step 4, extracting geometric outline information of the traffic identification from the constructed image-point cloud associated data by using the central point data obtained in the step 2, and extracting point cloud data in an outline range by using the outline information as a basis;
step 5, selecting image data corresponding to the traffic identification outline of the point cloud data, performing Gabor wavelet transformation on the image data, and then performing binarization mode extraction by using an LBP operator to obtain an LBP-Gabor texture feature image; dividing the LBP-Gabor texture feature image into n sub-image blocks, and performing histogram statistics and normalization processing on each sub-image block to generate n histogram feature vectors; after the n vectors are cascaded, the n vectors are used as the final texture features of the traffic identification for storage; wherein n is more than or equal to 4;
step 6, repeating the steps 2-4, obtaining the final texture features of all traffic identifications of the current road, obtaining a cognitive map semantic layer, and finishing cognitive map generation;
step 7, matching and positioning the current position traffic identification of the automatic driving vehicle and the characteristics in the semantic layer data of the cognitive map by adopting a two-stage search strategy so as to obtain the coordinate of the current driving traffic identification of the automatic driving vehicle in the cognitive map;
and 8, aligning the coordinates of the current automatic driving vehicle running traffic identification obtained in the step 7 in the cognitive map to a global coordinate system, and fusing the coordinates with a local positioning result of the laser radar odometer to obtain vehicle pose estimation.
Further, the process of calculating the coordinates of the center point of the traffic sign by using the geometric center estimation method in the step 2 is as follows:
judging that the shape of the traffic sign is circular or triangular according to the binary mask processing result; if the traffic sign center point data is judged to be circular, extracting the traffic sign center point data by adopting a Hough transformation algorithm to obtain a center point and radius data, and if the traffic sign center point data is judged to be triangular, obtaining the traffic sign center point data by adopting a small discrete area or HSV color space transformation method in a statistical extraction area;
further, the detailed process of obtaining the pose state estimation in step 7 is as follows:
step 7.1, first-stage matching: eliminating the interference of the same traffic identification in the same road section on the positioning result:
7.1.1, taking the current position traffic identification of the automatic driving vehicle as a center as an interested area according to a vehicle end observation result, and extracting a rectangular special frame; meanwhile, a rectangular frame is extracted for each traffic identification in the same road section cognitive map;
7.1.2 respectively adopting two measures of Bhattacharyya distance and perceptual hashing to calculate the similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame of the cognitive map,
7.1.3, respectively obtaining the comprehensive correlation coefficient of each rectangular frame of the cognitive map and the rectangular frame of the vehicle-end observation result according to the settlement result of the step 7.1.2. The integrated correlation coefficient is defined as:
7.1.4, sorting the comprehensive correlation coefficients calculated in the step 7.1.3 in a descending order, wherein the smallest numerical value is matched as the same traffic signboard object.
Step 7.2, second stage matching: and matching the textural features of each traffic identification stored in the semantic layer of the cognitive map according to the matching result of the first stage to obtain the corresponding position of the current vehicle running traffic identification in the cognitive map.
Further, the step 1 further comprises the steps of carrying out time synchronization processing on the GNSS, the laser radar and the camera, and aligning data collected by the camera and point cloud data of the laser radar in a time dimension through the time synchronization processing; the time synchronization process is as follows:
step 1.1, adding timestamps to each frame of camera image data and point cloud data acquired by a laser radar based on a high-frequency GPS signal of a clock;
and 1.2, searching the image data time stamp according to the point cloud data time stamp with lower frequency, and matching the image data time stamp with the nearest image data time stamp of the point cloud data time stamp with lower frequency to realize synchronous processing of the image data and the point cloud data.
Further, the detailed process of step 8 is as follows:
the laser radar odometer is designed to obtain relative poses as follows:
P＝{P 0 ,P 1 ,P 2 ...}
P i ＝[R|T]
wherein, P i Is the pose of frame i relative to frame i-1. R, T are position transformation matrix
The global positioning result obtained by solving the matched road signs arranged in the map is as follows:
G＝{G 0 ,G k ,...}
G＝[R|T]
wherein, G i For map global matching positioning result, R and T are pose transformation matrix
Assuming that the true pose of the camera to be solved is as follows:
X＝{X 0 ,X 1 ,X 2 ...}
X i ＝[R|T]
wherein X is the real pose of the camera to be solved, and R and T are pose transformation matrix
Constructing the fusion positioning problem as a maximum posterior probability problem:
X＝argmax(f(X|P,G))
under the condition that the observation G and the observation X are known, the posterior probability is maximized, and a pose graph is constructed to solve the real pose of the vehicle.
A cognitive map positioning based on sparse point cloud-texture information comprises a data acquisition module, a feature extraction module, a map generation module, a matching positioning module and a fusion positioning module.
The data acquisition module is respectively connected with the map feature extraction module and the matching positioning module; for acquiring image data and point cloud data; preprocessing the acquired image data to obtain a central point of the image data traffic identification; global coordinates are established by adopting a GNSS, and coordinates, time and space unification of a camera, a laser radar and the GNSS sensor is completed; in the embodiment, image data is collected by a camera, and point cloud data is collected by a laser radar.
The map feature extraction module is connected with the map generation module, extracts geometric prior information of the traffic identification by using the received central point data under the coordinate system of the unified camera and the laser radar provided by the data acquisition module, and extracts point cloud data in the traffic identification outline range on the basis of the geometric prior information, namely sparse point cloud generation; and selecting image data corresponding to the traffic identification outline of the point cloud data, and processing the image data by adopting Gabor wavelet transform, LBP operators, histogram statistics and normalization algorithm to obtain texture information of the traffic identification and provide the texture information to a map generation module.
The map generation module is connected with the matching positioning module and generates a cognitive map according to the received traffic identification texture signal;
the matching positioning module is connected with the fusion positioning module, and matches the traffic identification scanned near the current position of the automatic driving vehicle with the traffic identification of the cognitive map by adopting two-stage matching to obtain the coordinate of the traffic identification of the current position of the automatic driving vehicle in the cognitive map;
and the fusion positioning module aligns the coordinates of the traffic identification of the current position of the automatic driving vehicle in the cognitive map to a global coordinate system, and then fuses the coordinates with the local positioning result of the laser radar odometer to obtain the estimation of the vehicle pose.
According to the cognitive map positioning method and system based on sparse point cloud-texture information, provided by the invention, the accurate depth estimation capability of a laser radar and the good feature expression capability of a camera are fully utilized, the traffic identification geometric information is obtained by adopting a geometric center estimation algorithm, and the point cloud data and the picture data are fused through the traffic identification geometric information to generate a cognitive map. Based on the cognitive map, the obtained geometric center information of the traffic identification is utilized, a two-stage search strategy is adopted to match and position the traffic identification at the current position of the automatic driving vehicle and the traffic identification characteristics stored in the semantic layer data of the cognitive map, the corresponding position of the traffic identification of the current vehicle running in the cognitive map is obtained, and the corresponding position is fused with the local positioning result of the vehicle self-carried laser radar odometer, so that the high-precision positioning of the automatic driving vehicle is realized.
Compared with the prior art, the invention has the following advantages:
1. and reducing the detection deviation between the actual center of the traffic sign and the center of the detection frame by a geometric center estimation algorithm.
2. In the generation process of the cognitive map semantic layer, firstly, Gabor wavelets are adopted to extract the characteristics of image data in the image-point cloud associated data, and then, LBP operators are adopted to extract texture characteristics. Through the combination of Gabor and LBP operator, more abundant texture features can be obtained while the calculation amount is reduced.
3. When the cognitive map positioning is based, the problem that multiple identical traffic identifications possibly exist in the same road section and no matching exists is solved by adopting a two-stage searching strategy, and the positioning accuracy is improved by fusing a cognitive map positioning result and a vehicle self-contained laser radar odometer local positioning result.
Drawings
FIG. 1 is a flow chart of an embodiment geometric center estimation;
FIG. 2 is an embodiment of a texture feature extraction process;
FIG. 3 is a cognitive map generation diagram of the present invention;
FIG. 4 is a schematic diagram of a first stage matching of the embodiment;
FIG. 5 is a schematic diagram of embodiment pose graph fusion;
fig. 6 is a block diagram of the system of the present invention.
Detailed Description
The technical scheme of the invention is detailed below by combining the accompanying drawings and the embodiment.
The invention provides a cognitive map positioning method based on sparse point cloud-texture information, which comprises the following steps of:
step 1, collecting road data, wherein the data comprises image data collected by a camera and point cloud data collected by a laser radar. A global coordinate system is established using GNSS. Due to the fact that the working frequencies of the three sensors are different, time difference exists among the sensors, and therefore the three sensors including the GNSS sensor, the camera and the laser radar need to be synchronously processed in time and space, and the sensors are aligned in time dimension and space dimension. The time synchronization process is as follows:
step 1.1, adding timestamps to each frame of camera image data and point cloud data acquired by a laser radar based on a high-frequency GPS signal of a clock;
and step 1.2, searching the image data time stamp according to the point cloud data time stamp with lower frequency, and matching the image data time stamp with the time closest to the point cloud data time stamp with lower frequency to realize synchronous processing of the image data and the point cloud data.
And 2, extracting the traffic sign from the image data acquired in the step 1, performing binary mask processing on the extracted traffic sign by adopting an image segmentation algorithm, and calculating the central point data of the traffic sign by adopting a geometric center estimation algorithm.
In practical application, the traffic sign can be divided into a circle or a triangle, whether the traffic sign is the circle or the triangle is judged according to the binary mask processing result, and then different calculation modes are adopted according to different shapes. As shown in fig. 1: in this embodiment, an Otsu algorithm is first used to perform threshold segmentation on image data to obtain a binary mask region of a traffic identifier, and if the obtained binary mask region of the traffic identifier is circular, a Hough transform algorithm is used to extract a traffic identifier center point to obtain a center point and radius data. If the obtained traffic sign binary mask area is triangular, a mode of statistically extracting discrete small areas in the area is adopted: and extracting the discrete small region with the largest area from the statistical discrete region, then calculating the first moment of the discrete small region, and calculating the central point of the triangle according to the calculation result of the first moment. If the traffic identification binary mask region cannot be obtained by adopting the Otsu algorithm or the obtained traffic identification binary mask region is wrong, converting the detection region image from the RGB space to the HSV space by adopting an HSV color space conversion method to calculate the central point. Taking a triangle as an example, the detailed calculation process of the HSV color space transformation method is as follows:
carrying out region segmentation on the detection region according to the traffic sign color in practical application, and obtaining the maximum circumscribed triangle meeting the conditions as a mask region of the traffic sign; meanwhile, calculating the area of each discrete area in the detection area, selecting a triangular area with the largest area as a traffic sign area, and solving a first moment of the area to obtain a geometric center point of the traffic sign, wherein the geometric center point is used as geometric prior information of the traffic sign;
step 3, carrying out combined calibration on the laser radar and the camera, solving a laser radar-camera coordinate system transformation matrix, and unifying the coordinate systems of the camera and the laser radar by using the matrix; and (3) projecting the point cloud data obtained in the step (1) into the image data obtained in the step (1) by a cone projection guiding method to construct image-point cloud associated data. The detailed process is as follows:
and projecting the 2D mask region of the image data through the initial parameter prior information of the detection frame, searching the cloud coordinates of the points in the 3D coordinate space corresponding to the 2D detection frame, and extracting the target points on the plane traffic identifier in the geometric outline of the traffic identifier. Because the 2D area of the image detection frame lacks depth information, one point on the image is projected to a ray corresponding to the laser radar coordinate system, and therefore, the planar rectangular frame on the image corresponds to a pyramid in the laser radar coordinate system.
After the laser point cloud data is projected into the image data, the pixel coordinates of the point cloud data in the image are calculated according to the following formula through the one-to-one mapping relation of the point cloud data and the image data.
Wherein P is a camera internal reference matrix, and Tr _ velo _ to _ cam is a laser radar-camera transformation matrix obtained by combined calibration. The method comprises the steps of constructing image-point cloud associated data, wherein the associated data structure is [ u, v, x, y, z ], u/v is coordinates of point cloud projection to an image, determining whether corresponding original point cloud is reserved or not by judging whether a projection point coordinate value belongs to a traffic identification or not, achieving the purpose of only reserving point cloud data with cognitive information, and achieving light weight of the point cloud data.
And 4, extracting geometric outline information of the traffic identification from the constructed image-point cloud associated data by using the central point data obtained in the step 2, and extracting point cloud data in an outline range by taking the outline information as a basis.
And 5, projecting the point cloud data to the image to form discrete points so as to ensure semantic continuity. Therefore, it is necessary to select an image block region corresponding to the point cloud projection point and perform feature extraction. As shown in fig. 2, image data corresponding to the traffic identification contour of the point cloud data is selected, and after Gabor wavelet transformation is performed on the image data, an LBP operator is used for performing binarization mode extraction to generate an LBP-Gabor texture feature image. Dividing the LBP-Gabor texture feature image into n sub-image blocks, and performing histogram statistics and normalization processing on each sub-image block to generate n histogram feature vectors; and after cascading the n vectors, storing the n vectors as the final texture features of the traffic identification. The data calculation amount is reduced through Gabor wavelet transformation, and the robustness of the system is improved.
And 6, repeating the steps 2-4, obtaining the final texture features of all traffic identifications of the current road, obtaining a cognitive map semantic layer, and generating a cognitive map as shown in figure 3.
Step 7, matching and positioning the current position traffic identification scanned by the automatic driving vehicle and the characteristics in the cognitive map semantic layer data by adopting a two-stage searching strategy to obtain the coordinate of the current position traffic identification of the automatic driving vehicle in the cognitive map; the detailed process of matching and positioning by the two-stage search strategy is as follows:
as shown in fig. 4, the first stage matching includes the steps of:
7.1.1, taking the current position traffic identification of the automatic driving vehicle as a center as an interesting area according to a vehicle end observation result, and extracting a rectangular special frame; meanwhile, a rectangular frame is extracted for each traffic identification in the same road section cognitive map;
7.1.2 respectively adopting two measures of Bhattacharyya distance and perceptual hashing to calculate the similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame of the cognitive map,
7.1.3, respectively obtaining the comprehensive correlation coefficient of each rectangular frame of the cognitive map and the rectangular frame of the vehicle-end observation result according to the settlement result of the step 7.1.2. The integrated correlation coefficient is defined as:
7.1.4, sorting the comprehensive correlation coefficients calculated in the step 7.1.3 in a descending order, wherein the smallest numerical value is matched as the same traffic identification object.
And the second stage matching: and matching the textural features of each traffic identification stored in the semantic layer of the cognitive map according to the matching result of the first stage to obtain the corresponding position of the traffic identification scanned from the current position of the automatic driving vehicle in the cognitive map.
And 8, aligning the coordinates of the current automatic driving vehicle running traffic identification obtained in the step 7 in the cognitive map to a global coordinate system, and fusing the coordinates with a local positioning result of the laser radar odometer to obtain the pose estimation of the automatic driving vehicle. The detailed process is as follows:
referring to fig. 5, the vertex in the graph is formed by each frame pose, and the side is formed by laser radar odometer factor constraint and cognitive map matching result factor constraint; the alignment position and the constraint can be modeled into a joint probability distribution problem, and the nodes in the position and posture graph are assumed as the motion states x ═ x of the vehicle 0 ,x 1 ,...,x n And then:
wherein X is the vehicle state, z L ，z R And the measured values S of the vehicle states corresponding to the two constraint factors are measured value sets, and the measured values comprise local pose measurement of the laser radar odometer and pose measurement obtained by matching and positioning of the cognitive map. The essence of the pose graph is a maximum likelihood estimation problem, which can be expressed as follows, assuming that all probability measures in the same time period are independent:
assuming that the uncertainty observed in equation (2) follows a gaussian distribution,
then equation (3) can be obtained:
z k the method is characterized in that the measured value of the vehicle state, X is a state predicted value, h is an observation equation, observation is converted into a nonlinear least square problem through an equation (3), a target function is calculated to be the square sum of a constraint factor observed by a laser radar and a constraint factor source observed by a cognitive map in a global matching mode, and therefore the square sum is obtained
The observation result model is matched with the laser radar odometer and the cognitive map, and the position and posture between frames can be obtained through posture transformation between frames.
8.2, optimizing a laser radar odometer and a cognitive map matching observation result model by analyzing various residual factors in the pose graph:
(1) constraint factor for lidar observations
The invention solves the local pose through the laser radar odometer, the pose estimation in a short time is accurate and stable, and the pose relation between two adjacent frames is constructed as a factor of the global pose:
(2) Constraint factor for global matching observation of cognitive map
In the invention, the starting point of the vehicle is used as the origin of a coordinate system, and the position information obtained by map matching calculation is set as
For a state node observed by a traffic identifier, a map matching location constraint may be included, and the map matching factor is as follows:
the establishment of the global optimization pose graph is completed through the two constraint factors, and the essence of solving the optimization problem is to find a group of state variables to complete the matching of all the constraint factors, so that the sum of the Markov norms of all the error factors is minimum.
According to the derivation, the process of estimating the current pose of the autonomous vehicle in the embodiment is as follows:
step 8.1, setting a laser radar odometer to obtain a relative pose as follows:
P＝{P 0 ,P 1 ,P 2 ...}
P i ＝[R|T]
wherein, P i Is the pose of frame i relative to frame i-1. R and T are pose transformation matrix
The global positioning result obtained by solving the matched road signs arranged in the map is as follows:
G＝{G 0 ,G k ,...}
G＝[R|T]
wherein G is i For map global matching positioning result, R and T are pose transformation matrix
Assuming that the true pose of the camera to be solved is as follows:
X＝{X 0 ,X 1 ,X 2 ...}
X i ＝[R|T]
wherein X is the real pose of the camera to be solved, R and T are pose transformation matrix
Constructing the fusion positioning problem as a maximum posterior probability problem:
X＝argmax(f(X|P,G))
under the condition that the observation G and the observation X are known, the posterior probability is maximized, and a pose graph is constructed to solve the real pose of the vehicle.
The positioning method comprises two positioning sources of pose estimation of the laser radar odometer and vehicle positioning based on the traffic identification. The two positioning sources complement each other, and a high-precision positioning result of the automatic driving vehicle is obtained through the fusion of the two positioning sources. The laser radar odometer is a result output in the whole process, errors can be accumulated along with time, the result drifts in a certain range, the positioning method based on the cognitive map is a sparse positioning result, long-time drift errors of the laser radar odometer are corrected through fusion of the positioning results of the traffic identification, and accumulated errors of the laser radar odometer are eliminated. The fusion algorithm regards the positioning problem as an optimization problem under known observation, and constructs a pose graph optimization problem through a relative pose provided by the laser radar odometer and a global positioning result based on the traffic identification to obtain optimal positioning description.
A cognitive map positioning based on sparse point cloud-texture information comprises a data acquisition module, a feature extraction module, a map generation module, a matching positioning module and a fusion positioning module.
The data acquisition module is respectively connected with the map feature extraction module and the matching positioning module; for acquiring image data and point cloud data; preprocessing the acquired image data to obtain a central point of the image data traffic identification; global coordinates are established by adopting a GNSS, and coordinates, time and space unification of a camera, a laser radar and the GNSS sensor is completed; in the embodiment, image data is collected by a camera, and point cloud data is collected by a laser radar.
The map feature extraction module is connected with the map generation module, extracts geometric prior information of the traffic identification by using the received central point data under a coordinate system of a unified camera and a laser radar provided by the data acquisition module, and extracts point cloud data in a geometric prior information range on the basis of the geometric prior information, namely sparse point cloud generation; and selecting image data corresponding to the traffic identification outline of the point cloud data, and processing the image data by adopting Gabor wavelet transform, LBP operators, histogram statistics and normalization algorithm to obtain texture information of the traffic identification and provide the texture information to a map generation module.
The map generation module is connected with the matching positioning module and generates a cognitive map according to the received traffic identification texture signal;
the matching positioning module is connected with the fusion positioning module and is used for matching the traffic identification scanned near the current position of the automatic driving vehicle with the traffic identification of the cognitive map to obtain the coordinate of the traffic identification of the current position of the automatic driving vehicle in the cognitive map;
the fusion positioning module is connected with the traffic identification positioning module, aligns the coordinates of the traffic identification at the current position of the automatic driving vehicle in the cognitive map to a global coordinate system, and then fuses the coordinates with the local positioning result of the laser radar odometer to obtain the estimation of the vehicle pose.
Claims (6)
1. A cognitive map positioning method based on sparse point cloud-texture information is characterized by comprising the following steps: the method comprises the following steps:
step 1, collecting current road data, wherein the data comprises image data collected by a camera and point cloud data collected by a laser radar; establishing a global coordinate system by adopting a GNSS;
step 2, extracting a traffic sign from the image data acquired in the step 1, performing binary mask processing on the extracted traffic sign by adopting an image segmentation algorithm, and calculating central point data of the traffic sign by adopting a geometric center estimation algorithm to serve as geometric prior information of the traffic sign;
step 3, jointly calibrating the laser radar and the camera, solving a laser radar-camera coordinate system transformation matrix, and unifying the coordinate systems of the camera and the laser radar by using the matrix; projecting the point cloud data obtained in the step 1 into the image data obtained in the step 1 by a cone projection guiding method to construct image-point cloud associated data;
step 4, extracting geometric outline information of the traffic identification from the constructed image-point cloud associated data by using the central point data obtained in the step 2, and extracting point cloud data in an outline range by using the outline information as a basis;
step 5, selecting image data corresponding to the traffic identification outline of the point cloud data, performing Gabor wavelet transformation on the image data, and then performing binarization mode extraction by using an LBP operator to obtain an LBP-Gabor texture feature image; dividing the LBP-Gabor texture feature image into n sub-image blocks, and carrying out histogram statistics and normalization processing on each sub-image block to generate n histogram feature vectors; after the n vectors are cascaded, the n vectors are used as the final texture features of the traffic identification for storage;
step 6, repeating the steps 2-4, obtaining the final texture features of all traffic identifications of the current road, obtaining a cognitive map semantic layer, and finishing cognitive map generation;
step 7, matching and positioning the current position traffic identification of the automatic driving vehicle and the characteristics in the semantic layer data of the cognitive map by adopting a two-stage search strategy so as to obtain the coordinate of the current driving traffic identification of the automatic driving vehicle in the cognitive map;
and 8, aligning the coordinates of the current automatic driving vehicle driving traffic identification obtained in the step 7 in the cognitive map into a global coordinate system, and fusing the coordinates with the local positioning result of the laser radar odometer to obtain the vehicle pose estimation.
2. The cognitive map positioning method based on sparse point cloud-texture information as claimed in claim 1, wherein: the step 2 adopts a geometric center estimation algorithm to calculate the center point coordinates of the traffic sign, and the process comprises the following steps:
judging that the shape of the traffic sign is circular or triangular according to the binary mask processing result; if the traffic sign center point data is judged to be circular, a Hough transformation algorithm is adopted to extract the traffic sign center point data to obtain the center point data and the radius data, and if the traffic sign center point data is judged to be triangular, a discrete small area or an HSV color space transformation method in the statistical extraction area is adopted to obtain the traffic sign center point data.
3. The cognitive map positioning method based on sparse point cloud-texture information as claimed in claim 1, wherein: the detailed process of obtaining pose state estimation in the step 7 is as follows:
step 7.1, first-stage matching: eliminating the interference of the same traffic identification in the same road section on the positioning result:
7.1.1, taking the current position traffic identification of the automatic driving vehicle as a center as an interesting area according to a vehicle end observation result, and extracting a rectangular special frame; meanwhile, extracting a rectangular frame for each traffic identification in the cognitive map of the same road section;
7.1.2 respectively adopting two measures of Bhattacharyya distance and perceptual hashing to calculate the similarity between the rectangular frame of the vehicle-end observation result and each rectangular frame of the cognitive map,
7.1.3, respectively obtaining the comprehensive correlation coefficient of each rectangular frame of the cognitive map and the rectangular frame of the vehicle-end observation result according to the settlement result of the step 7.1.2. The integrated correlation coefficient is defined as:
7.1.4, sequencing the comprehensive correlation coefficients calculated in the step 7.1.3 in a descending order, wherein the smallest numerical value is matched as the same traffic signboard object;
step 7.2, second stage matching: and matching the textural features of each traffic identification stored in the semantic layer of the cognitive map according to the matching result of the first stage to obtain the corresponding position of the current vehicle running traffic identification in the cognitive map.
4. The cognitive map positioning method based on sparse point cloud-texture information as claimed in claim 1, wherein: the step 1 also comprises the steps of carrying out time synchronization processing on the GNSS, the laser radar and the camera, and aligning data acquired by the camera and point cloud data of the laser radar in a time dimension through the time synchronization processing; the time synchronization process is as follows:
step 1.1, adding timestamps to each frame of camera image data and point cloud data acquired by a laser radar based on a high-frequency GPS signal of a clock;
and step 1.2, searching the image data time stamp according to the point cloud data time stamp with lower frequency, and matching the image data time stamp with the time closest to the point cloud data time stamp with lower frequency to realize synchronous processing of the image data and the point cloud data.
5. The cognitive map positioning method based on sparse point cloud-texture information as claimed in claim 1, wherein: the detailed process of the step 8 is as follows:
the laser radar odometer is designed to obtain relative poses as follows:
P＝{P 0 ,P 1 ,P 2 …}
P i ＝[R|T]
wherein, P i Is the pose of frame i relative to frame i-1. R, T are position transformation matrix
The global positioning result obtained by solving the matched road signs arranged in the map is as follows:
G＝{G 0 ,G k ,…}
G＝[R|T]
wherein G is i For the global matching positioning result of the map, R and T are pose transformation matrixes
Assuming that the true pose of the camera to be solved is as follows:
X＝{X 0 ,X 1 ,X 2 ...}
X i ＝[R|T]
wherein X is the real pose of the camera to be solved, R and T are pose transformation matrix
Constructing the fusion positioning problem as a maximum posterior probability problem:
X＝argmax(f(X|P,G))
under the condition that the observation G and the observation X are known, the posterior probability is maximized, and a pose graph is constructed to solve the real pose of the vehicle.
6. A cognitive map positioning based on sparse point cloud-texture information comprises a data acquisition module, a feature extraction module, a map generation module, a matching positioning module and a fusion positioning module;
the data acquisition module is respectively connected with the map feature extraction module and the matching positioning module; for acquiring image data and point cloud data; preprocessing the acquired image data to obtain a central point of the image data traffic identification; global coordinates are established by adopting a GNSS, and coordinate, time and space unification of a camera, a laser radar and the GNSS sensor is completed; in the embodiment, image data is collected by a camera, and point cloud data is collected by a laser radar;
the map feature extraction module is connected with the map generation module, extracts geometric prior information of the traffic identification by using the received central point data under a coordinate system of a unified camera and a laser radar provided by the data acquisition module, and extracts point cloud data in a contour range on the basis of the geometric prior information, namely sparse point cloud generation; selecting image data corresponding to the traffic identification outline of the point cloud data, and obtaining texture information of the traffic identification after the image data is processed by Gabor wavelet transformation, LBP operators, histogram statistics and normalization algorithm and providing the texture information to a map generation module;
the map generation module is connected with the matching positioning module and generates a cognitive map according to the received traffic identification texture signal;
the matching positioning module is connected with the fusion positioning module, and matches the traffic identification scanned near the current position of the automatic driving vehicle with the traffic identification of the cognitive map by adopting two-stage matching to obtain the coordinate of the traffic identification of the current position of the automatic driving vehicle in the cognitive map;
and the fusion positioning module aligns the coordinates of the traffic identification of the current position of the automatic driving vehicle in the cognitive map to a global coordinate system, and then fuses the coordinates with the local positioning result of the laser radar odometer to obtain the estimation of the vehicle pose.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210612389.XA CN115031744A (en) | 2022-05-31 | 2022-05-31 | Cognitive map positioning method and system based on sparse point cloud-texture information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210612389.XA CN115031744A (en) | 2022-05-31 | 2022-05-31 | Cognitive map positioning method and system based on sparse point cloud-texture information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115031744A true CN115031744A (en) | 2022-09-09 |
Family
ID=83122874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210612389.XA Pending CN115031744A (en) | 2022-05-31 | 2022-05-31 | Cognitive map positioning method and system based on sparse point cloud-texture information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115031744A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115468576A (en) * | 2022-09-29 | 2022-12-13 | 东风汽车股份有限公司 | Automatic driving positioning method and system based on multi-mode data fusion |
-
2022
- 2022-05-31 CN CN202210612389.XA patent/CN115031744A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115468576A (en) * | 2022-09-29 | 2022-12-13 | 东风汽车股份有限公司 | Automatic driving positioning method and system based on multi-mode data fusion |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111144388B (en) | Monocular image-based road sign line updating method | |
| CN109631855B (en) | ORB-SLAM-based high-precision vehicle positioning method | |
| CN111273305B (en) | Multi-sensor fusion road extraction and indexing method based on global and local grid maps | |
| Xia et al. | Geometric primitives in LiDAR point clouds: A review | |
| CN111652179A (en) | Semantic high-precision map construction and positioning method based on dotted line feature fusion laser | |
| CN113989450B (en) | Image processing method, device, electronic equipment and medium | |
| CN111060924B (en) | SLAM and target tracking method | |
| CN110542908A (en) | laser radar dynamic object perception method applied to intelligent driving vehicle | |
| CN114413881B (en) | Construction method, device and storage medium of high-precision vector map | |
| CN111915517B (en) | Global positioning method suitable for RGB-D camera under indoor illumination unfavorable environment | |
| CN107679458B (en) | Method for extracting road marking lines in road color laser point cloud based on K-Means | |
| CN112434706B (en) | High-precision traffic element target extraction method based on image point cloud fusion | |
| CN113516664A (en) | Visual SLAM method based on semantic segmentation dynamic points | |
| CN113392169A (en) | High-precision map updating method and device and server | |
| CN115294293B (en) | Method for automatically compiling high-precision map road reference line based on low-altitude aerial photography result | |
| WO2023060632A1 (en) | Street view ground object multi-dimensional extraction method and system based on point cloud data | |
| CN111325184B (en) | Intelligent interpretation and change information detection method for remote sensing image | |
| CN114509065B (en) | Map construction method, system, vehicle terminal, server and storage medium | |
| CN117111085A (en) | Automatic driving automobile road cloud fusion sensing method | |
| CN115031744A (en) | Cognitive map positioning method and system based on sparse point cloud-texture information | |
| Lu et al. | A lightweight real-time 3D LiDAR SLAM for autonomous vehicles in large-scale urban environment | |
| CN113487741B (en) | Dense three-dimensional map updating method and device | |
| CN113838129B (en) | Method, device and system for obtaining pose information | |
| CN115468576A (en) | Automatic driving positioning method and system based on multi-mode data fusion | |
| CN112925322B (en) | Autonomous positioning method for unmanned vehicle in long-term scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |