WO2021017211A1

WO2021017211A1 - Vehicle positioning method and device employing visual sensing, and vehicle-mounted terminal

Info

Publication number: WO2021017211A1
Application number: PCT/CN2019/113488
Authority: WO
Inventors: 李天威; 徐抗; 刘一龙; 童哲航
Original assignee: 魔门塔(苏州)科技有限公司; 北京初速度科技有限公司
Priority date: 2019-07-29
Filing date: 2019-10-26
Publication date: 2021-02-04
Also published as: CN112308913B; CN112308913A

Abstract

Disclosed are a vehicle positioning method and device employing visual sensing, and a vehicle-mounted terminal. The method comprises: acquiring a road image captured by a camera apparatus; determining, according to data acquired by a motion detection apparatus, an initial positioning orientation corresponding to the road image; determining, from a preconfigured map and according to the initial positioning orientation, a target map point corresponding to the road image; extracting an edge feature image of the road image according to preconfigured edge intensity; and determining, according to the initial positioning orientation, a mapping difference between the target map point and a point in the edge feature image, and determining a vehicle positioning orientation according to a projection difference, wherein the initial positioning orientation is an orientation in a world coordinate system in which the preconfigured map is located, and each map point in the preconfigured map is acquired by performing three-dimensional reconstruction on a point in an edge feature image of a sample road image in advance and by performing selection. The solution provided by the embodiments of the invention improves the success rate of vehicle positioning performed on the basis of visual sensing.

Description

Vision-based vehicle positioning method, device and vehicle-mounted terminal

Technical field

The present invention relates to the technical field of intelligent driving, in particular to a vision-based vehicle positioning method, device and vehicle-mounted terminal.

Background technique

In the field of intelligent driving technology, positioning the vehicle is an important part of intelligent driving. Generally, when the vehicle is running, the vehicle's pose can be determined according to the satellite positioning system. However, when the vehicle is driving into a scene with weak or no satellite signal, in order to accurately determine the positioning pose of the vehicle, the positioning can be performed based on the visual positioning.

Vision-based positioning is usually based on the matching between the semantic information of the road image collected by the camera device and the semantic information in the high-precision map. The semantic information in the high-precision map is modeled based on common landmarks on the road. Markers generally include lane lines on the ground, ground marking lines, traffic signs, and street light poles.

When there are enough effective markers in the scene, the above-mentioned vision-based positioning method can effectively determine the vehicle positioning pose. However, when the markers in the scene are scarce or even there are no markers, it is difficult for the high-precision map to give enough effective information for visual positioning; or when the markers cannot be completely matched with the high-precision map due to occlusion or aging, visual positioning It may not be possible. All of the above results in low effectiveness of visual positioning.

Summary of the invention

The invention provides a vision-based vehicle positioning method, device and vehicle-mounted terminal to improve the effectiveness of vehicle positioning based on vision. The specific technical solution is as follows.

In the first aspect, an embodiment of the present invention discloses a vision-based vehicle positioning method, including:

Obtain road images collected by camera equipment;

Determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

According to the initial positioning pose, a target map point corresponding to the road image is determined from the preset map; wherein, each map point in the preset map is: an edge feature map of the sample road image in advance The points in are obtained after 3D reconstruction and selection;

Extracting the edge feature map of the road image according to the preset edge strength;

According to the initial positioning pose, the mapping difference between the target map point and the point in the edge feature map is determined, and the vehicle positioning pose is determined according to the projection difference.

Optionally, the step of determining the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determining the vehicle positioning pose according to the mapping difference includes:

Taking the initial positioning pose as the initial value of the estimated pose, and according to the value of the estimated pose, map the target map point and the point in the edge feature map to the same coordinate system, and determine the mapping to The mapping difference between the target map point and the point in the edge feature map in the same coordinate system;

When the mapping difference is greater than a preset difference threshold, modify the value of the estimated pose according to the mapping difference, return to execute the value of the estimated pose, and compare the target map point with the Steps of mapping points in the edge feature map to the same coordinate system;

When the mapping difference is less than the preset difference threshold, the vehicle positioning pose is determined according to the current value of the estimated pose.

Optionally, the target map point and the point in the edge feature map are mapped to the same coordinate system according to the value of the estimated pose, and the target map mapped to the same coordinate system is determined The step of mapping the difference between the point and the point in the edge feature map includes:

According to the value of the estimated pose, determine the conversion matrix between the world coordinate system and the camera coordinate system. According to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, The target map point is mapped to the image coordinate system to obtain a first mapping position of the target map point, and the first mapping position and the position of the point in the edge feature map in the image coordinate system are calculated Projection difference between; wherein the camera coordinate system is the three-dimensional coordinate system where the camera device is located, and the image coordinate system is the coordinate system where the road image is located;

or,

According to the value of the estimated pose, determine the conversion matrix between the world coordinate system and the camera coordinate system; according to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, The points in the edge feature map are mapped to the world coordinate system to obtain a second mapping position of the points in the edge feature map, and the second mapping position and the target map point are calculated in the world coordinate system The projection difference between the positions in.

Optionally, the step of determining a target map point corresponding to the road image from the preset map according to the initial positioning pose includes:

Taking the position of the initial positioning pose in the preset map as a center, and a map point contained in a sphere determined by using a preset distance as a radius as a candidate map point;

The map points within the collection range of the camera device are filtered from each candidate map point to obtain a target map point corresponding to the road image.

Optionally, the step of screening map points within the collection range of the camera device from each candidate map point to obtain a target map point corresponding to the road image includes:

Determine the conversion matrix between the world coordinate system and the camera coordinate system according to the initial positioning pose; wherein, the camera coordinate system is the three-dimensional coordinate system where the camera device is located;

Mapping each candidate map point to the camera coordinate system according to the conversion matrix to obtain a third mapping position of each candidate map point;

According to the filtering condition that the third mapping position is within the collection range of the camera device in the vertical height direction, a target map point corresponding to the road image is obtained by filtering from each candidate map point.

Optionally, the determined map points to be selected include the coordinate positions and normal vectors of the map points to be selected; the map points within the collection range of the camera device are selected from each map point to be selected, and the corresponding map points to the road are obtained. The steps of the target map point corresponding to the image include:

Determine the connection line between the camera device and each map point to be selected according to the coordinate position of each map point to be selected;

Calculate the angle between each line and the normal vector of the corresponding map point to be selected;

According to the screening condition that the included angle is within the preset included angle range, a target map point corresponding to the road image is obtained by filtering from each candidate map point.

Optionally, each map point in the preset map is constructed in the following manner:

Acquiring a sample road image, and extracting a sample edge feature map of the sample road image according to a preset edge strength;

Determine the sample positioning pose corresponding to the sample road image according to the data collected by the motion detection device; wherein the sample positioning pose is the pose in the world coordinate system;

Determining the position information of each point in the sample edge feature map in the world coordinate system based on a three-dimensional reconstruction algorithm and the sample positioning pose;

A map point is selected from each point of the sample edge feature map according to a preset point density, and the position information of each map point in the world coordinate system is added to the preset map.

In the second aspect, an embodiment of the present invention discloses a vision-based vehicle positioning device, including:

The road image acquisition module is configured to acquire the road image collected by the camera device;

The initial pose determination module is configured to determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

The map point determination module is configured to determine a target map point corresponding to the road image from the preset map according to the initial positioning pose; wherein, each map point in the preset map is: Perform three-dimensional reconstruction and selection of points in the edge feature map of the sample road image;

An edge feature extraction module configured to extract the edge feature map of the road image according to a preset edge strength;

The vehicle pose determination module is configured to determine the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determine the vehicle positioning pose according to the projection difference.

Optionally, the initial pose determination module is specifically configured as:

When the mapping difference is greater than a preset difference threshold, modify the value of the estimated pose according to the mapping difference, return to execute the value of the estimated pose, and compare the target map point with the The operation of mapping points in the edge feature map to the same coordinate system;

Optionally, the initial pose determination module maps the target map point and the point in the edge feature map to the same coordinate system according to the value of the estimated pose, and determines the mapping to the same coordinate system When the mapping difference between the target map point in and the point in the edge feature map includes:

or,

Optionally, the map point determination module is specifically configured as:

Optionally, when the map point determination module selects map points within the collection range of the camera device from each map point to be selected, and obtains the target map point corresponding to the road image, it includes:

Optionally, the determined to-be-selected map points include the coordinate positions and normal vectors of the to-be-selected map points; the map point determination module selects map points within the collection range of the camera device from each of the to-be-selected map points, When obtaining the target map point corresponding to the road image, it includes:

According to the screening condition that the included angle is within the preset included angle range, the target map point corresponding to the road image is obtained by filtering from each candidate map point.

Optionally, the device further includes: a map point construction module configured to construct each map point in the preset map by using the following operations:

In a third aspect, an embodiment of the present invention discloses a vehicle-mounted terminal, including: a processor, a camera device, and a motion detection device; the processor includes:

Road image acquisition module for acquiring road images collected by camera equipment;

The initial pose determination module is used to determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

The map point determination module is configured to determine a target map point corresponding to the road image from a preset map according to the initial positioning pose; wherein, each map point in the preset map is: pre-complying sample roads The points in the edge feature map of the image are obtained after three-dimensional reconstruction and selection;

The edge feature extraction module is used to extract the edge feature map of the road image according to the preset edge strength;

Optionally, the initial pose determination module is specifically used for:

or,

Optionally, the map point determination module is specifically used for:

Taking the position of the initial positioning pose on the preset map as the center, and the map point contained in the sphere determined by using the preset distance as the radius as the candidate map point;

Optionally, when the map point determination module selects map points within the collection range of the camera device from each to-be-selected map point to obtain the target map point corresponding to the road image, it includes:

Optionally, the processor further includes: a map point construction module, configured to construct each map point in the preset map by using the following operations:

It can be seen from the foregoing that the vision-based vehicle positioning method, device, and vehicle-mounted terminal provided by the embodiments of the present invention can determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device, and then according to the initial positioning pose from The target map point corresponding to the road image is determined in the preset map, and the vehicle positioning pose is determined according to the initial positioning pose and the mapping difference between the target map point and the points in the edge feature map of the road image. The map points are obtained by three-dimensional reconstruction and selection of points in the edge feature map of the sample road image in advance. Because the edge feature map of the road image can extract the structured features in the image, these structured features will be richer and anti-noise. Even if the markers in the scene are sparse or the markers are occluded, they can pass through the target map points and edge feature maps. The match between the points in determines the vehicle positioning pose. Therefore, the embodiments of the present invention can improve the effectiveness of vehicle positioning based on vision. Of course, implementing any product or method of the present invention does not necessarily need to achieve all the advantages described above at the same time.

The innovative points of the embodiments of the present invention include:

1. The edge feature map is the structural feature in the road image, which is rich and anti-noise, and is not easily affected by the number of lane lines, traffic signs, street light poles and other landmarks in the scene, and it is not easily affected by changes in the morning, evening, and weather. It is more robust due to the influence of illumination. Therefore, matching the points in the edge feature map with the map points in the preset map for vehicle positioning can improve the effectiveness of vehicle positioning.

2. When determining the vehicle positioning pose, continuously adjust the value of the estimated pose to obtain the mapping difference between the target map point and the point in the edge feature map, so that the value of the estimated pose gradually approaches the real Value, iteratively solve the vehicle positioning pose, making the solved vehicle positioning pose more accurate.

3. For the map points in the ball centered on the position of the initial positioning pose in the preset map, the map points within the collection range of the camera device are filtered as the target map points, which can be selected from the preset map. The map points can improve the accuracy of the determined vehicle positioning pose.

4. When constructing the preset map, extract the structured sample edge feature map from the sample road image, and extract the point cloud data from the sample edge feature map for map construction, which can get more dense map information and increase the preset map The amount of effective information in.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained from these drawings without creative work.

FIG. 1 is a schematic structural diagram of a vision-based vehicle positioning method provided by an embodiment of the present invention;

2 is a schematic diagram of the angle between the camera device and the normal vector of the map point provided by an embodiment of the present invention;

3 is a schematic diagram of an architecture of a vision-based vehicle positioning method provided by an embodiment of the present invention;

4 is a schematic structural diagram of a vision-based vehicle positioning device provided by an embodiment of the present invention;

Fig. 5 is a schematic structural diagram of a vehicle-mounted terminal provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

It should be noted that the terms "including" and "having" in the embodiments of the present invention and the drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, the process, method, system, product or device that contains a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.

The embodiment of the invention discloses a vision-based vehicle positioning method, a processor, and a vehicle-mounted terminal, which can improve the effectiveness of vehicle positioning based on vision. The embodiments of the present invention will be described in detail below.

FIG. 1 is a schematic flowchart of a vision-based vehicle positioning method provided by an embodiment of the present invention. This method is applied to electronic equipment. The electronic device may be an ordinary computer, a server, or an intelligent terminal device, etc., and may also be an in-vehicle terminal such as an in-vehicle computer or an in-vehicle industrial control computer (IPC). In this embodiment, the vehicle-mounted terminal may be installed in a vehicle, and the vehicle refers to a smart vehicle. A variety of sensors are installed in the vehicle, including sensors such as camera equipment and motion detection equipment. There may be one or more camera devices provided in the vehicle. The motion detection device may include sensors such as an Inertial Measurement Unit (IMU) and/or a wheel speedometer. The method specifically includes the following steps.

S110: Obtain a road image collected by the camera device.

Among them, the camera device can collect road images at a preset frequency. The road image may include road markers or image data of any other objects within the image collection range of the camera device.

In this embodiment, when there are multiple camera devices, the above-mentioned road image may be collected by one camera device installed in the front of the vehicle, or obtained by stitching images collected by multiple cameras installed in the front of the vehicle. The place where the vehicle is located can be outdoors or a parking lot.

The road image may be an image of the surrounding environment of the vehicle collected by the camera device when the vehicle is driving on various roads. Wherein, the road can be any place where vehicles can travel, such as urban roads, rural roads, mountain roads, parking lot roads, etc., and the images collected during the process of entering the parking space can also be included in the road image.

S120: Determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device.

Among them, the initial positioning pose is the pose in the world coordinate system where the preset map is located. The preset map may be a high-precision map installed in the vehicle. The time of the data collected by the motion detection device and the time of the road image collected by the camera device may be related to each other. For example, the collection time of the two may be the same time or a time with a short time difference.

According to the data collected by the motion detection device, determining the initial positioning pose corresponding to the road image may specifically include: obtaining the previous positioning pose, and determining the relationship with the road image based on the previous positioning pose and the data collected by the motion detection device The corresponding initial positioning pose. Among them, the last positioning pose may be the determined positioning pose of the vehicle at the last moment.

In another embodiment, this step may further include: according to the data collected by the motion detection device and the result of matching the road feature in the road image with the road feature in the preset map, determining the vehicle's corresponding to the road image Initial positioning pose. In this embodiment, matching the road feature in the road image with the road feature in the preset map corresponds to another vision-based positioning method.

In another implementation manner, the motion detection device may also include a global positioning system (Global Positioning System, GPS). When the motion detection equipment includes GPS, it is possible to eliminate the accumulated errors in the positioning process according to the IMU and/or wheel speedometer as much as possible, and improve the accuracy of the positioning pose.

The initial positioning pose determined in this step is the initial positioning pose of the vehicle at the current moment when the road image is collected, and it is used to determine a more accurate vehicle positioning pose at the current moment.

S130: Determine the target map point corresponding to the road image from the preset map according to the initial positioning pose.

Determining the target map point corresponding to the road image from the preset map can be understood as determining the location information of the target map point corresponding to the road image from the preset map. The target map point is a map point that may be observed in the road image. In this step, according to the position of the initial positioning pose in the world coordinate system, a map point within a preset range near the position in the preset map can be used as a target map point.

Among them, each map point in the preset map is obtained after three-dimensional reconstruction and selection of points in the edge feature map of the sample road image in advance. The corresponding relationship between each map point and location information is stored in a preset map, and the location information is the location information of the map point in the world coordinate system.

The position information of each map point in the world coordinate system includes: the coordinate position of each map point in the world coordinate system and the normal vector information of the map point. The normal vector information of the map point indicates the normal vector of the plane where the map point is located. In the world coordinate system, the coordinate position of the map point can be represented by three coordinates (a, b, c), and the normal vector information of the point can be represented by three parameters (A, B, C). The location information of each map point contains information of 6 dimensions. This representation can be equivalent to using a plane A(x-a)+B(y-b)+C(z-c)=0 to represent each map point.

S140: Extract the edge feature map of the road image according to the preset edge strength.

The edge in an image refers to a collection of pixels with a step change in the gray level of surrounding pixels. The gray values of the pixels on both sides of the edge points are significantly different. Edge strength can be understood as the magnitude of the edge point gradient. The edge feature map can be understood as data containing edge features consistent with the size of the road image. The edge feature map contains the position information of the edge lines in the road image. The edge line is composed of edge points.

The extracted edge feature map is the position information expressed in the image coordinate system where the road image is located. The edge points in the edge feature map are the positions in the image coordinate system.

The above extraction of the edge feature map of the road image can be understood as the feature extraction of the road image to obtain the edge feature map. Specifically, when extracting the edge feature map of the road image, the preset edge strength can be used as the threshold, and edges with edge strength greater than the threshold can be extracted from the road image to obtain the edge feature map. It is also possible to extract edges with greater edge strength in the local range of the road image. For example, in a column with a ridge, the gradient of the ridge inside the column may not be large in the entire road image, but The edge strength of the ridge line is relatively high in the range of the column, so it is also the object of edge extraction. The above two schemes can also be combined.

The edge feature map extracted according to the preset edge strength can reflect the structural features in the road image. Specifically, canny operator, sobel operator and LoG operator can be used to extract the edge feature map of the road image.

After the edge feature map is extracted, Gaussian Blur algorithm processing can also be performed on the values in the edge feature map, which can make the processed edge feature map smoother.

In another embodiment, this step may further include: extracting an edge feature map of the road image based on the edge feature extraction model.

Among them, the edge feature extraction model is obtained by training based on the sample road image and the edge feature map labeled according to the preset edge strength. The edge feature extraction model associates the road image with the corresponding edge feature map.

Based on the edge feature extraction model, extracting the edge feature map of the road image may include: inputting the road image into the edge feature extraction model, and obtaining the edge feature map of the road image output by the edge feature extraction model.

In the training phase, multiple sample road images and labeled edge features can be obtained, and the sample road images are input into the edge feature extraction model; the edge feature extraction model extracts the feature vector of the sample road image according to the model parameters, and regresses the feature vector to obtain The reference edge feature of the sample road image; compare the reference edge feature with the labeled edge feature to obtain the difference amount; when the difference amount is greater than the preset difference threshold, return to the step of inputting the sample road image into the edge feature extraction model; When the amount of difference is not greater than the preset difference threshold, it is determined that the edge feature extraction model training is completed.

Among them, the marked edge feature reflects the preset edge strength. In other words, the edge features in the sample road image are labeled according to the preset edge strength as the standard. The edge feature extraction model trained by the above-mentioned machine learning method can extract more accurate edge features. The edge feature extraction model can also be called an edge detector.

S150: Determine the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determine the vehicle positioning pose according to the projection difference.

The initial positioning pose can reflect the pose of the vehicle within a certain accuracy range. In order to determine the pose of the vehicle more accurately, the positioning pose of the vehicle is determined according to the above-mentioned projection difference in this embodiment. When the determined vehicle positioning pose is close to the true value, the mapping difference between the target map point and the point in the edge feature map is less than the preset difference threshold.

It can be seen from the above content that this embodiment can determine the initial positioning pose corresponding to the road image based on the data collected by the motion detection device, and determine the target map point corresponding to the road image from the preset map according to the initial positioning pose. The initial positioning pose and the mapping difference between the target map points and the points in the edge feature map of the road image are used to determine the vehicle positioning pose. The map points in the preset map are the points in the edge feature map of the sample road image in advance The three-dimensional reconstruction performed and selected. Because the edge feature map of the road image can extract the structured features in the image, these structured features will be richer and anti-noise. Even if the markers in the scene are sparse or the markers are occluded, they can pass through the target map points and edge feature maps. The match between the points in determines the vehicle positioning pose. Therefore, this embodiment can improve the effectiveness when positioning the vehicle based on vision.

To sum up, the difference between this embodiment and the traditional vision-based vehicle positioning method also includes that the traditional positioning method detects semantic features corresponding to artificial objects such as lane lines, sidewalks, traffic signs, and street light poles in road images. If these semantic features are not present in the scene, vehicle positioning may not be possible. However, the edge features extracted from the road image in this embodiment are more generalized, including the edge features of man-made objects and natural objects in the scene, and are more adaptable, which makes the vehicle positioning process more effective.

On the other hand, because the present embodiment extracts structured features, it is more robust to the influence of illumination when morning and evening, weather, etc. change. The traditional method of extracting feature points from the image is not anti-noise in the case of lighting changes; the traditional trained method does not generalize the scene and requires a lot of data training to ensure the effectiveness of the scheme. The solution of this embodiment can be used as a mapping and positioning solution across morning and evening and/or seasons.

In another embodiment of the present invention, based on the embodiment shown in FIG. 1, in step S150, the mapping difference between the target map point and the point in the edge feature map is determined according to the initial positioning pose, and the mapping difference is determined according to the mapping difference The steps for positioning the vehicle pose include the following steps 1a to 3a.

Step 1a: Use the initial positioning pose as the initial value of the estimated pose, and according to the estimated pose value, map the target map point and the point in the edge feature map to the same coordinate system, and determine the mapping to the same coordinate system The mapping difference between the target map point and the point in the edge feature map.

In this embodiment, the target map point is located in the world coordinate system, and the point in the edge feature map is located in the image coordinate system. The location information of the target map point can be mapped to the image coordinate system, or the point in the edge feature map can be mapped to In the world coordinate system. The same coordinate system mentioned above may be an image coordinate system, a world coordinate system, or other coordinate systems.

Determining the mapping difference between the target map point mapped to the same coordinate system and the point in the edge feature map can be understood as determining the location information of the target map point mapped to the same coordinate system and the point in the edge feature map The difference between the location information is regarded as the mapping difference. The above difference can also be called residual.

For example, the above-mentioned mapping difference can be calculated according to the formula e=I(u)-α. Among them, the pixel coordinate obtained after mapping the target map point to the image coordinate system is u, I(u) represents the response of the pixel coordinate u when the target map point is mapped to the edge feature map, and α can be based on the characteristics of the edge detector Set, for example, set to 255. If the edge detector makes statistical and regular observations for the position of each map point and the camera device’s viewing angle, it can be based on the observation generation function α=f(p _c ) to replace the fixed value of α, where p _c is the map point in the camera The position in the coordinate system.

Step 2a: When the mapping difference is greater than the preset difference threshold, modify the estimated pose value according to the mapping difference, and return to perform step 1a according to the estimated pose value to map the target map point and the point in the edge feature map to Steps in the same coordinate system.

When the mapping difference is greater than the preset difference threshold, it is considered that the estimated pose is still far from the real pose of the vehicle, and the estimated pose can be modified continuously, and the iterative process can be repeated.

When modifying the estimated pose value according to the mapping difference, it can be modified based on the Jacobian matrix, Hessian matrix, Gauss-Newton iteration method or Levenberg-Marquardt (LM) algorithm, according to the mapping difference. Estimate the value of the pose.

Step 3a: When the mapping difference is less than the preset difference threshold, the vehicle positioning pose is determined according to the current value of the estimated pose.

When the mapping difference is less than the preset difference threshold, it is considered that the estimated pose is very close to the real pose of the vehicle. According to the current value of the estimated pose, the vehicle positioning pose can be determined. Specifically, the current value of the estimated pose can be directly determined as the vehicle positioning pose, or the current value of the estimated pose can be determined as Vehicle positioning pose.

When the mapping difference is equal to the preset difference threshold, the value of the estimated pose may be modified according to the mapping difference, and step 1a may be returned to, or the vehicle positioning pose may be determined according to the current value of the estimated pose.

In summary, in this embodiment, when determining the vehicle positioning pose, the value of the estimated pose is continuously adjusted to obtain the mapping difference between the target map point and the point in the edge feature map, so that the estimated pose is The value gradually approaches the true value, so that the vehicle positioning pose is solved iteratively, so that the solved vehicle positioning pose is more accurate.

In another embodiment of the present invention, based on the embodiment shown in FIG. 1, in step 1a, according to the value of the estimated pose, map the target map point and the point in the edge feature map to the same coordinate system, and determine the mapping to The step of mapping differences between the target map points in the same coordinate system and the points in the edge feature map may include the following embodiments.

In the first embodiment, the conversion matrix between the world coordinate system and the camera coordinate system is determined according to the value of the estimated pose, and according to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, the target map The point is mapped to the image coordinate system to obtain the first mapping position of the target map point, and the projection difference between the first mapping position and the position of the point in the edge feature map in the image coordinate system is calculated.

The camera coordinate system is the three-dimensional coordinate system where the camera device is located, and the image coordinate system is the coordinate system where the road image is located. The estimated pose is the pose of the vehicle in the world coordinate system. According to the value of the estimated pose, the conversion matrix between the world coordinate system and the camera coordinate system can be determined.

In this embodiment, when the target map point is mapped to the image coordinate system, the target map point can be converted to the camera coordinate system according to the following formula:

Among them, p _c is the position of the target map point in the camera coordinate system, and p _w is the position of the target map point in the world coordinate system,

Is the conversion relationship between the world coordinate system and the car body coordinate system,

Is the conversion matrix between the vehicle body coordinate system and the camera coordinate system,

Represents the conversion matrix between the world coordinate system and the camera coordinate system. Then, according to the projection model of the camera device, the coordinates in the camera coordinate system are converted to the image coordinate system to obtain the pixel coordinate u=π(p _c ) of the target map point. Among them, π(.) represents the projection model of the camera device, and u represents the target map point in the image coordinate system.

The second embodiment is to determine the conversion matrix between the world coordinate system and the camera coordinate system according to the value of the estimated pose, and according to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, the edge feature map The point of is mapped to the world coordinate system to obtain the second mapping position of the point in the edge feature map, and the projection difference between the second mapping position and the position of the target map point in the world coordinate system is calculated.

In summary, in this embodiment, according to the mutual conversion relationship between the world coordinate system, the camera coordinate system, and the image coordinate system, the location information of the target map point can be mapped to the image coordinate system, or the point in the edge feature map can be mapped In the world coordinate system, a specific implementation method is provided for determining the mapping difference.

In another embodiment of the present invention, based on the embodiment shown in FIG. 1, step S130, the step of determining the target map point corresponding to the road image from the preset map according to the initial positioning pose, includes steps 1b and 2b.

Step 1b: Take the position of the initial positioning pose in the preset map as the center, and use the preset distance as the radius to determine the map point contained in the sphere as the candidate map point. Wherein, the preset distance may be a distance value determined based on experience. There can be multiple determined map points to be selected.

Step 2b: Filter the map points within the collection range of the camera device from each to-be-selected map point to obtain the target map point corresponding to the road image.

The target map points obtained through the above-mentioned screening process are map points that may be observed in the road image.

In summary, in this embodiment, for the map points in the ball centered on the position of the initial positioning pose in the preset map, the map points within the collection range of the camera device are filtered as the target map points. Select effective map points from the preset maps to improve the accuracy of the determined vehicle positioning pose.

In another embodiment of the present invention, based on the embodiment shown in FIG. 1, in step 2b, the map points within the collection range of the camera device are selected from the map points to be selected to obtain the target map point corresponding to the road image The steps include the following steps 2b-1 to 2b-3.

Step 2b-1: Determine the conversion matrix between the world coordinate system and the camera coordinate system according to the initial positioning pose. Among them, the camera coordinate system is the three-dimensional coordinate system where the camera device is located.

Step 2b-2: According to the conversion matrix, map each to-be-selected map point to the camera coordinate system to obtain the third mapping position of each to-be-selected map point.

Specifically, in this step, the map points to be selected can be mapped to the camera coordinate system according to the following formula:

p _b ＝Tp _w ,p∈A

Among them, p _b is the third mapping position of each candidate map point in the camera coordinate system, p _w is the position information of each candidate map point in the world coordinate system, T is the above conversion matrix, and p is any candidate Map points, A is a point set composed of map points to be selected.

Since the camera device is fixed in the vehicle, the map points to be selected can be screened by replacing the camera coordinate system with the vehicle body coordinate system according to the known conversion relationship between the camera coordinate system and the vehicle body coordinate system.

Step 2b-3: According to the filtering condition that the third mapping position is within the collection range of the camera device in the vertical height direction, the target map points corresponding to the road image are filtered from each candidate map point.

Wherein, the acquisition range of the camera device in the vertical height direction can be expressed as the range of the z axis [z1, z2]. For example, [z1, z2] can be [-0.1m, 4m]. Specifically, in this step, the map point to be selected whose value of the z-axis is within the collection range may be determined as the target map point. The filtered target map points are still expressed in the coordinates of the world coordinate system.

This step may specifically include filtering the candidate map points whose third mapping position is within the collection range of the camera device in the vertical height direction as target map points corresponding to the road image.

In summary, this embodiment filters the map points to be selected according to the range of the camera device in the height direction to obtain the target map points, which can filter out the map points outside the secondary height range from the map points to be selected.

In another embodiment of the present invention, based on the embodiment shown in FIG. 1, the determined map point to be selected includes the coordinate position and normal vector of the map point to be selected. The location information of each map point to be selected includes coordinate location and normal vector. Step 2b, screening the map points within the collection range of the camera device from each candidate map point to obtain the target map point corresponding to the road image, including the following steps 2b-4 to 2b-6.

Step 2b-4: Determine the connection line between the camera device and each candidate map point according to the coordinate position of each candidate map point.

The coordinate position of the map point to be selected is the position in the world coordinate system. The position of the camera device in the world coordinate system can be determined by the initial positioning pose.

Step 2b-5: Calculate the angle between each line and the normal vector of the corresponding map point to be selected.

For example, the four candidate map points A, B, C, and D are connected to the camera device O as AO, BO, CO, and DO, respectively. Calculate the clip between the connection AO and the normal vector of the candidate map point A Angle, calculate the angle between the line BO and the normal vector of the candidate map point B, calculate the angle between the line CO and the normal vector of the candidate map point C, calculate the line DO and the candidate map point D The angles between the normal vectors of, get four angles for the four candidate map points.

Step 2b-6: According to the above-mentioned screening condition that the included angle is within the preset included angle range, the target map points corresponding to the road image are filtered from each candidate map point.

This step may specifically include screening the candidate map points whose included angles are within a preset included angle range as target map points corresponding to the road image. Among them, the preset included angle range can be determined in advance based on experience.

Refer to FIG. 2, which is a schematic diagram of the angle between the camera device and the normal vector of the map point provided in this embodiment. Among them, the incident light is emitted from the map point, projected to the optical center of the camera device, and imaged on the imaging plane to obtain a road image. The line where the incident light is located can be used as the line where the connecting line between the camera device and the map point is located. The normal vector of the map point is perpendicular to the plane. When the relative positions of the camera device and the map point are different, the angle between the incident light and the normal vector is different. It can be seen from FIG. 2 that the map point corresponding to the angle 1 can be collected by the camera device, and the map point corresponding to the angle 2 cannot be collected by the camera device. Therefore, a reasonable setting of the preset angle range can filter out the map points that can be collected by the camera device from a large number of map points to be selected, and obtain the target map point.

To sum up, in this embodiment, the map points to be selected are filtered according to the normal vector of the map points to be selected, and the map points that cannot be observed by the camera are filtered out, thereby improving the accuracy of the target map points.

In another embodiment of the present invention, based on the embodiment shown in FIG. 1, each map point in the preset map is constructed using the following steps 1c to 4c.

Step 1c: Obtain a sample road image, and extract a sample edge feature map of the sample road image according to the preset edge strength.

For extracting the edge feature map of the sample in this step, refer to step S140.

Step 2c: Determine the sample positioning pose corresponding to the sample road image according to the data collected by the motion detection device.

Among them, the sample positioning pose is the pose in the world coordinate system. The sample positioning pose can be considered as the positioning pose of the vehicle at the current moment, and the position of the map point in the preset map can be constructed based on the sample positioning pose.

Step 3c: Determine the position information of each point in the world coordinate system in the sample edge feature map based on the three-dimensional reconstruction algorithm and the aforementioned sample positioning pose.

In this step, when determining the position information of each point in the sample edge feature map in the world coordinate system, it may include:

Based on the three-dimensional reconstruction algorithm, determine the three-dimensional coordinates of each point in the sample edge feature map in the camera coordinate system, determine the conversion matrix between the camera coordinate system and the world coordinate system according to the sample positioning pose, and calculate the above three-dimensional coordinates according to the conversion matrix. After conversion, the position information of each point in the sample edge feature map in the world coordinate system is obtained.

Step 4c: Select a map point from each point of the sample edge feature map according to the preset point density, and add the position information of each map point in the world coordinate system to the preset map.

This step may specifically include: constructing an octree cube grid in a preset map according to an octree algorithm with a preset voxel size; for each octree cube grid, starting from the octree cube grid Select a point from the points in the sample edge feature map as the map point corresponding to the octree cube grid.

The map points selected from each point of the sample edge feature map according to the preset point density form point cloud data in a three-dimensional world coordinate system.

In summary, in this embodiment, when constructing a preset map, a structured sample edge feature map is extracted from the sample road image, and point cloud data is extracted from the sample edge feature map for map construction, and a denser map can be obtained. Information, increase the amount of effective information in the preset map.

FIG. 3 is a schematic diagram of a framework of a vision-based vehicle positioning method provided by an embodiment of the present invention. Among them, the framework includes a front end and a relocation end. The front-end motion detection equipment may include sensors commonly used in smart vehicles such as IMU or wheel speedometer. The function of the front end is to estimate the initial positioning pose of the vehicle at the current moment according to the previous positioning pose of the vehicle and various sensor data, and input the initial positioning pose into the map manager and the MVS positioning optimizer. The initial positioning pose can be understood as the predicted global pose. When the front end estimates the initial positioning pose of the vehicle, it can also be based on road images.

The relocation terminal includes map manager, edge detector and MVS location optimizer. The map manager loads the preset map of the environment, and the map is managed by the octree. The map manager can query the map points that may be observed by the camera from the preset map according to the initial positioning pose. The map manager can also filter the queried map points according to the collection range of the camera device, and remove the map points beyond the range. The map manager inputs the filtered target map points into the MVS positioning optimizer.

The edge detector takes the road image as input, outputs the edge response of the road image, that is, the edge feature map, and inputs the value of the edge feature map to the MVS positioning optimizer.

After receiving the input initial positioning pose, target map point, and edge feature map, the MVS positioning optimizer adopts an iterative optimization solution method to determine the vehicle's performance based on the mapping difference between the target map point and the point in the edge feature map. Global pose. The global pose of the vehicle is the vehicle positioning pose mentioned above.

FIG. 4 is a schematic structural diagram of a vision-based vehicle positioning device provided by an embodiment of the present invention. This device embodiment corresponds to the embodiment shown in FIG. 1, and the device is applied to electronic equipment. The device includes:

The road image acquisition module 410 is configured to acquire the road image collected by the camera device;

The initial pose determination module 420 is configured to determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

The map point determination module 430 is configured to determine the target map point corresponding to the road image from the preset map according to the initial positioning pose; wherein, each map point in the preset map is: pre-compare the edge features of the sample road image The points in the figure are obtained after three-dimensional reconstruction and selection;

The edge feature extraction module 440 is configured to extract the edge feature map of the road image according to the preset edge strength;

The vehicle pose determination module 450 is configured to determine the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determine the vehicle positioning pose according to the projection difference.

In another embodiment of the present invention, based on the embodiment shown in FIG. 4, the initial pose determination module 420 is specifically configured as follows:

Take the initial positioning pose as the initial value of the estimated pose, and according to the value of the estimated pose, map the target map point and the point in the edge feature map to the same coordinate system, and determine the target map point mapped to the same coordinate system Mapping difference between the points in the edge feature map;

When the mapping difference is greater than the preset difference threshold, modify the value of the estimated pose according to the mapping difference, and return to perform the operation of mapping the target map point and the point in the edge feature map to the same coordinate system according to the value of the estimated pose ；

In another embodiment of the present invention, based on the embodiment shown in FIG. 4, the initial pose determination module 420 maps the target map point and the point in the edge feature map to the same coordinate system according to the value of the estimated pose , When determining the mapping difference between the target map point mapped to the same coordinate system and the point in the edge feature map, including:

According to the value of the estimated pose, determine the conversion matrix between the world coordinate system and the camera coordinate system, and map the target map point to the image coordinate system according to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system , The first mapping position of the target map point is obtained, and the projection difference between the first mapping position and the position of the point in the edge feature map in the image coordinate system is calculated; the camera coordinate system is the three-dimensional coordinate system where the camera device is located, The image coordinate system is the coordinate system where the road image is located;

or,

According to the estimated pose value, determine the conversion matrix between the world coordinate system and the camera coordinate system; according to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, map the points in the edge feature map to the world In the coordinate system, the second mapping position of the point in the edge feature map is obtained, and the projection difference between the second mapping position and the position of the target map point in the world coordinate system is calculated.

In another embodiment of the present invention, based on the embodiment shown in FIG. 4, the map point determination module 430 is specifically configured as:

Take the position of the initial positioning pose on the preset map as the center, and use the preset distance as the radius to determine the map point contained in the sphere as the candidate map point;

The map points within the collection range of the camera device are filtered from each candidate map point to obtain the target map point corresponding to the road image.

In another embodiment of the present invention, based on the embodiment shown in FIG. 4, the map point determination module 430 selects the map points within the collection range of the camera device from the map points to be selected to obtain the target corresponding to the road image When the map points, include:

According to the initial positioning pose, determine the conversion matrix between the world coordinate system and the camera coordinate system; where the camera coordinate system is the three-dimensional coordinate system where the camera device is located;

According to the conversion matrix, map each candidate map point to the camera coordinate system to obtain the third mapping position of each candidate map point;

According to the screening condition that the third mapping position is within the collection range of the camera device in the vertical height direction, a target map point corresponding to the road image is obtained by screening from each candidate map point.

In another embodiment of the present invention, based on the embodiment shown in FIG. 4, the determined map point to be selected includes the coordinate position and normal vector of the map point to be selected; the map point determination module 430 filters each map point to be selected When the map points within the collection range of the camera device are obtained, the target map points corresponding to the road image include:

Determine the connection line between the camera device and each candidate map point according to the coordinate position of each candidate map point;

According to the screening condition that the included angle is within the preset included angle range, the target map point corresponding to the road image is filtered from each candidate map point.

In another embodiment of the present invention, based on the embodiment shown in FIG. 4, the device further includes: a map point construction module (not shown in the figure), configured to use the following operations to construct each map point in the preset map :

Obtain the sample road image, and extract the sample edge feature map of the sample road image according to the preset edge strength;

According to the data collected by the motion detection equipment, determine the sample positioning pose corresponding to the sample road image; where the sample positioning pose is the pose in the world coordinate system;

Based on the three-dimensional reconstruction algorithm and the sample positioning pose, determine the position information of each point in the sample edge feature map in the world coordinate system;

Select map points from each point in the sample edge feature map according to the preset point density, and add the location information of each map point in the world coordinate system to the preset map.

The foregoing device embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment. For specific description, refer to the method embodiment. The device embodiment is obtained based on the method embodiment, and the specific description can be found in the method embodiment part, which is not repeated here.

Fig. 5 is a schematic structural diagram of a vehicle-mounted terminal provided by an embodiment of the present invention. The vehicle-mounted terminal includes: a processor 510, a camera device 520, and a motion detection device 530; the processor 510 includes:

Road image acquisition module (not shown in the figure), used to acquire road images collected by the camera device 520;

The initial pose determination module (not shown in the figure) is used to determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device 530; where the initial positioning pose is the world coordinate system where the preset map is located Pose in

The map point determination module (not shown in the figure) is used to determine the target map point corresponding to the road image from the preset map according to the initial positioning pose; wherein, each map point in the preset map is: The points in the edge feature map of the road image are obtained after three-dimensional reconstruction and selection;

The edge feature extraction module (not shown in the figure) is used to extract the edge feature map of the road image according to the preset edge strength;

The vehicle pose determination module (not shown in the figure) is used to determine the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determine the vehicle positioning pose according to the projection difference.

In another embodiment of the present invention, based on the embodiment shown in FIG. 5, the initial pose determination module is specifically used for:

In another embodiment of the present invention, based on the embodiment shown in FIG. 5, the initial pose determination module maps the target map point and the point in the edge feature map to the same coordinate system according to the value of the estimated pose, When determining the mapping difference between the target map point mapped to the same coordinate system and the point in the edge feature map, it includes:

or,

In another embodiment of the present invention, based on the embodiment shown in FIG. 5, the map point determination module is specifically used for:

The map points within the collection range of the camera device 520 are filtered from each candidate map point to obtain the target map point corresponding to the road image.

In another embodiment of the present invention, based on the embodiment shown in FIG. 5, the map point determination module selects the map points within the collection range of the camera device 520 from the map points to be selected to obtain the target corresponding to the road image When the map points, include:

According to the initial positioning pose, determine the conversion matrix between the world coordinate system and the camera coordinate system; where the camera coordinate system is the three-dimensional coordinate system where the camera device 520 is located;

According to the filtering condition that the third mapping position is within the collection range of the camera device 520 in the vertical height direction, the target map point corresponding to the road image is filtered from each candidate map point.

In another embodiment of the present invention, based on the embodiment shown in FIG. 5, the determined map point to be selected includes the coordinate position and normal vector of the map point to be selected; the map point determination module filters the map points to be When the map points within the collection range of the camera device 520 obtain the target map point corresponding to the road image, it includes:

Determine the connection line between the camera device 520 and each candidate map point according to the coordinate position of each candidate map point;

In another embodiment of the present invention, based on the embodiment shown in FIG. 5, the processor 510 further includes: a map point construction module, configured to construct each map point in the preset map by using the following operations:

This embodiment of the terminal and the embodiment of the method shown in FIG. 1 are embodiments based on the same inventive concept, and relevant points may be referred to each other. The foregoing terminal embodiment corresponds to the method embodiment, and has the same technical effect as the method embodiment. For specific description, refer to the method embodiment.

Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention.

A person of ordinary skill in the art can understand that the modules in the device in the embodiment may be distributed in the device in the embodiment according to the description of the embodiment, or may be located in one or more devices different from this embodiment with corresponding changes. The modules of the above-mentioned embodiments can be combined into one module or further divided into multiple sub-modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A vision-based vehicle positioning method is characterized in that it includes:

Obtain road images collected by camera equipment;

Determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

According to the initial positioning pose, a target map point corresponding to the road image is determined from the preset map; wherein, each map point in the preset map is: an edge feature map of the sample road image in advance The points in are obtained after 3D reconstruction and selection;

Extracting the edge feature map of the road image according to the preset edge strength;

According to the initial positioning pose, the mapping difference between the target map point and the point in the edge feature map is determined, and the vehicle positioning pose is determined according to the projection difference.
The method according to claim 1, wherein said determining the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determining according to the mapping difference The steps for vehicle positioning and pose include:

Taking the initial positioning pose as the initial value of the estimated pose, and according to the value of the estimated pose, map the target map point and the point in the edge feature map to the same coordinate system, and determine the mapping to The mapping difference between the target map point and the point in the edge feature map in the same coordinate system;

When the mapping difference is greater than a preset difference threshold, modify the value of the estimated pose according to the mapping difference, return to execute the value of the estimated pose, and compare the target map point with the Steps of mapping points in the edge feature map to the same coordinate system;

When the mapping difference is less than the preset difference threshold, the vehicle positioning pose is determined according to the current value of the estimated pose.
The method of claim 2, wherein the target map point and the point in the edge feature map are mapped to the same coordinate system according to the value of the estimated pose, and the mapping is determined to be The step of mapping the difference between the target map point and the point in the edge feature map in the same coordinate system includes:

According to the value of the estimated pose, determine the conversion matrix between the world coordinate system and the camera coordinate system. According to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, The target map point is mapped to the image coordinate system to obtain a first mapping position of the target map point, and the first mapping position and the position of the point in the edge feature map in the image coordinate system are calculated Projection difference between; wherein the camera coordinate system is the three-dimensional coordinate system where the camera device is located, and the image coordinate system is the coordinate system where the road image is located;

or,

According to the value of the estimated pose, determine the conversion matrix between the world coordinate system and the camera coordinate system; according to the conversion matrix and the projection relationship between the camera coordinate system and the image coordinate system, The points in the edge feature map are mapped to the world coordinate system to obtain a second mapping position of the points in the edge feature map, and the second mapping position and the target map point are calculated in the world coordinate system The projection difference between the positions in.
The method according to claim 1, wherein the step of determining a target map point corresponding to the road image from the preset map according to the initial positioning pose comprises:

Taking the position of the initial positioning pose in the preset map as a center, and a map point contained in a sphere determined by using a preset distance as a radius as a candidate map point;

The map points within the collection range of the camera device are filtered from each candidate map point to obtain a target map point corresponding to the road image.
The method according to claim 4, wherein the step of screening map points within the collection range of the camera device from each candidate map point to obtain a target map point corresponding to the road image, include:

Determine the conversion matrix between the world coordinate system and the camera coordinate system according to the initial positioning pose; wherein, the camera coordinate system is the three-dimensional coordinate system where the camera device is located;

According to the conversion matrix, map each candidate map point to the camera coordinate system to obtain the third mapping position of each candidate map point;

According to the filtering condition that the third mapping position is within the collection range of the camera device in the vertical height direction, a target map point corresponding to the road image is obtained by filtering from each candidate map point.
The method according to claim 4, wherein the determined map points to be selected include the coordinate positions and normal vectors of the map points to be selected; and the selection of each map point to be selected is within the collection range of the camera device The step of obtaining the target map point corresponding to the road image includes:

Determine the connection line between the camera device and each map point to be selected according to the coordinate position of each map point to be selected;

Calculate the angle between each line and the normal vector of the corresponding map point to be selected;

According to the screening condition that the included angle is within the preset included angle range, a target map point corresponding to the road image is obtained by filtering from each candidate map point.
The method according to claim 1, wherein each map point in the preset map is constructed in the following manner:

Acquiring a sample road image, and extracting a sample edge feature map of the sample road image according to a preset edge strength;

Determine the sample positioning pose corresponding to the sample road image according to the data collected by the motion detection device; wherein the sample positioning pose is the pose in the world coordinate system;

Determining the position information of each point in the sample edge feature map in the world coordinate system based on a three-dimensional reconstruction algorithm and the sample positioning pose;

A map point is selected from each point of the sample edge feature map according to a preset point density, and the position information of each map point in the world coordinate system is added to the preset map.
A vision-based vehicle positioning device is characterized in that it comprises:

The road image acquisition module is configured to acquire the road image collected by the camera device;

The initial pose determination module is configured to determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

The map point determination module is configured to determine a target map point corresponding to the road image from the preset map according to the initial positioning pose; wherein, each map point in the preset map is: Perform three-dimensional reconstruction and selection of points in the edge feature map of the sample road image;

An edge feature extraction module configured to extract the edge feature map of the road image according to a preset edge strength;

The vehicle pose determination module is configured to determine the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determine the vehicle positioning pose according to the projection difference.
The device according to claim 8, wherein the initial pose determination module is specifically configured to:

Taking the initial positioning pose as the initial value of the estimated pose, and according to the value of the estimated pose, map the target map point and the point in the edge feature map to the same coordinate system, and determine the mapping to The mapping difference between the target map point and the point in the edge feature map in the same coordinate system;

When the mapping difference is greater than a preset difference threshold, modify the value of the estimated pose according to the mapping difference, return to execute the value of the estimated pose, and compare the target map point with the The operation of mapping points in the edge feature map to the same coordinate system;

When the mapping difference is less than the preset difference threshold, the vehicle positioning pose is determined according to the current value of the estimated pose.
A vehicle-mounted terminal is characterized by comprising: a processor, a camera device and a motion detection device; the processor includes:

Road image acquisition module for acquiring road images collected by camera equipment;

The initial pose determination module is used to determine the initial positioning pose corresponding to the road image according to the data collected by the motion detection device; wherein the initial positioning pose is the pose in the world coordinate system where the preset map is located;

The map point determination module is configured to determine a target map point corresponding to the road image from a preset map according to the initial positioning pose; wherein, each map point in the preset map is: pre-complying sample roads The points in the edge feature map of the image are obtained after three-dimensional reconstruction and selection;

The edge feature extraction module is used to extract the edge feature map of the road image according to the preset edge strength;

The vehicle pose determination module is configured to determine the mapping difference between the target map point and the point in the edge feature map according to the initial positioning pose, and determine the vehicle positioning pose according to the projection difference.