WO2023123837A1 - Map generation method and apparatus, electronic device, and storage medium - Google Patents

Abstract

A map generation method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to point cloud trajectory data of a second type of vehicle; obtaining determined second point cloud trajectory data, wherein the second point cloud trajectory data corresponds to point cloud trajectory data of a first type of vehicle; generating a point cloud base map according to the first point cloud trajectory data; using the point cloud base map as an alignment medium and performing alignment and merger on the second point cloud trajectory data and the point cloud base map, and obtaining a map. A map can be quickly generated for region having a weak navigation signal.

Description

Map generation method, device, electronic device and storage medium

This application claims the priority of the Chinese patent application submitted to the State Intellectual Property Office on December 30, 2021, with the application number 202111646647.8 and the application name "Map generation method, device, electronic equipment and storage medium", the entire content of which is passed References are incorporated in this application.

technical field

The present application relates to the field of map technology, and in particular to a map generation method, device, electronic equipment, and storage medium.

Background technique

In related technologies, in some indoor areas where GPS (Global Positioning System, Global Positioning System) signals or GNSS (Global Navigation Satellite System, Global Navigation Satellite System) signals are weak, such as indoor parking lots, there is generally no map. Therefore, when the user's vehicle enters the indoor parking lot, since there is no map of the indoor parking lot, if the user is not familiar with the internal environment of the indoor parking lot, the user is likely to get lost in the indoor parking lot, causing the user to spend a lot of time Find a parking space.

Contents of the invention

In order to solve or partially solve the problems existing in the related technologies, the embodiment of the present application proposes a map generation method, device, electronic equipment and storage medium, which can quickly generate maps for areas with weaker signals and facilitate parking for users.

According to an aspect of an embodiment of the present application, a method for generating a map is provided, including: acquiring generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to the point cloud trajectory of a second type of vehicle data; obtain the determined second point cloud track data, wherein the second point cloud track data corresponds to the point cloud track data of the first type of vehicle; generate a point cloud base map according to the first point cloud track data; with The point cloud base map is used as an alignment medium, and the second point cloud trajectory data is aligned and fused with the point cloud base map to obtain a map.

In one embodiment, the first point cloud trajectory data is generated in the following manner: a plurality of trajectory data are acquired, and the trajectory data includes the pose information of the vehicle and the bird's-eye view stitching image associated with the vehicle pose information; The road element recognition is carried out on the stitched image of the bird's-eye view, and the road elements in each stitched image of the bird's-eye view are determined; according to the road elements in each stitched image of the bird's-eye view and the vehicle pose information associated with each stitched image of the bird's-eye view, the first-order road element corresponding to each trajectory data is generated. A point cloud trajectory data; said generating a point cloud base map according to said first point cloud trajectory data includes: splicing the first point cloud trajectory data corresponding to a plurality of trajectory data to obtain a point cloud base map.

According to an aspect of an embodiment of the present application, a map generation device is provided, including: a data acquisition module, configured to acquire the generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to the second type The point cloud track data of the vehicle; Acquire the determined second point cloud track data, wherein the second point cloud track data corresponds to the point cloud track data of the first type of vehicle; The splicing module is used for according to the first The point cloud track data generates a point cloud base map; the fusion module is used to use the point cloud base map as an alignment medium to align and fuse the second point cloud track data and the point cloud base map to obtain a map.

In one embodiment, the data acquisition module includes an acquisition module, an identification module, and a generation module; the acquisition module is used to acquire a plurality of trajectory data, and the trajectory data includes the pose information of the vehicle and the bird's-eye view associated with the vehicle pose information Angle of view mosaic image; identification module, used to identify road elements in each bird's-eye view angle mosaic image, and determine road elements in each bird's-eye view angle mosaic image; generation module, used for splicing road elements and each bird's-eye angle of view according to each bird's-eye view angle of view mosaic image Stitching the vehicle pose information associated with the images to generate first point cloud track data corresponding to each track data; the splicing module splicing the first point cloud track data corresponding to multiple track data to obtain a point cloud base map.

According to an aspect of an embodiment of the present application, there is provided an electronic device, including: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the above The method for generating the map.

According to an aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method for generating the above-mentioned map is implemented.

In the solution of this application, the first point cloud trajectory data is constructed by stitching images based on the vehicle’s bird’s-eye view angle and the vehicle’s pose information, and the point cloud base map is generated according to the first point cloud trajectory data, and the point cloud The base map is used as an alignment medium, and the second point cloud trajectory data is aligned and fused with the point cloud base map to generate a map. The solution of the present application can be applied to generate maps for areas with weak GNSS signals or weak GPS signals, so as to facilitate parking for users. For example, a map of an indoor parking lot can be generated, thereby solving the problem in the related art that users lose their way because there is no map of the indoor parking lot. In the embodiment of the present application, the point cloud base map is constructed by using the first point cloud track data with lower precision, and then the second point cloud track data with higher precision and more comprehensive perceived road elements is fused with the point cloud base map, Generating a map can improve the efficiency of map generation and ensure the accuracy and precision of the map.

Moreover, in the scheme of the embodiment of the present application, because the point cloud base map as the alignment medium is generated first, the point cloud base map basically reflects the overall situation of the geographical environment area, thereby ensuring that the second point cloud track data is consistent with the point cloud base The alignment success rate of the graph reduces the probability of misalignment.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Description of drawings

The above and other objects, features and advantages of the present application will become more apparent by describing the exemplary embodiments of the present application in more detail with reference to the accompanying drawings, wherein, in the exemplary embodiments of the present application, the same reference numerals generally represent same parts.

Fig. 1 is a schematic diagram showing an application scenario of the solution of the present application according to an embodiment of the present application.

Fig. 2 is a flowchart of a method for generating a map according to an embodiment of the present application.

FIG. 3 shows a schematic diagram of splicing images along a bird's-eye view to obtain a bird's-eye view stitching.

FIG. 4 is a schematic diagram of a bird's-eye view stitching image obtained by continuously collecting images collected in a vehicle trajectory along a bird's-eye view.

Fig. 5 is a schematic diagram of the first point cloud trajectory data corresponding to the bird's-eye view stitched image according to a specific embodiment.

Fig. 6 is a schematic diagram showing the projection of the point cloud model of each road element in the point cloud trajectory data on the vertical projection plane according to an embodiment of the present application.

Fig. 7 is a flow chart of generating first point cloud trajectory data according to an embodiment of the present application.

Fig. 8 is a schematic diagram of a point cloud base map according to an embodiment of the present application.

Fig. 9 is a schematic diagram of splicing two first point cloud trajectory data according to an embodiment.

Fig. 10 is a flowchart showing steps before step 250 according to an embodiment of the present application.

Fig. 11 is a flow chart of updating a point cloud base map according to candidate point cloud trajectory data according to an embodiment of the present application.

Fig. 12 is a flowchart of a method for generating a map according to a specific embodiment of the present application.

Fig. 13 is a block diagram of an apparatus for generating a map according to an embodiment of the present application.

Fig. 14 is a structural block diagram of an electronic device according to an embodiment of the present application.

Detailed ways

Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of this application to those skilled in the art.

The terminology used in this application is for the purpose of describing particular embodiments only, and is not intended to limit the application. As used in this application and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It should be understood that although the terms "first", "second", "third" and so on may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present application, first information may also be called second information, and similarly, second information may also be called first information. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the present application, "plurality" means two or more, unless otherwise specifically defined.

The technical solution of the present application will be described in detail below in conjunction with the accompanying drawings.

Fig. 1 is a schematic diagram showing an application scenario of the solution according to an embodiment of the present application. As shown in FIG. 1 , the application scenario includes a vehicle 110 , a server 120 and a terminal 130 . Wherein, the vehicle 110 and the server 120 may establish a communication connection through a wired or wireless network, and the terminal 130 and the server 120 may establish a communication connection through a wired or wireless network.

Based on the communication connection between the vehicle 110 and the server 120, the vehicle 110 can report its own track data to the server 120, so that the server 120 can generate a map according to the method of the embodiment of the present application based on the track data of the vehicle. The server 120 may be an independent physical server or a cloud server, which is not specifically limited here.

After the server 120 generates the point cloud base map and/or map, it can also send the generated point cloud base map and/or map to the terminal 130, and the point cloud base map and/or map can be checked and verified by the user at the terminal 130. For editing and modification, the server 120 receives the terminal 130 to review and edit the modified point cloud base map and/or map. Wherein, the terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., which are not specifically limited here.

In some embodiments, the server 120 may also send the generated map to each vehicle 110 for display on the vehicle-mounted display device of the vehicle 110 , or send it to the terminal 130 where the user is located.

An embodiment of the present application provides a method for generating a map, including:

1) Acquiring the generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to the point cloud trajectory data of the second type of vehicle.

In this embodiment of the present application, a vehicle without a lidar and an intelligent perception module may be referred to as a vehicle of the second type. The first point cloud trajectory data may refer to point cloud trajectory data corresponding to the second type of vehicle. For example, the first point cloud trajectory data may be a point cloud trajectory constructed for pose information of the second type of vehicle according to the stitched image of the bird's-eye view of the second type of vehicle and the associated vehicle. In the embodiment of the present application, based on the images collected by the image acquisition device corresponding to multiple viewing angles in the first type of vehicle, a bird's-eye view stitching image can be obtained, and collected by a GNSS module (or GPS module), an IMU, and a wheel speedometer Get the pose information of the vehicle.

2) Acquiring the determined second point cloud trajectory data, where the second point cloud trajectory data corresponds to the point cloud trajectory data of the first type of vehicle.

In this embodiment of the present application, a vehicle equipped with a laser radar and/or an intelligent perception module may be referred to as a vehicle of the first type. The second point cloud trajectory data may refer to point cloud trajectory data corresponding to the first type of vehicle. For example, the second point cloud track data can be combined with the information collected by the lidar and/or the environmental perception results and positioning results of the intelligent perception module, the visual information collected by the image acquisition device, IMU, wheel speedometer, GNSS ( Or GPS) module and other information to build.

3) Generate a point cloud base map according to the first point cloud trajectory data.

4) Using the point cloud base map as the alignment medium, align and fuse the second point cloud trajectory data with the point cloud base map to obtain a map.

It can be seen that in the embodiment of the present application, the first point cloud trajectory data is constructed first, and the point cloud base map is generated according to the first point cloud trajectory data, and the point cloud base map is used as an alignment medium, and the second point cloud trajectory data and the point cloud The base map is aligned and fused to generate a map, so that maps can be quickly generated for areas with weak GNSS signals or weak GPS signals. In the embodiment of the present application, the point cloud base map is constructed by using the first point cloud track data with lower precision, and then the second point cloud track data with higher precision and more comprehensive perceived road elements is fused with the point cloud base map, Generating a map can improve the efficiency of map generation and ensure the accuracy and precision of the map.

Fig. 2 is a flowchart of a method for generating a map according to an embodiment of the present application. The method can be executed by a computer device with processing capabilities, such as a server, a cloud server, etc., and is not specifically limited here.

Referring to Figure 2, the method at least includes steps 210 to 250, which are described in detail as follows:

Step 210, acquiring a plurality of trajectory data, the trajectory data including vehicle pose information and a bird's-eye view stitching image associated with the vehicle pose information.

The trajectory data is collected by the vehicle during driving, wherein multiple trajectory data can come from one vehicle or multiple vehicles. The plurality of track data may be collected during multiple driving of the vehicle.

The pose information of the vehicle can be determined according to the information collected by the GNSS (Global Navigation Satellite System) module, GPS module, IMU (Inertial Measurement Unit, inertial measurement unit) module and wheel speedometer in the vehicle . Among them, the IMU is a module composed of various sensors such as a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. The wheel speedometer is used to detect the distance that the wheel moves within a certain period of time, so as to calculate the change of the relative pose (position and heading) of the vehicle.

The pose information of the vehicle may indicate the position information of the vehicle and the attitude information of the vehicle, and the attitude of the vehicle may include a pitch angle, a yaw angle, and a roll angle of the vehicle.

The location information of the vehicle can be determined by the GNSS module according to the collected GNSS signals, or by the GPS module according to the collected GPS signals. When the vehicle enters a place where the GNSS signal or GPS signal is weak (such as entering an indoor area such as an underground parking lot), the vehicle can use the position information of the location point where the GNSS signal or GPS signal is lower than the set threshold and the wheel speedometer , The information collected by the IMU module is used for dead reckoning, and the position information of each position point during the driving process is obtained.

The bird's-eye view stitching image is obtained by stitching the images collected by the vehicle in at least two viewing angles along the bird's-eye view. Multiple image capture devices can be installed in the vehicle to capture images of the surrounding environment of the vehicle from multiple perspectives during driving. Wherein, the image collected can be the image of the environment directly in front of the vehicle, the image of the left side, the image of the right side, the image of the left rear, the image of the right rear, etc. In one embodiment, three image acquisition devices are provided in the vehicle, which respectively acquire images from three perspectives of the front, left side, and right side of the vehicle. Certainly, in other embodiments, images under other viewing angles may also be collected.

In some embodiments, the vehicle can splice images collected at various points of view under multiple viewing angles along a bird's eye view (Birds Eye View, BEV) to obtain a bird's eye view stitching image. In other embodiments, the vehicle can also upload the collected images under multiple viewing angles to the server, and the server can stitch them together along the bird's-eye view to obtain the stitched image from the bird's-eye view.

FIG. 3 shows a schematic diagram of splicing images along a bird's-eye view to obtain a bird's-eye view stitching. Figures 3A-3C are captured by different cameras on the vehicle at the same location. Wherein, the image in FIG. 3A is the collected image of the environment directly in front of the vehicle, FIG. 3B is the collected image of the environment in front of the left side of the vehicle, and FIG. 3C is the collected image of the environment in front of the vehicle right. By splicing Fig. 3A-Fig. 3C along the bird's-eye view angle, the bird's-eye view stitching image shown in Fig. 3D can be obtained. The image in Fig. 3B belongs to side forward looking perception.

During the driving process of the vehicle, the vehicle can collect images at multiple locations in real time. Therefore, multiple images collected by the vehicle can be continuously spliced along the bird's-eye view angle to obtain continuous bird's-eye view stitching images reflecting the surrounding environment of the vehicle's driving track.

In the embodiment of the present application, the stitched bird's-eye view image in the trajectory data is obtained by continuously stitching the stitched bird's-eye view images of multiple locations along the driving track. For the convenience of distinction, the image obtained by splicing the images collected at each location point under multiple viewing angles along the bird's-eye view is called the bird's-eye view stitching sub-image.

FIG. 4 is a schematic diagram of a bird's-eye view stitched image obtained by continuously stitching images collected in a vehicle trajectory along a bird's-eye view. In the stitching process, for example, in the stitching process of stitching sub-images corresponding to different positions, especially when stitching sub-images corresponding to the bird’s-eye view at corners, orientation optimization can be performed to reduce black edges (in the figure indicated by marker 401). Among them, the black border is caused by the incomplete joint of two adjacent bird's-eye view stitching sub-images. During the stitching process, free stitching is performed on the bird's-eye view stitching sub-images corresponding to multiple position points on the straight line trajectory to reduce turning distortion (indicated by mark 402 in the figure).

In step 220, road element identification is performed on each stitched image from a bird's-eye view, and road elements in each stitched image from a bird's-eye view are determined.

Road elements may include lane lines (such as solid lane lines and dashed lane lines), road arrows, stop lines, speed bumps, parking space borderlines in the parking lot, parking space entrance lines, etc., which are not specifically limited here. Identifying road elements refers to determining the pixel area where the road elements are located in the stitched image from the bird's-eye view. It can be understood that, on the one hand, the recognition result obtained from road element recognition indicates which road elements are specifically included in the bird's-eye view stitched image, and on the other hand, indicates the position of each road element in the bird's-eye view stitched image.

For example, a neural network model can be used to identify road elements on stitched images from a bird's-eye view. In some embodiments, Mask R-CNN (Mask Recycle Convolutional Neural Network, Mask Recycle Convolutional Neural Network), PANet (Path Aggregation Network, Path Aggregation Network), FCIS (Fully Convolutional Instance-aware Semantic Segmentation, full The convolutional instance-aware semantic segmentation) network is used to segment each road element in the bird's-eye view stitching image, so as to determine the position of each road element in the bird's-eye view stitching image.

In some embodiments, step 220 may include: inputting each stitched bird's-eye view image into the road element recognition model; performing road element recognition by the road element recognition model, and outputting road element information corresponding to each stitched bird's-eye view image. Indicates the road element in the stitched image corresponding to the bird's-eye view. In some embodiments, the road element recognition model may be constructed by one or more neural networks among convolutional neural network, fully connected neural network, feedforward neural network, long short-term memory network, and recurrent neural network. In other embodiments, the road element recognition model may be Mask R-CNN, PANet, FCIS, etc. as listed above.

In order to ensure the accuracy of road element recognition, the road element recognition model can be trained with training data before road element recognition. Wherein, the training data includes multiple sample bird's-eye view stitched images and annotation information of the sample bird's-eye view stitched images. The annotation information is used to indicate the road elements in the stitched image from the bird's-eye view of the corresponding sample. In the embodiment of the present application, the bird's-eye view stitched image used for training the road element recognition model is referred to as a sample bird's-eye view stitched image. During the training process, the stitched image of the bird's-eye view of the sample is input into the road element recognition model, and the road element recognition model recognizes the road element on the stitched image of the bird's-eye view of the sample, and outputs the predicted road element information, which is used to indicate the sample Road elements in a bird's eye view stitched image. It can be understood that the predicted road element information not only indicates the position information of the identified road element in the sample bird's-eye view stitched image, but also indicates the semantics of the road element (that is, indicates what kind of road element, such as lane line , speed bumps or stop lines, etc.). Afterwards, based on the annotation information and predicted road element information of the stitched image from the bird's-eye view of the sample, the loss value of the loss function is calculated, and the parameters of the road element recognition model are reversely adjusted according to the loss value. It can be understood that the annotation information of the sample bird's-eye view stitched image also indicates the position information of each road element in the sample bird's-eye view stitched image.

Wherein, the loss function may be set according to actual needs, for example, the loss function may be a cross-entropy loss function, a logarithmic loss function, etc., which are not specifically limited here.

After completing the training of the road element recognition model, the road element recognition model can be applied online to accurately identify road elements.

Step 230, according to the road elements in each bird's-eye view stitching image and the vehicle pose information associated with each bird's-eye view stitching image, generate first point cloud trajectory data corresponding to each trajectory data.

In this application, for the convenience of distinction, the point cloud trajectory data used to construct the point cloud base map and constructed based on the stitched images from the bird's-eye view and the associated pose information is referred to as the first point cloud trajectory data. The first point cloud trajectory data includes position information of each position point in the trajectory path and a point cloud model of road elements on the trajectory path. It can be understood that the relative positional relationship between the point cloud models of different road elements in the first point cloud trajectory is basically the same as the relative positional relationship presented by the bird's-eye view stitching image.

The point cloud model of road elements is a collection of massive points expressing the spatial distribution of road elements and the target surface characteristics in the same spatial reference system. After obtaining the spatial coordinates of each sampling point of road elements, the road elements are placed on the Arrange all the sampling points of the road element to obtain the point cloud model of the road element.

In some embodiments, step 230 may include: according to the vehicle pose information associated with each bird's-eye view stitching image, perform three-dimensional reconstruction on each road element in each bird's-eye view stitching image, and obtain the corresponding The first point cloud trajectory data.

The 3D point cloud model of the corresponding road elements can be obtained by 3D reconstruction of the road elements in the bird's-eye view stitching image. On this basis, combining the pose information associated with the stitched image from the bird's-eye view and the obtained position information of the road elements in the stitched image from the bird's-eye view, the location information of the road element in the geographic space can be determined, so that according to the geographic location of the road element The position information in the space is arranged by arranging the 3D point cloud models of the road elements in the bird's-eye view stitching image, and correspondingly obtaining the first point cloud trajectory data corresponding to the bird's-eye view stitching image.

In some embodiments, deep learning may be used to perform three-dimensional reconstruction on the road elements in the bird's-eye view stitched image. For example, the neural network model used to generate the 3D point cloud model can be trained (for the sake of distinction, the neural network model used to generate the 3D point cloud model is called the 3D reconstruction model), and then the bird's-eye view can be obtained through the 3D reconstruction model. Each road element in the perspective stitching image is reconstructed in 3D.

In some embodiments, the three-dimensional reconstruction model may be a model constructed by a convolutional neural network, a fully connected neural network, or the like. In some embodiments, the three-dimensional reconstruction model may be an Im2Avatar model, a confrontation network, a generation network, etc., which are not specifically limited here.

Fig. 5 is a schematic diagram of the first point cloud trajectory data corresponding to the bird's-eye view stitched image according to an embodiment. In Figure 5, although the edges of each road element appear to be lines, they are actually point sequences. Since the points are relatively dense, the visual effect feels like lines. In some embodiments, in the first point cloud trajectory data, different road elements can be represented by different point clouds, for example, lane lines are represented by blue point clouds, parking space lines are represented by green point clouds, and parking spaces are represented by red point clouds. The point cloud represents arrows, etc.

When the vehicle is driving on a plane with a large difference in height, it is also necessary to pay attention to the height of the vehicle in the vertical direction. For example, the height difference can be sensed through the pitch angle of the vehicle, and then the height of the vehicle in the vertical direction can be determined. For example, when the vehicle is on the first underground floor and the second underground floor of the underground parking lot, its height in the vertical direction is different. Further, when the vehicle is running on the slope, the slope of the slope where the vehicle is located can also be calculated according to the pitch angle of the vehicle.

Fig. 6 is a schematic diagram showing the projection of the point cloud model of each road element in the point cloud trajectory data on the vertical projection plane according to an embodiment of the present application. As shown in FIG. 6 , it can be clearly seen that there is a height difference between the first plane 610 and the second plane 620 and the third plane 630 in the vertical direction. Therefore, the first plane 610 , the second plane 620 , and the third plane 630 correspond to different floors, and the white shaded parts in FIG. 6 represent road elements in the corresponding floors. In FIG. 6 , the first oblique line 621 , the second oblique line 622 and the third oblique line 623 between the first plane 610 and the second plane 620 may indicate that the first floor 610 and the second floor 620 are connected at different positions. Similarly, the fourth oblique line 631 between the second plane 620 and the third plane 630 represents the inclined road connecting the second plane 620 and the third plane 630 .

Fig. 7 is a flow chart of generating first point cloud trajectory data according to an embodiment of the present application. As shown in Figure 7, including:

Step 710, stitching images along the bird's-eye view. Wherein, the camera used for collecting images on the vehicle may be a surround-view camera, and the surround-view camera may collect images of the surrounding environment during the driving of the vehicle based on fish-eye imaging. By setting multiple surround-view cameras, images from multiple perspectives can be collected. In some embodiments, in the process of splicing images under different viewing angles, there may be distortions, in this case, the distorted region is regarded as ROI (Region of Interest, region of interest), Distortion correction is also performed to avoid inaccurate subsequent generated maps due to the distortion caused by the splicing process. Furthermore, in the process of splicing, it also involves inverse perspective transformation of images collected under multiple viewing angles, so as to project the images into a bird's-eye view, and then splicing to obtain a bird's-eye view stitching image.

Step 720, identifying road elements.

Each road element in the bird's-eye view stitched image is determined through step 720 .

Step 730, three-dimensional reconstruction.

Through three-dimensional reconstruction, the point cloud model of each road element in the bird's-eye view stitching image is obtained, and then the point cloud models are combined to obtain the first point cloud trajectory data.

For the specific implementation process of steps 710-730, refer to the above description, which will not be repeated here.

Please continue to refer to FIG. 2 , step 240 , splicing the first point cloud trajectory data corresponding to multiple trajectory data to obtain a point cloud base map.

Different first point cloud trajectory data cover different environmental regions. Therefore, multiple first point cloud trajectory data can be spliced to obtain a point cloud base map reflecting the basic global region.

For example, step 240 may include: determining the first target road element representing the same geographic location in any two of the multiple first point cloud trajectories; based on the first target road element, splicing multiple first point cloud track data, Get the point cloud base map.

The first target road element refers to road elements representing the same geographic location in any two of the plurality of first point cloud trajectory data.

Different driving trajectories may have overlapping trajectories, so there may be road elements representing the same geographic location (ie, the first target road element) in the first point cloud trajectory data constructed based on different trajectory data.

Since the first point cloud trajectory data not only shows the point cloud model of each road element, but also corresponds to the element semantics of the road element (the element semantics indicates which kind of road element it is) and the position information of the road element, Therefore, based on the element semantics and position information of each road element in the point cloud trajectory data and the relative positional relationship between other road elements near the road element, each road element in any two first point cloud trajectory data can be compared. Thus, the first target road element representing the same geographic location is determined.

On this basis, the first target road elements in different first point cloud trajectory data can be coincident by moving the first point cloud trajectory data. It can be understood that after the movement, the position of the overlapping first target road element is the splicing joint of different first point cloud trajectory data.

By splicing multiple first point cloud trajectory data, the road elements in the multiple first point cloud trajectory data after splicing can reflect road elements in more geographical environment areas, and then obtain the road elements that reflect the overall situation of the geographical environment area. Point cloud basemap. Fig. 8 is a schematic diagram of a point cloud base map according to an embodiment of the present application.

Step 250, using the point cloud base map as an alignment medium, aligning and fusing the second point cloud trajectory data with the point cloud base map to obtain a map.

In this application, the point cloud base map can reflect the whole field map skeleton of the geographical environment area (especially the indoor area with weak GNSS signal and GPS signal), and this method of constructing the point cloud base map is relatively fast. However, since the point cloud base map is constructed through the first point cloud trajectory data, and the first point cloud trajectory data is generated by splicing images from a bird's-eye view and the pose information of the associated vehicle, in practice, due to the vehicle The angle of view that the image acquisition device on the Internet can perceive is limited, and the point cloud trajectory data generated only by splicing images from the bird’s-eye view and the corresponding pose information may not be able to fully reflect all road elements in the geographical environment. Therefore, in this application In the embodiment, the second point cloud trajectory data is further obtained, and the map is generated by fusing the second point cloud trajectory data capable of expressing more road elements with the point cloud base map.

Compared with the point cloud base map, the second point cloud trajectory data may include a point cloud model with more road elements. Through fusion, the point cloud model of road elements (such as some road signs, pillars, ultrasonic obstacles, walls, gates, railings, zebra crossings, etc.) that exist in the second point cloud trajectory data but do not exist in the point cloud base map Add it to the point cloud base map, so as to fuse and update the point cloud base map, and then get the map.

In some embodiments, step 250 may include: using the point cloud basemap as an alignment medium, aligning the second point cloud trajectory data with the point cloud basemap, and determining the second point cloud trajectory data compared with the point cloud basemap The newly added road element; the point cloud model of the newly added road element is added to the point cloud base map to obtain a map.

Wherein, compared with the newly added road elements in the point cloud base map, the second point cloud track data refers to road elements that exist in the second point cloud track data but do not exist in the point cloud base map.

As described above, since the point cloud basemap can basically reflect the global skeleton of the geographical environment area, the probability of road elements representing the same geographic location in the second point cloud trajectory data and the point cloud basemap is relatively high, thus, based on this The road elements representing the same geographic location can semantically align the second point cloud trajectory data with the point cloud basemap, so as to realize the positioning of the second point cloud trajectory data on the point cloud basemap. After alignment, the second point cloud trajectory data is compared with the point cloud base map, thereby determining the newly added road element compared to the point cloud base map, and based on the newly added road element in the second point cloud track data The position information of the newly added road element is determined in the point cloud base map, and according to the determined position information, the point cloud model of the newly added road element is added to the point cloud base map, and then the map can be obtained.

To sum up, in the solution of the embodiment of this application, the first point cloud trajectory data is constructed according to the bird’s-eye view angle stitching image of the vehicle and the vehicle’s pose information, and the point cloud base is generated according to the first point cloud trajectory data The point cloud base map is used as the alignment medium, and the second point cloud trajectory data is aligned and fused with the point cloud base map to generate a map. The solutions of the embodiments of the present application can be applied to quickly generate maps for areas with weak GNSS signals or weak GPS signals.

Moreover, in the solution of the embodiment of the present application, because the point cloud base map as the alignment medium is generated first, the point cloud base map basically reflects the overall situation of the geographical environment area, thereby ensuring that the second point cloud track data is consistent with the point cloud The alignment success rate of the basemap reduces the probability of misalignment.

In some embodiments, there are differences in hardware configurations on different vehicles. For example, some vehicles are equipped with lidar, and/or, an intelligent perception module (also called a perception chip, wherein the intelligent perception module can be used in real time according to the acquired The image of the vehicle and the information collected by other sensors on the vehicle (such as the information collected by the wheel speedometer, IMU, etc.) to identify things in the environment around the vehicle, such as road elements, etc.), while some vehicles are not equipped with lidar and intelligent perception module. It is understandable that, since the bird’s-eye view stitching image is obtained through a series of inverse perspective transformations and stitching, some information in the original image may be lost during the inverse perspective transformation and stitching process. Therefore, compared with stitching images based on bird’s-eye view To identify road elements, the accuracy of identifying road elements based on real-time images collected by the vehicle combined with information from other sensors on the vehicle is higher. Therefore, there are differences in the precision and accuracy of point cloud trajectory data from vehicles with different hardware configurations.

In this embodiment of the application, for the convenience of distinction, vehicles equipped with lidar and/or intelligent perception modules are referred to as vehicles of the first type; vehicles not equipped with lidar and intelligent perception modules are referred to as is a vehicle of type II.

LiDAR can detect the size and location of objects in the environment around the vehicle. It can be understood that, for a vehicle equipped with a lidar, images collected by the vehicle, signals detected by the lidar, and other wheel speedometers, IMU, GNSS modules (or GPS modules) can be combined to correspond to the vehicle's driving trajectory The point cloud trajectory data has more reference information, and the detection accuracy and sensing range of lidar are wider. Therefore, comparatively speaking, the precision and accuracy of point cloud trajectory data from vehicles equipped with lidar is higher.

The intelligent perception module (sensing chip) can combine the information collected by the sensors set in the vehicle (such as image acquisition device, wheel speed information, IMU, GNSS module (or GPS module)) in real time to realize the scene perception and understanding of the environment in real time, such as obstacles Semantic classification of data such as types of objects, road signs and markings, detection of pedestrians and vehicles, traffic signals, etc., and then positioning based on the results of perception and understanding, so as to help vehicles more accurately understand their position relative to their environment.

Generally speaking, the precision and accuracy of the point cloud trajectory data from the first type of vehicle is higher than that of the point cloud trajectory data from the second type of vehicle, but in the market, there are the first type of The number of users of the vehicle is much lower than that of the second type of vehicle. Therefore, if the map is constructed only by point cloud trajectory data from the first type of vehicle, the map construction period will be long. Therefore, in this case, the method of the present application can be used to construct the map.

For example, in this application scenario, the first point cloud trajectory data may be a point cloud trajectory constructed for pose information of the second type of vehicle based on the stitched image from the bird's-eye view of the second type of vehicle and the associated vehicle. At this time, in this case, the bird's-eye view stitching image can be obtained based on the images collected by the image acquisition device corresponding to multiple viewing angles in the first type of vehicle, and the GNSS module (or GPS module), IMU, and The wheel speedometer is used to collect the pose information of the vehicle.

The second point cloud trajectory data may refer to point cloud trajectory data corresponding to the first type of vehicle. In some embodiments, the second point cloud trajectory data can be combined with the information collected by the lidar and/or the environment perception results and positioning results of the intelligent perception module, the visual information collected by the image acquisition device, IMU, wheel speed It can be built with various information such as meter, GNSS (or GPS) module, etc. It can be understood that the second point cloud track data also correspondingly indicates the driving track of the vehicle, and each road element and position information of the road element in the driving environment.

In this case, the point cloud base map is constructed by using the first point cloud trajectory data with lower precision, and then the second point cloud trajectory data with higher accuracy and more comprehensive perceived road elements is compared with the point cloud base map. Fusion and map generation can improve map generation efficiency and ensure map accuracy and precision.

Moreover, if there is no point cloud base map, the scheme of directly splicing different point cloud trajectory data to generate a map is easy when there are no trajectory intersection points between two point cloud trajectory data, or there are few trajectory intersection points. As a result, point cloud trajectory data splicing fails, or semantic alignment fails, and if the solution of the embodiment of the present application is adopted, since the point cloud base map that can reflect the basic global situation in the environment area is pre-built, this problem can be effectively solved, ensuring The second point cloud trajectory data can be fused with the point cloud base map.

In the embodiment of the present application, the point cloud base map is constructed based on the first point cloud track data of the second type of vehicle, and the point cloud base map is optimized and updated by using the second point cloud track data of the first type of vehicle. As described above, in the related art, there are many vehicles belonging to the second type, and if only the first point cloud track data corresponding to the vehicles belonging to the second type is used to directly generate the map, the period of map generation will be long. However, with the method of the embodiment of the present application, since the first point cloud trajectory data corresponding to vehicles belonging to the second type with a higher quantity is larger, and the coverage of the geographical environment area is larger, the first point cloud is used first. track data to construct the point cloud base map, so as to ensure the coverage of the point cloud base map; on this basis, the point cloud base map is optimized and updated by the second point cloud track data corresponding to the vehicle belonging to the first type, and the map is obtained , so that a compromise can be made between shortening the generation cycle of the map and improving the accuracy of the map. Moreover, the trajectory data corresponding to vehicles with different hardware configurations can be used to generate maps, making the data sources more comprehensive.

In addition, when a new semantic element needs to be added to the map, it only needs to be aligned with the point cloud base map to be quickly fused and generated, and there is no need to re-splicing based on the new version of the data to generate the map.

The solutions of the embodiments of the present application can be applied to constructing maps of areas with weak GNSS signals (or GPS signals), such as maps of indoor parking lots.

In practice, if the trajectory corresponding to the two first point cloud trajectory data does not have an intersection trajectory point, the two first point cloud trajectory data may not be spliced, or there are still some areas in the point cloud base map that are not correct after splicing. A continuous situation, such as a road may be a disconnected situation. Fig. 9 is a schematic diagram of splicing two first point cloud trajectory data according to an embodiment. As shown in Figure 9, after the splicing of the first point cloud trajectory data I and the first point cloud trajectory data II, there is a disconnected area 910, and according to conventional judgments, the disconnected area 910 may be different from the actual geographical environment area. matched.

Therefore, in this case, it is possible to manually participate in the construction process of the point cloud base map. In this case, after step 240, the method may also include: sending the point cloud base map to the client, so that the user can splice and edit the point cloud base map on the client, and then receive the point cloud base map from the client. The point cloud base map after splicing and editing. Therefore, by editing the point cloud base map, manually edit the part of the point cloud base map where there is a disconnected area, thereby improving the point cloud base map.

In some embodiments, as shown in FIG. 10, before step 250, the method may further include:

Step 1010, acquire a candidate point cloud trajectory data from the candidate point cloud trajectory data set.

In some embodiments, the candidate point cloud trajectory data in the candidate point cloud trajectory data set may be point cloud trajectory data from the first type of vehicle. In some other embodiments, the candidate point cloud trajectory data set may also include point cloud trajectory data from the first type of vehicle and the second type of vehicle.

In some embodiments, the candidate point cloud trajectory data may be acquired from the candidate point cloud trajectory data randomly or in a set order.

In some other embodiments, step 1010 may include: according to the priority corresponding to each candidate point cloud trajectory data in the candidate point cloud trajectory data set, according to the order of priority from high to low, obtain a Candidate point cloud trajectory data.

For example, the priority corresponding to each candidate point cloud trajectory data can be set according to the vehicle information of the vehicle from which the candidate point cloud trajectory data originates, wherein the hardware modules installed in the vehicle can be determined according to the vehicle information.

In some embodiments, it can be set that the priority corresponding to the candidate point cloud track data originating from a vehicle equipped with a laser radar and an intelligent perception module is the first priority, and that originating from a vehicle equipped with a laser radar or an intelligent perception module The priority corresponding to the candidate point cloud trajectory data of the vehicle is the second priority, and the corresponding priority of the candidate point cloud trajectory data from the vehicle without lidar and no intelligent perception module is the third priority, among which, the first The priority is higher than the second priority, and the second priority is higher than the third priority.

Step 1020, determine the coverage of the candidate point cloud trajectory data relative to the point cloud base map.

For example, according to the position information of each road element indicated by the candidate point cloud trajectory data, the length of the target part track located in the point cloud base map in the driving track indicated by the candidate point cloud track data can be determined, and then the length of the target part track Divide it with the total length of the driving trajectory indicated by the candidate point cloud trajectory data, and use the obtained ratio as the coverage of the candidate point cloud trajectory data relative to the point cloud base map.

Step 1030, if the coverage is greater than the set threshold, the candidate point cloud trajectory data is used as the second point cloud trajectory data.

When the coverage is greater than the set threshold, it indicates that the candidate point cloud trajectory data has a higher coverage relative to the point cloud base map, which also indicates that the candidate point cloud trajectory data contributes less to improving the coverage of the point cloud base map. Therefore, at this time, the candidate point cloud trajectory data can be used as the second point cloud trajectory data, so as to align and fuse the candidate point cloud trajectory data with the point cloud base map.

In some embodiments, after step 1020, the method may further include: if the coverage is not greater than the set threshold, splicing the candidate point cloud trajectory data with the point cloud base map to update the point cloud base map.

If the coverage is not greater than the set threshold, it indicates that the candidate point cloud trajectory data has a high contribution to the coverage of the coverage area of the point cloud base map. Therefore, at this time, the candidate point cloud trajectory data is used as the first point cloud trajectory data. It is used to update the point cloud basemap.

In some embodiments, as shown in FIG. 11 , splicing the candidate point cloud trajectory data with the point cloud base map, so that the step of updating the point cloud base map further includes:

Step 1110, determine the target vehicle type of the vehicle corresponding to the candidate point cloud trajectory data.

The target vehicle type refers to the vehicle type to which the source vehicle of the candidate point cloud trajectory data belongs.

In this embodiment, the vehicles may be classified according to the types of hardware configured on the vehicles. In a specific embodiment, the vehicle type set based on the hardware on the vehicle includes the first type and the second type, wherein, the vehicle belonging to the first type is equipped with a laser radar and/or an intelligent perception module; The second type of vehicle is not equipped with lidar and intelligent perception modules.

Step 1120, based on the correspondence between vehicle types and weights, determine the target weight corresponding to the target vehicle type.

The target weight refers to the weight corresponding to the target vehicle type. Wherein, the corresponding relationship between vehicle types and weights can be set according to actual needs.

Step 1130, if the weight of the target is greater than the weight threshold, move the second target road element in the point cloud base map so that the second target road element in the point cloud base map is consistent with the second target in the candidate point cloud trajectory data The road elements overlap; wherein, the second target road element refers to the road element representing the same geographic location in the point cloud base map and the candidate point cloud trajectory data.

Step 1140, if the weight of the target is not greater than the weight threshold, then move the second target road in the candidate point cloud trajectory data so that the second target road element in the point cloud base image is consistent with the second target road element in the candidate point cloud trajectory data. Target road elements overlap.

Step 1150, combining the moved point cloud base map and candidate point cloud trajectory data as an updated point cloud base map.

In some embodiments, higher weights may be assigned to vehicles with higher accuracy of point cloud trajectory data, and lower weights may be assigned to vehicles with lower accuracy of point cloud trajectory data. For example, compared with vehicles belonging to the second type, vehicles belonging to the first type are assigned a higher weight. Therefore, during the splicing process, it can be ensured that the point cloud trajectory data with higher precision moves less during the splicing process. On the contrary, the point cloud trajectory data with lower precision moves more during the splicing process, so as to avoid moving the original precision during the splicing process. Higher point cloud trajectory data results in reduced accuracy and precision of the point cloud trajectory data.

In this embodiment, the corresponding target weight is determined according to the target vehicle type of the vehicle corresponding to the candidate point cloud trajectory data, and then the object to be moved during the splicing process is determined according to the target weight, so that the original accuracy of moving during the splicing process can be avoided. Higher point cloud trajectory data, thereby ensuring the position accuracy of each road element in the point cloud base map.

Fig. 12 is a flowchart of a method for generating a map according to a specific embodiment of the present application. The steps filled in gray in FIG. 12 may be steps participated by equipment or manually. As shown in Figure 12, it includes: uploading the trajectory data to the server, and the server identifies road elements based on the trajectory data and generates corresponding point cloud trajectory data; then, data screening is performed, and the process of data screening can be referred to in Figure 10 The process shown is to determine the coverage of the point cloud trajectory data relative to the point cloud base map. If the coverage is not greater than the set threshold, it indicates that the coverage contribution of the point cloud trajectory data is high, and the point cloud trajectory The data is used for splicing to generate a point cloud base map. In the process of splicing and generating a point cloud base map, the point cloud base map can be sent to the client, and the user can splice and edit the point cloud base map.

Conversely, if the coverage is greater than the set threshold, it indicates that the coverage contribution of the point cloud trajectory data is low, and the point cloud trajectory data can be used for alignment and fusion with the point cloud base map to generate a map.

Layers in a map include positioning layers and logical layers. Further, after the map is generated, the map can be sent to the client, so that the user can edit the positioning layer and/or edit the logical layer based on the map displayed on the client. After the map is edited, the map can be further inspected by technicians to obtain a high-precision and high-accuracy map.

The following introduces the device embodiments of the embodiments of the present application, which can be used to implement the methods in the foregoing embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the foregoing method embodiments of the present application.

An embodiment of the present application provides a map generation device, including: a data acquisition module, a splicing module, and a fusion module. The data acquisition module is used to obtain the generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to the point cloud trajectory data of the second type of vehicle; obtain the determined second point cloud trajectory data, wherein the second point cloud trajectory data The cloud track data corresponds to the point cloud track data of the first type of vehicle; the splicing module is used to generate a point cloud base map according to the first point cloud track data; the fusion module is used to use the point cloud base map as an alignment medium to convert the second The two point cloud trajectory data are aligned and fused with the point cloud base map to obtain a map.

In one embodiment, the data acquisition module may include an acquisition module, an identification module and a generation module. The device for generating the map will be described in detail below in conjunction with the accompanying drawings.

Fig. 13 is a block diagram of a device for generating a map according to an embodiment of the present application. As shown in Fig. 13, the device for generating a map includes: an acquisition module 1310 for acquiring multiple trajectory data, the trajectory data including the pose of the vehicle Information and the bird's-eye view stitching image associated with the vehicle pose information; the identification module 1320 is used to identify road elements in each bird's-eye view stitching image, and determines the road elements in each bird's-eye view stitching image; the generation module 1330 is used to The road elements in each bird's-eye view stitching image and the vehicle pose information associated with each bird's-eye view stitching image generate the first point cloud track data corresponding to each track data; the stitching module 1340 is used to combine multiple track data. The first point cloud track data is spliced to obtain a point cloud base map; the fusion module 1350 is used to use the point cloud base map as an alignment medium to align and fuse the second point cloud track data with the point cloud base map to obtain a map.

In some embodiments, the recognition module 1320 includes: an input unit, configured to input the spliced images of bird's-eye view angles into the road element recognition model; Road element information corresponding to the image, where the road element information is used to indicate the corresponding road element in the bird's-eye view stitched image.

In some embodiments, the generation module 1330 is further configured to: according to the vehicle pose information associated with each bird's-eye view stitching image, perform three-dimensional reconstruction on each road element in each bird's-eye view stitching image to obtain each bird's-eye view stitching image The first point cloud trajectory data corresponding to the image.

In some embodiments, the splicing module 1340 includes: a first target road element determining unit, configured to determine first target road elements representing the same geographic location in any two of a plurality of first point cloud trajectories; Based on the first target road element, multiple first point cloud trajectory data are spliced to obtain a point cloud base map.

In some embodiments, the map generation device further includes: a sending module, configured to send the point cloud base map to the client, so that the user can stitch and edit the point cloud base map on the client.

In some embodiments, the map generation device also includes: a candidate point cloud trajectory data acquisition module, used to obtain a candidate point cloud trajectory data from the candidate point cloud trajectory data set; a coverage determination module, used to determine the candidate point cloud The coverage of the trajectory data relative to the point cloud base map; the second point cloud trajectory data determination module is used to use the candidate point cloud trajectory data as the second point cloud trajectory data if the coverage is greater than the set threshold.

In some embodiments, the generating device of the map also includes: an update module, used for splicing the candidate point cloud trajectory data with the point cloud base map if the coverage is not greater than the set threshold, so as to update the point cloud base map . In some embodiments, the update module includes: a target vehicle type determination unit, configured to determine the target vehicle type of the vehicle corresponding to the candidate point cloud trajectory data; a target weight determination unit, configured to determine the vehicle type based on the corresponding relationship between the weight , to determine the target weight corresponding to the target vehicle type; the first moving unit is used to move the second target road element in the point cloud base map if the target weight is greater than the weight threshold, so that the second target road element in the point cloud base map The target road element overlaps with the second target road element in the candidate point cloud track data; wherein, the second target road element refers to the road element representing the same geographic location in the point cloud base map and the candidate point cloud track data; the second mobile unit, It is used to move the second target road in the candidate point cloud trajectory data if the target weight is not greater than the weight threshold, so that the second target road element in the point cloud base map is consistent with the second target in the candidate point cloud trajectory data The road elements overlap; the combination unit is used to combine the moved point cloud base map and candidate point cloud trajectory data as the updated point cloud base map.

In some embodiments, the candidate point cloud trajectory data acquisition module is further configured to: according to the priority corresponding to each candidate point cloud trajectory data in the candidate point cloud trajectory data set, according to the order of priority from high to low, from the candidate points The cloud trajectory data set obtains a candidate point cloud trajectory data.

In some embodiments, the fusion module includes: a new road element determination unit, configured to use the point cloud basemap as an alignment medium, align the second point cloud trajectory data with the point cloud basemap, and determine the second point cloud trajectory The data is compared with the newly added road elements in the point cloud base map; the adding unit is used to add the point cloud model of the newly added road elements to the point cloud base map to obtain a map.

Fig. 14 is a structural block diagram of an electronic device according to an embodiment of the present application. The electronic device may be a physical server, a cloud server, etc., which is not specifically limited here. As shown in FIG. 14 , the electronic device in this application may include: a processor 1410 and a memory 1420, where computer-readable instructions are stored on the memory 1420, and when the computer-readable instructions are executed by the processor 1410, any of the above method embodiments may be implemented method in .

Processor 1410 may include one or more processing cores. The processor 1410 uses various interfaces and lines to connect various parts of the entire electronic device, and executes electronic operations by running or executing instructions, programs, code sets or instruction sets stored in the memory 1420, and calling data stored in the memory 1420. Various functions and processing data of the device. Optionally, the processor 1410 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware. The processor 1410 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the displayed content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 1410, but may be realized by a communication chip alone.

The memory 1420 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory). The memory 1420 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 1420 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, an alarm function, etc.), and for implementing the following Instructions and the like of the various method embodiments described above. The storage data area can also store data created by the electronic device during use (such as a disguised response command, obtained process status) and the like.

The present application also provides a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method in any one of the foregoing method embodiments is implemented. The computer readable storage medium may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for computer-readable instructions for performing any method steps in the methods described above. These computer readable instructions can be read from or written into one or more computer program products. Computer readable instructions may, for example, be compressed in a suitable form.

According to an aspect of the embodiments of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method in any of the above embodiments.

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. Actually, according to the embodiment of the present application, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided to be embodied by a plurality of modules or units.

Through the description of the above implementations, those skilled in the art can easily understand that the example implementations described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, server, touch terminal, or network device, etc.) execute the method according to the embodiment of the present application. Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in the application . It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (15)

1. A method for generating a map, comprising:

   Acquiring the generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to the point cloud trajectory data of the second type of vehicle;

   Obtaining the determined second point cloud trajectory data, wherein the second point cloud trajectory data corresponds to the point cloud trajectory data of the first type of vehicle;

   Generate a point cloud base map according to the first point cloud trajectory data;

   Using the point cloud base map as an alignment medium, align and fuse the second point cloud trajectory data with the point cloud base map to obtain a map.
2. The method according to claim 1, characterized in that,

   The first point cloud trajectory data is generated in the following manner:

   Acquiring a plurality of trajectory data, the trajectory data including vehicle pose information and a bird's-eye view stitching image associated with the vehicle pose information, the vehicle being a vehicle corresponding to the second type;

   Carrying out road element identification on each of the bird's-eye view stitching images, and determining the road elements in each of the bird's-eye view stitching images;

   According to the road elements in each of the bird's-eye view stitching images and the vehicle pose information associated with each of the bird's-eye view stitching images, generate the first point cloud track data corresponding to each of the track data;

   The generating point cloud base map according to the first point cloud trajectory data includes:

   The first point cloud trajectory data corresponding to the plurality of trajectory data are spliced to obtain a point cloud base map.
3. The method according to claim 2, wherein the road element identification is carried out on each of the bird's-eye view stitched images, and determining the road elements in each of the bird's-eye view stitched images includes:

   Input each stitched image of the bird's-eye view into the road element recognition model;

   Road element identification is performed by the road element identification model, and road element information corresponding to each stitched bird's-eye view image is output, and the road element information is used to indicate the road element in the corresponding stitched bird's-eye view image.
4. The method according to claim 2, characterized in that, according to the road elements in each of the bird's-eye view stitching images and the vehicle pose information associated with each of the bird's-eye view stitching images, each of the trajectory data corresponding The first point cloud trajectory data, including:

   According to the vehicle pose information associated with each of the bird's-eye view stitching images, perform three-dimensional reconstruction on each road element in each of the bird's-eye view stitching images, and obtain the first point cloud trajectory corresponding to each of the bird's-eye view stitching images data.
5. The method according to claim 2, wherein the splicing of the first point cloud trajectory data corresponding to the plurality of first trajectory data to obtain a point cloud base map includes:

   Determining first target road elements representing the same geographic location in any two of the plurality of first point cloud trajectory data;

   Based on the first target road element, multiple pieces of the first point cloud trajectory data are spliced to obtain the point cloud base map.
6. The method according to claim 2, wherein the first point cloud trajectory data corresponding to the plurality of trajectory data is spliced, and after obtaining the point cloud base map, the method also includes:

   Send the point cloud base map to the client, and receive the point cloud base map after the client splices and edits the point cloud base map.
7. The method according to claim 1, wherein the second point cloud track data is determined in the following manner:

   Obtain a candidate point cloud trajectory data from the candidate point cloud trajectory data set;

   Determine the coverage of the candidate point cloud trajectory data relative to the point cloud base map;

   If the coverage is greater than a set threshold, the candidate point cloud trajectory data is used as the second point cloud trajectory data.
8. The method according to claim 7, wherein after said determining the degree of coverage of said candidate point cloud trajectory data relative to said point cloud base map, said method further comprises:

   If the coverage is not greater than the set threshold, the candidate point cloud trajectory data is spliced with the point cloud base map to update the point cloud base map.
9. The method according to claim 8, wherein the splicing the candidate point cloud trajectory data with the point cloud base map to update the point cloud base map includes:

   Determine the target vehicle type of the vehicle corresponding to the candidate point cloud trajectory data;

   determining the target weight corresponding to the target vehicle type based on the correspondence between the vehicle type and the weight;

   If the target weight is greater than the weight threshold, the second target road element in the point cloud base map is moved, so that the second target road element in the point cloud base map is consistent with the candidate point cloud trajectory data The second target road element overlaps; wherein, the second target road element refers to the road element representing the same geographic location in the point cloud base map and the candidate point cloud trajectory data; or,

   If the target weight is not greater than the weight threshold, then move the second target road in the candidate point cloud trajectory data so that the second target road element in the point cloud base map is consistent with the candidate point cloud trajectory The second target road element in the data overlaps;

   Combining the moved point cloud base map and the candidate point cloud trajectory data as an updated point cloud base map.
10. The method according to claim 7, wherein said obtaining a candidate point cloud trajectory data from the candidate point cloud trajectory data set comprises:

    According to the priority corresponding to each candidate point cloud trajectory data in the candidate point cloud trajectory data set, a candidate point cloud trajectory data is obtained from the candidate point cloud trajectory data set in order of priority from high to low.
11. The method according to claim 1, wherein the point cloud base map is used as an alignment medium, and the second point cloud trajectory data is aligned and fused with the point cloud base map to obtain a map, including:

    Using the point cloud base map as an alignment medium, align the second point cloud trajectory data with the point cloud base map, and determine that the second point cloud trajectory data is compared with the point cloud base map. Added road elements;

    The point cloud model of the newly added road element is added to the point cloud base map to obtain the map.
12. A device for generating a map, comprising:

    The data acquisition module is used to obtain the generated first point cloud trajectory data, wherein the first point cloud trajectory data corresponds to the point cloud trajectory data of the second type of vehicle; obtain the determined second point cloud trajectory data, wherein the The second point cloud trajectory data corresponds to the point cloud trajectory data of the first type of vehicle;

    A splicing module, configured to generate a point cloud base map according to the first point cloud trajectory data;

    The fusion module is configured to use the point cloud base map as an alignment medium to align and fuse the second point cloud trajectory data with the point cloud base map to obtain a map.
13. The device according to claim 12, characterized in that:

    The data acquisition module includes an acquisition module, an identification module and a generation module;

    An acquisition module, configured to acquire a plurality of trajectory data, the trajectory data including vehicle pose information and a bird's-eye view stitching image associated with the vehicle pose information;

    An identification module, configured to perform road element identification on each stitched image from a bird's-eye view, and determine road elements in each stitched image from a bird's-eye view;

    A generating module, configured to generate the first point cloud trajectory data corresponding to each of the trajectory data according to the road elements in each of the bird's-eye perspective stitching images and the vehicle pose information associated with each of the bird's-eye perspective stitching images;

    The splicing module splices the first point cloud track data corresponding to the plurality of track data to obtain a point cloud base map.
14. An electronic device, characterized in that it comprises:

    processor;

    A memory, where computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1-11 is realized.
15. A computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the method according to any one of claims 1-11 is implemented.

Images