WO2022199195A1 - Map updating method and system, vehicle-mounted terminal, server, and storage medium - Google Patents

Abstract

Provided in the present application are a map updating method and system, a vehicle-mounted terminal, a server, and a storage medium. The method comprises: on the basis of vehicle trajectory data of a vehicle and a vehicle visual image corresponding to the vehicle trajectory data, obtaining internal and external parameters of a driving recorder on the vehicle, and sending data of the driving recorder to a server, such that the server updates an HD Map of the vehicle on the basis of the data of the driving recorder. In this way, the problem of an existing HD Map not being updated in a timely manner, which leads to the incorrect understanding of information of the environment in which the autonomous vehicle is located, thus resulting in a larger problem for autonomous driving is solved. In addition, according to the method, since HD Map updating is performed on the basis of data from a driving recorder, data in a larger range can be obtained, thereby improving the accuracy and timeliness of HD Map updating, and also reducing the cost of HD Map updating.

Description

Map update method, system, vehicle terminal, server and storage medium

This application claims the priority of the Chinese patent application filed on March 26, 2021 with the application number 202110330793.3 and the application name is "Map Update Method, System, Vehicle Terminal, Server and Storage Medium", the entire content of which is approved by Reference is incorporated in this application.

technical field

The present application relates to the technical field of navigation electronic maps, and more particularly, to a map updating method, system, vehicle terminal, server and storage medium.

Background technique

With the development of technology, people's requirements for maps have been further improved. High Definition Map (HD Map) has been applied to many aspects of people's lives. It is different from ordinary electronic navigation maps and can meet the needs of automatic driving, such as automatic driving. Driving, path planning, high-precision navigation and positioning, etc.

In related technologies, taking autonomous driving as an example, an autonomous vehicle needs to understand the surrounding environment first. Then, the optimal driving trajectory of the vehicle is determined, and then the vehicle is controlled to travel according to the planned driving trajectory. Among them, autonomous vehicles can obtain environmental information near the vehicle through sensors (such as cameras, lidars, etc.). However, due to the limitations of the sensing range of the sensor and the performance of the processor, the ability of the self-driving car to perceive the surrounding environment will be reduced. To overcome this limitation, self-driving cars can utilize pre-built HD Maps to obtain environmental information.

However, the real world is constantly changing. If the HD Map is not updated in time, the self-driving car will have an incorrect understanding of the environment information, which will cause major problems to the self-driving car. For example, the positioning of the self-driving car will fail because the HD Map is not updated in time.

SUMMARY OF THE INVENTION

The present application provides a map update method, system, on-board terminal, server and storage medium, which can update the HD Map of the vehicle in time, so that the autonomous vehicle can correctly understand the environmental information in which it is located, and reduce the occurrence of accidents in autonomous driving. question.

In a first aspect, an embodiment of the present application provides a method for updating a map, and the method includes the following steps:

From the controller area network (CAN) bus and driving recorder of the target vehicle, the data stream of the target vehicle is obtained, and each frame of data in the data stream includes vehicle trajectory data, and data related to the vehicle The vehicle visual image corresponding to the trajectory;

Determine the correspondence between the vehicle visual image and the HD Map to be updated according to the internal parameter parameters of the driving recorder and the vehicle trajectory data, wherein the HD Map to be updated corresponds to the target vehicle;

obtaining the first internal and external parameters of the driving recorder according to the corresponding relationship;

The first internal and external parameters are sent to the server, so that the server updates the HD Map to be updated according to the first internal and external parameters.

In a possible implementation manner, determining the corresponding relationship between the vehicle visual image and the HD Map to be updated according to the internal parameter parameters of the driving recorder and the vehicle trajectory data, including:

According to the vehicle trajectory data, determine the HD Map to be updated corresponding to the target vehicle, and determine the second internal and external parameters of the driving recorder according to the internal parameter parameters of the driving recorder and the vehicle trajectory data;

According to the second internal and external parameters, the corresponding relationship between the vehicle visual image and the HD Map to be updated is determined.

In a possible implementation manner, determining the corresponding relationship between the vehicle visual image and the HD Map to be updated according to the second internal and external parameters includes:

According to the second internal and external parameters, the to-be-updated HD Map is projected onto the corresponding vehicle visual image, and the two-dimensional coordinates of each element in the to-be-updated HD Map are obtained;

matching the two-dimensional coordinates with the corresponding vehicle visual image;

The corresponding relationship is determined according to the matching result between the two-dimensional coordinates and the corresponding vehicle visual image.

In a possible implementation manner, the obtaining the first internal and external parameters of the driving recorder according to the corresponding relationship includes:

According to the corresponding relationship, obtain the projection matrix of each frame of the driving recorder;

Obtaining estimated values of internal and external parameters of each frame of the driving recorder according to the projection matrix;

According to the estimated values of the internal and external parameters of the initial frame of the driving recorder, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.

In a possible implementation manner, according to the estimated values of the internal and external parameters of the initial frame of the driving recorder, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters, including:

According to the estimated value of the internal parameter in the estimated value of the internal and external parameters of the initial frame, project the HD Map to be updated on the corresponding visual image of the vehicle, and calculate the projection error;

The projection error is used as a loss function, and based on the loss function, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.

In a second aspect, the embodiments of the present application provide another method for updating a map, and the method includes the following steps:

Receive the first internal and external parameters of the driving recorder of the target vehicle sent by the vehicle terminal;

According to the first internal and external parameters, the HD Map to be updated is updated, and the HD Map to be updated corresponds to the target vehicle;

Wherein, the first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the CAN bus of the target vehicle and the driving recorder, according to the internal parameter parameters of the driving recorder and the driving recorder Vehicle trajectory data in the data stream, determine the corresponding relationship between the vehicle visual image in the data stream and the high-precision map to be updated, and obtain the first internal and external parameters according to the corresponding relationship, wherein the data stream Each frame of data in includes the vehicle trajectory data and the vehicle visual image corresponding to the vehicle trajectory.

In a possible implementation manner, the updating of the HD Map to be updated according to the first internal and external parameters includes:

Determine the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters;

According to the pose, construct the target HD Map;

According to the target HD Map, the HD Map to be updated is updated.

In a possible implementation manner, the determining the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters includes:

According to the internal parameter parameters in the first internal and external parameters, the inter-frame fundamental matrix of the driving recorder is decomposed to obtain the inter-frame pose estimation value;

According to the estimated value of the pose between frames, the external parameter parameters in the first internal and external parameters are optimized to obtain the pose.

In a possible implementation manner, the constructing the target HD Map according to the pose includes:

According to the pose, obtain the three-dimensional space coordinate value of each object feature point;

According to the three-dimensional space coordinate value of each object feature point, the target HD Map is constructed.

In a third aspect, an embodiment of the present application provides a map update device, including:

The acquisition module is used to acquire the data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle, and each frame of data in the data stream includes vehicle trajectory data and vehicle vision corresponding to the vehicle trajectory image;

A determination module, configured to determine the corresponding relationship between the vehicle visual image and the HD Map to be updated according to the internal parameter parameters of the driving recorder and the vehicle trajectory data, wherein the HD Map to be updated corresponds to the target vehicle ;

an obtaining module for obtaining the first internal and external parameters of the driving recorder according to the corresponding relationship;

A sending module, configured to send the first internal and external parameters to a server, so that the server updates the HD Map to be updated according to the first internal and external parameters.

In a possible implementation manner, the determining module is specifically used for:

According to the vehicle trajectory data, determine the HD Map to be updated corresponding to the target vehicle, and determine the second internal and external parameters of the driving recorder according to the internal parameter parameters of the driving recorder and the vehicle trajectory data;

According to the second internal and external parameters, the corresponding relationship between the vehicle visual image and the HD Map to be updated is determined.

In a possible implementation manner, the determining module is specifically used for:

According to the second internal and external parameters, the to-be-updated HD Map is projected onto the corresponding vehicle visual image, and the two-dimensional coordinates of each element in the to-be-updated HD Map are obtained;

matching the two-dimensional coordinates with the corresponding vehicle visual image;

The corresponding relationship is determined according to the matching result between the two-dimensional coordinates and the corresponding vehicle visual image.

In a possible implementation manner, the obtaining module is specifically used for:

According to the corresponding relationship, obtain the projection matrix of each frame of the driving recorder;

Obtaining estimated values of internal and external parameters of each frame of the driving recorder according to the projection matrix;

According to the estimated values of the internal and external parameters of the initial frame of the driving recorder, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.

In a possible implementation manner, the obtaining module is specifically used for:

According to the estimated value of the internal parameter in the estimated value of the internal and external parameters of the initial frame, project the HD Map to be updated on the corresponding visual image of the vehicle, and calculate the projection error;

The projection error is used as a loss function, and based on the loss function, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.

In a fourth aspect, the embodiments of the present application provide another map update device, including:

The receiving module is used to receive the first internal and external parameters of the driving recorder of the target vehicle sent by the vehicle terminal;

an update module, configured to update the HD Map to be updated according to the first internal and external parameters, and the HD Map to be updated corresponds to the target vehicle;

Wherein, the first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the CAN bus of the target vehicle and the driving recorder, according to the internal parameter parameters of the driving recorder and the driving recorder Vehicle trajectory data in the data stream, determine the corresponding relationship between the vehicle visual image in the data stream and the high-precision map to be updated, and obtain the first internal and external parameters according to the corresponding relationship, wherein the data stream Each frame of data in includes the vehicle trajectory data and the vehicle visual image corresponding to the vehicle trajectory.

In a possible implementation manner, the update module is specifically used for:

Determine the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters;

According to the pose, construct the target HD Map;

According to the target HD Map, the HD Map to be updated is updated.

In a possible implementation manner, the update module is specifically used for:

According to the internal parameter parameters in the first internal and external parameters, the inter-frame fundamental matrix of the driving recorder is decomposed to obtain the inter-frame pose estimation value;

According to the estimated value of the pose between frames, the external parameter parameters in the first internal and external parameters are optimized to obtain the pose.

In a possible implementation manner, the update module is specifically used for:

According to the pose, obtain the three-dimensional space coordinate value of each object feature point;

According to the three-dimensional space coordinate value of each object feature point, the target HD Map is constructed.

In a fifth aspect, an embodiment of the present application provides a map update system, including:

The vehicle-mounted terminal is used to obtain the data stream of the target vehicle from the controller area network bus and the driving recorder of the target vehicle, and each frame of data in the data stream includes vehicle trajectory data and corresponds to the vehicle trajectory According to the internal parameter parameters of the driving recorder and the vehicle trajectory data, the corresponding relationship between the vehicle visual image and the HD Map to be updated is determined, wherein the HD Map to be updated corresponds to the target vehicle ; obtain the first internal and external parameters of the driving recorder according to the corresponding relationship; send the first internal and external parameters to the server;

The server is configured to update the HD Map to be updated according to the first internal and external parameters.

In a sixth aspect, an embodiment of the present application provides a vehicle-mounted terminal, including:

Sending devices, receiving devices, processors, memories, and computer programs;

Wherein, the transmitting device and the receiving device are respectively connected with the processor, the computer program is stored in the memory and configured to be executed by the processor, the computer program includes a program for executing Instructions for the method of the first aspect.

In a seventh aspect, an embodiment of the present application provides a server, including:

processor;

memory; and

Computer program;

Wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of the second aspect.

In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program enables the vehicle-mounted terminal to execute the method described in the first aspect.

In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program causes a server to execute the method of the second aspect.

In a tenth aspect, an embodiment of the present application provides a computer program product, including computer instructions, where the computer instructions are executed by a processor to execute the method of the first aspect.

In an eleventh aspect, an embodiment of the present application provides a computer program product, including computer instructions, where the computer instructions are executed by a processor to execute the method described in the second aspect.

The map update method, system, vehicle terminal, server and storage medium provided by the embodiments of the present application, the method obtains the data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle, and each frame of data in the data stream is obtained. Including the vehicle trajectory data and the vehicle visual image corresponding to the vehicle trajectory, and then, according to the internal parameter parameters of the above-mentioned driving recorder and the vehicle trajectory data, determine the corresponding relationship between the above-mentioned vehicle visual image and the HD Map to be updated, the HD Map to be updated and The above-mentioned target vehicle corresponds to, and according to the above-mentioned corresponding relationship, the first internal and external parameters of the driving recorder are obtained, so that the first internal and external parameters are sent to the server, so that the server can perform the above-mentioned HD Map to be updated according to the first internal and external parameters. Update, that is, the embodiment of the present application obtains the internal and external parameters of the above-mentioned vehicle driving recorder based on the vehicle trajectory data of the vehicle and the vehicle visual image corresponding to the vehicle trajectory data, and sends the data of the driving recorder to the server, so that The server updates the HD Map of the above-mentioned vehicle based on the data of the above-mentioned driving recorder, which solves the problem that the existing HD Map is not updated in time, which leads to an incorrect understanding of the environmental information of the autonomous vehicle and causes great problems to the autonomous driving. question. Moreover, the embodiment of the present application performs HD Map update based on the driving recorder data, which can obtain data in a wider range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve the timeliness.

Description of drawings

1 is a schematic diagram of a map update system architecture provided by an embodiment of the present application;

2 is a schematic flowchart of a method for updating a map according to an embodiment of the present application;

3 is a schematic flowchart of another map update method provided by an embodiment of the present application;

4 is a schematic flowchart of still another map updating method provided by an embodiment of the present application;

5 is a schematic flowchart of another map update method provided by an embodiment of the present application;

6 is a schematic flowchart of another map update method provided by an embodiment of the present application;

7 is a schematic flowchart of a map update provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a map updating apparatus provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of another map updating apparatus provided by an embodiment of the present application;

10A is a schematic diagram of the basic hardware architecture of a vehicle-mounted terminal provided by the application;

FIG. 10B is a schematic diagram of a basic hardware architecture of a server provided by this application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The terms "first", "second", "third" and "fourth", etc. (if any) in the description and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances such that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

In related technologies, taking autonomous driving as an example, an autonomous vehicle needs to understand the surrounding environment first. Then, the optimal driving trajectory of the vehicle is determined, and then the vehicle is controlled to travel according to the planned driving trajectory. Among them, autonomous vehicles can obtain environmental information near the vehicle through sensors (such as cameras, lidars, etc.). However, due to the limitations of the sensing range of the sensor and the performance of the processor, the ability of the self-driving car to perceive the surrounding environment will be reduced. To overcome this limitation, self-driving cars can utilize pre-built HD Maps to obtain environmental information.

However, the real world is constantly changing. If the HD Map is not updated in time, the self-driving car will have an incorrect understanding of the environment information, which will cause major problems to the self-driving car. For example, the positioning of the self-driving car will fail because the HD Map is not updated in time.

In order to solve the above problems, there are two main ways to update the HD Map: the first one: according to the construction, road maintenance and other information, determine the spatial range of the map change and drive the surveying and mapping-level mobile acquisition vehicle to update. The second: use professional visual acquisition equipment (monocular or binocular) and positioning equipment, and use the depth estimation method based on deep learning to complete the map construction.

However, since the first method adopts an intelligence-driven approach, it is difficult to obtain relevant information for unplanned changes, which may easily lead to insufficient update range, which seriously affects the accuracy of HD Map updates. At the same time, the follow-up processing of the data collected by the mobile collection vehicle is complex and time-consuming, and it is difficult to ensure the timeliness of the HD Map update. In the second method, the visual equipment needs to be calibrated regularly, which will result in large maintenance costs and high costs.

Therefore, the embodiment of the present application proposes a map update method, which updates the HD Map based on the data of the driving recorder, which solves the problem that the existing HD Map is not updated in time, which causes the autonomous vehicle to incorrectly understand the environment information in which it is located. Driving a problem that causes bigger problems. Moreover, the embodiment of the present application performs HD Map update based on driving recorder data, which can obtain data in a wider range, improves the accuracy of HD Map update, and is compared with HD Map update based on professional visual acquisition equipment and positioning equipment. Method, the embodiment of the present application adopts the low-cost driving recorder data, which greatly reduces the cost of HD Map update. In addition, compared with the HD Map update method based on the surveying-level mobile acquisition vehicle, the embodiment of the present application uses the driving recorder data. Significantly reduces computing power requirements and subsequent point cloud processing procedures, enables on-device computing, and improves the timeliness of HD Map updates.

Optionally, the map update method provided in the embodiment of the present application is applied to the process of map update in the process of HD Map mapping, and specifically, can be applied to the system for generating HD Map by performing information exchange between the vehicle terminal and the server. 1 is a schematic diagram of the architecture of a map update system provided by an embodiment of the present application. As shown in FIG. 1 , the system includes a vehicle terminal 11 and a server 12 . Here, the in-vehicle terminal 11 may include a transmitting device and a receiving device. Exemplarily, which devices on the vehicle are specific to the vehicle-mounted terminal 11 may be determined according to the actual situation. For example, in some cases, the vehicle-mounted terminal 11 may be a navigation terminal on the vehicle, which is not particularly limited in this embodiment of the present application.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the architecture of the map update system. In some other feasible embodiments of the present application, the above architecture may include more or less components than shown in the figure, or combine some components, or separate some components, or arrange different components, depending on the actual application. The scene is determined, and there is no restriction here. The components shown in Figure 1 may be implemented in hardware, software, or a combination of software and hardware.

In a specific implementation process, the vehicle-mounted terminal 11 may send an acquisition instruction to the driving recorder through the sending device, and then, based on the acquisition instruction, acquire data from the driving recorder through the receiving device. Wherein, the above-mentioned obtaining instruction may be sent by the vehicle-mounted terminal 11 after obtaining the position of the vehicle and determining that the position needs to be updated according to the position. Here, the vehicle is further equipped with a positioning device, and the vehicle-mounted terminal 11 obtains the position of the vehicle through the positioning device.

The in-vehicle terminal 11 acquires the internal and external parameters of the driving recorder of the vehicle, and sends the data of the driving recorder to the server 12. The server 12 updates the HD Map of the above-mentioned vehicle based on the data of the above-mentioned driving recorder, so as to solve the problem that the existing HD Map is not updated in time, which leads to an incorrect understanding of the information of the environment in which the autonomous driving vehicle is located, and causes great problems to the autonomous driving. question. Moreover, the embodiment of the present application performs HD Map update based on the driving recorder data, which can obtain data in a larger range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve the timeliness.

Wherein, after the on-board terminal 11 sends the data of the vehicle driving recorder to the server 12, the server 12 updates the HD Map of the above-mentioned vehicle based on the data, so that the updated HD Map can be applied to automatic driving, route planning, Navigation and positioning, etc.

It should be noted that in FIG. 1 , the server 12 may also send the location area that needs to be updated to the vehicle terminal 11 , so that the vehicle terminal 11 sends an acquisition instruction to the driving record when determining that the vehicle is located in the above location area. to obtain data from the dash cam.

In the embodiments of the present application, the above-mentioned vehicle-mounted terminals may be handheld devices, vehicle-mounted devices, wearable devices, computing devices, and various forms of user equipment (User Equipment, UE).

In addition, the system architecture and business scenarios described in the embodiments of the present application are for the purpose of illustrating the technical solutions of the embodiments of the present application more clearly, and do not constitute limitations on the technical solutions provided by the embodiments of the present application. With the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The technical solutions of the present application are described below by taking several embodiments as examples, and the same or similar concepts or processes may not be repeated in some embodiments.

FIG. 2 provides a schematic flowchart of a map update method according to an embodiment of the present application. The execution body of this embodiment may be the vehicle-mounted terminal in the embodiment shown in FIG. 1 . As shown in FIG. 2 , the method may include:

S201: Obtain a data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle, where each frame of data in the data stream includes vehicle trajectory data and a vehicle visual image corresponding to the vehicle trajectory.

The target vehicle is a vehicle that needs to be updated on the map, which can be determined according to the actual situation. The above-mentioned vehicle-mounted terminal obtains the data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle. Exemplarily, the above vehicle terminal may send an acquisition instruction to the CAN bus and the driving recorder of the target vehicle, and then, based on the acquisition instruction, acquire data from the CAN bus and the driving recorder. Here, the above-mentioned obtaining instruction may be sent by the vehicle-mounted terminal after obtaining the position of the target vehicle and determining that the position needs to be updated according to the position.

In the embodiment of the present application, each frame of data in the above-mentioned data stream includes vehicle trajectory data, and a vehicle visual image corresponding to the vehicle trajectory, wherein the corresponding can be understood as the vehicle trajectory and driving information obtained by the CAN bus of the target vehicle at the same time. The visual image of the vehicle acquired by the recorder.

S202: Determine the corresponding relationship between the above-mentioned vehicle visual image and the high-precision map to be updated according to the internal parameter parameters of the above-mentioned driving recorder and the above-mentioned vehicle trajectory data, wherein the to-be-updated HD Map corresponds to the above-mentioned target vehicle.

Here, the above-mentioned on-board terminal can determine the above-mentioned HD Map to be updated corresponding to the target vehicle according to the above-mentioned vehicle trajectory data. For example, the on-board terminal obtains a certain range of HD Maps from the existing HD Maps of the target vehicle according to the above-mentioned vehicle trajectory data as the above-mentioned HD Maps to be updated. Exemplarily, the above-mentioned vehicle-mounted terminal may obtain, from the existing HD Map of the target vehicle, an HD Map within a radius of 200m with the target vehicle as the center according to the plane coordinate value in the above-mentioned vehicle trajectory data as the above-mentioned HD Map to be updated.

When the above-mentioned vehicle-mounted terminal determines the corresponding relationship between the above-mentioned vehicle visual image and the HD Map to be updated, it can determine the second internal and external parameters of the above-mentioned driving recorder according to the internal parameter parameters of the above-mentioned driving recorder and the above-mentioned vehicle trajectory data, and then, according to the The second internal and external parameters are used to determine the corresponding relationship between the above-mentioned vehicle visual image and the above-mentioned HD Map to be updated. Exemplarily, the above-mentioned vehicle-mounted terminal may calculate the heading angle between the two points according to the plane coordinate values of the vehicle trajectory data of the previous frame and the current frame, and use this as the initial heading angle of the current frame, thereby determining the current heading angle of the target vehicle. The external parameter parameters (plane coordinate value, initial heading angle, pitch and roll angle are set to 0), and then combined with the internal parameter parameters of the driving recorder to determine the second internal and external parameters of the driving recorder.

S203: Obtain the first internal and external parameters of the above-mentioned driving recorder according to the above-mentioned corresponding relationship.

In the embodiment of the present application, the above-mentioned vehicle-mounted terminal can project the above-mentioned HD Map to be updated onto a corresponding vehicle visual image according to the above-mentioned second internal and external parameters, so as to obtain the two-dimensional coordinates of each element in the above-mentioned HD Map to be updated, and then, The two-dimensional coordinates are matched with the above-mentioned corresponding vehicle visual image, and according to the matching result, the corresponding relationship between the above-mentioned vehicle visual image and the above-mentioned HD Map to be updated is determined.

Exemplarily, the above-mentioned on-board terminal can project the above-mentioned HD Map to be updated onto the vehicle visual image of the current frame of the driving recorder, transform the three-dimensional coordinates in the to-be-updated HD Map into two-dimensional coordinates on the image, and complete the to-be-updated HD Map. The dimensionality reduction of the data provides the possibility to match the vehicle visual image and the HD Map projection result. After the HD Map to be updated is reprojected, the above-mentioned vehicle terminal can project the two-dimensional coordinates (such as traffic signs and lane lines) of each element in the HD Map to be updated and the visual image of the vehicle according to their respective attributes, positions and aspect ratios, etc. The rules are matched to obtain the corresponding relationship between the vehicle visual image and the HD Map to be updated, and the 2D-3D initial matching between the vehicle visual image and the HD Map to be updated is completed.

In order to improve the accuracy of subsequent processing results, the vehicle-mounted terminal may optimize the correspondence between the vehicle visual image and the HD Map to be updated, that is, to further optimize the 2D-3D feature matching result between the vehicle visual image and the HD Map to be updated. Exemplarily, the above-mentioned vehicle-mounted terminal may combine the above-mentioned corresponding relationship (feature matching result), and use the optical flow method to perform object-level semantic matching on the images of the front and rear frames, determine the spatial correlation of the elements in the front and rear frames, and use this as a constraint, further. The correspondence between the above-mentioned vehicle visual image and the HD Map to be updated is to optimize the 2D-3D feature matching results.

Here, after determining the correspondence between the vehicle visual image and the HD Map to be updated, the vehicle-mounted terminal may obtain the first internal and external parameters of the driving recorder according to the correspondence. Wherein, when the vehicle-mounted terminal obtains the first internal and external parameters of the driving recorder according to the corresponding relationship, it may obtain the estimated value of the internal and external parameters of each frame of the driving recorder according to the above-mentioned corresponding relationship, and perform a calculation on the estimated value of the internal and external parameters. Optimization, to obtain the first internal and external parameters of the above driving recorder.

S204: Send the above-mentioned first internal and external parameters to the server, so that the server updates the above-mentioned HD Map to be updated according to the above-mentioned first internal and external parameters.

Here, after receiving the first internal and external parameters sent by the above-mentioned vehicle-mounted terminal, the above-mentioned server may first construct a target HD Map according to the above-mentioned first internal and external parameters, and then, according to the above-mentioned target HD Map, update the above-mentioned HD Map to be updated. Exemplarily, the above-mentioned server may perform a difference between the above-mentioned target HD Map and the above-mentioned HD Map to be updated, and find the changed part, so as to update the above-mentioned HD Map to be updated.

In this embodiment of the present application, a data stream of the target vehicle is obtained from the CAN bus and the driving recorder of the target vehicle through the on-board terminal. Each frame of data in the data stream includes vehicle trajectory data and a vehicle visual image corresponding to the vehicle trajectory. Further, according to the internal parameters of the driving recorder and the vehicle trajectory data, the corresponding relationship between the above-mentioned vehicle visual image and the above-mentioned HD Map to be updated is determined, the HD Map to be updated corresponds to the above-mentioned target vehicle, and the driving record is obtained according to the above-mentioned corresponding relationship. Therefore, the first internal and external parameters are sent to the server, so that the server can update the above-mentioned HD Map to be updated according to the first internal and external parameters. Track data, and the vehicle visual image corresponding to the vehicle track data, obtain the internal and external parameters of the above-mentioned vehicle driving recorder, and send the data of the driving recorder to the server, so that the server is based on the data of the driving recorder. The updated HD Map solves the problem that the existing HD Map is not updated in time, which leads to the incorrect understanding of the information of the environment in which the self-driving car is located, which causes big problems for automatic driving. Moreover, the embodiment of the present application performs HD Map update based on the driving recorder data, which can obtain data in a larger range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve the timeliness.

In addition, in the embodiment of the present application, when the vehicle-mounted terminal obtains the first internal and external parameters of the driving recorder according to the corresponding relationship, it can obtain the projection matrix of each frame of the driving recorder according to the corresponding relationship, and further, according to the corresponding relationship Projection matrix to obtain the estimated values of the internal and external parameters of each frame of the above-mentioned driving recorder, according to the estimated values of the internal and external parameters of the initial frame of the above-mentioned driving recorder, optimize the estimated values of the internal and external parameters of each of the above-mentioned frames, and obtain the first number of the above-mentioned driving recorder. One internal and external parameter, which solves the problem of estimating the internal and external parameters of the driving recorder, reduces the maintenance cost caused by the internal parameter calibration of the driving recorder and the equipment cost of the high-precision positioning module, and is suitable for application. FIG. 3 is a schematic flowchart of another map updating method proposed by an embodiment of the present application. As shown in Figure 3, the method includes:

S301: Obtain a data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle, where each frame of data in the data stream includes vehicle trajectory data and a vehicle visual image corresponding to the vehicle trajectory.

S302: Determine the corresponding relationship between the above-mentioned vehicle visual image and the HD Map to be updated according to the internal parameter parameters of the above-mentioned driving recorder and the vehicle trajectory data, wherein the to-be-updated HD Map corresponds to the target vehicle.

Wherein, for steps S301-S302, reference may be made to the relevant descriptions of the above-mentioned steps S201-S202, which will not be repeated here.

S303: Obtain the projection matrix of each frame of the above-mentioned driving recorder according to the above-mentioned corresponding relationship.

S304: According to the above-mentioned projection matrix, obtain the estimated value of the internal and external parameters of each frame of the above-mentioned driving recorder.

Here, after determining the corresponding relationship between the above-mentioned vehicle visual image and the above-mentioned HD Map to be updated, that is, after obtaining the above-mentioned vehicle visual image and the 2D-3D feature matching result of each frame of the to-be-updated HD Map, the vehicle-mounted terminal can carry out the driving recorder. The pose is solved, the frame-by-frame projection matrix is obtained, and the internal and external parameters are obtained.

Among them, the above-mentioned vehicle terminal can obtain the matching relationship pairs of all 2D image coordinates of the current frame and the corresponding spatial 3D coordinates according to the 2D-3D feature matching results of each frame, and use the direct linear transformation method (Direct Linear Transformation, DLT) to calculate each The projection matrix of the frame, that is, the product of the internal parameter matrix and the rotation matrix, and then use RQ decomposition to decompose the projection matrix into two parts: internal and external parameter matrices, complete frame-by-frame internal and external parameter estimation, and solve the internal and external parameters of the driving recorder. The problem of estimation reduces the maintenance cost caused by the internal parameter calibration of the driving recorder and the equipment cost of the high-precision positioning module.

S305: According to the estimated values of the internal and external parameters of the initial frame of the driving recorder, optimize the estimated values of the internal and external parameters of each frame to obtain the first internal and external parameters of the driving recorder.

In the embodiment of the present application, the above-mentioned vehicle terminal can be based on the assumption that the optical properties of the same optical device remain unchanged in a short period of time (that is, the internal parameters of the above-mentioned driving recorder remain unchanged in a short period of time), and according to the estimated values of the internal and external parameters of the initial frame in a sequence, The estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.

Exemplarily, the above-mentioned vehicle-mounted terminal may project the above-mentioned HD Map to be updated on the corresponding vehicle visual image according to the estimated value of the internal parameter parameter in the estimated value of the internal and external parameters of the above-mentioned initial frame, calculate the projection error, and then use the projection error as the loss. function, and based on the loss function, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters. For example, the above-mentioned vehicle-mounted terminal can reproject the HD Map to the remaining frames according to the estimated values of the internal parameters in the estimated values of the internal and external parameters of the initial frame, and calculate the re-projection error of each frame, so as to use the cooperation of the re-projection errors of each frame in the sequence. For the loss function, the least squares method is used to estimate the internal parameters of the camera, so that the loss function locally converges, so as to incrementally optimize the internal parameters, and then recalculate the external parameters frame by frame through the optimized internal parameter results and the frame-by-frame projection matrix. parameters, so as to complete the optimization of internal and external parameters, and obtain the above-mentioned first internal and external parameters.

S306: Send the above-mentioned first internal and external parameters to the server, so that the server updates the above-mentioned HD Map to be updated according to the above-mentioned first internal and external parameters.

Wherein, for step S306, reference is made to the relevant description of the above-mentioned step S204, and details are not repeated here.

The embodiment of the present application uses the HD Map to be updated to complete the estimation of the internal and external parameters of the driving recorder, solves the problem of estimating the internal and external parameters of the driving recorder, and reduces the maintenance cost and high-precision positioning module caused by the calibration of the internal parameters of the driving recorder. The equipment cost is suitable for the application. Moreover, in the embodiment of the present application, the vehicle-mounted terminal obtains the internal and external parameters of the above-mentioned vehicle driving recorder based on the vehicle trajectory data of the vehicle and the vehicle visual image corresponding to the vehicle trajectory data, and sends the data of the driving recorder to the server. , so that the server updates the HD Map of the above vehicle based on the data of the above-mentioned driving recorder, which solves the problem that the existing HD Map is not updated in time, resulting in an incorrect understanding of the environmental information of the autonomous vehicle, which will cause greater damage to autonomous driving. problem problem. In addition, the embodiment of the present application performs HD Map update based on driving recorder data, which can obtain data in a wider range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve timeliness.

The above embodiments describe the map updating method according to the embodiments of the present application in detail from the vehicle terminal side, and the following describes the map updating method according to the embodiments of the present application from the server side in detail with reference to the following embodiments. It should be understood that some concepts, characteristics, etc. described on the server side correspond to the description on the terminal side of the vehicle terminal, and for the sake of brevity, repeated descriptions are appropriately omitted.

FIG. 4 is a schematic flowchart of still another map updating method provided by an embodiment of the present application. The execution body of this embodiment may be the server in FIG. 1 . As shown in FIG. 4 , the method may include the following steps:

S401: Receive the first internal and external parameters of the driving recorder of the target vehicle sent by the in-vehicle terminal.

S402: According to the above-mentioned first internal and external parameters, update the HD Map to be updated, and the HD Map to be updated corresponds to the above-mentioned target vehicle.

Wherein, the above-mentioned first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the CAN bus of the target vehicle and the above-mentioned driving recorder, according to the internal parameter parameters of the above-mentioned driving recorder and the vehicle trajectory data in the above-mentioned data stream, determine the above-mentioned The corresponding relationship between the vehicle visual image in the data stream and the above-mentioned HD Map to be updated, and the first internal and external parameters obtained according to the corresponding relationship, wherein each frame of data in the above-mentioned data stream includes the above-mentioned vehicle trajectory data, and the above-mentioned vehicle trajectory data. The above-mentioned vehicle visual image corresponding to the vehicle trajectory.

In the embodiment of the present application, the on-board terminal obtains the internal and external parameters of the above-mentioned vehicle driving recorder based on the vehicle trajectory data of the vehicle and the vehicle visual image corresponding to the vehicle trajectory data, and sends the data of the driving recorder to the server, so that The server updates the HD Map of the above-mentioned vehicle based on the data of the above-mentioned driving recorder, which solves the problem that the existing HD Map is not updated in time, which leads to an incorrect understanding of the environmental information of the autonomous vehicle and causes great problems to the autonomous driving. question. Moreover, the embodiment of the present application performs HD Map update based on the driving recorder data, which can obtain data in a larger range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve the timeliness.

In addition, in the embodiment of the present application, when the above-mentioned server updates the above-mentioned HD Map to be updated according to the above-mentioned first internal and external parameters, it can determine the pose of the above-mentioned driving recorder in the HD Map coordinate system according to the above-mentioned first internal and external parameters, Furthermore, the target HD Map is constructed according to the above pose, so that the constructed HD Map is more accurate, so that the above-mentioned HD Map to be updated is updated according to the target HD Map, and the accuracy of the update result is improved. FIG. 5 is a schematic flowchart of another map updating method proposed by an embodiment of the present application. As shown in Figure 5, the method includes:

S501: Receive the first internal and external parameters of the driving recorder of the target vehicle sent by the vehicle-mounted terminal.

S502: Determine the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters.

Here, the server may decompose the inter-frame fundamental matrix of the driving recorder according to the internal parameter parameters in the first internal and external parameters to obtain an estimated value of the pose between frames, and then, according to the estimated value of the pose between frames, perform a The external parameter parameters in the first internal and external parameters are optimized to obtain the above pose.

Exemplarily, the above-mentioned server may perform inter-frame feature point matching on the consecutive frames of vehicle visual images obtained by the above-mentioned driving recorder through a scale-invariant feature transform (SIFT) and an optical flow method, and perform feature point matching between frames according to the epipolar geometry. The constraints are used to solve the inter-frame fundamental matrix of the inter-frame feature point matching results. Then, the basic matrix is decomposed through the internal parameter parameters in the above-mentioned first internal and external parameters to complete the pose estimation between frames. And take this as a constraint, optimize the external parameters in the above-mentioned first internal and external parameters, eliminate the influence of jumps, etc., and obtain the optimized absolute pose, that is, the pose of the above-mentioned driving recorder in the HD Map coordinate system, and then, According to the pose, the target HD Map is constructed to make the constructed HD Map more accurate.

S503: According to the above pose, construct a target HD Map.

In the embodiment of the present application, the above server may obtain the three-dimensional space coordinate value of each object feature point according to the above-mentioned pose, and then construct the target HD Map according to the three-dimensional space coordinate value of each object feature point.

Exemplarily, the above-mentioned server can obtain the three-dimensional space coordinate value of each object feature point according to the above-mentioned pose, complete the local mapping work, and make a difference with the above-mentioned HD Map result to be updated, find the changed part, and perform the above-mentioned HD Map to be updated. to update.

S504: According to the above target HD Map, update the HD Map to be updated, and the HD Map to be updated corresponds to the target vehicle.

Wherein, the above-mentioned first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the CAN bus of the target vehicle and the above-mentioned driving recorder, according to the internal parameter parameters of the above-mentioned driving recorder and the vehicle trajectory data in the above-mentioned data stream, determine the above-mentioned The corresponding relationship between the vehicle visual image in the data stream and the above-mentioned HD Map to be updated, and the first internal and external parameters obtained according to the corresponding relationship, wherein each frame of data in the above-mentioned data stream includes the above-mentioned vehicle trajectory data, and the above-mentioned vehicle trajectory data. The above-mentioned vehicle visual image corresponding to the vehicle trajectory.

In the embodiment of the present application, when updating the HD Map to be updated according to the first internal and external parameters, the server may determine the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters, and further, According to the above pose, the target HD Map is constructed, so that the constructed HD Map is more accurate. Therefore, according to the target HD Map, the above-mentioned HD Map to be updated is updated to improve the accuracy of the update result. Moreover, in the embodiment of the present application, the vehicle-mounted terminal obtains the internal and external parameters of the above-mentioned vehicle driving recorder based on the vehicle trajectory data of the vehicle and the vehicle visual image corresponding to the vehicle trajectory data, and sends the data of the driving recorder to the server. , so that the server updates the HD Map of the above vehicle based on the data of the above-mentioned driving recorder, which solves the problem that the existing HD Map is not updated in time, resulting in an incorrect understanding of the environmental information of the autonomous vehicle, which will cause greater damage to autonomous driving. problem problem. Moreover, the embodiment of the present application performs HD Map update based on the driving recorder data, which can obtain data in a larger range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve the timeliness.

In addition, the embodiment of the present application also provides another map update method, which is described from the interaction between the vehicle terminal and the server. As shown in FIG. 6 , the method may include:

S601: The vehicle-mounted terminal obtains a data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle. Each frame of data in the data stream includes vehicle trajectory data and a vehicle visual image corresponding to the vehicle trajectory.

Exemplarily, as shown in FIG. 7 , each frame of data (single frame data) in the above-mentioned data stream acquired by the vehicle-mounted terminal includes vehicle trajectory data and vehicle visual images corresponding to the vehicle trajectory. The above-mentioned vehicle-mounted terminal can obtain a certain range of HD Maps from the existing HD Map of the target vehicle according to the above-mentioned vehicle trajectory data as the above-mentioned HD Map to be updated, and can also calculate the two values according to the plane coordinate values of the vehicle trajectory data of the previous frame and the current frame. The yaw angle between the points is used as the initial heading angle of the current frame, so as to determine the current external parameter parameters of the target vehicle (the plane coordinate value, the initial heading angle, the pitch and roll angles are set to 0), and then combined with the driving The internal parameters of the recorder determine the second internal and external parameters of the driving recorder.

S602: The in-vehicle terminal determines the corresponding relationship between the above-mentioned vehicle visual image and the high-precision map to be updated according to the internal parameter parameters of the above-mentioned driving recorder and the above-mentioned vehicle trajectory data, wherein the to-be-updated HD Map corresponds to the above-mentioned target vehicle.

Here, the in-vehicle terminal can project the above-mentioned HD Map to be updated on the corresponding vehicle visual image according to the above-mentioned second internal and external parameters, obtain the two-dimensional coordinates of each element in the above-mentioned HD Map to be updated, and then combine the two-dimensional coordinates with the above-mentioned two-dimensional coordinates. The corresponding vehicle visual image is matched, and according to the matching result, the corresponding relationship between the above-mentioned vehicle visual image and the above-mentioned HD Map to be updated is determined. For example, as shown in Figure 7, the vehicle terminal can project the above-mentioned HD Map to be updated onto the vehicle visual image of the current frame of the driving recorder, transform the three-dimensional coordinates in the HD Map to be updated into two-dimensional coordinates on the image, and complete the to-be-updated The dimensionality reduction of HD Map data provides the possibility to match vehicle visual images and HD Map projection results. After the HD Map to be updated is reprojected, the above-mentioned vehicle terminal can project the two-dimensional coordinates (such as traffic signs and lane lines) of each element in the HD Map to be updated and the visual image of the vehicle according to their respective attributes, positions and aspect ratios, etc. The rules are matched to obtain the corresponding relationship between the vehicle visual image and the HD Map to be updated, and the 2D-3D initial matching between the vehicle visual image and the HD Map to be updated is completed.

In order to improve the accuracy of subsequent processing results, the vehicle-mounted terminal may optimize the correspondence between the vehicle visual image and the HD Map to be updated, that is, to further optimize the 2D-3D feature matching result between the vehicle visual image and the HD Map to be updated. Exemplarily, the above-mentioned vehicle-mounted terminal may combine the above-mentioned corresponding relationship (feature matching result), and use the optical flow method to perform object-level semantic matching on the images of the front and rear frames, determine the spatial correlation of the elements in the front and rear frames, and use this as a constraint, further. The correspondence between the above-mentioned vehicle visual image and the HD Map to be updated is to optimize the 2D-3D feature matching results.

S603: The in-vehicle terminal obtains the first internal and external parameters of the above-mentioned driving recorder according to the above-mentioned corresponding relationship.

Here, the vehicle-mounted terminal can obtain the projection matrix of each frame of the above-mentioned driving recorder according to the above-mentioned correspondence, obtain the estimated values of internal and external parameters of each frame of the above-mentioned driving recorder according to the above-mentioned projection matrix, and further, according to the initial frame of the above-mentioned driving recorder The estimated values of the internal and external parameters are optimized, and the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters of the above-mentioned driving recorder.

Exemplarily, as shown in Figure 7, the vehicle-mounted terminal can obtain the matching relationship pairs of all 2D image coordinates of the current frame and the corresponding spatial 3D coordinates according to the 2D-3D feature matching results of each frame, and use DLT to calculate the projection of each frame. Matrix, that is, the product of the internal parameter matrix and the rotation matrix, and then use RQ decomposition to decompose the projection matrix into two parts, the internal and external parameter matrices, to complete the frame-by-frame internal and external parameter estimation.

Based on the assumption that the optical properties of the same optical device do not change in a short time (that is, the internal parameters of the above-mentioned driving recorder do not change in a short time), the vehicle-mounted terminal can calculate the internal and external parameters of each frame according to the estimated values of the internal and external parameters of the initial frame in a sequence. The estimated value is optimized to obtain the above-mentioned first internal and external parameters.

S604: The vehicle-mounted terminal sends the first internal and external parameters to the server.

S605: The server receives the above-mentioned first internal and external parameters sent by the in-vehicle terminal.

S606: The server updates the above-mentioned HD Map to be updated according to the above-mentioned first internal and external parameters.

The server can determine the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters, build a target HD Map according to the above pose, and then update the to-be-updated HD Map according to the target HD Map. .

Exemplarily, as shown in FIG. 7 , the server may perform inter-frame feature point matching on consecutive frames of vehicle visual images obtained by the above-mentioned driving recorder through SIFT and optical flow methods, and match the inter-frame feature point matching results according to epipolar geometric constraints. Perform an inter-frame fundamental matrix solution. Then, the basic matrix is decomposed through the internal parameter parameters in the above-mentioned first internal and external parameters to complete the pose estimation between frames. And take this as a constraint, optimize the external parameters in the above-mentioned first internal and external parameters, eliminate the influence of jumps, etc., and obtain the optimized absolute pose, that is, the pose of the above-mentioned driving recorder in the HD Map coordinate system, and then, According to the pose, obtain the three-dimensional space coordinate value of each object feature point, complete the local mapping work, and make a difference with the above HD Map result to be updated, find the changed part, and update the above HD Map to be updated.

It can be seen from the above description that in the embodiment of the present application, the vehicle-mounted terminal obtains the internal and external parameters of the above-mentioned vehicle driving recorder based on the vehicle trajectory data of the vehicle and the vehicle visual image corresponding to the vehicle trajectory data, and converts the data of the driving recorder to the driving recorder. Send it to the server, so that the server can update the HD Map of the above vehicle based on the data of the above driving recorder, which solves the problem that the existing HD Map is not updated in time, which causes the autonomous vehicle to incorrectly understand the environmental information in which it is located. Problems that cause bigger problems. Moreover, the embodiment of the present application performs HD Map update based on the driving recorder data, which can obtain data in a larger range, improve the accuracy of HD Map update, reduce the cost of HD Map update, and improve the timeliness.

Corresponding to the map update method of the above embodiment, FIG. 8 is a schematic structural diagram of a map update apparatus provided by an embodiment of the present application. For the convenience of description, only the parts related to the embodiments of the present application are shown. FIG. 8 is a schematic structural diagram of a map updating apparatus provided by an embodiment of the present application. The map updating apparatus 80 includes: an obtaining module 801 , a determining module 802 , an obtaining module 803 and a sending module 804 . The map update device here may be the above-mentioned vehicle-mounted terminal itself, or a chip or integrated circuit that realizes the functions of the above-mentioned vehicle-mounted terminal. It should be noted here that the division of the acquisition module, the determination module, the acquisition module and the transmission module is only a division of logical functions, and the two may be physically integrated or independent.

Wherein, the acquisition module 801 is used to acquire the data stream of the target vehicle from the CAN bus and the driving recorder of the target vehicle, and each frame of data in the data stream includes vehicle trajectory data and corresponds to the vehicle trajectory. vehicle visual image.

A determination module 802, configured to determine the corresponding relationship between the vehicle visual image and the HD Map to be updated according to the internal parameter parameters of the driving recorder and the vehicle trajectory data, wherein the HD Map to be updated and the target vehicle correspond.

The obtaining module 803 is configured to obtain the first internal and external parameters of the driving recorder according to the corresponding relationship.

The sending module 804 is configured to send the first internal and external parameters to the server, so that the server updates the HD Map to be updated according to the first internal and external parameters.

In a possible implementation manner, the determining module 802 is specifically configured to:

According to the vehicle trajectory data, determine the HD Map to be updated corresponding to the target vehicle, and determine the second internal and external parameters of the driving recorder according to the internal parameter parameters of the driving recorder and the vehicle trajectory data;

According to the second internal and external parameters, the corresponding relationship between the vehicle visual image and the HD Map to be updated is determined.

In a possible implementation manner, the determining module 802 is specifically configured to:

According to the second internal and external parameters, the to-be-updated HD Map is projected onto the corresponding vehicle visual image, and the two-dimensional coordinates of each element in the to-be-updated HD Map are obtained;

matching the two-dimensional coordinates with the corresponding vehicle visual image;

The corresponding relationship is determined according to the matching result between the two-dimensional coordinates and the corresponding vehicle visual image.

In a possible implementation manner, the obtaining module 803 is specifically used for:

According to the corresponding relationship, obtain the projection matrix of each frame of the driving recorder;

Obtaining estimated values of internal and external parameters of each frame of the driving recorder according to the projection matrix;

According to the estimated values of the internal and external parameters of the initial frame of the driving recorder, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.

In a possible implementation manner, the obtaining module 803 is specifically used for:

According to the estimated value of the internal parameter in the estimated value of the internal and external parameters of the initial frame, project the HD Map to be updated on the corresponding visual image of the vehicle, and calculate the projection error;

The projection error is used as a loss function, and based on the loss function, the estimated values of the internal and external parameters of each frame are optimized to obtain the second internal and external parameters.

The apparatus provided in the embodiment of the present application can be used to implement the technical solution of the method embodiment described in FIG. 2 or FIG. 3 , and the implementation principle and technical effect thereof are similar, and are not described again in the embodiment of the present application.

FIG. 9 is a schematic structural diagram of another map updating apparatus provided by an embodiment of the present application. The map updating apparatus 90 includes a receiving module 901 and an updating module 902 . The map updating device here may be the above-mentioned server itself, or a chip or integrated circuit that implements the functions of the above-mentioned server. It should be noted here that the division of the receiving module and the updating module is only a division of logical functions, and the two may be physically integrated or independent.

The receiving module 901 is configured to receive the first internal and external parameters of the driving recorder of the target vehicle sent by the vehicle terminal.

The updating module 902 is configured to update the HD Map to be updated according to the first internal and external parameters, and the HD Map to be updated corresponds to the target vehicle.

Wherein, the first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the CAN bus of the target vehicle and the driving recorder, according to the internal parameter parameters of the driving recorder and the driving recorder Vehicle trajectory data in the data stream, determine the corresponding relationship between the vehicle visual image in the data stream and the high-precision map to be updated, and obtain the first internal and external parameters according to the corresponding relationship, wherein the data stream Each frame of data in includes the vehicle trajectory data and the vehicle visual image corresponding to the vehicle trajectory.

In a possible implementation manner, the update module 902 is specifically used for:

Determine the pose of the driving recorder in the HD Map coordinate system according to the first internal and external parameters;

According to the pose, construct the target HD Map;

According to the target HD Map, the HD Map to be updated is updated.

In a possible implementation manner, the update module 902 is specifically used for:

According to the internal parameter parameters in the first internal and external parameters, the inter-frame fundamental matrix of the driving recorder is decomposed to obtain the inter-frame pose estimation value;

According to the estimated value of the pose between frames, the external parameter parameters in the first internal and external parameters are optimized to obtain the pose.

In a possible implementation manner, the update module 902 is specifically used for:

According to the pose, obtain the three-dimensional space coordinate value of each object feature point;

According to the three-dimensional space coordinate value of each object feature point, the target HD Map is constructed.

The apparatus provided in the embodiment of the present application can be used to implement the technical solution of the method embodiment described in FIG. 4 or FIG. 5 , and the implementation principle and technical effect thereof are similar, and details are not described herein again in the embodiment of the present application.

Optionally, FIGS. 10A and 10B schematically provide a possible basic hardware architecture of the vehicle-mounted terminal and the server described in this application, respectively.

Referring to FIGS. 10A and 10B , the in-vehicle terminal and the server include at least one processor 1001 and a communication interface 1003 . Further optionally, a memory 1002 and a bus 1004 may also be included. The in-vehicle terminal further includes a sending device and a receiving device, and the sending device and the receiving device are respectively connected to the processor 1001 .

Wherein, in the vehicle-mounted terminal and the server, the number of processors 1001 may be one or more, and FIGS. 10A and 10B only illustrate one of the processors 1001 . Optionally, the processor 1001 may be a central processing unit (central processing unit, CPU), a graphics processing unit (graphics processing unit, GPU) or a digital signal processor (digital signal processor, DSP). If the in-vehicle terminal and the server have multiple processors 1001, the types of the multiple processors 1001 may be different, or may be the same. Optionally, multiple processors 1001 of the in-vehicle terminal and the server may also be integrated into a multi-core processor.

The memory 1002 stores computer instructions and data; the memory 1002 can store computer instructions and data required for implementing the above-mentioned map updating method provided by the present application, for example, the memory 1002 stores instructions for implementing the steps of the above-mentioned map updating method. The memory 1002 may be any one or any combination of the following storage media: non-volatile memory (eg read only memory (ROM), solid state drive (SSD), hard disk (HDD), optical disk), volatile memory.

Communication interface 1003 may provide information input/output for the at least one processor. It may also include any one or any combination of the following devices: a network interface (eg, an Ethernet interface), a wireless network card, and other devices with network access functions.

Optionally, the communication interface 1003 can also be used for the vehicle-mounted terminal and the server to perform data communication with other computing devices or vehicle-mounted terminals.

Further optionally, Figures 10A and 10B represent bus 1004 with a thick line. The bus 1004 may connect the processor 1001 with the memory 1002 and the communication interface 1003 . In this way, through the bus 1004, the processor 1001 can access the memory 1002, and can also use the communication interface 1003 to perform data interaction with other computing devices or vehicle-mounted terminals.

In the present application, the in-vehicle terminal and the server execute the computer instructions in the memory 1002, so that the in-vehicle terminal and the server implement the above-mentioned map updating method provided in this application, or make the in-vehicle terminal and the server deploy the above-mentioned map updating apparatus.

From the perspective of logical function division, exemplarily, as shown in FIG. 10A , the memory 1002 may include an obtaining module 801 , a determining module 802 , an obtaining module 803 and a sending module 804 . The inclusion here only refers to that when the instructions stored in the memory are executed, the functions of the acquiring module, the determining module, the acquiring module and the transmitting module can be implemented respectively, and are not limited to the physical structure.

A possible design, as shown in FIG. 10B , the memory 1002 includes a receiving module 901 and an updating module 902, and the inclusion here only involves that when the instructions stored in the memory are executed, the functions of the receiving module and the updating module can be implemented respectively, but not. Constraints are physical structures.

In addition, the above-mentioned map updating apparatus can be implemented as a hardware module, or as a circuit unit, in addition to being implemented by software as in the above-mentioned FIGS. 10A and 10B .

The present application provides a computer-readable storage medium, and the computer program product includes computer instructions, the computer instructions instructing a computing device to execute the above-mentioned map updating method provided by the present application.

The present application provides a computer program product, comprising computer instructions, the computer instructions are executed by a processor to execute the above-mentioned map updating method provided by the present application.

The present application provides a chip comprising at least one processor and a communication interface, wherein the communication interface provides information input and/or output for the at least one processor. Further, the chip may also include at least one memory for storing computer instructions. The at least one processor is configured to invoke and execute the computer instructions to execute the above-mentioned map updating method provided by the present application.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

Claims (13)

1. A method for updating a map, comprising:

   Obtain a data stream of the target vehicle from the controller area network bus and the driving recorder of the target vehicle, where each frame of data in the data stream includes vehicle trajectory data and a vehicle visual image corresponding to the vehicle trajectory;

   Determine the correspondence between the vehicle visual image and the high-precision map to be updated according to the internal reference parameters of the driving recorder and the vehicle trajectory data, wherein the high-precision map to be updated corresponds to the target vehicle;

   obtaining the first internal and external parameters of the driving recorder according to the corresponding relationship;

   The first internal and external parameters are sent to the server, so that the server updates the high-precision map to be updated according to the first internal and external parameters.
2. The method according to claim 1, wherein the determining the corresponding relationship between the vehicle visual image and the high-precision map to be updated according to the internal parameter parameters of the driving recorder and the vehicle trajectory data, comprising:

   According to the vehicle trajectory data, the high-precision map to be updated corresponding to the target vehicle is determined, and the second internal and external parameters of the driving recorder are determined according to the internal parameter parameters of the driving recorder and the vehicle trajectory data. ;

   According to the second internal and external parameters, the corresponding relationship between the vehicle visual image and the high-precision map to be updated is determined.
3. The method according to claim 2, wherein the determining the corresponding relationship between the vehicle visual image and the high-precision map to be updated according to the second internal and external parameters comprises:

   According to the second internal and external parameters, project the high-precision map to be updated on the corresponding vehicle visual image, and obtain the two-dimensional coordinates of each element in the high-precision map to be updated;

   matching the two-dimensional coordinates with the corresponding vehicle visual image;

   The corresponding relationship is determined according to the matching result between the two-dimensional coordinates and the corresponding vehicle visual image.
4. The method according to any one of claims 1 to 3, wherein the obtaining the first internal and external parameters of the driving recorder according to the corresponding relationship comprises:

   According to the corresponding relationship, obtain the projection matrix of each frame of the driving recorder;

   Obtaining estimated values of internal and external parameters of each frame of the driving recorder according to the projection matrix;

   According to the estimated values of the internal and external parameters of the initial frame of the driving recorder, the estimated values of the internal and external parameters of each frame are optimized to obtain the first internal and external parameters.
5. A method for updating a map, comprising:

   Receive the first internal and external parameters of the driving recorder of the target vehicle sent by the vehicle terminal;

   According to the first internal and external parameters, the high-precision map to be updated is updated, and the high-precision map to be updated corresponds to the target vehicle;

   Wherein, the first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the controller area network bus of the target vehicle and the driving recorder, according to the internal parameter parameters of the driving recorder and the vehicle trajectory data in the data stream, determine the corresponding relationship between the vehicle visual image in the data stream and the high-precision map to be updated, and obtain the first internal and external parameters according to the corresponding relationship, wherein the Each frame of data in the data stream includes the vehicle trajectory data and the vehicle visual image corresponding to the vehicle trajectory.
6. The method according to claim 5, wherein the updating the high-precision map to be updated according to the first internal and external parameters comprises:

   Determine the pose of the driving recorder in the high-precision map coordinate system according to the first internal and external parameters;

   According to the pose, construct a high-precision map of the target;

   The high-precision map to be updated is updated according to the target high-precision map.
7. A map updating device, comprising:

   The acquisition module is used to acquire the data stream of the target vehicle from the controller area network bus and the driving recorder of the target vehicle, and each frame of data in the data stream includes vehicle trajectory data and corresponds to the vehicle trajectory vehicle visual image;

   A determination module, configured to determine the corresponding relationship between the vehicle visual image and the high-precision map to be updated according to the internal parameter parameters of the driving recorder and the vehicle trajectory data, wherein the high-precision map to be updated and the target vehicle correspondence;

   an obtaining module for obtaining the first internal and external parameters of the driving recorder according to the corresponding relationship;

   A sending module, configured to send the first internal and external parameters to a server, so that the server updates the high-precision map to be updated according to the first internal and external parameters.
8. A map updating device, comprising:

   The receiving module is used to receive the first internal and external parameters of the driving recorder of the target vehicle sent by the vehicle terminal;

   an update module, configured to update the high-precision map to be updated according to the first internal and external parameters, and the high-precision map to be updated corresponds to the target vehicle;

   Wherein, the first internal and external parameter is that after the on-board terminal obtains the data stream of the target vehicle from the controller area network bus of the target vehicle and the driving recorder, according to the internal parameter parameters of the driving recorder and the vehicle trajectory data in the data stream, determine the corresponding relationship between the vehicle visual image in the data stream and the high-precision map to be updated, and obtain the first internal and external parameters according to the corresponding relationship, wherein the Each frame of data in the data stream includes the vehicle trajectory data and the vehicle visual image corresponding to the vehicle trajectory.
9. A map update system, characterized in that it includes:

   The vehicle-mounted terminal is used to obtain the data stream of the target vehicle from the controller area network bus and the driving recorder of the target vehicle, and each frame of data in the data stream includes vehicle trajectory data and corresponds to the vehicle trajectory the vehicle visual image; determine the corresponding relationship between the vehicle visual image and the high-precision map to be updated according to the internal parameters of the driving recorder and the vehicle trajectory data, wherein the high-precision map to be updated and the target vehicle correspondence; obtain the first internal and external parameters of the driving recorder according to the corresponding relationship; send the first internal and external parameters to the server;

   The server is configured to update the high-precision map to be updated according to the first internal and external parameters.
10. A vehicle-mounted terminal, comprising:

    Sending devices, receiving devices, processors, memories, and computer programs;

    Wherein, the transmitting device and the receiving device are respectively connected with the processor, the computer program is stored in the memory and configured to be executed by the processor, the computer program includes a program for executing Instructions for the method of any of claims 1-4.
11. A server, characterized in that it includes:

    processor;

    memory; and

    Computer program;

    wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of claim 5 or 6.
12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program enables the vehicle-mounted terminal to execute the method of any one of claims 1-4, or the computer program The server is caused to perform the method of claim 5 or 6.
13. A computer program, characterized in that it includes program code, and when a computer runs the computer program, the program code executes the method according to any one of claims 1-4, or the program code executes the method according to claim 1 The method described in 5 or 6.

Images