CN117252997A

CN117252997A - Road three-dimensional reconstruction method, device, equipment and readable storage medium

Info

Publication number: CN117252997A
Application number: CN202210657162.7A
Authority: CN
Inventors: 刘坤
Original assignee: Beijing Co Wheels Technology Co Ltd
Current assignee: Beijing Co Wheels Technology Co Ltd
Priority date: 2022-06-10
Filing date: 2022-06-10
Publication date: 2023-12-19

Abstract

The present disclosure relates to a road three-dimensional reconstruction method, apparatus, device, and readable storage medium. According to the method, the pose of a plurality of associated images of a target road to be rebuilt and the pose of each associated image shot in a world coordinate system are obtained, a three-dimensional rebuilding model of the sub road section in the image coordinate system and the pose of a camera in the three-dimensional rebuilding model when each associated image is shot are determined, the three-dimensional rebuilding models of different sub road sections are converted into the pose conversion relations in the world coordinate system, and then the three-dimensional model of the target road in the world coordinate system is rebuilt based on the pose conversion relations and the three-dimensional rebuilding models of different sub road sections. Because three-dimensional reconstruction is firstly carried out on each sub-road section, and then the pose of the three-dimensional reconstruction model of each sub-road section under the world coordinate system is adjusted, the adjusted pose is more in line with the pose of the sub-road section in the target road section, the reconstructed three-dimensional model of the target road under the world coordinate system is more accurate, and the effect of three-dimensional reconstruction of the road can be improved.

Description

Road three-dimensional reconstruction method, device, equipment and readable storage medium

Technical Field

The disclosure relates to the field of image processing, and in particular relates to a road three-dimensional reconstruction method, device and equipment and a readable storage medium.

Background

In an autopilot scenario, a three-dimensional reconstruction of the road environment around the vehicle is required.

At present, a three-dimensional reconstruction technology of image feature points based on a vehicle-mounted camera generally collects an image sequence through an image collection device, the image sequence is subjected to three-dimensional modeling to generate a visual point cloud, and the visual point cloud can be fused with a high-precision map to complete three-dimensional road reconstruction.

However, the three-dimensional road reconstructed by the three-dimensional reconstruction technology based on the image feature points of the vehicle-mounted camera has poor effect, and how to adjust the pose of the three-dimensional reconstruction model of each sub-road segment under the world coordinate system by utilizing the pose of each sub-road segment under the image coordinate system, so that the effect of three-dimensional reconstruction of the road is improved.

Disclosure of Invention

In order to solve the technical problems, the disclosure provides a road three-dimensional reconstruction method, a device, equipment and a readable storage medium.

In a first aspect, an embodiment of the present disclosure provides a road three-dimensional reconstruction method, the method including:

acquiring a plurality of associated images of a target road to be reconstructed and the pose of each associated image under a world coordinate system when the associated images are shot;

Determining a three-dimensional reconstruction model of the sub-road section under an image coordinate system and the pose of a camera in the three-dimensional reconstruction model when each associated image is shot on the basis of the plurality of associated images aiming at any sub-road section included in the target road;

for any sub-road section included in the target road, according to the pose of each associated image under the world coordinate system of the camera and the pose of each associated image in the image coordinate system of the three-dimensional reconstruction model of the different sub-road section when the associated image is shot, obtaining the coordinate corresponding relation of the any sub-road section under the image coordinate system and the world coordinate system, and determining the pose conversion relation of the image coordinate system of the three-dimensional reconstruction model of the different sub-road section under the world coordinate system according to the coordinate corresponding relation of the any sub-road section under the image coordinate system and the world coordinate system;

and reconstructing the three-dimensional model of the target road under the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system.

In some embodiments, the acquiring a plurality of associated images of the target road to be reconstructed and a pose of each of the associated images in the world coordinate system of the camera includes:

acquiring an image sequence of the target road shot by the camera and positioning information of the camera when each image in the image sequence is shot;

and determining a plurality of associated images and the pose of each camera in the world coordinate system when the associated images are shot from the image sequence based on the positioning information of the camera when each image in the image sequence is shot.

In some embodiments, before determining a three-dimensional reconstruction model of any sub-road segment included in the target road in the image coordinate system based on the plurality of associated images, the method further includes:

the target road is divided into a plurality of sub-links based on the plurality of associated images.

In some embodiments, for any sub-road section included in the target road, according to a pose of each associated image in a world coordinate system of a camera when the associated image is captured and a pose of each associated image in an image coordinate system where a three-dimensional reconstruction model of a different sub-road section is located, a coordinate corresponding relation of the any sub-road section in the image coordinate system and the world coordinate system is obtained, and according to a coordinate corresponding relation of the any sub-road section in the image coordinate system and the world coordinate system, a pose conversion relation of the image coordinate system where the three-dimensional reconstruction model of the different sub-road section is located is converted into the world coordinate system, including:

Converting the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the sub-road section when each associated image is shot into the world coordinate system based on the initial pose conversion relation of the pose of the three-dimensional reconstruction model of the sub-road section in the image coordinate system into the world coordinate system;

and based on the pose of each associated image in the world coordinate system of the camera, the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections included in the same associated image, and the pose of the camera in the world coordinate system, the coefficients of the initial pose conversion relationship of the pose conversion of the three-dimensional reconstruction model of different sub-road sections in the image coordinate system to the world coordinate system are adjusted, so that the pose conversion relationship of the image coordinate system of the three-dimensional reconstruction model of different sub-road sections to the world coordinate system is obtained.

In some embodiments, for any sub-road section included in the target road, according to a pose of each associated image in a world coordinate system of a camera when the associated image is captured and a pose of each associated image in an image coordinate system where a three-dimensional reconstruction model of a different sub-road section is located, a coordinate corresponding relation of the any sub-road section in the image coordinate system and the world coordinate system is obtained, and according to a coordinate corresponding relation of the any sub-road section in the image coordinate system and the world coordinate system, determining a pose conversion relation of converting the image coordinate system where the three-dimensional reconstruction model of the different sub-road section is located into the world coordinate system includes:

According to the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, converting the pose of the camera in the world coordinate system and the pose of the camera in the world coordinate system when each associated image is shot, and constructing a first constraint relation;

according to the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of each of the two sub-road sections included in the image when any one of the associated images is shot, converting the pose into the pose under the world coordinate system, and constructing a second constraint relation;

and determining pose conversion relations from the image coordinate system of the three-dimensional reconstruction model of different sub-road sections to the world coordinate system based on the first constraint relation and the second constraint relation.

In some embodiments, after determining the pose conversion relationship of the image coordinate system of the three-dimensional reconstruction model of the different sub-road segments to the pose conversion relationship of the world coordinate system based on the pose of the camera in the world coordinate system when each of the associated images is captured and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the different sub-road segments when each of the associated images is captured, the method further includes:

Determining an initial scale conversion relation from an image coordinate system of the three-dimensional reconstruction model of the different sub-road sections to a world coordinate system;

the three-dimensional model of the target road under the world coordinate system is reconstructed based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system and the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system, comprising:

and reconstructing the three-dimensional model of the target road under the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system, the initial scale conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system, and the pose and scale of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system.

In some embodiments, the reconstructing the three-dimensional model of the target road in the world coordinate system based on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road sections in the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system, includes:

Converting the three-dimensional reconstruction models of different sub-road sections into a world coordinate system based on the pose conversion relation of the three-dimensional reconstruction models of the different sub-road sections into the world coordinate system, so as to obtain the poses of the three-dimensional reconstruction models of different sub-road sections in the world coordinate system;

and merging the three-dimensional reconstruction models of the different sub-road sections based on the pose of the three-dimensional reconstruction models of the different sub-road sections under the world coordinate system to obtain the three-dimensional model of the target road under the world coordinate system.

In a second aspect, an embodiment of the present disclosure provides a road three-dimensional reconstruction apparatus, including:

the acquisition module is used for acquiring a plurality of associated images of a target road to be reconstructed and the pose of each associated image under a world coordinate system when the associated images are shot;

the first determining module is used for determining a three-dimensional reconstruction model of any sub-road section included in the target road under an image coordinate system and the pose of a camera in the three-dimensional reconstruction model when each associated image is shot based on the plurality of associated images;

the second determining module is used for obtaining the coordinate corresponding relation of any sub-road section under the image coordinate system and the world coordinate system according to the pose of each associated image under the world coordinate system of the camera when being shot and the pose of each associated image in the image coordinate system of the three-dimensional reconstruction model of the different sub-road section, and determining the pose conversion relation of the image coordinate system of the three-dimensional reconstruction model of the different sub-road section under the world coordinate system according to the coordinate corresponding relation of the any sub-road section under the image coordinate system and the world coordinate system;

The reconstruction module is used for reconstructing the three-dimensional model of the target road under the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

a memory;

a processor; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method according to the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium having stored thereon a computer program for execution by a processor to implement the method of the first aspect.

In a fifth aspect, embodiments of the present disclosure provide a vehicle comprising:

a memory;

a processor; and

a computer program;

In a sixth aspect, the presently disclosed embodiments also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement the method as described in the first aspect.

According to the road three-dimensional reconstruction method, the device, the equipment and the readable storage medium, the pose of a camera in a three-dimensional reconstruction model of a sub-road section in an image coordinate system is determined according to the multiple associated images for any sub-road section included in a target road by acquiring the multiple associated images of the target road to be reconstructed and the pose of each associated image in the world coordinate system when the images are shot. Further, based on the pose of each associated image in the world coordinate system of the camera and the pose of each associated image in the three-dimensional reconstruction model of the camera in different sub-road sections when the associated images are shot, the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections converted into the world coordinate system is determined, and based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections converted into the world coordinate system and the three-dimensional reconstruction model of the different sub-road sections, the three-dimensional model of the target road in the world coordinate system is reconstructed. Because three-dimensional reconstruction is carried out on each sub-road section, and then the pose of the three-dimensional reconstruction model of each sub-road section under the world coordinate system is adjusted, the adjusted pose is more in line with the pose of the sub-road section in the target road section, the reconstructed three-dimensional model of the target road under the world coordinate system is more accurate, the effect of three-dimensional reconstruction of the road can be improved, and the three-dimensional reconstruction model is applied to a high-precision map and better assists in automatic driving.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1 is a flowchart of a road three-dimensional reconstruction method provided by an embodiment of the present disclosure;

FIG. 2 is a flowchart of a road three-dimensional reconstruction method according to another embodiment of the present disclosure;

FIG. 3 is a flowchart of a road three-dimensional reconstruction method according to another embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of a road three-dimensional reconstruction device according to an embodiment of the disclosure;

fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.

At present, the three-dimensional road reconstructed by the three-dimensional reconstruction technology based on the image feature points of the vehicle-mounted camera has poor effect. In view of this problem, embodiments of the present disclosure provide a road three-dimensional reconstruction method, which is described below with reference to specific embodiments.

Fig. 1 is a flowchart of a road three-dimensional reconstruction method provided in an embodiment of the disclosure. As shown in fig. 1, the method comprises the following steps:

s101, acquiring a plurality of associated images of a target road to be reconstructed and the pose of each associated image under a world coordinate system when the associated images are shot.

The terminal acquires driving data of a target road to be rebuilt, wherein the driving data comprises a road image shot by a camera and positioning information of the camera when the road image is shot. And further acquiring a plurality of associated images of the target road to be reconstructed and the pose of each associated image under the world coordinate system when the associated images are shot.

Optionally, the definition of the associated image is: the two images are on the same path at a distance and orientation angle close enough, for example less than 20m and less than 30 °. The case of non-associated pictures is: the two images are too far apart; an image of a main road and an image of an auxiliary road; one upstream image and one downstream image; one image on the bridge and one image under the bridge.

S102, determining a three-dimensional reconstruction model of the sub-road section under an image coordinate system and the pose of a camera in the three-dimensional reconstruction model when each associated image is shot according to any sub-road section included in the target road based on the plurality of associated images.

For any sub-road section included in the target road, a three-dimensional reconstruction model of the sub-road section under an image coordinate system and the pose of a camera in the three-dimensional reconstruction model when each associated image is photographed can be determined based on the plurality of associated images. The pose information of the camera may contain six-degree-of-freedom (the object has six degrees of freedom in space, i.e., a degree of freedom of movement in the directions of three rectangular coordinate axes of x, y, z and a degree of freedom of rotation about these three coordinate axes) information of the camera, i.e., three-dimensional coordinates of the camera and three rotation angle data.

Optionally, each sub-section is reconstructed in three dimensions by means of recovering structures (Structure from motion, SFM) from motion based on a plurality of associated images. There are many open source software available in the SFM field, such as one three-dimensional reconstruction software (colmap), another three-dimensional reconstruction software (openSfM), etc.

S103, for any sub-road section included in the target road, according to the pose of each associated image under the world coordinate system of the camera and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, obtaining the coordinate corresponding relation of any sub-road section under the image coordinate system and the world coordinate system, and according to the coordinate corresponding relation of any sub-road section under the image coordinate system and the world coordinate system, determining the pose conversion relation of the three-dimensional reconstruction model of different sub-road sections under the world coordinate system.

The terminal determines a pose conversion relation from an image coordinate system of the three-dimensional reconstruction model of different sub-road sections to a world coordinate system based on the pose of each associated image in the world coordinate system of a camera when the associated images are shot and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when the associated images are shot. The pose conversion relation between the image coordinate system of the three-dimensional reconstruction model of the sub-road section and the world coordinate system can be determined through the pose of the camera under the world coordinate system and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the sub-road section.

S104, reconstructing the three-dimensional model of the target road under the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system.

After determining the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system, the terminal converts the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections in the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system, and reconstructs the three-dimensional model of the target road in the world coordinate system. Firstly, three-dimensional reconstruction is carried out on each sub-road section, then the pose of each sub-road section is adjusted, and a three-dimensional model of the target road under the world coordinate system is reconstructed, so that the reconstructed road three-dimensional model is more accurate.

According to the method and the device, the multiple associated images of the target road to be rebuilt and the pose of each associated image under the world coordinate system of the camera are obtained, and for any sub-road section included in the target road, the three-dimensional rebuilding model of the sub-road section under the image coordinate system and the pose of the camera in the three-dimensional rebuilding model when each associated image is shot are determined based on the multiple associated images. Further, based on the pose of each associated image in the world coordinate system of the camera and the pose of each associated image in the three-dimensional reconstruction model of the camera in different sub-road sections when the associated images are shot, the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections converted into the world coordinate system is determined, and based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections converted into the world coordinate system and the three-dimensional reconstruction model of the different sub-road sections, the three-dimensional model of the target road in the world coordinate system is reconstructed. Because three-dimensional reconstruction is carried out on each sub-road section, and then the pose of the three-dimensional reconstruction model of each sub-road section under the world coordinate system is adjusted, the adjusted pose is more in line with the pose of the sub-road section in the target road section, the reconstructed three-dimensional model of the target road under the world coordinate system is more accurate, the effect of three-dimensional reconstruction of the road can be improved, and the three-dimensional reconstruction model is applied to a high-precision map and better assists in automatic driving.

Fig. 2 is a flowchart of a road three-dimensional reconstruction method according to another embodiment of the present disclosure, as shown in fig. 2, the method includes the following steps:

s201, acquiring an image sequence of a road three-dimensional reconstruction target road shot by a road three-dimensional reconstruction camera and positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot.

Optionally, the terminal acquires driving data of the target road to be reconstructed, where the driving data includes a road image captured by the camera and positioning information of the camera when the road image is captured. And further acquiring an image sequence of the road three-dimensional reconstruction target road shot by the camera and positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot from the driving data.

S202, determining a plurality of associated images and the pose of each road three-dimensional reconstruction associated image in a world coordinate system based on the positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot.

After the image sequence of the road three-dimensional reconstruction target road shot by the camera and the positioning information, such as GPS information, of the camera when each image in the road three-dimensional reconstruction image sequence is shot are obtained, a plurality of associated images are determined from the road three-dimensional reconstruction image sequence, and the pose of the camera under the world coordinate system when each road three-dimensional reconstruction associated image is shot is determined based on the positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot.

Optionally, acquiring driving data in an area, where the driving data includes: image sequence of the vehicle-mounted camera with a time stamp; the GPS coordinates with the time stamps may be considered to be aligned for simplicity of discussion, and the GPS coordinates are those of the camera; each image has a segmentation result, and the type of a vehicle body drivable area (freespace) in the segmentation result is mainly used; and calibrating results such as internal parameters of the camera, distortion coefficients, external parameters between the camera and the vehicle body and the like.

And judging the association method: 1) If the two images are adjacent frames of the driving image, the two images are necessarily related to each other (eliminating error association operation is not needed); 2) If the GPS coordinate horizontal distance of the two images is less than 20m, the height distance is less than 5m, and the direction angle is less than 30 degrees, the two pictures are correlated (error correlation is easily generated, and a rejection operation is required).

In the case of a mis-association, for example, when one image is on the main road and one image is on the auxiliary road, the two images should not be associated, but because of the proximity of the GPS coordinates, a mis-association is likely to occur.

The method for eliminating the error association comprises the following steps: 1) And projecting the related AB two images, namely the position of the vehicle body corresponding to the B image into the pixel coordinate system of the A image, and if the B vehicle body is not in the freespace of the A, the two images are in error association. Otherwise, reversely judging again, and judging again, wherein the two times of judgment pass through to be considered as correct association. 2) And extracting characteristic points in the two images, matching, calculating the number of the successfully matched characteristic points, and if the number of the successfully matched characteristic points is less than a certain threshold value, considering the two images as being in error association.

S203, based on the three-dimensional reconstruction of the road, dividing the three-dimensional reconstruction target road into a plurality of sub-road segments.

The target road is divided according to the plurality of associated images, the target road can be divided into a plurality of sub-road sections, and an overlapping area exists between the sub-road sections.

Optionally, a whole picture association diagram is established: regarding each picture as a vertex (vertex) of a graph structure, connecting the vertices of the associated pictures by using edges (edge), thus obtaining a whole picture associated graph: g _all 。

Extracting a connected subgraph: extracting all connected pictures in the whole picture association pictures: g _all ->G ₁ ,G ₂ ,G ₃ .. it should be noted that, because one association graph may include uplink and downlink data of a road, and there is no association between the two data, the overall association graph may be divided into two communication graphs.

Extracting a 100m range subgraph with overlapping regions: for each connected graph, the result of the segmentation is a sub-graph with an overlap region in the range of 100 m:the specific segmentation method comprises the following steps: 1) Extracting the connected graph G ₁ Forms a seed set S ₁ The method comprises the steps of carrying out a first treatment on the surface of the 2) Taking a point from the seed set, taking the point as a starting point, and connecting the point with the graph G ₁ Finding out all the vertexes with the sum of the side lengths smaller than 50m from the point, and forming a sub-graph by the vertexes with the range of 50m and seed points>3) From seed set S, all vertices in the subgraph ₁ Delete in the middle; 4) Repeating the steps 2-3 untilTo seed set S ₁ For null, each sub-graph obtained represents a sub-segment. Based on this, the road three-dimensional reconstruction target road is divided into a plurality of sub-links.

S204, converting the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the sub-road section when each associated image is shot into the world coordinate system based on the initial pose conversion relation of the pose conversion of the three-dimensional reconstruction model of the sub-road section in the image coordinate system into the world coordinate system aiming at any sub-road section included in the target road.

For each sub-road section, according to the pose conversion relation between the image coordinate system of the three-dimensional reconstruction model of the sub-road section and the world coordinate system, the pose of the camera in the three-dimensional reconstruction model of the sub-road section when the three-dimensional reconstruction related image of each road is shot can be converted into the world coordinate system.

S205, based on the pose of each associated image in the world coordinate system of the camera, the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections included in the same associated image, the pose of the camera in the world coordinate system, and the coefficients of the initial pose conversion relationship of the pose of the three-dimensional reconstruction model of different sub-road sections in the image coordinate system, in which the pose of the three-dimensional reconstruction model of different sub-road sections is converted, are adjusted, so that the pose conversion relationship of the image coordinate system of the three-dimensional reconstruction model of different sub-road sections, in which the pose of the three-dimensional reconstruction model of different sub-road sections is converted, in the world coordinate system, is obtained.

Based on the pose of each road three-dimensional reconstruction related image under the world coordinate system of a camera, and the pose of each road three-dimensional reconstruction related image under the world coordinate system of a camera when being shot, the pose of each road three-dimensional reconstruction related image in the three-dimensional reconstruction model of different sub-road segments is converted into the pose under the world coordinate system, the pose of the sub-road segment three-dimensional reconstruction model is optimized, and then the initial pose conversion relation of the three-dimensional reconstruction model of each sub-road segment under the world coordinate system is optimized. For example, the initial pose conversion relation is y=ax, and the pose of the three-dimensional reconstruction model of different sub-road segments in the image coordinate system is converted intoAdjusting coefficients of initial pose conversion relations under the world coordinate system to obtain the pose conversion relations of y=a, wherein the pose conversion relations of the image coordinate system of the three-dimensional reconstruction model of different sub-road sections under the world coordinate system are converted into the pose conversion relations under the world coordinate system ₀ And x, because an overlapping area exists between the sub-road sections, an overlapping part also exists between the constructed sub-road section three-dimensional reconstruction model and the sub-road section three-dimensional reconstruction model. There is a same associated image that includes both sub-segment a and sub-segment B. According to the constraint relation, the pose of the camera in the three-dimensional reconstruction model of the sub-road section A when the associated image is shot and the pose of the camera in the three-dimensional reconstruction model of the sub-road section B when the associated image is shot are the same, the pose of the three-dimensional reconstruction model of the sub-road section is optimized based on the pose, and then the initial pose conversion relation of the three-dimensional reconstruction model of each sub-road section to the world coordinate system is optimized, and the pose conversion relation of the three-dimensional reconstruction model of different sub-road sections to the world coordinate system is obtained through twice optimization.

S206, reconstructing the three-dimensional model of the target road under the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system.

Specifically, the implementation process and principle of S206 and S104 are consistent, and will not be described herein.

According to the method, the image sequence of the road three-dimensional reconstruction target road shot by the road three-dimensional reconstruction camera and the positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot are obtained, based on the positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot, the pose of a plurality of associated images and the pose of a camera when each road three-dimensional reconstruction associated image is shot in a world coordinate system are determined from the road three-dimensional reconstruction image sequence, and based on the road three-dimensional reconstruction of the plurality of associated images, the road three-dimensional reconstruction target road is divided into a plurality of sub-road sections. Further, for any sub-road section included in the road three-dimensional reconstruction target road, based on an initial pose conversion relation that the pose of the three-dimensional reconstruction model of the sub-road section is converted into a world coordinate system, the pose of the camera in the three-dimensional reconstruction model of the sub-road section when each road three-dimensional reconstruction associated image is shot is converted into the world coordinate system. And then based on the pose of each road three-dimensional reconstruction related image under the world coordinate system of a camera, the pose of each road three-dimensional reconstruction related image in the three-dimensional reconstruction model of different sub-road sections is converted into the pose under the world coordinate system when the three-dimensional reconstruction related image is shot, and the pose of the camera in the three-dimensional reconstruction model of different sub-road sections included in the same road three-dimensional reconstruction related image is converted into the pose under the world coordinate system, optimizing the initial pose conversion relation of the pose conversion of the three-dimensional reconstruction model of different sub-road sections into the world coordinate system, and obtaining the pose conversion relation of the three-dimensional reconstruction model of different sub-road sections into the world coordinate system. And then, based on the three-dimensional reconstruction model of the different sub-road sections, the three-dimensional reconstruction model of the road three-dimensional reconstruction target road under the world coordinate system is converted into the pose conversion relation under the world coordinate system and the three-dimensional reconstruction model of the different sub-road sections. Because three-dimensional reconstruction is carried out on each sub-road section, and then the pose of the three-dimensional reconstruction model of the sub-road section is optimized, the pose conversion relation of the three-dimensional reconstruction model of different sub-road sections under the world coordinate system is obtained, then the pose of the three-dimensional reconstruction model of each sub-road section under the world coordinate system is adjusted, and the adjusted pose is more in line with the pose of the sub-road section in the target road section, the reconstructed three-dimensional model of the target road under the world coordinate system is more accurate, the effect of three-dimensional reconstruction of the road can be improved, and the three-dimensional reconstruction method is applied to a high-precision map and better assists in automatic driving.

In some embodiments, determining the pose conversion relationship of converting the three-dimensional reconstruction model of different sub-road segments into the world coordinate system based on the pose of each road three-dimensional reconstruction-related image in the world coordinate system of the camera and the pose of each road three-dimensional reconstruction-related image in the three-dimensional reconstruction model of different sub-road segments when being photographed, includes, but is not limited to, the following steps:

s1001, according to the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, converting the pose of the camera in the world coordinate system into the pose of the camera in the world coordinate system, and constructing a first constraint relation when each associated image is shot.

The first constraint relation represents the absolute constraint of the GPS, theoretically, the pose of the camera in the image coordinate system where the three-dimensional reconstruction model of the ith sub-road section is located when the image is shot is converted into the world coordinate system, and the pose of the camera in the world coordinate system when the image containing the real GPS information is shot is the same as the pose of the camera in the world coordinate system, so that the smaller the residual error is, the better the residual error is.

S1002, converting the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of each of two sub-road sections included in any associated image when the image is shot into the pose under the world coordinate system, and constructing a second constraint relation.

The second constraint relation represents the relative constraint of the overlapping area of the two sub-sections, in theory, the pose of the camera when the same picture is shot is converted into the pose under the world coordinate system through the pose in the image coordinate system where the three-dimensional reconstruction models of the two sub-sections are located, so that the smaller the residual error is, the better.

S1003, determining a pose conversion relation from an image coordinate system where the three-dimensional reconstruction model of different sub-road sections is located to a world coordinate system based on the first constraint relation and the second constraint relation.

And determining the pose conversion relation of the image coordinate system of the three-dimensional reconstruction model of the different sub-road sections to the world coordinate system according to the absolute constraint of the GPS and the relative constraint of the overlapping area of the two sub-road sections.

In some embodiments, the minimum value of the objective function is solved, and the pose conversion relation of the image coordinate system where the three-dimensional reconstruction model of different sub-road segments is located is obtained, wherein the pose conversion relation is converted into the world coordinate system, and the three-dimensional reconstruction objective function of the road is as follows:

wherein,representing the absolute constraint of GPS, theoretically, the pose of the camera in the three-dimensional reconstruction model of the ith sub-link when taking an image is converted into the world coordinate system, and the pose of the camera in the world coordinate system should be the same as that of the camera when the image containing real GPS information is taken, so the smaller the residual is, the better. / >For the pose of the camera in the three-dimensional reconstruction model of the ith sub-section when the kth relevant image was recorded,/->The representation is based on S _i Will->Conversion to pose under world coordinate System, S _i Converting the pose of the three-dimensional reconstruction model of the ith sub-road section into an initial pose conversion relation under a world coordinate system; />The pose of a camera under a world coordinate system is taken for the kth associated image;

wherein,representing the relative constraint of the overlapping areas of the two sub-sections, theoretically, the pose of the camera when the same picture is taken under the world coordinate system by the pose conversion in the three-dimensional reconstruction model of the two sub-sections should be the same, so the smaller the residual is, the better. Wherein (1)>The representation is based on S _j Will->Transition toPose in world coordinate system +.>For the pose of the camera in the three-dimensional reconstruction model of the jth sub-road section when the kth related image is captured, S _j Converting the pose of the three-dimensional reconstruction model of the jth sub-road section into an initial pose conversion relation under a world coordinate system; the ith sub-road segment and the jth sub-road segment are different sub-road segments included in the same associated image.

In some embodiments, after determining the pose conversion relationship of the image coordinate system of the three-dimensional reconstruction model of the different sub-road segments to the pose conversion relationship of the world coordinate system based on the pose of the camera under the world coordinate system when each associated image is captured and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the different sub-road segments when each associated image is captured, the method further comprises: and determining an initial scale conversion relation of converting an image coordinate system where the three-dimensional reconstruction model of different sub-road sections is positioned into a world coordinate system.

The initial scale conversion relationship is a proportional relationship between the three-dimensional reconstruction model of different sub-road segments and the sub-road segments in the world coordinate system, for example, the proportional relationship between the three-dimensional reconstruction model of the sub-road segments and the sub-road segments is 1:10.

In some embodiments, reconstructing the three-dimensional model of the target road in the world coordinate system based on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system to the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system, including: based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system, the initial scale conversion relation of the three-dimensional reconstruction model of the different sub-road sections in the world coordinate system, and the pose and scale of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system, the three-dimensional model of the target road in the world coordinate system is reconstructed.

Optionally, S _i The method also comprises the step of converting the three-dimensional reconstruction model of the ith sub-road section into an initial scale conversion relation under a world coordinate system; s is S _j The method also comprises the step of converting the three-dimensional reconstruction model of the jth sub-road section into an initial scale conversion relation under a world coordinate system.

According to the method, the target function is constructed, the target function is solved for zero through Gaussian Newton method or other methods, the pose and the scale of the three-dimensional reconstruction model of each sub-road section are obtained, the reconstructed road three-dimensional model is more accurate, and the road three-dimensional reconstruction effect is further improved.

Fig. 3 is a flowchart of a road three-dimensional reconstruction method according to another embodiment of the present disclosure, as shown in fig. 3, the method includes the following steps:

s301, acquiring a plurality of associated images of a target road to be reconstructed and the pose of each associated image under a world coordinate system when the associated images are shot.

Specifically, the implementation process and principle of S301 and S101 are identical, and will not be described herein.

S302, determining a three-dimensional reconstruction model of the sub-road section under an image coordinate system and the pose of a camera in the three-dimensional reconstruction model when each associated image is shot according to any sub-road section included in the target road based on a plurality of associated images.

Specifically, the implementation process and principle of S302 and S102 are consistent, and will not be described herein.

S303, for any sub-road section included in the target road, according to the pose of each associated image under the world coordinate system of the camera and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, obtaining the coordinate corresponding relation of any sub-road section under the image coordinate system and the world coordinate system, and according to the coordinate corresponding relation of any sub-road section under the image coordinate system and the world coordinate system, determining the pose conversion relation of the image coordinate system of the three-dimensional reconstruction model of different sub-road sections under the world coordinate system.

Specifically, the implementation process and principle of S303 and S103 are identical, and will not be described herein.

S304, converting the three-dimensional reconstruction models of different sub-road sections into the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction models of different sub-road sections in the world coordinate system, and obtaining the poses of the three-dimensional reconstruction models of different sub-road sections in the world coordinate system.

After determining the pose conversion relation of the three-dimensional reconstruction model of each sub-road section in the world coordinate system, converting the three-dimensional reconstruction model of each sub-road section into the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of each sub-road section in the world coordinate system, and obtaining the pose of the three-dimensional reconstruction model of each sub-road section in the world coordinate system.

S305, combining the three-dimensional reconstruction models of the road three-dimensional reconstruction different sub-road segments based on the pose of the three-dimensional reconstruction models of the different sub-road segments under the world coordinate system to obtain the three-dimensional model of the road three-dimensional reconstruction target road under the world coordinate system.

And combining the three-dimensional reconstruction models of all the sub-road segments based on the pose of the three-dimensional reconstruction model of each sub-road segment under the world coordinate system to obtain the three-dimensional model of the road three-dimensional reconstruction target road under the world coordinate system.

According to the method and the device, the multiple associated images of the target road to be rebuilt and the pose of each associated image under the world coordinate system of the camera are obtained, and for any sub-road section included in the target road, the three-dimensional rebuilding model of the sub-road section under the image coordinate system and the pose of the camera in the three-dimensional rebuilding model when each associated image is shot are determined based on the multiple associated images. Further, based on the pose of each associated image in the world coordinate system of the camera and the pose of each associated image in the three-dimensional reconstruction model of different sub-road segments when the associated images are shot, the pose conversion relation of the three-dimensional reconstruction model of different sub-road segments to the world coordinate system is determined. And then converting the three-dimensional reconstruction models of the different sub-road sections into the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction models of the different sub-road sections in the world coordinate system to obtain the poses of the three-dimensional reconstruction models of the different sub-road sections in the world coordinate system, and merging the three-dimensional reconstruction models of the road three-dimensional reconstruction different sub-road sections based on the poses of the three-dimensional reconstruction models of the different sub-road sections in the world coordinate system to obtain the three-dimensional model of the road three-dimensional reconstruction target road in the world coordinate system. Because three-dimensional reconstruction is carried out on each sub-road section, and then the pose of the three-dimensional reconstruction model of the sub-road section is optimized, the pose conversion relation of the three-dimensional reconstruction model of different sub-road sections under the world coordinate system is obtained, then the pose of the three-dimensional reconstruction model of each sub-road section under the world coordinate system is adjusted, and the adjusted pose is more in line with the pose of the sub-road section in the target road section, the reconstructed three-dimensional model of the target road under the world coordinate system is more accurate, the effect of three-dimensional reconstruction of the road can be improved, the three-dimensional reconstruction method is applied to a high-precision map, and automatic driving is better realized.

Fig. 4 is a schematic structural diagram of a road three-dimensional reconstruction device according to an embodiment of the disclosure. The road three-dimensional reconstruction device may be a terminal as in the above embodiments, or the road three-dimensional reconstruction device may be a part or assembly in the terminal. The road three-dimensional reconstruction device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the road three-dimensional reconstruction method, as shown in fig. 4, the road three-dimensional reconstruction device 40 includes: an acquisition module 41, a first determination module 42, a second determination module 43, a reconstruction module 44; the acquiring module 41 is configured to acquire a plurality of associated images of a target road to be reconstructed and a pose of each road three-dimensional reconstruction associated image under a world coordinate system when the camera is photographed; the first determining module 42 is configured to determine, for any sub-road segment included in the road three-dimensional reconstruction target road, a three-dimensional reconstruction model of the sub-road segment under an image coordinate system and a pose of a camera in the road three-dimensional reconstruction model when each road three-dimensional reconstruction associated image is captured, based on the road three-dimensional reconstruction multiple associated images; the second determining module 43 is configured to obtain, for any sub-road segment included in the target road, a coordinate correspondence of the any sub-road segment in the image coordinate system and the world coordinate system according to a pose of each associated image in the world coordinate system of the camera when the associated image is captured and a pose of each associated image in the image coordinate system of the three-dimensional reconstruction model of the different sub-road segment, and determine a pose conversion relationship of the image coordinate system of the three-dimensional reconstruction model of the different sub-road segment to the world coordinate system according to the coordinate correspondence of the any sub-road segment in the image coordinate system and the world coordinate system; the reconstruction module 44 is configured to reconstruct a three-dimensional model of the target road in the world coordinate system based on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the world coordinate system and the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system.

Optionally, the road three-dimensional reconstruction obtaining module 41 is specifically configured to, when obtaining a plurality of associated images of a target road to be reconstructed and a pose of each road three-dimensional reconstruction associated image under a world coordinate system when the camera is photographed: acquiring an image sequence of a road three-dimensional reconstruction target road shot by a road three-dimensional reconstruction camera and positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot; and determining a plurality of associated images and the pose of each road three-dimensional reconstruction associated image in a world coordinate system when the camera is shot from the road three-dimensional reconstruction image sequence based on the positioning information of the camera when each image in the road three-dimensional reconstruction image sequence is shot.

Optionally, the road three-dimensional reconstruction device 40 further includes: a dividing module 45; the dividing module 45 is configured to divide the road three-dimensional reconstruction target road into a plurality of sub-road segments based on the road three-dimensional reconstruction multiple associated images.

Optionally, the second determining module 43 obtains, for any sub-road segment included in the target road, a coordinate corresponding relationship of the any sub-road segment in the image coordinate system and the world coordinate system according to a pose of each associated image in the world coordinate system of the camera when the associated image is captured and a pose of each associated image in the image coordinate system of the three-dimensional reconstruction model of the different sub-road segment, where the three-dimensional reconstruction model of the different sub-road segment is located, and determines, according to the coordinate corresponding relationship of the any sub-road segment in the image coordinate system and the world coordinate system, a pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segment when the three-dimensional reconstruction model of the different sub-road segment is converted into the world coordinate system: converting the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the sub-road section when each associated image is shot into the world coordinate system based on the initial pose conversion relation of the pose conversion of the three-dimensional reconstruction model of the sub-road section in the image coordinate system into the world coordinate system aiming at any sub-road section included in the target road; and based on the pose of each associated image in the world coordinate system when the camera is shot, the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections when each associated image is shot, and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections included in the same associated image, and the pose of the camera in the world coordinate system, adjusting the coefficient of the initial pose conversion relationship of the pose of the three-dimensional reconstruction model of different sub-road sections in the image coordinate system, and obtaining the pose conversion relationship of the three-dimensional reconstruction model of different sub-road sections in the image coordinate system of the three-dimensional reconstruction model of different sub-road sections.

Optionally, the second determining module 43 obtains, for any sub-road segment included in the target road, a coordinate corresponding relationship of the any sub-road segment in the image coordinate system and the world coordinate system according to a pose of each associated image in the world coordinate system of the camera when the associated image is captured and a pose of each associated image in the image coordinate system of the three-dimensional reconstruction model of the different sub-road segment, where the three-dimensional reconstruction model of the different sub-road segment is located, and determines, according to the coordinate corresponding relationship of the any sub-road segment in the image coordinate system and the world coordinate system, a pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segment when the three-dimensional reconstruction model of the different sub-road segment is converted into the world coordinate system: according to the pose of a camera in an image coordinate system of a three-dimensional reconstruction model of different sub-road sections when each associated image is shot, converting the pose of the camera in the world coordinate system and the pose of each associated image shooting camera in the world coordinate system, and constructing a first constraint relation; according to the pose of a camera in an image coordinate system of each of two sub-road sections included in the image when any associated image is shot, converting the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of each sub-road section into the pose under the world coordinate system, and constructing a second constraint relation; and determining a pose conversion relation from an image coordinate system where the three-dimensional reconstruction model of different sub-road sections is positioned to a world coordinate system based on the first constraint relation and the second constraint relation.

Optionally, the second determining module 43 is further configured to: determining an initial scale conversion relation from an image coordinate system of the three-dimensional reconstruction model of the different sub-road sections to a world coordinate system;

the reconstruction module 44 is specifically configured to reconstruct the three-dimensional model of the target road in the world coordinate system based on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the world coordinate system and the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system, where the three-dimensional reconstruction relationship is that: and reconstructing the three-dimensional model of the target road under the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system, the initial scale conversion relation of the three-dimensional reconstruction model of the different sub-road sections under the world coordinate system, and the pose and scale of the three-dimensional reconstruction model of the different sub-road sections in the image coordinate system.

Optionally, the reconstruction module 44 is specifically configured to, when reconstructing the three-dimensional model of the target road in the world coordinate system, based on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the world coordinate system to the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the world coordinate system, and on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system: converting the three-dimensional reconstruction models of different sub-road sections into the world coordinate system based on the pose conversion relation of the three-dimensional reconstruction models of different sub-road sections into the world coordinate system, so as to obtain the poses of the three-dimensional reconstruction models of different sub-road sections in the world coordinate system; and combining the three-dimensional reconstruction models of the different sub-road sections based on the pose of the three-dimensional reconstruction models of the different sub-road sections under the world coordinate system to obtain the three-dimensional model of the target road under the world coordinate system.

The road three-dimensional reconstruction device of the embodiment shown in fig. 4 may be used to implement the technical solution of the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein again.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. The electronic device may be a terminal as described in the above embodiments. The electronic device provided in the embodiment of the present disclosure may execute the processing flow provided in the embodiment of the road three-dimensional reconstruction method, as shown in fig. 5, where the electronic device 50 includes: memory 51, processor 52, computer programs and communication interface 53; wherein the computer program is stored in the memory 51 and configured to be executed by the processor 52 for the road three-dimensional reconstruction method as described above.

In addition, the embodiment of the present disclosure also provides a computer-readable storage medium having stored thereon a computer program that is executed by a processor to implement the road three-dimensional reconstruction method described in the above embodiment.

The disclosed embodiments also provide a vehicle including: memory, processor, computer program and communication interface; wherein the computer program is stored in the memory and configured to be executed by the processor for performing the road three-dimensional reconstruction method as described above.

Furthermore, the disclosed embodiments also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implements a road three-dimensional reconstruction method as described above.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:

In addition, the electronic device may also perform other steps in the road three-dimensional reconstruction method as described above.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for three-dimensional reconstruction of a roadway, the method comprising:

2. The method according to claim 1, wherein the acquiring a plurality of associated images of the target road to be reconstructed and a pose of each of the associated images in a world coordinate system of a camera is performed, comprises:

3. The method of claim 1, wherein prior to determining a three-dimensional reconstruction model of any sub-link included for the target link in an image coordinate system based on the plurality of associated images, the method further comprises:

4. The method according to claim 1, wherein the determining, for any sub-link included in the target link, a pose conversion relationship from an image coordinate system where the three-dimensional reconstruction model of the different sub-links is located to a world coordinate system according to a pose of each of the associated images in the world coordinate system where the three-dimensional reconstruction model of the different sub-links is located when each of the associated images is captured, and the coordinate conversion relationship from the image coordinate system and the world coordinate system of the any sub-link according to the coordinate correspondence from the any sub-link to the world coordinate system includes:

5. The method according to claim 4, wherein the determining, for any sub-link included in the target link, the pose conversion relationship from the image coordinate system of the three-dimensional reconstruction model of the different sub-links to the world coordinate system according to the pose of each associated image in the world coordinate system of the camera when each associated image is captured and the pose of the camera in the image coordinate system of the three-dimensional reconstruction model of the different sub-link according to the coordinate correspondence of the any sub-link in the image coordinate system and the world coordinate system, includes:

6. The method according to claim 1, wherein for any sub-link included in the target link, according to a pose of each associated image in a world coordinate system of a camera when the associated image is captured and a pose of each associated image in an image coordinate system where a three-dimensional reconstruction model of a different sub-link is located, a coordinate correspondence of the any sub-link in the image coordinate system and the world coordinate system is obtained, and according to a coordinate correspondence of the any sub-link in the image coordinate system and the world coordinate system, determining a pose conversion relationship of the three-dimensional reconstruction model of the different sub-link in the image coordinate system to the world coordinate system, the method further comprises:

7. The method according to claim 1, wherein reconstructing the three-dimensional model of the target road in the world coordinate system based on the pose conversion relationship of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system to the world coordinate system and the pose of the three-dimensional reconstruction model of the different sub-road segments in the image coordinate system, comprises:

8. A road three-dimensional reconstruction device, the device comprising:

9. An electronic device, comprising:

a memory;

a processor; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any of claims 1-7.

10. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any of claims 1-7.

11. A vehicle, characterized by comprising:

a memory;

a processor; and

a computer program;