CN114820784A

CN114820784A - Guideboard generation method and device and electronic equipment

Info

Publication number: CN114820784A
Application number: CN202210359557.9A
Authority: CN
Inventors: 赵飞翔; 朱磊; 李正旭; 贾双成
Original assignee: Zhidao Network Technology Beijing Co Ltd
Current assignee: Zhidao Network Technology Beijing Co Ltd
Priority date: 2022-04-07
Filing date: 2022-04-07
Publication date: 2022-07-29

Abstract

The application relates to a method and a device for generating a guideboard and electronic equipment. The method comprises the following steps: acquiring a plurality of frames of target images; respectively extracting characteristic points in each target image, and matching the characteristic points in two adjacent frames of target images to obtain the relative pose of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the characteristic points; respectively determining the absolute poses of the other target images according to the preset absolute pose and the relative poses of one frame of target image; carrying out graph optimization according to the absolute pose, the three-dimensional coordinates of the characteristic points and the pixel coordinates to obtain an optimized absolute pose; obtaining an optimized relative pose corresponding to each target image according to the optimized absolute pose; and triangularizing according to the optimized relative pose and the pixel coordinates corresponding to the corner points of the target guideboard to obtain optimized three-dimensional coordinates of the corner points. The scheme provided by the application can improve the coordinate precision of the corner points of the guideboard, and has higher robustness to deal with the processing of each guideboard.

Description

Guideboard generation method and device and electronic equipment

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for generating a guideboard, and an electronic device.

Background

Compared with the traditional electronic map, the high-precision map can show the real road characteristic information such as lanes, guardrails, traffic signboards and the like, and provides lane-level navigation.

In the related art, various mapping companies collect road characteristic information by using special mapping vehicles, which generally use laser radar technology, to create a vehicle route navigation map on a global scale. However, taking the guideboards in the traffic signboards as an example, the guideboards are usually only accurate to a few meters when being collected, and the positions of the guideboards cannot be accurately located in a high-precision map. Although such a vehicle route guidance map is sufficient for route guidance, the position of the guideboard in such a map is not accurate enough, which is disadvantageous for a vehicle traveling at high speed to accurately identify the guideboard on the roadside in time.

Disclosure of Invention

In order to solve or partially solve the problems in the related art, the application provides a method and a device for generating a guideboard and electronic equipment, which can improve the display position accuracy of the guideboard in a high-precision map.

The first aspect of the present application provides a method for generating a guideboard, including:

acquiring a plurality of frames of target images; wherein each target image comprises a target guideboard;

respectively extracting feature points in each target image, and matching the feature points in two adjacent frames of the target images to obtain the relative poses of the two adjacent frames of the target images and three-dimensional coordinates corresponding to the feature points;

respectively determining the absolute poses of the rest of the target images according to the preset absolute pose and each relative pose of one frame of the target images;

carrying out graph optimization according to the absolute pose, the three-dimensional coordinates of the characteristic points and the pixel coordinates to obtain an optimized absolute pose;

obtaining an optimized relative pose corresponding to each target image according to the optimized absolute pose;

and triangularizing according to the optimized relative pose and the pixel coordinates corresponding to the corner points of the target guideboard to obtain optimized three-dimensional coordinates of the corner points.

In some embodiments, the acquiring a plurality of frames of target images includes:

acquiring multiple frames of target images according to a preset interval distance in sequence, wherein the number of the target images is more than or equal to 3.

In some embodiments, after the extracting feature points in each of the target images, and matching the feature points in two adjacent frames of the target images to obtain the relative poses of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points, the method further includes:

and screening the characteristic points according to the depth values of the three-dimensional coordinates, and deleting the characteristic points with negative depth values.

In some embodiments, performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose includes:

determining common characteristic points among different target images, and taking the mean value of corresponding three-dimensional coordinates as the three-dimensional coordinates of the common characteristic points;

acquiring three-dimensional coordinates of each feature point including the common feature point, absolute poses of each target image and corresponding relations between pixel coordinates of each feature point in the corresponding target image;

determining a reprojection error of each common characteristic point in a corresponding target image through a preset graph optimization model by taking the absolute pose and the three-dimensional coordinates of the characteristic points as vertexes and taking the corresponding relation as a connecting edge;

deleting the connecting edges with the reprojection errors larger than a preset value, iterating through a preset graph optimization model, and adjusting the absolute pose of each target image and the three-dimensional coordinates of the feature points until a preset iteration termination condition is reached;

and taking the absolute pose after iteration termination as an optimized absolute pose.

In some embodiments, the preset iteration termination condition includes a preset number of iterations and/or a preset total weight projection error threshold.

In some embodiments, after said extracting the feature points in each of the target images, the method further includes:

acquiring an angular point corresponding to the target guideboard and three-dimensional coordinates of the angular point from the characteristic points; and/or

After triangularization is performed according to the optimized relative pose and the pixel coordinates corresponding to the corner points of the target guideboard to obtain the optimized three-dimensional coordinates of the corner points, the method further comprises the following steps:

and acquiring a geographical coordinate corresponding to the optimized three-dimensional coordinate according to the optimized three-dimensional coordinate of the angular point, a preset camera parameter and a corresponding geographical coordinate and a corresponding course angle.

A second aspect of the present application provides a guideboard generation apparatus, including:

the first acquisition module is used for acquiring multi-frame target images; wherein each target image comprises a target guideboard;

the second acquisition module is used for respectively extracting feature points in each target image, matching the feature points in two adjacent frames of the target images and acquiring the relative poses of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points;

the first pose conversion module is used for respectively determining the absolute poses of the rest target images according to the preset absolute pose and each relative pose of one frame of the target image;

the optimization module is used for carrying out graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose;

the second pose conversion module is used for acquiring the optimized relative pose corresponding to each target image according to the optimized absolute pose;

and the coordinate optimization module is used for triangularizing according to the optimized relative pose and the pixel coordinates corresponding to the corner points of the target guideboard to obtain the optimized three-dimensional coordinates of the corner points.

In some embodiments, the second obtaining module is further configured to obtain, from the feature points, corner points corresponding to the target guideboard and three-dimensional coordinates of the corner points; and/or

The device also comprises a third acquisition module for acquiring the camera parameters and the corresponding geographic coordinates and course angles; and/or

The device also comprises a coordinate conversion module which is used for acquiring world coordinates corresponding to the optimized three-dimensional coordinates according to the optimized three-dimensional coordinates of the angular points, the camera parameters and the corresponding geographic coordinates and course angles.

A third aspect of the present application provides an electronic device comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method as described above.

A fourth aspect of the present application provides a computer-readable storage medium having stored thereon executable code, which, when executed by a processor of an electronic device, causes the processor to perform a method as described above.

The technical scheme provided by the application can comprise the following beneficial effects:

according to the technical scheme, the initial relative poses of two adjacent frames and the initial three-dimensional coordinates of the feature points can be obtained by extracting and matching the feature points in the multi-frame target images, then the initial absolute coordinates corresponding to each target image and the pixel coordinates corresponding to each three-dimensional coordinate are obtained, the optimized absolute coordinates are obtained through image optimization and converted into the optimized relative coordinates, and then the optimized three-dimensional coordinates corresponding to the guideboard in each target image can be determined. The design can improve the coordinate precision of the corner points of the guideboard, has higher robustness to deal with the processing of each guideboard, and is convenient for generating the guideboard at an accurate position in a high-precision map more accurately.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.

Fig. 1 is a schematic flow chart of a method for generating a guideboard according to an embodiment of the present disclosure;

fig. 2 is another schematic flow chart of a method for generating a guideboard according to an embodiment of the present disclosure;

fig. 3 is a schematic diagram of a corresponding relationship between an absolute pose, a feature point, and a reprojection error according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a guideboard generation apparatus according to an embodiment of the present disclosure;

fig. 5 is another schematic structural diagram of a guideboard generation apparatus according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.

Detailed Description

Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

In the related art, the position of a guideboard displayed in a high-precision map is not accurate enough, so that the accuracy of the display content of the high-precision map is influenced.

In view of the above problems, embodiments of the present application provide a method for generating a guideboard, which can improve the accuracy of a display position of the guideboard in a high-precision map.

The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a schematic flow chart of a method for generating a guideboard according to an embodiment of the present disclosure.

Referring to fig. 1, a method for generating a guideboard according to an embodiment of the present application includes:

s110, acquiring a plurality of frames of target images; wherein each target image comprises a target guideboard.

In this step, the target image may be acquired using an image capture device on the mapping vehicle, which may be a monocular camera. In the process of driving of the surveying and mapping vehicle, the environment along the road can be collected and shot in real time to form a shooting video. Multiple frames of target images can be selected from the shot video, wherein the target images comprise the target guideboards.

For convenience of data processing, the target images can be arranged in sequence according to the acquisition time sequence.

And S120, respectively extracting the feature points in each target image, and matching the feature points in two adjacent frames of target images to obtain the relative poses of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points.

In this step, the feature points in each target image may be identified respectively according to the related art. Because the guideboard generally presents the shapes of a square, a triangle or a circle, the feature points corresponding to the corner points of the guideboard can be identified and obtained through a related algorithm. After the feature points in each target image are extracted, the feature points in two adjacent frames of target images can be sequentially matched according to a correlation algorithm, so that the feature points of the same object in different images form correlation, and then matched point pairs between the two frames of target images are obtained.

It will be appreciated that the camera motion may be different for capturing different target images. According to the point pairs, the relative pose of the camera in the collected two adjacent frames of target images is estimated through the epipolar geometry according to the camera parameters, and the three-dimensional coordinates of each feature point in the camera coordinate system corresponding to each frame of target image are obtained through triangularization processing according to the relative pose and the related camera parameters.

And S130, respectively determining the absolute poses of the other target images according to the preset absolute pose and the relative poses of one frame of target image.

In order to obtain the absolute pose of each target image, in this step, one of the target images in the frame may be selected, for example, the target image in the frame 1 may be selected as a reference, and the absolute pose of the target image may be set in advance, for example, the preset absolute pose may be represented by a rotation matrix. And then, taking the preset absolute pose of the selected target image as a reference, and respectively calculating the absolute poses of the cameras corresponding to the remaining target images of each frame according to the relative poses of the target images of each two adjacent frames.

And S140, carrying out graph optimization according to the absolute pose, the three-dimensional coordinates of the characteristic points and the pixel coordinates to obtain an optimized absolute pose.

In this step, an optimization graph may be constructed by a correlation technique according to the absolute pose and the three-dimensional coordinates of each feature point, for example, an optimization model may be constructed by the g20 graph, in which the absolute pose and the three-dimensional coordinates of each feature point are used as vertices, and then the three-dimensional coordinates of each absolute pose and each feature point and the corresponding pixel coordinates are used as connecting edges, thereby constructing the optimization graph. And iteratively adjusting the absolute pose and each three-dimensional coordinate in a graph optimization mode to reduce the reprojection error and finally obtain the coincident absolute pose as the optimized absolute pose.

It is understood that the pixel coordinates may be obtained by three-dimensional coordinate conversion of the feature points according to the related art, and then the pixel coordinates of each feature point in the corresponding target image are obtained.

And S150, acquiring the optimized relative pose corresponding to each target image according to the optimized absolute pose.

In the step, the optimized absolute pose can be converted into the optimized relative pose of the two adjacent frames of target images according to the optimized absolute pose. That is, the optimized relative pose is more accurate than the relative pose of step S120.

And S160, triangularization is carried out according to the optimized relative pose and the pixel coordinates corresponding to the corner points of the target guideboard, and optimized three-dimensional coordinates of the corner points are obtained.

It is understood that although the step S120 may obtain the three-dimensional coordinates corresponding to each corner point of the target guideboard, it is obvious that the three-dimensional coordinates are not accurate. In this step, according to the optimized relative pose and the pixel coordinates of each corner point in each target image, the optimized three-dimensional coordinates of each corner point in the camera coordinate system in the corresponding target image can be obtained through triangularization calculation.

And optimizing the three-dimensional coordinates according to each corner point, and converting to obtain the geographic coordinates of each corner point according to the camera parameters and the geographic coordinates corresponding to the camera. Wherein the geographic coordinates are coordinates in a world coordinate system. According to the clear geographic coordinates, the specific display position of the guideboard in the high-precision map can be clear, and therefore the guideboard at the corresponding position can be accurately generated in the high-precision map.

As can be seen from this example, in the method for generating a guideboard according to the present application, the feature points in the multiple frames of target images are extracted and matched to obtain the initial relative poses of two adjacent frames and the initial three-dimensional coordinates of each feature point, then obtain the initial absolute coordinates corresponding to each target image and the pixel coordinates corresponding to each three-dimensional coordinate, obtain the optimized absolute coordinates through image optimization, and convert the optimized absolute coordinates into the optimized relative coordinates, so as to determine the optimized three-dimensional coordinates corresponding to the guideboard in each target image. The design can improve the coordinate precision of the corner points of the guideboard, has higher robustness to deal with the processing of each guideboard, and is convenient for generating the guideboard at an accurate position in a high-precision map more accurately.

Fig. 2 is another flow chart of the method for generating a guideboard according to the embodiment of the present disclosure.

Referring to fig. 2, a method for generating a guideboard according to an embodiment of the present application includes:

s210, obtaining multiple frames of target images according to a preset interval distance in sequence, wherein the number of the target images is greater than or equal to 3, and each image comprises a target guideboard.

In this embodiment, the monocular camera mounted on the surveying and mapping vehicle may capture the guideboard along the way, and the second frame, the third frame, the fourth frame, and the nth frame of target image may be sequentially selected every 3 meters to 5 meters, for example, from the time of obtaining the first frame of target image including the target guideboard according to the travel route and the capture time. It can be understood that the geographical positions of the cameras corresponding to each selected frame of the target image are different from each other. By selecting the target images shot at different positions, mismatching of subsequent characteristic points caused by too far separation of two frames of images is avoided, more target images are obtained through proper separation distance, the obtained patterns are richer, the images are prevented from being too monotonous, and the matching precision of the characteristic points in the subsequent steps is improved.

And S220, respectively extracting the feature points in each target image, and matching the feature points in two adjacent frames of target images to obtain the relative poses of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points.

After extracting the feature points in the target images of the frames, for example, the feature points in the 1 st frame and the 2 nd frame are respectively matched, the feature points in the 2 nd frame and the 3 rd frame are matched, the feature points in the 3 rd frame and the 4 th frame are matched, and so on. It can be understood that, since the shooting position of the camera of each target image is different, the objects contained in each target image may be partially the same and partially different. Automatically matching every two adjacent images in each frame of target image through a correlation algorithm, and obtaining the phase of each two adjacent frames of target images through epipolar geometric constraint according to a camera internal reference matrixAnd (5) relative pose of the machine. Further, calculating according to the matrix of the relative pose of the camera to obtain the corresponding three-dimensional coordinates (X) of the feature points successfully matched in each target image in the camera coordinate system _c ，Y _c ，Z _c ) Wherein Z is _c The corresponding depth value of the feature point in the camera coordinate system.

After the feature points of the target image are extracted, in an embodiment, the corner points and the three-dimensional coordinates of the corner points corresponding to the target guideboard are obtained from the feature points. That is to say, the target image has a large number of feature points, but based on the particularity of the shape of the target guideboard, the corner points and the corresponding three-dimensional coordinates of the corresponding target guideboard can be obtained from the feature points according to the related image recognition algorithm, thereby facilitating the calculation in the subsequent steps. For example, when the guideboard is a quadrangle, 4 corresponding corner points and three-dimensional coordinates thereof can be obtained; when the guideboard is triangular, 3 corresponding corner points and three-dimensional coordinates thereof can be obtained.

And S230, screening the feature points according to the depth values of the three-dimensional coordinates, and deleting the feature points with the depth values being negative values.

And if the depth value calculated after the matching of the pair of feature points has a negative value, the matching failure of the set of feature points is indicated. That is, the depth values of the same object in the camera coordinate systems corresponding to the different images should all be positive values. Therefore, in the present step, in the single-frame target image, the feature point whose depth value in the three-dimensional coordinates is a negative value is deleted, thereby reducing the calculation error in the subsequent step.

And S240, respectively determining the absolute poses of the other target images according to the preset absolute pose and the relative poses of one frame of target image.

The step can refer to the related description of step S130, which is not described herein.

S250, carrying out graph optimization according to the absolute pose, the three-dimensional coordinates of the characteristic points and the pixel coordinates, and adjusting according to the reprojection error absolute pose and the three-dimensional coordinates; and when the iteration is carried out until the preset iteration termination condition is met, obtaining an optimized absolute pose.

For ease of understanding, the following is illustrated in a specific embodiment:

and S251, determining common characteristic points among different target images, and taking the mean value of the corresponding three-dimensional coordinates as the three-dimensional coordinates of the common characteristic points.

The common-view relationship between different target images can be found through the feature points, that is, the feature points of the same object in different target images are found, that is, the feature points successfully matched are used as the common feature points. As shown in FIG. 3, C1 shows the absolute pose of the target image at frame 1, C2 shows the absolute pose of the target image at frame 2, and C3 shows the absolute pose of the target image at frame 3. P1 to P3 indicate feature points in the 1 st frame target image, P2 to P4 indicate feature points in the 2 nd frame target image, and P3 to P6 indicate feature points in the 3 rd frame target image. The reprojection error e11 represents the position deviation of the pixel coordinates of the original projection position and the adjusted projection position of the feature point P1 in the 1 st frame target image; e12 represents the positional deviation of the pixel coordinates of the original projection position and the adjusted projection position of the feature point P2 in the 1 st frame target image, and so on.

As shown in fig. 3, feature points P2 and P3 are common feature points of the target images of the 1 st and 2 nd frames, feature point P3 is a common feature point of the target images of the 1 st to 3 rd frames, and P4 is a common feature point of the target images of the 2 nd and 3 rd frames. In this step, for the common feature point, an average value may be calculated according to the three-dimensional coordinates of the feature point corresponding to the original target image, so that the common feature point has a uniform three-dimensional coordinate. And when a certain characteristic point is not a common characteristic point, the original three-dimensional coordinate is adopted to participate in the related calculation.

And S252, acquiring the three-dimensional coordinates of each feature point including the common feature point, the absolute pose of each target image and the corresponding relation between the pixel coordinates of each feature point in the corresponding target image.

In order to facilitate the subsequent graph optimization processing, in this step, the three-dimensional coordinates of each feature point in the camera coordinate system, the pixel coordinates corresponding to the pixel coordinate system, and the absolute pose of the target image where the feature point is located form a corresponding relationship. In order to facilitate rapid data acquisition during calculation, IDs can be used for identification, so that the correspondence relationship can be determined according to the IDs.

As shown in table 1 below, with reference to fig. 3, taking 3 frames of target images as an example, in order to facilitate acquisition of data in subsequent calculation, ID labels such as 1, 2, and 3 may be set for absolute poses corresponding to each frame of target images. The feature points in each frame of target image have respective corresponding IDs, and when the feature points are common feature points, the same ID is displayed in different target images, for example, the feature point P2 appears in the 1 st frame and the 2 nd frame of target images at the same time, and the corresponding IDs in the two frame of target images are both 2. The reprojection error of each feature point in the corresponding target image is represented according to the respective ID, for example, the reprojection error of the 1 st feature point P1 in the 1 st frame target image is e11, for example, the reprojection error of the 3 rd feature point P3 in the 2 nd frame target image is e23, and so on. Therefore, all the feature points can form corresponding relations with corresponding absolute poses, three-dimensional coordinates and pixel coordinates based on the target images in which the feature points are located.

TABLE 1

And S253, determining the reprojection error of each common characteristic point in the corresponding target image through a preset graph optimization model by taking the absolute pose and the three-dimensional coordinates of the characteristic points as vertexes and the corresponding relation as connecting edges.

In this embodiment, according to a correlation algorithm, taking the g2o graph optimization model as an example, the absolute pose and the three-dimensional coordinates of the feature points are input into the model as vertices, and corresponding IDs are set. And then, the corresponding relation in the step S252 is used as a connecting edge to be added into the model, the model can automatically calculate the corresponding reprojection error, and the absolute pose and the three-dimensional coordinate are synchronously adjusted in the iteration process, so that the reprojection error is smaller and smaller.

And S254, deleting the connecting edges with the reprojection errors larger than the preset value, iterating through the preset graph optimization model, and adjusting the absolute pose of each target image and the three-dimensional coordinates of the feature points until a preset iteration termination condition is reached.

In an embodiment, when the reprojection error is greater than 1, the corresponding connecting edge is deleted, so as to reduce the influence of the data with larger error on the calculation. After the initial 1-2 times of iterative optimization, the connecting edges with larger errors can be removed, and the rest connecting edges are continuously added into the model and continuously subjected to iterative optimization, so that the three-dimensional coordinates of each absolute pose and each feature point are more and more accurate.

In an embodiment, the preset iteration termination condition includes a preset iteration number and/or a preset total weight projection error threshold. For example, the preset iteration number is 15-20. For example, the total weight projection error threshold is set to 100. The total weight projection error is the sum of the re-projection errors corresponding to the pixel coordinates of all the feature points. When the iteration reaches one of the preset iteration termination conditions, the iteration can be stopped.

And S255, taking the absolute pose after iteration termination as an optimized absolute pose.

It can be understood that after the iteration is finished, the absolute pose obtained by the last optimization is the optimized absolute pose. The optimized absolute pose is more accurate than the initial absolute pose.

And S260, acquiring the optimized relative pose corresponding to each target image according to the optimized absolute pose.

It can be understood that after the optimized absolute pose of each frame of target image is obtained, calculation conversion can be correspondingly performed to obtain the optimized relative pose of the camera corresponding to each two adjacent frames of target images.

And S270, triangularizing according to the optimized relative pose and the pixel coordinates corresponding to the corner points of the target guideboard to obtain optimized three-dimensional coordinates of the corner points.

It can be understood that, after extracting each feature point in the foregoing step S220, the corner point of the corresponding target guideboard and the three-dimensional coordinates of each frame of target image in the camera coordinate system are correspondingly obtained. According to the related art, according to the three-dimensional coordinates and the camera parameters in the camera coordinate system, the corresponding pixel coordinates in the pixel coordinate system can be converted accordingly.

In this step, triangularization processing can be performed according to the optimized relative pose and the pixel coordinates of each corner point, and the corresponding optimized three-dimensional coordinates in the camera coordinate system are obtained through calculation according to the related technology. Obviously, the optimized three-dimensional coordinates are more accurate than the three-dimensional coordinates of step S220.

And S280, acquiring a geographical coordinate corresponding to the optimized three-dimensional coordinate according to the optimized three-dimensional coordinate of the corner point, the preset camera parameter, the corresponding geographical coordinate and the corresponding course angle.

It can be understood that when the cameras capture corresponding target images at different positions, the mapping vehicle where the cameras are located has corresponding vehicle poses, i.e. geographic coordinates and heading angles of the vehicle. Therefore, when the camera shoots and obtains the target image at different positions, the camera has corresponding geographic coordinates and a heading angle, the geographic coordinates can be represented by world coordinates corresponding to a world coordinate system, and the heading angle can be obtained according to related instruments in the surveying and mapping vehicle.

In this step, the optimized three-dimensional coordinates of each corner point can be calculated and converted into the geographic coordinates corresponding to each corner point in each frame of target image according to the camera external parameters, the geographic coordinates and the course angle. The unique geographic coordinates of the same corner point can be obtained by averaging the geographic coordinates of the same corner point in each frame of target image.

And S290, generating a guideboard at the corresponding position in the high-precision map according to the geographic coordinates corresponding to the corner points.

It can be understood that when the geographical coordinates with accurate angular points are provided, the spatial form and the display position of the guideboard can be determined, that is, the corresponding guideboard can be accurately generated and displayed at the map position of the corresponding geographical coordinates of the high-precision map.

As can be seen from this example, in the guideboard generation method of the present application, relevant conditions are set from the target image selection stage, the target images are selectively screened, and the measurement and calculation accuracy is improved from the source; and determining a corresponding initial absolute pose according to the initial relative pose, performing graph optimization according to the mean three-dimensional coordinates and corresponding pixel coordinates of the common feature points in each target image, adjusting the absolute pose and the three-dimensional coordinates through iterative optimization until a preset iteration termination condition is reached, obtaining the optimal optimized absolute pose, converting the optimal absolute pose into the optimized relative pose, and further obtaining an accurate optimized three-dimensional coordinate by combining the pixel coordinates of the angular points. By the design, the robustness of a system calculation result can be effectively improved, and compared with a three-dimensional coordinate with larger initial error, the geographic coordinates of the corresponding corner points can be obtained according to the optimized three-dimensional coordinate and the related technology, so that a more accurate spatial form of the corner points of the guideboard and a display position in a high-precision map can be obtained.

Corresponding to the embodiment of the application function implementation method, the application also provides a guideboard generation device, electronic equipment and a corresponding embodiment.

Fig. 4 is a schematic structural diagram of a guideboard generation apparatus according to an embodiment of the present application.

Referring to fig. 4, an embodiment of the present application provides a device for generating a road sign, which includes:

a first obtaining module 410, configured to obtain multiple frames of target images; wherein each target image comprises a target guideboard.

The second obtaining module 420 is configured to extract feature points in each target image, match the feature points in two adjacent frames of target images, and obtain the relative pose of the two adjacent frames of target images and the three-dimensional coordinates corresponding to the feature points.

And the first pose conversion module 430 is configured to determine the absolute poses of the other target images according to the preset absolute pose and each relative pose of one frame of target image.

And the optimization module 440 is configured to perform graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose.

And the second pose conversion module 450 is configured to obtain an optimized relative pose corresponding to each target image according to the optimized absolute pose.

And the coordinate optimization module 460 triangulates the pixel coordinates corresponding to the corner points of the optimized relative pose and the target guideboard to obtain the optimized three-dimensional coordinates of the corner points.

Further, the first obtaining module 410 is configured to obtain multiple frames of target images in sequence according to a preset interval distance, where the number of the target images is greater than or equal to 3 frames.

The second obtaining module 420 is further configured to obtain, from the feature points, corner points and three-dimensional coordinates of the corner points corresponding to the target guideboard. After obtaining the three-dimensional coordinates corresponding to each feature point, the second obtaining module 420 may be further configured to filter the feature points according to the depth values of the three-dimensional coordinates, and delete the feature points with the depth values being negative values.

In a specific embodiment, the optimization module 440 is configured to determine common feature points between different target images, and use a mean value of corresponding three-dimensional coordinates as three-dimensional coordinates of the common feature points; acquiring three-dimensional coordinates of each feature point including the common feature point, absolute poses of each target image and corresponding relations among pixel coordinates of each feature point in the corresponding target image; determining the reprojection error of each common characteristic point in the corresponding target image by taking the absolute pose and the three-dimensional coordinates of the characteristic points as vertexes and connecting edges according to the corresponding relation through a preset graph optimization model; deleting the connecting edges with the reprojection errors larger than a preset value, iterating through a preset graph optimization model, and adjusting the absolute pose of each target image and the three-dimensional coordinates of the feature points until a preset iteration termination condition is reached; and taking the absolute pose after iteration termination as an optimized absolute pose. The preset iteration termination condition comprises a preset iteration number and/or a preset total weight projection error threshold value.

Referring to fig. 5, the apparatus for generating a guideboard of the present application further includes a third obtaining module 470 and a coordinate transforming module 480. The third obtaining module 470 is used for obtaining the camera parameters and the corresponding geographic coordinates and heading angle.

The coordinate transformation module 480 is configured to obtain a geographic coordinate corresponding to the optimized three-dimensional coordinate according to the optimized three-dimensional coordinate of the corner point, the preset camera parameter, and the corresponding geographic coordinate and the corresponding heading angle. That is, the coordinate conversion module can calculate and convert the optimized three-dimensional coordinates of each corner point into the geographic coordinates corresponding to each corner point according to the external parameters of the camera, the geographic coordinates and the course angle.

In conclusion, the device for generating the guideboard can improve the coordinate precision of the corner points of the guideboard, has higher robustness to deal with the processing of each guideboard, and is convenient for generating the guideboard at an accurate position in a high-precision map more accurately.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Referring to fig. 6, the electronic device 1000 includes a memory 1010 and a processor 1020.

The Processor 1020 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 1010 may include various types of storage units, such as system memory, Read Only Memory (ROM), and permanent storage. Wherein the ROM may store static data or instructions that are needed by the processor 1020 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at runtime. Further, the memory 1010 may comprise any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, among others. In some embodiments, memory 1010 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a digital versatile disc read only (e.g., DVD-ROM, dual layer DVD-ROM), a Blu-ray disc read only, an ultra-dense disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disk, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.

The memory 1010 has stored thereon executable code that, when processed by the processor 1020, may cause the processor 1020 to perform some or all of the methods described above.

Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.

Alternatively, the present application may also be embodied as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) having executable code (or a computer program or computer instruction code) stored thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the various steps of the above-described method according to the present application.

Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating a guideboard, comprising:

2. The method of claim 1, wherein the obtaining multiple frames of target images comprises:

3. The method according to claim 1, wherein after the extracting the feature points in each of the target images respectively and matching the feature points in two adjacent frames of the target images to obtain the relative poses of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points, the method further comprises:

4. The method of claim 1, wherein performing graph optimization according to the absolute pose and the three-dimensional coordinates and pixel coordinates of the feature points to obtain an optimized absolute pose comprises:

5. The method of claim 4, wherein:

the preset iteration termination condition comprises a preset iteration number and/or a preset total weight projection error threshold value.

6. The method according to any one of claims 1 to 5, wherein after the extracting the feature points in each of the target images, further comprises:

7. A guideboard generation apparatus, comprising:

the second acquisition module is used for respectively extracting feature points in each target image, matching the feature points in two adjacent frames of the target images and acquiring the relative pose of the two adjacent frames of the target images and the three-dimensional coordinates corresponding to the feature points;

8. The apparatus of claim 7,

the second obtaining module is further configured to obtain, from the feature points, corner points corresponding to the target guideboard and three-dimensional coordinates of the corner points; and/or

9. An electronic device, comprising:

a processor; and

a memory having executable code stored thereon, which when executed by the processor, causes the processor to perform the method of any one of claims 1-6.

10. A computer-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the method of any of claims 1-6.