CN114840703A - Pose information acquisition method, device, equipment, medium and product - Google Patents
Pose information acquisition method, device, equipment, medium and product Download PDFInfo
- Publication number
- CN114840703A CN114840703A CN202210334642.XA CN202210334642A CN114840703A CN 114840703 A CN114840703 A CN 114840703A CN 202210334642 A CN202210334642 A CN 202210334642A CN 114840703 A CN114840703 A CN 114840703A
- Authority
- CN
- China
- Prior art keywords
- live
- action image
- camera
- action
- projection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000009466 transformation Effects 0.000 claims abstract description 56
- 238000005457 optimization Methods 0.000 claims abstract description 38
- 238000005096 rolling process Methods 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims abstract description 17
- 238000004364 calculation method Methods 0.000 claims abstract description 16
- 230000001131 transforming effect Effects 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 33
- 230000006870 function Effects 0.000 claims description 17
- 238000005259 measurement Methods 0.000 claims description 17
- 238000005295 random walk Methods 0.000 claims description 16
- 238000013519 translation Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 230000010354 integration Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 241000274965 Cyrestis thyodamas Species 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/56—Information retrieval; Database structures therefor; File system structures therefor of still image data having vectorial format
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
- G01C21/3804—Creation or updating of map data
- G01C21/3833—Creation or updating of map data characterised by the source of data
- G01C21/3841—Data obtained from two or more sources, e.g. probe vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Databases & Information Systems (AREA)
- Radar, Positioning & Navigation (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Computer Graphics (AREA)
- Navigation (AREA)
Abstract
The embodiment of the disclosure discloses a pose information acquisition method, a pose information acquisition device, a pose information acquisition equipment, a pose information acquisition medium and a pose information acquisition product, wherein the method comprises the following steps: acquiring a live-action image and three-dimensional coordinates of feature points on the live-action image; calculating a motion compensation quantity based on the rolling shutter exposure time delay and the angular speed of the live-action image by taking the camera speed of the live-action image as a variable; compensating the transformation process of transforming the three-dimensional coordinates of the characteristic points to the coordinates of the projection pixel points by using the motion compensation quantity to obtain the coordinates of the projection pixel points of the characteristic points projected to the live-action image; calculating the difference value between the projection pixel point coordinates on the live-action image and the real pixel point coordinates on the live-action image to obtain the re-projection error of the characteristic points; and establishing a re-projection constraint according to the re-projection errors of the feature points, and optimizing the initial pose data of the n frames of live-action images participating in optimization to obtain the optimized pose information of the live-action images. According to the technical scheme, high-precision pose data can be obtained when pose calculation is carried out on data collected by a low-cost sensor.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of high-precision maps, in particular to a pose information acquisition method, a pose information acquisition device, pose information acquisition equipment, a pose information acquisition medium and a pose information acquisition product.
Background
With the development of science and technology, advanced assistant driving and automatic driving become a hot spot of technical research because of the ability to help drivers to drive safely. Currently, one technique for advanced driver assistance and autonomous driving requires reliance on "high-precision maps" in the path. Compared with a common map, the high-precision map can be more refined and more accurately express the real world, the large-range, high-frequency and high-precision updating of the high-precision map is the most critical problem in the field of the high-precision map, and the functional safety of the map in driving is determined.
In the field of high-precision map making, a collection vehicle carrying high-precision combined inertial navigation, a laser radar and other sensors is generally adopted to collect map data to make a high-precision map at present; however, these sensors are too costly to deploy on a large scale and cannot meet the frequent updating requirements. Therefore, a low-cost data acquisition scheme is provided at present, acquisition equipment integrated with cheap visual inertial navigation equipment, a visual sensor (such as a camera) and a positioning module and other sensor modules can be configured on a crowdsourcing vehicle for data acquisition, then the update of a high-precision map is completed according to the acquired data, the crowdsourcing vehicle is widely distributed, the data can be acquired in daily trip, the data acquisition is widely and rapidly distributed, the large-range and high-frequency update of the high-precision map can be met, but because the accuracy of the cheap sensors is limited, the accuracy of the data acquired by the sensors is insufficient, and the high-precision requirement of the high-precision map cannot be met.
Disclosure of Invention
The embodiment of the disclosure provides a pose information acquisition method, a pose information acquisition device, pose information acquisition equipment, a pose information acquisition medium and a pose information acquisition product.
In a first aspect, an embodiment of the present disclosure provides a pose information acquisition method.
Specifically, the pose information acquiring method includes:
acquiring a live-action image acquired by a camera and three-dimensional coordinates of feature points on the live-action image;
calculating a motion compensation quantity based on the rolling shutter exposure time delay of the camera and the angular speed corresponding to the live-action image by taking the camera speed corresponding to the live-action image as a variable;
compensating the transformation process of transforming the three-dimensional coordinates of the feature points to the coordinates of the projection pixel points by using the motion compensation amount to obtain the coordinates of the projection pixel points projected by the feature points to the live-action image;
calculating the difference value between the projection pixel point coordinates on the live-action image and the corresponding real pixel point coordinates on the live-action image to obtain the reprojection error of the characteristic points;
and establishing a re-projection constraint according to the re-projection errors of the feature points, and optimizing initial pose data of n frames of live-action images participating in optimization to obtain optimized pose information of the live-action images, wherein n is an integer greater than 1.
In a possible implementation manner, the calculating a motion compensation amount based on a rolling exposure delay and an angular velocity of the camera with a camera velocity as a variable includes:
calculating the product of the camera speed and the roller shutter exposure time delay to obtain a translation compensation amount;
calculating to obtain rotation compensation quantity by taking e as a base number and taking the product of the angular speed and the shutter exposure time delay as an index;
and constructing a motion compensation amount based on the translation compensation amount and the rotation compensation amount.
In a possible implementation manner, the compensating, by using the motion compensation amount, a transformation process of transforming the feature point from the three-dimensional coordinate to a projection pixel point coordinate to obtain a projection pixel point coordinate of the feature point projected onto a live-action image includes:
compensating a coordinate transformation matrix based on the motion compensation amount to obtain a compensation transformation matrix, wherein the coordinate transformation matrix comprises a transformation matrix from a world coordinate system to a camera coordinate system;
and carrying out coordinate change on the three-dimensional coordinates of the characteristic points by using the compensation transformation matrix and the coordinate transformation function to obtain the coordinates of the projection pixel points.
In a possible implementation manner, the establishing of the reprojection constraint according to the reprojection error of the feature point, and the optimizing of the pose data of the n frames of live-action images participating in the optimization, to obtain the pose information of the live-action image includes:
calculating the sum of pre-integral errors between adjacent live-action images in the n frames of live-action images and the sum of random walk errors of inertial navigation deviation rate;
accumulating the sum of the reprojection errors of each feature point in the n frames of live-action images, the sum of the pre-integration errors and the sum of the random walk errors to obtain an error sum;
and when the error sum is minimum, calculating the corresponding camera pose, the camera speed and the inertial navigation measurement value deviation of each live-action image.
In one possible implementation manner, the acquiring three-dimensional coordinates of the feature point on the live-action image includes:
obtaining a three-dimensional vector map containing the geographical position reflected by the live-action image;
establishing a matching relation between vector map elements in the three-dimensional vector map and image elements in the live-action image;
and obtaining the three-dimensional coordinates of the feature points on the live-action image in the three-dimensional vector map based on the matching relation.
In a second aspect, an embodiment of the present disclosure provides a pose information acquisition apparatus.
Specifically, the pose information acquiring apparatus includes:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is configured to acquire a live-action image acquired by a camera and three-dimensional coordinates of feature points on the live-action image;
the compensation module is configured to calculate a motion compensation amount based on a rolling shutter exposure time delay of a camera and an angular speed corresponding to the live-action image by taking a camera speed corresponding to the live-action image as a variable;
the projection module is configured to compensate a transformation process of the feature point from the three-dimensional coordinate to a projection pixel point coordinate by using the motion compensation amount, and obtain a projection pixel point coordinate of the feature point projected on a real image;
the calculation module is configured to calculate a difference value between a projection pixel point coordinate on the live-action image and a corresponding real pixel point coordinate on the live-action image, so as to obtain a re-projection error of the feature point;
and the optimization module is configured to establish a re-projection constraint according to the re-projection error of the feature point, optimize initial pose data of n frames of live-action images participating in optimization, and obtain optimized pose information of the live-action images, wherein n is an integer greater than 1.
In one possible implementation, the compensation module may be configured to:
calculating the product of the camera speed and the roller shutter exposure time delay to obtain a translation compensation amount;
calculating to obtain rotation compensation quantity by taking e as a base number and taking the product of the angular speed and the shutter exposure time delay as an index;
and constructing a motion compensation amount based on the translation compensation amount and the rotation compensation amount.
In one possible implementation, the projection module may be configured to:
compensating a coordinate transformation matrix based on the motion compensation amount to obtain a compensation transformation matrix, wherein the coordinate transformation matrix comprises a transformation matrix from a world coordinate system to a camera coordinate system;
and carrying out coordinate change on the three-dimensional coordinates of the characteristic points by using the compensation transformation matrix and the coordinate transformation function to obtain the coordinates of the projection pixel points.
In one possible implementation, the optimization module may be configured to:
calculating the sum of pre-integral errors between adjacent live-action images in the n frames of live-action images and the sum of random walk errors of inertial navigation deviation rate;
accumulating the sum of the reprojection errors of each feature point in the n frames of live-action images, the sum of the pre-integration errors and the sum of the random walk errors to obtain an error sum;
and when the error sum is minimum, calculating the corresponding camera pose, the camera speed and the inertial navigation measurement value deviation of each live-action image.
In one possible implementation, the obtaining module is configured to:
obtaining a three-dimensional vector map containing the geographical position reflected by the live-action image;
establishing a matching relation between vector map elements in the three-dimensional vector map and image elements in the live-action image;
and obtaining the three-dimensional coordinates of the feature points on the live-action image in the three-dimensional vector map based on the matching relationship.
In a third aspect, the disclosed embodiments provide an electronic device, including a memory for storing one or more computer instructions that support the above apparatus to perform the above method, and a processor configured to execute the computer instructions stored in the memory.
In a fourth aspect, the disclosed embodiments provide a computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method steps of any of the above aspects.
In a fifth aspect, the disclosed embodiments provide a computer program product comprising computer programs/instructions, wherein the computer programs/instructions, when executed by a processor, implement the method steps of any of the above aspects.
In a sixth aspect, an embodiment of the present disclosure provides a navigation method, where a navigation route calculated based on at least a starting point, an end point, and a road condition is obtained based on a high-precision map, and navigation guidance is performed based on the navigation route, where the high-precision map is implemented by reconstructing a map based on pose data obtained in any one of the above methods.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
according to the technical scheme, when pose calculation is carried out by using data collected by a low-cost sensor, re-projection constraint is constructed to optimize initial pose data, when re-projection constraint is constructed, the influence of the camera pose and speed on re-projection is considered while the shutter exposure time delay of a camera is considered, the calculation precision of re-projection errors can be improved, the optimization precision of the pose data is further improved, a good basis is provided for high-precision map element updating, and therefore a high-precision map can be updated conveniently.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of embodiments of the disclosure.
Drawings
Other features, objects, and advantages of embodiments of the disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:
fig. 1 shows a flowchart of a pose information acquisition method according to an embodiment of the present disclosure.
Fig. 2 illustrates a pose optimization factor graph according to an embodiment of the present disclosure.
Fig. 3 shows a block diagram of the configuration of a pose information acquisition apparatus according to an embodiment of the present disclosure.
Fig. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure.
FIG. 5 is a block diagram of a computer system suitable for use in implementing the methods according to embodiments of the present disclosure.
Detailed Description
Hereinafter, exemplary embodiments of the disclosed embodiments will be described in detail with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.
In the embodiments of the present disclosure, it is to be understood that terms such as "including" or "having", etc., are intended to indicate the presence of the features, numerals, steps, actions, components, parts, or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, numerals, steps, actions, components, parts, or combinations thereof are present or added.
It should be further noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
As mentioned above, with the development of technology, advanced assistant driving and automatic driving become a hot spot of technical research because they can help drivers to drive safely. Currently, one technique for advanced driver assistance and autonomous driving requires reliance on "high-precision maps" in the path. Compared with a common map, the high-precision map can be more refined and more accurately express the real world, the large-range, high-frequency and high-precision updating of the high-precision map is the most critical problem in the field of the high-precision map, and the functional safety of the map in driving is determined. In the field of high-precision map making, a collection vehicle carrying high-precision combined inertial navigation, a laser radar and other sensors is generally adopted to collect map data to make a high-precision map at present; however, these sensors are too costly to deploy on a large scale and cannot meet the frequent updating requirements. Therefore, a low-cost data acquisition scheme is provided at present, acquisition equipment integrated with cheap visual inertial navigation equipment, a visual sensor (such as a camera) and a positioning module and other sensor modules can be configured on a crowdsourcing vehicle for data acquisition, then the update of a high-precision map is completed according to the acquired data, the crowdsourcing vehicle is widely distributed, the data can be acquired in daily trip, the data acquisition is widely and rapidly distributed, the large-range and high-frequency update of the high-precision map can be met, but because the accuracy of the cheap sensors is limited, the accuracy of the data acquired by the sensors is insufficient, and the high-precision requirement of the high-precision map cannot be met.
In order to solve the above problems, the present disclosure provides a pose information acquisition scheme, where when pose calculation is performed using data acquired by a low-cost sensor, a re-projection constraint is constructed to optimize initial pose data, and when the re-projection constraint is constructed, influence of camera pose and speed on re-projection is also considered while considering shutter exposure delay of a camera, so that calculation accuracy of a re-projection error can be improved, optimization accuracy of pose data is improved, a good basis is provided for high-precision map element update, and a high-precision map is updated.
According to the technical scheme provided by the disclosure, when the real-scene images of the road traffic area are obtained, the pose information of each real-scene image (for continuous video frames, each video frame) can be calculated with high precision. The pose information of the live-action image is actually the position and angle information of the camera that acquired the live-action image at the time of acquisition. After the high-precision pose information is acquired, the pose data of the live-action image can be applied to update the map of the high-precision map, and certainly, the high-precision pose information can be used for positioning calculation in positioning scenes of automatic driving or auxiliary driving or indoor navigation positioning scenes of a robot.
Fig. 1 shows a flowchart of a pose information acquisition method according to an embodiment of the present disclosure, and as shown in fig. 1, the map data update method includes the following steps S101 to S105:
in step S101, a live-action image acquired by a camera and three-dimensional coordinates of feature points on the live-action image are acquired;
in step S102, calculating a motion compensation amount based on a rolling exposure delay of a camera and an angular velocity corresponding to the live-action image, with a camera velocity corresponding to the live-action image as a variable;
in step S103, compensating a transformation process of transforming the feature point from the three-dimensional coordinate to a projection pixel point coordinate by using the motion compensation amount, so as to obtain a projection pixel point coordinate of the feature point projected onto the live-action image;
in step S104, calculating a difference between the coordinates of the projection pixel points on the live-action image and the coordinates of the corresponding real pixel points on the live-action image, to obtain a reprojection error of the feature points;
in step S105, a reprojection constraint is established according to the reprojection error of the feature point, and the initial pose data of the n frames of live-action images participating in optimization is optimized to obtain the optimized pose information of the live-action images, where n is an integer greater than 1.
In one possible embodiment, the pose information acquisition method is applicable to a computer, a computing device, an electronic device, a server, a service cluster, and the like that can perform pose information acquisition.
In one possible embodiment, a low-cost visual inertial navigation sensor, a visual sensor and a positioning sensor can be integrated in the acquisition device of the present disclosure. The visual Inertial navigation sensor may be an IMU (Inertial Measurement Unit) for measuring angular velocity and acceleration of the object; the visual sensor can be a camera and is used for shooting and obtaining a live-action image of a road traffic area, the exposure mode of the camera in the disclosure is rolling shutter exposure, and the rolling shutter exposure refers to sequential line-by-line exposure; the Positioning module may be a Global Positioning System (GPS) module for Positioning a current position. Based on the data collected by the sensors, initial pose data of each live-action image can be calculated by a VIO (Visual Inertial odometer) algorithm.
In a possible embodiment, due to the limited accuracy of the low-cost sensors, the accuracy of the pose data calculated by the data collected by such sensors is insufficient, and the high accuracy requirement cannot be met, so that the initial pose data needs to be optimized, and the current pose optimization includes pose optimization based on reprojection constraint. The real three-dimensional space point corresponds to a difference value between an actual coordinate of a pixel point on the live-action image and a re-projection coordinate (namely, a projection pixel point coordinate projected onto the live-action image by the real three-dimensional space point calculated by an internal reference model of the camera and a camera pose), because of the problems of the camera pose, the three-dimensional space point coordinate precision and the like, the calculated re-projection coordinate does not completely accord with the actual coordinate, namely, the difference value cannot be exactly zero, and at the moment, the sum of the difference values needs to be minimized to be used as a re-projection constraint so as to obtain an optimized camera pose parameter and a coordinate of the three-dimensional space point and obtain optimized pose data.
In a possible implementation, when performing the reprojection constraint, a live-action image captured by a camera and three-dimensional coordinates of feature points on the live-action image may be obtained, where the feature points refer to points that can be used to identify some target objects such as road signs on the live-action image, and generally refer to points with a drastically changed gray value or points with a large curvature (e.g., intersection of two edges) on an edge of the image as feature points of the image. The feature points comprise two parts, namely Key points (Key-points) and descriptors (Descriptors), wherein the Key points express three-dimensional position coordinates of the feature points, and the descriptors describe visual characteristics of the feature points and are mostly in a vector form. Here, when performing VIO calculation, the three-dimensional coordinates of the feature point can be directly calculated.
In a possible implementation mode, the method not only considers the shutter exposure time delay of the camera, but also considers the influence of the pose and the speed of the camera on the reprojection when constructing the reprojection constraint, and compensates the transformation process of the feature point from the three-dimensional coordinate to the coordinate of the projection pixel point, for example, the calculation function of the motion compensation quantity can be M compensate (v，w m T), where v in the function is the camera speed, belonging to the variable, w m And measuring the angular velocity corresponding to the live-action image where the characteristic point is located by the visual inertial navigation sensor when the camera shoots the live-action image, wherein t is the rolling exposure time delay of the camera.
In a possible implementation manner, after the motion compensation amount is used to compensate the transformation process of the feature point from the three-dimensional coordinate to the projection pixel point coordinate, and the projection pixel point coordinate of the feature point projected onto the real image is obtained, the difference between the projection pixel point coordinate of the real image and the corresponding real pixel point coordinate of the real image can be calculated, so as to obtain the re-projection error of the feature point; if the real-scene image participating in optimization has n frames, the reprojection errors of all the feature points in the n frames of image can be calculated, reprojection constraint is constructed, and the sum of the difference values is minimized to obtain the optimization pose information of the real-scene image; it should be noted here that the method for optimizing pose information using reprojection constraint is clear to those skilled in the art and is not described here again.
When the re-projection constraint is constructed, the roller shutter exposure time delay of the camera is considered, and the influence of the position and the speed of the camera on the re-projection is also considered, so that the calculation precision of the re-projection error can be improved, the optimization precision of the position and attitude data is improved, high-precision camera position and attitude information is provided for positioning during high-precision map updating, positioning during automatic driving or auxiliary driving, or positioning in an indoor navigation positioning scene of a robot, and the positioning precision is improved.
In a possible implementation manner, the step S102 of the pose information acquiring method, which is to calculate the motion compensation amount based on the rolling exposure delay and the angular velocity of the camera with the camera velocity as a variable, includes:
calculating the product of the camera speed and the roller shutter exposure time delay to obtain a translation compensation amount;
calculating to obtain rotation compensation quantity by taking e as a base number and taking the product of the angular speed and the shutter exposure time delay as an index;
and constructing a motion compensation amount based on the translation compensation amount and the rotation compensation amount.
In this embodiment, the motion compensation amount M can be calculated by the following formula compensate (v，w m ，t)：
Wherein, Rptation is Rotation compensation quantity, and Rotation Exp (w) m *t)，w m The angular velocity, t, translation, v is the camera velocity, v is the variable, and w is the translation compensation amount m The rolling shutter exposure time delay t is a preset parameter of the camera, and is an angular velocity measured by the vision inertial navigation sensor when the camera shoots a live-action image of the characteristic point.
In a possible implementation manner, in step S103 of the pose information acquiring method, the step of using the motion compensation amount to compensate a transformation process of transforming the feature point from the three-dimensional coordinate to a projection pixel point coordinate to obtain a projection pixel point coordinate of the feature point projected on the real-scene image includes the following steps:
compensating a coordinate transformation matrix based on the motion compensation amount to obtain a compensation transformation matrix, wherein the coordinate transformation matrix comprises a transformation matrix from a world coordinate system to a camera coordinate system;
and carrying out coordinate change on the three-dimensional coordinates of the characteristic points by using the compensation transformation matrix and the coordinate transformation function to obtain the coordinates of the projection pixel points.
In this embodiment, the world coordinate system (wcs) is a convention for defining coordinates [0, 0, 0] in 3D virtual space and three unit axes orthogonal to each other, which is the present initial meridian of the 3D scene, a reference for measuring any other point or any other arbitrary coordinate system, and the origin of the world coordinates is fixed; the three-dimensional coordinates of the feature points obtained in the present disclosure are three-dimensional coordinates of the feature points in a world coordinate system. The camera coordinate system is a three-dimensional coordinate system constructed with the camera as the origin of the coordinate system.
In this embodiment, the coordinate transformation matrix from the world coordinate system to the camera coordinate system is constructed by the variable camera pose x to be solved, which may be denoted as m (x), and the coordinate transformation function from the camera coordinate system to the pixel coordinate is proj (); it should be noted that the specific formulas of m (x) and proj () are clear to those skilled in the art and are not described herein.
In this embodiment, the compensation transformation matrix may be the product of the motion compensation amount and the coordinate transformation matrix, i.e., M compensate (v m ，w m T) M (x); assuming that P is the three-dimensional coordinate of the feature point, the compensation transformation matrix is first used to convert the three-dimensional coordinate of the feature point from the world coordinate system to the camera coordinate system to obtain M compensate (v m ，w m T) M (x) P; then, using the coordinate transformation function proj (), for the featureConverting the three-dimensional coordinates of the point in the camera coordinate system into the coordinates of the projection pixel point, and obtaining the coordinates of the projection pixel point as proj (M) compensate (v m ，w m ，t)M(x)P)。
Thus, the difference value between the projection pixel point coordinate on the real scene image and the corresponding real pixel point coordinate p on the real scene image is calculated to obtain the reprojection error of the characteristic point reprojection ：
error reprojection ＝proj(M compensate (v m ，w m ，t)M(x)P)-p。
In a possible implementation manner, the step S105 of the pose information acquiring method, namely, establishing a reprojection constraint according to the reprojection error of the feature point, and optimizing the pose data of the n frames of live-action images participating in the optimization to obtain the pose information of the live-action image, includes the following steps:
calculating the sum of pre-integral errors between adjacent live-action images in the n frames of live-action images and the sum of random walk errors of inertial navigation deviation rate bias;
accumulating the sum of the reprojection errors of each feature point in the n frames of live-action images, the sum of the pre-integration errors and the sum of the random walk errors to obtain an error sum;
and when the error sum is minimum, calculating the camera pose, the camera speed and the inertial navigation measurement value deviation corresponding to each live-action image.
In this embodiment, FIG. 2 shows a pose optimization factor graph according to an embodiment of the present disclosure, where 201 is the camera pose variable { x } 1 ，x 2 ，…，x n 202 is a camera speed variable { v } 1 ，v 2 ，…，v n 203 is inertial navigation measurement value deviation variable (bias) of inertial navigation accelerometer and gyroscope 1 ，b 2 ，…，b n 204 is a reprojection constraint, 205 is an inertial navigation pre-integration constraint, and 206 is an inertial navigation bias random walk constraint; therefore, the pose optimization can be described by the following equation:
{x 1 ，x 2 ，...，x n ，v 1 ，v 2 ，...，v n ，b 1 ，b 2 ，...，b n }＝argmin error sum ；
wherein n is the number of the live-action images participating in optimization, m is the total number of the feature points in the n frames of live-action images, and each live-action image corresponds to one camera pose, one camera speed and one bias vector; the corresponding camera pose variable of the n frames of live-action images is { x } 1 ，x 2 ，…，x n The camera speed variable is { v } 1 ，v 2 ，…，v n The deviation rate variable of the inertial navigation measured value is { b } 1 ，b 2 ，…，b n }。error reprojection Is the reprojection error of each feature point, which can be calculated according to the scheme provided in the above embodiment, error preintegration Is the pre-integration error between adjacent live-action images; error rw Random walk error which is the inertial navigation measurement value deviation between adjacent images; the inertial navigation measurement value deviation error preintegration And inertial navigation measurement deviation error rw The calculation process is well known to those skilled in the art and is not limited herein. The sum of pre-integration errors between adjacent frames of live-action images in the n frames of live-action images can be calculated as
The sum of random walk errors of inertial navigation measured value deviations among n live-action images is
The sum of the reprojection errors of m characteristic points in the n frames of live-action images is
The error sum error can be obtained by adding the three error values sum 。
In this embodiment, in the execution postureWhen optimizing, the error sum error can be calculated sum Minimization to obtain error sum Minimized variable x 1 ，x 2 ，...，x n ，v 1 ，v 2 ，...，v n ，b 1 ，b 2 ，...，b n Solving by a common nonlinear optimization algorithm; therefore, the camera pose, the camera speed and the inertial navigation measurement value deviation corresponding to each live-action image can be obtained, and pose optimization is completed.
In a possible implementation manner, the step S101 of acquiring three-dimensional coordinates of feature points on the live-action image in the pose information acquiring method may include the following steps:
obtaining a three-dimensional vector map containing the geographical position reflected by the live-action image;
establishing a matching relation between vector map elements in the three-dimensional vector map and image elements in the live-action image;
and obtaining the three-dimensional coordinates of the feature points on the live-action image in the three-dimensional vector map based on the matching relation.
In this embodiment, the three-dimensional coordinates of each feature point in the live-action map may be derived from the three-dimensional coordinates of known feature points in the high-precision map.
In this embodiment, the three-dimensional vector map is a three-dimensional map that has been established in advance, and may be, for example, a high-precision map that is currently being used by a user.
In this embodiment, when a live-action image is obtained by shooting with a camera, initial pose information of the live-action image can be obtained according to positioning information measured by a positioning module and inertial navigation information measured by a visual inertial navigation sensor during obtaining; the position of the live-action image can be determined according to the initial pose information, so that a part of the three-dimensional vector map containing the geographic position reflected by the live-action image can be selected from a three-dimensional vector map library.
In this embodiment, the vector map elements and the image elements may be lane lines and guideboards, a matching relationship may be established between the guideboard elements "500 m apart from a distance a" in the three-dimensional vector map and the guideboard elements "500 m apart from a distance a" in the live-action image, and the matching relationship between the three-dimensional vector map and the guideboards in the live-action image may be established if the three-dimensional vector map and the guideboards in the live-action image are determined to be the same guideboard.
In this embodiment, based on the matching relationship, a matching point of a feature point on the live-action image in the three-dimensional vector map can be obtained, and the three-dimensional coordinates of the feature point on the live-action image are the three-dimensional coordinates of the corresponding matching point in the three-dimensional vector map.
The three-dimensional coordinates of the feature points obtained in the embodiment are more accurate, and more accurate reprojection errors can be calculated, so that the pose information for positioning is more accurate.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.
Fig. 3 is a block diagram showing a configuration of a pose information acquisition apparatus according to an embodiment of the present disclosure, which may be implemented as part or all of an electronic device by software, hardware, or a combination of both. As shown in fig. 3, the map data updating apparatus includes:
an acquisition module 301 configured to acquire a live-action image acquired by a camera and three-dimensional coordinates of feature points on the live-action image;
a compensation module 302 configured to calculate a motion compensation amount based on a rolling exposure delay of a camera and an angular velocity corresponding to the live-action image, with a camera velocity corresponding to the live-action image as a variable;
a projection module 303, configured to compensate a transformation process of transforming the feature point from the three-dimensional coordinate to a projection pixel point coordinate by using the motion compensation amount, so as to obtain a projection pixel point coordinate of the feature point projected onto the live-action image;
a calculating module 304, configured to calculate a difference between a projection pixel coordinate on the live-action image and a corresponding real pixel coordinate on the live-action image, so as to obtain a re-projection error of the feature point;
and the optimizing module 305 is configured to establish a re-projection constraint according to the re-projection error of the feature point, and optimize initial pose data of n frames of live-action images participating in optimization to obtain optimized pose information of the live-action images, where n is an integer greater than 1.
In one possible embodiment, the pose information acquisition apparatus is applicable to a computer, a computing device, an electronic device, a server, a service cluster, or the like that can perform pose information acquisition.
In one possible embodiment, a low-cost visual inertial navigation sensor, a visual sensor and a positioning sensor can be integrated in the acquisition device of the present disclosure. The visual Inertial navigation sensor may be an IMU (Inertial Measurement Unit) for measuring angular velocity and acceleration of the object; the visual sensor can be a camera and is used for shooting and obtaining a live-action image of a road traffic area, the exposure mode of the camera in the disclosure is rolling shutter exposure, and the rolling shutter exposure refers to sequential line-by-line exposure; the Positioning module may be a Global Positioning System (GPS) module for Positioning a current position. Based on the data collected by the sensors, initial pose data of each live-action image can be calculated by a VIO (Visual Inertial odometer) algorithm.
In a possible embodiment, due to the limited accuracy of the low-cost sensors, the accuracy of the pose data calculated by the data collected by such sensors is insufficient, and the high accuracy requirement cannot be met, so that the initial pose data needs to be optimized, and the current pose optimization includes pose optimization based on reprojection constraint. The real three-dimensional space point corresponds to a difference value between an actual coordinate of a pixel point on the live-action image and a re-projection coordinate (namely, a projection pixel point coordinate projected onto the live-action image by the real three-dimensional space point calculated by an internal reference model of the camera and a camera pose), because of the problems of the camera pose, the three-dimensional space point coordinate precision and the like, the calculated re-projection coordinate does not completely accord with the actual coordinate, namely, the difference value cannot be exactly zero, and at the moment, the sum of the difference values needs to be minimized to be used as a re-projection constraint so as to obtain an optimized camera pose parameter and a coordinate of the three-dimensional space point and obtain optimized pose data.
In a possible implementation, when performing the reprojection constraint, a live-action image captured by a camera and three-dimensional coordinates of feature points on the live-action image may be obtained, where the feature points refer to points that can be used to identify some target objects such as road signs on the live-action image, and generally refer to points with a drastically changed gray value or points with a large curvature (e.g., intersection of two edges) on an edge of the image as feature points of the image. The feature points comprise Key points (Key-points) and descriptors (Descriptors), wherein the Key points express three-dimensional position coordinates of the feature points, and the descriptors are description of visual characteristics of the feature points and mostly in a vector form. Here, when performing VIO calculation, the three-dimensional coordinates of the feature point can be directly calculated.
In a possible implementation mode, the method not only considers the shutter exposure time delay of the camera, but also considers the influence of the pose and the speed of the camera on the reprojection when constructing the reprojection constraint, and compensates the transformation process of the feature point from the three-dimensional coordinate to the coordinate of the projection pixel point, for example, the calculation function of the motion compensation quantity can be M compensate (v，w m T), where v in the function is the camera speed, belonging to the variable, w m And measuring the angular velocity corresponding to the live-action image where the characteristic point is located by the visual inertial navigation sensor when the camera shoots the live-action image, wherein t is the rolling exposure time delay of the camera.
In a possible implementation manner, after the motion compensation amount is used to compensate the transformation process of the feature point from the three-dimensional coordinate to the projection pixel point coordinate, and the projection pixel point coordinate of the feature point projected onto the real image is obtained, the difference between the projection pixel point coordinate of the real image and the corresponding real pixel point coordinate of the real image can be calculated, so as to obtain the re-projection error of the feature point; if the real-scene image participating in optimization has n frames, the reprojection errors of all the feature points in the n frames of image can be calculated, reprojection constraint is constructed, and the sum of the difference values is minimized to obtain the optimization pose information of the real-scene image; it should be noted here that the method for optimizing pose information using reprojection constraint is clear to those skilled in the art and is not described here again.
When the re-projection constraint is constructed, the roller shutter exposure time delay of the camera is considered, and the influence of the position and the speed of the camera on the re-projection is also considered, so that the calculation precision of the re-projection error can be improved, the optimization precision of the position and attitude data is improved, high-precision camera position and attitude information is provided for positioning during high-precision map updating, positioning during automatic driving or auxiliary driving, or positioning in an indoor navigation positioning scene of a robot, and the positioning precision is improved.
In one possible implementation, the compensation module 302 may be configured to:
calculating the product of the camera speed and the roller shutter exposure time delay to obtain a translation compensation amount;
calculating to obtain rotation compensation quantity by taking e as a base number and taking the product of the angular speed and the shutter exposure time delay as an index;
and constructing a motion compensation amount based on the translation compensation amount and the rotation compensation amount.
In this embodiment, the motion compensation amount M can be calculated by the following formula compensate (v，w m ，t)：
Wherein Rotation is a Rotation compensation amount, and Rotation is Exp (w) m *t)，w m The angular velocity, t, translation, v is the camera velocity, v is the variable, and w is the translation compensation amount m The rolling shutter exposure time delay t is a preset parameter of the camera, and is an angular velocity measured by the vision inertial navigation sensor when the camera shoots a live-action image of the characteristic point.
In one possible implementation, the projection module 303 may be configured to:
compensating a coordinate transformation matrix based on the motion compensation amount to obtain a compensation transformation matrix, wherein the coordinate transformation matrix comprises a transformation matrix from a world coordinate system to a camera coordinate system;
and carrying out coordinate change on the three-dimensional coordinates of the characteristic points by using the compensation transformation matrix and the coordinate transformation function to obtain the coordinates of the projection pixel points.
In this embodiment, the world coordinate system (wcs) is a convention for defining coordinates [0, 0, 0] in 3D virtual space and three unit axes orthogonal to each other, which is the present initial meridian of the 3D scene, a reference for measuring any other point or any other arbitrary coordinate system, and the origin of the world coordinates is fixed; the three-dimensional coordinates of the feature points obtained in the present disclosure are three-dimensional coordinates of the feature points in a world coordinate system. The camera coordinate system is a three-dimensional coordinate system constructed with the camera as the origin of the coordinate system.
In this embodiment, the coordinate transformation matrix from the world coordinate system to the camera coordinate system is constructed by the variable camera pose x to be solved, which may be denoted as m (x), and the coordinate transformation function from the camera coordinate system to the pixel coordinate is proj (); it should be noted that the specific formulas of m (x) and proj () are clear to those skilled in the art and are not described herein.
In this embodiment, the compensation transformation matrix may be the product of the motion compensation amount and the coordinate transformation matrix, i.e., M compensate (v m ，w m T) M (x); assuming that P is the three-dimensional coordinate of the feature point, the compensation transformation matrix is first used to convert the three-dimensional coordinate of the feature point from the world coordinate system to the camera coordinate system to obtain M compensate (v m ，w m T) M (x) P; then, the coordinate transformation function proj () is used to convert the three-dimensional coordinates of the feature points in the camera coordinate system into the projection pixel point coordinates, and the obtained projection pixel point coordinates are proj (M) compensate (v m ，w m ，t)M(x)P)。
Thus, the coordinates of the projection pixel points on the real image and the corresponding coordinates on the real image are calculatedObtaining the reprojection error of the characteristic point by the difference value between the real pixel point coordinates p reprojection ：
error reprojection ＝proj(M compensate (v m ，w m ，t)M(x)P)-p。
In one possible implementation, the optimization module 305 may be configured to:
calculating the sum of pre-integral errors between adjacent live-action images in the n frames of live-action images and the sum of random walk errors of inertial navigation deviation rate;
accumulating the sum of the reprojection errors of each feature point in the n frames of live-action images, the sum of the pre-integration errors and the sum of the random walk errors to obtain an error sum;
and when the error sum is minimum, calculating the corresponding camera pose, the camera speed and the inertial navigation measurement value deviation of each live-action image.
In this embodiment, FIG. 2 shows a pose optimization factor graph according to an embodiment of the present disclosure, where 201 is the camera pose variable { x } 1 ，x 2 ，…，x n 202 is a camera speed variable { v } 1 ，v 2 ，…，v n 203 is inertial navigation measurement value deviation variable (bias) of inertial navigation accelerometer and gyroscope 1 ，b 2 ，…，b n 204 is a reprojection constraint, 205 is an inertial navigation pre-integration constraint, and 206 is an inertial navigation bias random walk constraint; therefore, the pose optimization can be described by the following equation:
{x 1 ，x 2 ，...，x n ，v 1 ，v 2 ，...，v n ，b 1 ，b 2 ，...，b n }＝argminerror sum
wherein n is the number of the live-action images participating in optimization, m is the total number of the feature points in the n frames of live-action images, and each live-action image corresponds to one camera pose, oneA camera speed, and a bias vector; the corresponding camera pose variable of the n frames of live-action images is { x } 1 ，x 2 ，…，x n The camera speed variable is { v } 1 ，v 2 ，…，v n The deviation rate variable of the inertial navigation measured value is { b } 1 ，b 2 ，…，b n }。error reprojection Is the reprojection error of each feature point, which can be calculated according to the scheme provided in the above embodiment, error preintegration Is the pre-integration error between adjacent live-action images; error rw Random walk error which is the inertial navigation measurement value deviation between adjacent images; the inertial navigation measurement value deviation error preintegration And inertial navigation measurement deviation error rw The calculation process is well known to those skilled in the art and is not limited herein. The sum of the pre-integration errors between adjacent live-action images in the n live-action images can be calculated as
The sum of random walk errors of inertial navigation measured value deviations among n live-action images is
The sum of the reprojection errors of m characteristic points in the n frames of live-action images is
The error sum error can be obtained by adding the three error values sum 。
In this embodiment, the error sum error can be used for pose optimization sum Minimization to obtain error sum Minimized variable x 1 ，x 2 ，...，x n ，v 1 ，v 2 ，...，v n ，b 1 ，b 2 ，...，b n Solving by a common nonlinear optimization algorithm; therefore, the camera pose, the camera speed and the inertial navigation measurement value deviation corresponding to each live-action image can be obtained, and pose optimization is completed.
In one possible implementation, the obtaining module 301 is configured to:
obtaining a three-dimensional vector map containing the geographical position reflected by the live-action image;
establishing a matching relation between vector map elements in the three-dimensional vector map and image elements in the live-action image;
and obtaining the three-dimensional coordinates of the feature points on the live-action image in the three-dimensional vector map based on the matching relation.
In this embodiment, the three-dimensional vector map is a three-dimensional map that has been established in advance, and may be, for example, a high-precision map that is currently being used by a user.
In this embodiment, when a live-action image is obtained by shooting with a camera, initial pose information of the live-action image can be obtained according to positioning information measured by a positioning module and inertial navigation information measured by a visual inertial navigation sensor during obtaining; the position of the live-action image can be determined according to the initial pose information, so that a part of the three-dimensional vector map containing the geographic position reflected by the live-action image can be selected from a three-dimensional vector map library.
In this embodiment, the vector map elements and the image elements may be lane lines and guideboards, a matching relationship may be established between the guideboard elements "500 m apart from a distance a" in the three-dimensional vector map and the guideboard elements "500 m apart from a distance a" in the live-action image, and the matching relationship between the three-dimensional vector map and the guideboards in the live-action image may be established if the three-dimensional vector map and the guideboards in the live-action image are determined to be the same guideboard.
In this embodiment, based on the matching relationship, a matching point of a feature point on the live-action image in the three-dimensional vector map can be obtained, and the three-dimensional coordinates of the feature point on the live-action image are the three-dimensional coordinates of the corresponding matching point in the three-dimensional vector map.
The three-dimensional coordinates of the feature points obtained in the embodiment are more accurate, and more accurate reprojection errors can be calculated, so that the pose information for positioning is more accurate.
The embodiment of the disclosure also discloses a navigation service, wherein the positioning of the carrier is determined based on the carrier positioning method, and the navigation guidance service of the corresponding scene is provided for the carrier based on the positioning of the carrier. Wherein, the corresponding scene is one or a combination of more of AR navigation, overhead navigation or main and auxiliary road navigation.
The embodiment of the disclosure also discloses a navigation method, wherein a navigation route calculated at least based on a starting point, an end point and a road condition is obtained based on the high-precision map, and navigation guidance is carried out based on the navigation route, and the high-precision map is realized by reconstructing the map based on the pose data obtained by the method.
The present disclosure also discloses an electronic device, fig. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure, and as shown in fig. 4, the electronic device 400 includes a memory 401 and a processor 402; wherein the content of the first and second substances,
the memory 401 is used to store one or more computer instructions that are executed by the processor 402 to implement the above-described method steps.
FIG. 5 is a schematic block diagram of a computer system suitable for use in implementing methods according to embodiments of the present disclosure.
As shown in fig. 5, the computer system 500 includes a processing unit 501 that can execute various processes in the above-described embodiments according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the system 500 are also stored. The processing unit 501, the ROM502, and the RAM503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 55 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 55 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary. The processing unit 501 may be implemented as a CPU, a GPU, a TPU, an FPGA, an NPU, or other processing units.
In particular, the above described methods may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a medium readable thereby, the computer program comprising program code for performing the method described above. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.
As another aspect, the disclosed embodiment also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the embodiments of the present disclosure.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept. For example, the above features and (but not limited to) the features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (10)
1. A pose information acquisition method includes:
acquiring a live-action image acquired by a camera and three-dimensional coordinates of feature points on the live-action image;
calculating a motion compensation quantity based on the rolling shutter exposure time delay of the camera and the angular speed corresponding to the live-action image by taking the camera speed corresponding to the live-action image as a variable;
compensating the transformation process of transforming the three-dimensional coordinates of the feature points to the coordinates of the projection pixel points by using the motion compensation amount to obtain the coordinates of the projection pixel points projected by the feature points to the live-action image;
calculating the difference value between the projection pixel point coordinates on the live-action image and the corresponding real pixel point coordinates on the live-action image to obtain the reprojection error of the characteristic points;
and establishing a re-projection constraint according to the re-projection errors of the feature points, and optimizing initial pose data of n frames of live-action images participating in optimization to obtain optimized pose information of the live-action images, wherein n is an integer greater than 1.
2. The method of claim 1, wherein the calculating a motion compensation amount based on a rolling exposure delay and an angular velocity of the camera with a camera velocity as a variable comprises:
calculating the product of the camera speed and the roller shutter exposure time delay to obtain a translation compensation amount;
calculating to obtain rotation compensation quantity by taking e as a base number and taking the product of the angular speed and the shutter exposure time delay as an index;
and constructing a motion compensation amount based on the translation compensation amount and the rotation compensation amount.
3. The method according to claim 1, wherein the compensating the transformation process of the feature point from the three-dimensional coordinates to projected pixel point coordinates by using the motion compensation amount to obtain projected pixel point coordinates of the feature point projected on the real image comprises:
compensating a coordinate transformation matrix based on the motion compensation amount to obtain a compensation transformation matrix, wherein the coordinate transformation matrix comprises a transformation matrix from a world coordinate system to a camera coordinate system;
and carrying out coordinate change on the three-dimensional coordinates of the characteristic points by using the compensation transformation matrix and the coordinate transformation function to obtain the coordinates of the projection pixel points.
4. The method according to claim 1, wherein the establishing of the reprojection constraint according to the reprojection error of the feature point, and the optimizing of the pose data of the n frames of live-action images participating in the optimization to obtain the pose information of the live-action images comprises:
calculating the sum of pre-integral errors between adjacent live-action images in the n frames of live-action images and the sum of random walk errors of inertial navigation deviation rate;
accumulating the sum of the reprojection errors of each feature point in the n frames of live-action images, the sum of the pre-integration errors and the sum of the random walk errors to obtain an error sum;
and when the error sum is minimum, calculating the corresponding camera pose, the camera speed and the inertial navigation measurement value deviation of each live-action image.
5. The method of claim 1, wherein the obtaining three-dimensional coordinates of feature points on the live-action image comprises:
obtaining a three-dimensional vector map containing the geographical position reflected by the live-action image;
establishing a matching relation between vector map elements in the three-dimensional vector map and image elements in the live-action image;
and obtaining the three-dimensional coordinates of the feature points on the live-action image in the three-dimensional vector map based on the matching relation.
6. A pose information acquisition apparatus, comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is configured to acquire a live-action image acquired by a camera and three-dimensional coordinates of feature points on the live-action image;
the compensation module is configured to calculate a motion compensation amount based on a rolling shutter exposure time delay of a camera and an angular speed corresponding to the live-action image by taking a camera speed corresponding to the live-action image as a variable;
the projection module is configured to compensate a transformation process of the feature point from the three-dimensional coordinate to a projection pixel point coordinate by using the motion compensation amount, so as to obtain a projection pixel point coordinate of the feature point projected on a real image;
the calculation module is configured to calculate a difference value between a projection pixel point coordinate on the live-action image and a corresponding real pixel point coordinate on the live-action image, so as to obtain a re-projection error of the feature point;
and the optimization module is configured to establish a re-projection constraint according to the re-projection error of the feature point, optimize initial pose data of n frames of live-action images participating in optimization, and obtain optimized pose information of the live-action images, wherein n is an integer greater than 1.
7. An electronic device comprising a memory and at least one processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the at least one processor to implement the method steps of any one of claims 1-5.
8. A computer readable storage medium having stored thereon computer instructions which, when executed by a processor, carry out the method steps of any of claims 1-5.
9. A computer program product comprising computer programs/instructions, wherein the computer programs/instructions, when executed by a processor, implement the method steps of any of claims 1-5.
10. A navigation method, wherein a navigation route calculated at least based on a starting point, an end point and a road condition is obtained based on a high-precision map, navigation guidance is carried out based on the navigation route, and the high-precision map is realized by carrying out map reconstruction based on pose data obtained in any one of the methods of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210334642.XA CN114840703A (en) | 2022-03-30 | 2022-03-30 | Pose information acquisition method, device, equipment, medium and product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210334642.XA CN114840703A (en) | 2022-03-30 | 2022-03-30 | Pose information acquisition method, device, equipment, medium and product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114840703A true CN114840703A (en) | 2022-08-02 |
Family
ID=82564699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210334642.XA Pending CN114840703A (en) | 2022-03-30 | 2022-03-30 | Pose information acquisition method, device, equipment, medium and product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114840703A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110415344A (en) * | 2019-06-24 | 2019-11-05 | 武汉大学 | Motion compensation process based on event camera |
| US20200284590A1 (en) * | 2019-03-05 | 2020-09-10 | DeepMap Inc. | Distributed processing of pose graphs for generating high definition maps for navigating autonomous vehicles |
| WO2021057743A1 (en) * | 2019-09-27 | 2021-04-01 | Oppo广东移动通信有限公司 | Map fusion method, apparatus, device and storage medium |
| CN113838129A (en) * | 2021-08-12 | 2021-12-24 | 高德软件有限公司 | Method, device and system for obtaining pose information |
-
2022
- 2022-03-30 CN CN202210334642.XA patent/CN114840703A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200284590A1 (en) * | 2019-03-05 | 2020-09-10 | DeepMap Inc. | Distributed processing of pose graphs for generating high definition maps for navigating autonomous vehicles |
| CN110415344A (en) * | 2019-06-24 | 2019-11-05 | 武汉大学 | Motion compensation process based on event camera |
| WO2021057743A1 (en) * | 2019-09-27 | 2021-04-01 | Oppo广东移动通信有限公司 | Map fusion method, apparatus, device and storage medium |
| CN113838129A (en) * | 2021-08-12 | 2021-12-24 | 高德软件有限公司 | Method, device and system for obtaining pose information |
Non-Patent Citations (1)
| Title |
|---|
| 张慧智;高箴;周健;: "基于激光视觉技术的运动目标位姿测量与误差分析", 激光杂志, no. 04, 25 April 2020 (2020-04-25) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190219700A1 (en) | Iterative closest point process based on lidar with integrated motion estimation for high definition maps | |
| EP2372656A2 (en) | Method and apparatus for vision aided navigation using image registration | |
| CN113406682B (en) | Positioning method, positioning device, electronic equipment and storage medium | |
| US11619724B2 (en) | Calibration of multiple lidars mounted on a vehicle using localization based on a high definition map | |
| US20200364883A1 (en) | Localization of a mobile unit by means of a multi-hypothesis kalman filter method | |
| CN113447949B (en) | Real-time positioning system and method based on laser radar and prior map | |
| CN111241224B (en) | Method, system, computer device and storage medium for target distance estimation | |
| CN114111776B (en) | Positioning method and related device | |
| WO2024027350A1 (en) | Vehicle positioning method and apparatus, computer device and storage medium | |
| CN112347205A (en) | Method and device for updating error state of vehicle | |
| CN114880334A (en) | Map data updating method and electronic equipment | |
| CN115560760A (en) | Unmanned aerial vehicle-oriented vision/laser ranging high-altitude navigation method | |
| JP4986883B2 (en) | Orientation device, orientation method and orientation program | |
| CN113744308B (en) | Pose optimization method, pose optimization device, electronic equipment, medium and program product | |
| CN114690229A (en) | GPS-fused mobile robot visual inertial navigation method | |
| CN114022561A (en) | Urban area monocular mapping method and system based on GPS constraint and dynamic correction | |
| CN113959437A (en) | Measuring method and system for mobile measuring equipment | |
| CN113838129A (en) | Method, device and system for obtaining pose information | |
| CN114812601A (en) | State estimation method and device of visual inertial odometer and electronic equipment | |
| CN114840703A (en) | Pose information acquisition method, device, equipment, medium and product | |
| US11835342B2 (en) | Calibration of inertial measurement units of vehicles using localization | |
| CN112530270B (en) | Mapping method and device based on region allocation | |
| WO2020223868A1 (en) | Terrain information processing method and apparatus, and unmanned vehicle | |
| Deng et al. | Measurement model and observability analysis for optical flow-aided inertial navigation | |
| CN115205828B (en) | Vehicle positioning method and device, vehicle control unit and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |