MOTION SOLVER APPARATUS AND METHOD THEREOF Technical field The invention relates to a motion solver apparatus for estimating a relative planar motion between two camera poses, a method of estimating a relative planar motion between two camera poses, a corresponding computer program, a corresponding computer-readable data carrier, and a corresponding data carrier signal. Background Relative pose estimation is the process of determining the relative orientation and translation of a camera from at least two images captured by the camera. In general, there are six degrees of freedom in which a camera can be positioned in 3D space, three translational components (x, y, z) and three rotational components (yaw, pitch, roll). In scenarios such as autonomous driving, where cameras are used, among other sensors, to build a model of the environment through which an autonomous vehicle such as a car or a robot is moving, the camera motion is often constrained, e.g., by fixating the camera on the car. Since the camera is mounted rigidly to the vehicle, the only rotation of the camera is caused by the vehicle turning. In other words, the camera is moving in a (2D) plane, and the motion is referred to as planar motion. Consequently, the degrees of freedom, or motion parameters, are reduced to three: the translation in z (forward/backward) and x (lateral), and an angle of rotation θ around the y- axis. Note that even though the road or other surface over which the vehicle is moving may be non-planar, e.g., due to hills and other types of curvature, the motion of the camera can locally be approximated by a planar motion, in particular given the relatively small distances which autonomous vehicles
move between capturing subsequent images. This is the case because autonomous driving applications typically require images to be captured rather frequently for algorithmic stability, but also to be able to detect other vehicles and pedestrians in time. Another example where planar motion is a suitable approximation of the real motion is aerial surveillance with Unmanned Aerial Vehicle (UAVs) which are travelling at a constant altitude. This is common in, e.g., agricultural applications and city surveillance. Furthermore, it is common practice to refine the final solution obtained under the assumption of planar motion, allowing for minor deviations from the planar motion model. When incorporated in a complete Simultaneous Localization And Mapping (SLAM) framework, loop-closure detection and bundle adjustment typically compensate for such imperfections over time. A common approach for estimating the change in pose of a camera from two images captured by the camera is based on the following steps (aka pipeline): 1. Detecting points of interest or keypoints in both images. 2. Calculating visual descriptors for these keypoints. 3. Matching the features (the tuples of keypoint and descriptor) by visual appearance, using the visual descriptors, to obtain a set of tentatively pair-wise corresponding features in the two images. 4. Estimating the relative movement of the camera between the two images using the tentative correspondences obtained from the previous step. Remaining outliers among the correspondences need to be pruned to improve the motion estimation. In practice, this may be achieved using the RANSAC framework (RANdom SAmple Consensus). This results in a final set of pair-wise corresponding features (the inliers) and an estimated relative pose, i.e., the motion of the camera between the two images. 5. Optionally, refining the solution obtained in the previous set using the final set obtained in the previous step for the given motion model.
For calibrated cameras, the relative pose of the camera can be estimated up to a similarity transformation. In the case of planar motion, which is illustrated in Fig. 1, this means that the absolute scale of the translation of the camera between two captured images, i.e., the translation between pose i and pose j (illustrated by a dashed arrow), remains ambiguous. In other words, it is not possible to determine the distance travelled by the camera (the length of the dashed arrow between pose i and pose j in Fig. 1) from two images alone, only the direction of the translation can be obtained. This missing piece of information can often be inferred from other sensors, e.g., odometers. Consequently, it is common practice to represent the translation of the camera as an angle φ, corresponding to the direction of the translation. In conclusion, the planar motion model can be represented using two degrees of freedom, which can be expressed as two angles (also referred to as motion parameters): the direction of translation φ of the camera, and its orientation θ (both relative to the camera pose at the starting point of the translation). In Fig. 1, the respective field-of-view of the camera is illustrated as triangles oriented along the z-axis, which in case of an autonomous vehicle is the centerline of the vehicle. The relative motion of a camera, i.e., the two angles φ and θ, between two images can be estimated using, e.g., the approach described in “Fast and reliable minimal relative pose estimation under planar motion”, by S. Choi and J.-H. Kim, Image and Vision Computing, vol. 69, pages 103–112, Elsevier, 2018 (in the remainder referred to as “Choi and Kim”). The solver proposed by Choi and Kim requires two pair-wise correspondences between features in two images to estimate the motion parameters (the relative translation φ and the relative orientation θ). An alternative solution for estimating the relative motion of a camera uses affine features (“Minimal Solutions for Relative Pose With a Single Affine Correspondence”, by B. Guan, J. Zhao, Z. Li, F. Sun, and
F. Fraundorfer, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1926–1935, 2020, doi:10.1109/CVPR42600.2020.00200, in the remainder referred to as “Guan et al.”). An affine correspondence consists of a pair-wise correspondence between features in two images in addition to an affine (linear) transformation which maps image points in the neighborhood of the feature in the first image to image points in the neighborhood of the corresponding feature in the second image. That is, an affine correspondence not only represents the visual similarity between the respective neighborhoods around two features, also referred to as point correspondence, but also how the neighborhood deforms between the two images. The proposed solver only requires a single affine pair-wise correspondence between features in the two images. This reduces the number of required RANSC iterations and opens the possibility to employ other methods than RANSAC, such as histogram voting, which is computationally about 10 to 100 times more efficient than RANSAC based approaches (see, e.g., “Performance evaluation of 1-point-RANSAC visual odometry”, by D. Scaramuzza, Journal of Field Robotics, vol. 28, pages 792– 811, Wiley, 2011, doi: doi.org/10.1002/rob.20411). On the other hand, computing affine correspondences is computationally considerably more demanding, therefore offsetting the gain in computational efficiency by employing histogram voting or reducing the number of RANSAC iterations. For example, according to “ASIFT: An Algorithm for Fully Affine Invariant Comparison” (by G. Yu and J.-M. Morel, Image Processing On Line, vol. 1, pages 11–38, 2011, doi:10.5201/ipol.2011.my-asift), the complexity of extracting ASIFT features is about 13.5 times that of extracting SIFT features, and matching ASIFT features is about 180 times that of matching SIFT features. Recently, D. Barath and Z. Kukelova demonstrated (“Relative Pose from SIFT Features“, Computer Vision – ECCV 2022: 17th European Conference, pages 454–469, 2022, doi:10.1007/978-3-031-19824-3_27, in
the remainder referred to as “Barath and Kukelova”) that it is possible to interpret SIFT features as affine features. This is the case since SIFT features contain information about scale and orientation. More specifically, the local affine transformation, which can be represented by a 2 × 2 linear transformation ^, can be expressed as ^ = ^^^^^ ^ , where ^^ = ^^^^ for ^ = 1,2 are Jacobians matrices, ^^ is a rotation matrix corresponding to the rotation of the SIFT feature within the i-th image, and ^^ contains information about the scale of the SIFT feature within the i-th image: ^ ^ ^,^ ^^ ^ = ^ 0 ^^,^ ^
where ^^,^ and ^^,^ are the scales of the feature along the image axes ^ and ^, respectively, and
is the shear. From this observation, constraints relating the SIFT features and the affine transformation which they approximate are obtained, which in turn are used to derive novel constraints for epipolar geometry (two distinct camera poses viewing the same scene). Using SIFT features together with an approximation of the affine correspondence derived from the SIFT features allows using fewer point correspondences to estimate the complete motion model, compared to using only point correspondences. Summary It is an object of the invention to provide an improved alternative to the above techniques and prior art. More specifically, it is an object of the invention to provide improved solutions for estimating the relative pose of a camera, i.e., the change in pose of the camera between two camera poses, for planar motion, using two images captured by the camera with the respective poses.
These and other objects of the invention are achieved by means of different aspects of the invention, as defined by the independent claims. Embodiments of the invention are characterized by the dependent claims. According to a first aspect of the invention, a motion solver apparatus for estimating a relative planar motion between two camera poses is provided. The motion solver apparatus comprises processing circuitry causing the motion solver apparatus to be operative to acquire a set of pair- wise corresponding scale- and orientation covariant features present in two images of a real-world scene. The two images are captured with the two respective camera poses. The motion solver apparatus is further operative to, for each pair of the set of pair-wise corresponding features, compute the relative planar motion between the two camera poses independent of other ones of the set of pair-wise corresponding features. The motion solver apparatus is further operative to determine the estimated relative planar motion between the two camera poses as the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features. According to a second aspect of the invention, a method of estimating a relative planar motion between two camera poses is provided. The method comprises acquiring a set of pair-wise corresponding scale- and orientation covariant features present in two images of a real-world scene. The two images are captured with the two respective camera poses. The method further comprises, computing, for each pair of the set of pair-wise corresponding features, the relative planar motion between the two camera poses independent of other ones of the set of pair-wise corresponding features. The method further comprises determining the estimated relative planar motion between the two camera poses as the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features.
According to a third aspect of the invention, a computer program is provided. The computer program comprises instructions which, when the computer program is executed by one or more processors comprised in a computing device, cause the computing device to carry out the method according to an embodiment of the second aspect of the invention. According to a fourth aspect of the invention, a computer-readable data carrier provided. The computer-readable data carrier has stored thereon the computer program according to the third aspect of the invention. According to a fifth aspect of the invention, a data carrier signal is provided. The data carrier signal carries the computer program according to the third aspect of the invention. Even though advantages of the invention have in some cases been described with reference to embodiments of the first aspect of the invention, corresponding reasoning applies to embodiments of other aspects of the invention. Further objectives of, features of, and advantages with, the invention will become apparent when studying the following detailed disclosure, the drawings, and the appended claims. Those skilled in the art realize that different features of the invention can be combined to create embodiments other than those described in the following. Brief description of the drawings The above, as well as additional objects, features and advantages of the invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the appended drawings, in which: Fig. 1 schematically illustrates the geometry and parameters of planar motion, in accordance with embodiments of the invention.
Fig. 2 schematically illustrates a motion solver apparatus for estimating a relative planar motion between two camera poses, in accordance with embodiments of the invention. Fig. 3 shows a flow chart illustrating a method of estimating a relative planar motion between two camera poses, in accordance with embodiments of the invention. Fig.4 illustrates histogram voting, in accordance with embodiments of the invention. Fig. 5 illustrates cumulative density functions of the angular error for translation and rotation, respectively, and processing time, for the motion solver apparatus in accordance with embodiments of the invention in comparison with conventional motion solvers. All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested. Detailed description The invention will now be described more fully herein after with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In the following, solutions for solving the planar motion model in epipolar geometry, i.e., estimating the relative planar motion of a camera between two camera poses (pose i and pose j illustrated in Fig. 1) based on images of a real-world scene captured with the two respective camera poses, are described.
Although Barath and Kukelova have demonstrated that using SIFT features together with an approximation of the affine correspondence derived from the SIFT features allows using fewer point correspondences to estimate the complete motion model, as compared to using only the point correspondences, they have not addressed the situation of relative planar motion. In the following, SIFT constraints for planar motion are derived using an explicit parameterization. It turns out that the planar motion problem can be reduced to a quadratic equation in one variable. As a result, the number of pair-wise correspondences between features in the two images is reduced to a single correspondence. Different from the method proposed by Barath and Kukelova, which are depending on RANSAC-based approaches for a robust estimation of the complete motion model, embodiments of the invention can utilize histogram voting instead of RANSAC, owing to the fact that only a single pair-wise correspondence between features in the two images is required. In comparison with known solutions, the proposed solution is advantageous in that it is computationally less complex and therefore relatively fast in terms of processing speed, with respect to both computing and matching features to obtain sets of tentatively corresponding features (i.e., detecting keypoints in the two images, calculating visual descriptors for the detected keypoints, and pair-wise matching features by comparing their visual descriptors), as well as solving the motion model. More specifically, whereas the solution by Choi and Kim is relatively fast in computing and matching features, it requires two pair-wise correspondences between features in two images to estimate the motion parameters using a RANSAC approach, which is relatively slow in terms of processing speed. On the other hand, the solution by Guan et al. relies on computing and matching affine features, which is computationally relatively slow, but uses the relatively fast histogram voting to solve the planar motion model since only a single correspondence between affine features is required. If implemented in
autonomous vehicle applications, e.g., factory robots and the like, embodiments of the invention are advantageous in that they result in a reduced power consumption and shorter response times. Assuming that the camera is moving in a plane, the coordinate system can be chosen such that ^ = 0, and any translation of the camera can be expressed as ^ = [^^ 0 ^^]T. Further, the only rotation of the camera may occur around the y-axis, which can be expressed by a rotation matrix ^^. Since only the relative motion is to be determined, only the direction ^ of the translation ^ can be obtained, and the problem can be parametrized as:
where the translation scale ^ is unknown and ^ is the relative orientation of the camera (aka the yaw angle) (see Fig. 1). Hence, the relative planar motion has two degrees of freedom. The corresponding essential matrix between the views captured in the two images can be written as: ^ = ^^[^]× (3), where [^]× is the matrix representation of the cross product of ^. Eq. (3) can be expressed explicitly as:
Fixing the scale, i.e., ^ = 1, Eq. (4) can be written as:
to define the non-zero elements ^^, ^ = 1 … 4, of the essential matrix. The well-known rank constraint det(^) = 0 is trivially fulfilled, and the known trace constraint 2^^T^ − tr(^^T)^ = 0 is reduced to a single equation: ^^ ^ ^ ^ − ^^ − ^^ + ^^ ^ = 0 (6).
Considering homogenous keypoints, a single linear equation in the elements ^^ (Eq. (5)) can be obtained by using the epipolar constraint between the two views. Given two corresponding homogenous points ^^ ↔ ^^, = [^^, ^^, 1]T for ^ = 1,2 are respective positions of the corresponding keypoints in the two images, the linear equation is given by ^T ^^^^ = 0. For relative planar motion, this can be reduced to:
Barath and Kukelova have derived a new constraint for estimating the fundamental matrix which leverages properties of scale- and orientation covariant features: ^^^^^^^ + ^^^^^^^ + ^^^^^^^ + ^^^^^^^ + ^^^^^ + ^^^^^ + ^^^^^^ + ^^^^^^ + ^^^^^^ + ^^^^^^ + ^^^^ + ^^^^ = 0 (8).
are orientations of the pair-wise corresponding features in the two images, ^ = ^^/^^, where ^^ and ^^ are respective sizes of the features in the two images, and ^^ are the elements of the 3 × 3 fundamental matrix in row-major order. The new constraint of Eq. (8) relies on orientation and scale of the matched features, and can therefore not be used with orientation covariant features that are not scale covariant, e.g., ORB features. By using the new constraint in combination with the epipolar constraint for point correspondences from Eq. (7), a second linear equation is added in the elements of the fundamental matrix for every pair of matched scale- and orientation covariant features. Hence, the number of pair-wise correspondences between scale- and orientation covariant features which are necessary for solving the planar motion model is reduced from ^ to ⌈^/2⌉ (where ⌈^⌉ denotes the ceiling of ^). For example, for the general fundamental matrix, 7 point correspondences are needed, resulting in ⌈^/2⌉ = 4 correspondences between scale- and orientation covariant features. For the essential matrix, 5 point correspondences are needed, resulting in ⌈5/2⌉ = 3 correspondences between scale- and orientation
covariant features. Finally, in the planar motion scenario, 2 point correspondences are required, and therefore only a single correspondence (⌈2/2⌉ = 1) between scale- and orientation covariant features is needed to solve the planar motion model. Since scale- and orientation covariant features contain information about scale and rotation, expressed by ^^, ^^, and ^, defined hereinbefore, they have a relation to the affine correspondence. These relations can be expressed as:
^^^^ + ^^^^ − ^^^ = 0 (9b), and ^^^^ + ^^^^ − ^^^ = 0 (9c), where
are the elements of ^ representing the local affine transformation:
From the above equations, a new constraint for planar motion can be derived:
which together with the known constraint for point correspondences in Eq. (7) results in two equations for each pair-wise correspondence between scale- and orientation covariant features. As is described further below, these two equations can be cast into a quadratic equation in a single variable, which can be solved relatively efficiently. In the following, embodiments of a motion solver apparatus 210 for estimating a relative planar motion, also referred to as relative pose or change in pose, between two camera poses are now described with reference to Fig. 2. The motion solver apparatus 210 may be a computing device, e.g., a control unit of an autonomous vehicle or a server, in particular an edge server. In particular, the motion solver apparatus 210 may be comprised in an autonomous vehicle 200 which further comprises a camera 220 arranged to capture images of real-word scenes while the autonomous vehicle is moving. In this case, the relative motion of the vehicle
can be estimated based on two images of the environment which are captured by the camera 220. In order to estimate the absolute motion of the vehicle, in particular the distance travelled between pose i and pose j in Fig. 1, additional sensor data, e.g., from an odometer, may be used. The motion solver apparatus 210 comprises processing circuitry 220 which causes the motion solver apparatus 210 to become operative in accordance with embodiments of the invention described herein. The processing circuitry 220 may comprise one or more processors 221, such as Central Processing Units (CPUs), microprocessors, application processors, application-specific processors, Graphics Processing Units (GPUs), and Digital Signal Processors (DSPs) including image processors, or a combination thereof, and a memory 222 comprising a computer program 223, i.e., software, comprising instructions. When executed by the processor(s) 221, the instructions cause the motion solver apparatus 210 to be operative in accordance with embodiments of the invention described herein. The memory 222 may, e.g., be a Random- Access Memory (RAM), a Read-Only Memory (ROM), a Flash memory, or the like. The computer program 223 may be downloaded to the memory 222 by means of a communications interface circuitry comprised in the motion solver apparatus 210, as a data carrier signal carrying the computer program 223. The processing circuitry 220 may alternatively or additionally comprise one or more Application-Specific Integrated Circuits (ASICs), Field- Programmable Gate Arrays (FPGAs), or the like, which are operative to cause the motion solver apparatus 210 to be operative in accordance with embodiments of the invention described herein. More specifically, the motion solver apparatus 210 becomes operative to acquire a set of pair-wise corresponding scale- and orientation covariant features present in two images of a real-world scene. The two images have been captured with the two respective camera poses, either by the same camera 220 which moves in-between the two camera poses, or by two
different cameras having different poses. The first scenario is common in autonomous driving applications in which images captured by a camera 220, which is movable with the autonomous vehicle 200, are used for controlling the autonomous vehicle. Typically, images are captured frequently so that the vehicle has only moved a short distance in-between two consecutive images, and a considerable number of details in the real-world scenes are captured in both images. The scale- and orientation covariant features may, e.g., be features which have been detected using the known Scale-Invariant Feature Transform (SIFT) computer-vision algorithm (see, e.g., D. G. Lowe, "Object recognition from local scale-invariant features", Proceedings of the International Conference on Computer Vision, vol. 2, pages 1150–1157, 1999, doi:10.1109/ICCV.1999.790410). Pair-wise correspondences between such features, which may contain outliers, may be obtained by matching of visual descriptors calculated for these features, as is known in the art. The motion solver apparatus 210 is further operative to compute, for each pair of the set of pair-wise corresponding features, the relative planar motion between the two camera poses independent of other ones of the set of pair-wise corresponding features. Preferably, the relative planar motion between the two camera poses is computed for each pair of the set of pair- wise corresponding features using a planar motion constraint. The planar motion constraint encodes a relationship between pair-wise corresponding scale- and orientation covariant features and the relative pose problem under the planar motion assumption. More specifically, if the relative planar motion between the two camera poses is represented by the direction ^ of the relative translation (aka the translation direction) and the relative orientation ^ of the camera (the change in orientation of the field-of-view of the camera 220), the planar motion constraint can be expressed as in Eq. (11). Advantageously, the respective orientations ^^ and ^^ of the pair-wise corresponding features in the two images, and the respective sizes ^^ and ^^
of the features in the two images, or their ratio ^, approximate the affine transformation between the two corresponding features. The motion solver apparatus 210 may further be operative to compute the relative planar motion between the two camera poses for each pair of the set of pair-wise corresponding features further using an epipolar constraint for homogenous points. This constraint is commonly known as the “epipolar constraint” and applies to homogeneous points, i.e., does not consider orientation nor scale of corresponding features. More specifically, if the relative planar motion between the two camera poses is represented by the direction ^ of the relative translation and the relative orientation ^, as described above, the epipolar constraint for homogenous points can be expressed as in Eq. (7). The motion solver apparatus 210 may further be operative to compute the relative planar motion between the two camera poses for each pair of the set of pair-wise corresponding features by solving a quadratic equation derived from the planar motion constraint (Eq. (11)), the epipolar constraint for homogenous points (Eq. (7)), and the trace constraint (Eq. (6)). In order to derive the quadratic equation, Eqs. (7) and (11) can be expressed as a system ^^ = 0:
The nullspace of ^ can be computed analytically, using singular value decomposition, or using QR decomposition. If ^ denotes a matrix of size 4 × 2 which spans the nullspace
denoting the elements of ^, the quadratic equation can be written, with ^ = ^^^ + ^^^ and fixing the scale by setting ^ = 1, as:
Expanding this expression results in:
where:
^^ = ^^^^^^ − ^^^^^^ − ^^^^^^ + ^^^^^^ (15b), and ^^ = ^^ ^^ − ^^ ^^ − ^^ ^^ + ^^ ^ ^ (15c), which can be solved using the standard formula for finding the roots of a quadratic equation. For its two roots ^^ and ^^, the two solutions for the vector ^ are: ^(^) = ^^^^ + ^^ (16a) and
where ^^ and ^^ are the first and second column of ^, respectively. From this, two possible solutions for the motion parameters, i.e., two pairs (^^, ^^) and (^^, ^^), corresponding to ^(^) and ^(^), respectively, can be obtained through the following trigonometric identities:
for ^ = 1,2, where arctan2(^, ^) is the angle between the x-axis and a ray from the origin to a point (^, ^) in the Cartesian plane. The motion solver apparatus 210 is further operative to determine the estimated relative planar motion between the two camera poses as the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features. Preferably, the motion solver apparatus may be operative to determine the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features as one of: a most frequent range, an average, and a median, of the set of relative planar motions computed for each pair of the set of pair-wise corresponding features. In practice, in order to determine the relative planar motion which statistically best represents the relative planar motions as a most frequent
range of the set of relative planar motions computed for each pair of the set of pair-wise corresponding features, the set of computed planar motions may be analyzed using histogram voting, as is illustrated in Fig. 4. More specifically, the computed values for the direction φ of the relative translation and the relative orientation θ are distributed in bins of histograms, one histogram for each parameter, of typically equal range. For each histogram, the bin with most values is determined. This bin, or rather a value representing the bin, e.g., the center value of the bin, is the value which statistically best represents the relative planar motion of the camera. In this way, the values for φ and θ which statistically best represent the relative planar motion of the camera are obtained. As an alternative, the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features may be determined by calculating median values, one for the direction φ of the relative translation and one for the relative orientation θ. Calculating median values is typically computationally less complex than histogram voting while at the same time providing a good approximation, as is illustrated in Fig. 4. The motion solver apparatus 210 may further be operative to iteratively remove outliers from the set of pair-wise corresponding features, and determine the estimated relative planar motion between the two camera poses as the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of remaining pair- wise corresponding features, i.e., the set of inliers. The removed outliers are pair-wise corresponding features with respective computed relative planar motions which deviate from the determined relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features. Advantageously, by removing outliers from the set of pair-wise corresponding features, the planar motion between the two camera poses may be estimated more accurately.
The motion solver apparatus 210 may further be operative to acquire the set of pair-wise corresponding scale- and orientation covariant features present in the two images of a real-world scene by detecting scale- and orientation covariant keypoints in the two images, calculating visual descriptors for the detected keypoints, and matching the calculated visual descriptors to obtain the set of pair-wise corresponding scale- and orientation covariant features. The scale- and orientation covariant features may be detected using a known computer vision algorithm for detecting local features such as SIFT, SURF, LFNet, HesAffNet, CovDet, or S3Esti. The solution proposed herein has been tested using the well-known KITTI dataset odometry (A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite”, 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3354–3361, IEEE, 2012, doi:10.1109/CVPR.2012.6248074), which consists of 11 image sequences of a car navigating urban environments, annotated with their ground-truth poses. In order to compare the performance of the proposed solution with known solutions, the following metrics have been used: - The translation error ^^ as defined by Choi and Kim. - The rotation error ^^ as defined by Choi and Kim. - The percentage of inliers detected. - The processing time of the solver. The below table summarizes the performance of three different methods using the GC-RANSAC framework (D. Barath and J. Matas, “Graph- Cut RANSAC”, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6733–6741, IEEE, 2018, doi:10.1109/CVPR.2018.00704) with sequence 04 of the KITTI dataset. GC-RANSAC is a state-of-the-art RANSAC-type framework which applies non-minimal refinement to the best solution iteratively, which improves performance (but also impacts computational complexity). Furthermore, it optimizes how to select inliers in a
more sophisticated way than only selecting inliers uniformly at random. The table shows the median performance for the whole sequence of images. Number of Inliers Processing Method ^ [deg] ^ [deg] correspondences ^ ^ [%] time [ms] Choi and 2 point 0.8959 0.3509 80.79 10.83 Kim correspondences Guan et 1 affine 0.5838 0.2409 81.87 15.55 al. correspondence 1 scale- and Current orientation 0.5692 0.2717 84.25 8.57 proposal covariant correspondence As can be seen, the solution proposed herein is relatively fast and performs best for most metrics. The performance of the proposed solution can further be assessed by considering the cumulative density functions for the angular error in translation and rotation, respectively, as well as for the processing time. These are illustrated in Fig. 5 for sequence 04 of the KITTI dataset, using the GC-RANSAC framework. A method is considered accurate (for angular error) or fast (for processing time) if its curve is on the left side of the diagram. While all methods demonstrate to achieve similar accuracy, the proposed method is relatively faster. For example, the proposed method has an 80% of probability of achieving the solution in approximately 8 ms, while others require at least approximately 10 ms. In the following, embodiments of a method 300 of estimating a relative planar motion between two camera poses (pose i and pose j in Fig. 1) are described with reference to Fig. 3. Embodiments of the method 300 may,
e.g., be performed by a computing device such as an autonomous vehicle, a robot, a server, a motion solver apparatus, or the like. The method 300 comprises acquiring 301 a set of pair-wise corresponding scale- and orientation covariant features present in two images of a real-world scene. The two images are captured with the two respective camera poses. The method 300 further comprises computing 302, for each pair of the set of pair-wise corresponding features, the relative planar motion between the two camera poses independent of other ones of the set of pair-wise corresponding features. The method 300 further comprises determining 303 the estimated relative planar motion between the two camera poses as the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features. The relative planar motion between the two camera poses may be computed 302, for each pair of the set of pair-wise corresponding features, using a planar motion constraint which encodes a relationship between pair- wise corresponding scale- and orientation covariant features and the relative pose problem under the planar motion assumption. Preferably, if the relative planar motion between the two camera poses is represented by a direction ^ of a relative translation and a relative orientation ^, the planar motion constraint can be expressed as in Eq. (11). The relative planar motion between the two camera poses may optionally be computed 302, for each pair of the set of pair-wise corresponding features, further using an epipolar constraint for homogenous points as in Eq. (7). The relative planar motion between the two camera poses may optionally be computed, for each pair of the set of pair-wise corresponding features, by solving a quadratic equation derived from the planar motion constraint (Eq. (11)), the epipolar constraint for homogenous points (Eq. (7), and a trace constraint as in Eq. (6).
The method 300 may further comprise removing 304 outliers from the set of pair-wise corresponding features, and determining 303 the estimated relative planar motion between the two camera poses as the relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of remaining pair-wise corresponding features. The outliers may be removed by removing pair-wise corresponding features with respective computed relative planar motions which deviate from the determined relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features. The relative planar motion which statistically best represents the relative planar motions computed for each pair of the set of pair-wise corresponding features may be determined 303 as one of: a most frequent range, an average, and a median, of the set of relative planar motions computed for each pair of the set of pair-wise corresponding features. The acquiring 301 the set of pair-wise corresponding scale- and orientation covariant features present in the two images of a real-world scene may comprise detecting scale- and orientation covariant keypoints in the two images using a computer vision algorithm for detecting local features, calculating visual descriptors for the detected keypoints, and matching the calculated visual descriptors to obtain the set of pair-wise corresponding scale- and orientation covariant features. It will be appreciated that the method 300 may comprise additional, alternative, or modified, steps in accordance with what is described throughout this disclosure. An embodiment of the method 300 may be implemented as the computer program 223 comprising instructions which, when the computer program 223 is executed by one or more processor(s) 221 comprised in a computing device 210, such as a motion solver apparatus, cause the computing device 210 to carry out the method 300 and become operative in accordance with embodiments of the
invention described herein. The computer program 223 may be stored in a computer-readable data carrier, such as the memory 222. Alternatively, the computer program 223 may be carried by a data carrier signal, e.g., downloaded to the memory 222 via a communications interface circuitry comprised in the computing device 210. The person skilled in the art realizes that the invention by no means is limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.