CN114492652B - Outlier removing method and device, vehicle and storage medium - Google Patents
Outlier removing method and device, vehicle and storage medium Download PDFInfo
- Publication number
- CN114492652B CN114492652B CN202210114108.8A CN202210114108A CN114492652B CN 114492652 B CN114492652 B CN 114492652B CN 202210114108 A CN202210114108 A CN 202210114108A CN 114492652 B CN114492652 B CN 114492652B
- Authority
- CN
- China
- Prior art keywords
- point
- rotation
- point matching
- pair
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 85
- 239000011159 matrix material Substances 0.000 claims abstract description 110
- 230000001133 acceleration Effects 0.000 claims abstract description 20
- 230000001351 cycling effect Effects 0.000 claims abstract description 10
- 238000006073 displacement reaction Methods 0.000 claims description 46
- 238000005096 rolling process Methods 0.000 claims description 26
- 238000004364 calculation method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 3
- 125000004122 cyclic group Chemical group 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 description 17
- 238000005070 sampling Methods 0.000 description 8
- 238000013519 translation Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000017105 transposition Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an outlier removing method, which is used for calculating triaxial rotation quantity in a time interval between a first image frame and a second image frame of a shooting device according to triaxial angular speed, triaxial acceleration and a rotation matrix; randomly extracting a pair of second characteristic point matching pairs from a first characteristic point matching pair formed by a first characteristic point obtained from a first image and a second characteristic point obtained from a second image adjacent to the first image, and fitting according to the triaxial rotation quantity and the second characteristic point matching pairs to obtain a first basic matrix; identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix; obtaining a second basic matrix according to the first interior point fitting; identifying a second interior point and a second outlier in the first feature point matching pair using a second basis matrix; and (3) cycling the steps for N times, and acquiring one second inner point with the largest number of feature point matching pairs from the obtained M second inner points as a target point. By the embodiment of the invention, the mismatching point pairs can be effectively and rapidly identified.
Description
Technical Field
The present invention relates to the field of computer vision, and in particular, to a method, an apparatus, a device, and a storage medium for outlier removal.
Background
VISLAM (visual-inertial-SLAM, AR-oriented monocular visual inertial simultaneous localization and mapping) refers to a technology of a robot-mounted sensor, tracking the position and posture of a subject in a motion process and simultaneously constructing an environmental structure consistency map by combining image information acquired by the visual sensor and motion information acquired by an Inertial Measurement Unit (IMU). VISLAM is widely applied to robots, unmanned aerial vehicles, unmanned vehicles, AR and VR, and image feature matching and feature mismatching identification in an algorithm are used as key links of the algorithm, and the overall performance of VISLAM can be directly affected by the performance of the algorithm. The features in the image can significantly affect the effect of the visual odometer, and the more features, the more stable the motion estimation effect, which requires the key points to be distributed as evenly as possible in the image.
But well-matched point clouds are often contaminated with outliers, i.e., erroneous data correlations. When an outlier appears, the outlier identification method is mainly realized by adopting a random sampling consistency (RANSAC) method based on a basic matrix, and mainly comprises a single-point random sampling consistency (one-point RANSAC), a two-point random sampling consistency method (two-point RANSAC), a five-point method (five-point RANSAC), an eight-point method (eight-point RANSAC) and other methods for checking whether characteristic point matching pairs are matched correctly. The core idea of RANSAC is to randomly extract the hypothesis premise of the data point calculation model, and then verify the hypothesis in other data points. If other data consistently verifies this assumption, it is an efficient solution. For two-view motion estimation in a visual odometer, the estimated model is the relative motion (rotation and translation) between the two camera positions and the data points for candidate feature matching.
However, with respect to the advantages and disadvantages of the eight-point method, the five-point method, the two-point method and the single-point method, on the one hand, the more the former points are sampled, the more complex the calculation amount (the more the CPU overhead is), and on the other hand, the more the former points are sampled, the greater the probability that the points sampled contain outliers (outliers, mismatching points) is, which leads to the greater possibility that the algorithm is disabled or the error is increased.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a method, apparatus, device, and storage medium for outlier removal, so as to ensure that the method not only includes the advantage of the single-point method, but also can widen the application scenario of the single-point method, so that the algorithm is more robust.
To achieve the above object, the present invention provides an outlier removing method, comprising the steps of:
S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
s3: identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix;
S4: obtaining a second basic matrix according to the first interior point fitting;
s5: identifying a second interior point and a second outlier in the first feature point matching pair using the second basis matrix;
S6: cycling the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points;
s7: and acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
Optionally, the step S2 includes the following steps:
acquiring the rotation amount from a vehicle coordinate system to a camera coordinate system;
Acquiring the motion displacement of a vehicle in a time interval of the first image frame and the second image frame;
Randomly extracting a pair of feature point matching pairs from the first feature point matching pairs;
A first basis matrix is obtained according to the epipolar geometry principle.
Optionally, before step S1, the method further includes the following steps:
judging whether the vehicle has pitching rotation and rolling rotation;
if the vehicle has no pitch and roll rotations, steps S1 to S7 are performed.
Optionally, the determining whether the vehicle has pitch rotation and/or roll rotation includes the steps of:
Calculating the pitching rotation variable quantity and the rolling rotation variable quantity of the vehicle in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix;
If the absolute value of the pitching rotation variation is not larger than a first preset value, confirming that the vehicle does not have pitching rotation; if the absolute value of the pitching rotation variation is larger than or equal to a first preset value, confirming that the vehicle has pitching rotation;
if the absolute value of the roll rotation variation is not greater than a second preset value, confirming that the vehicle does not have roll rotation; and if the absolute value of the rolling rotation variation is larger than or equal to a second preset value, confirming that the vehicle has rolling rotation.
Optionally, the step S3 includes the following steps:
A1: randomly extracting a pair of third feature point matching pairs from the first feature point matching pairs;
A2: using the first basic matrix, and acquiring a first matching degree of the third characteristic point matching pair according to a epipolar geometry principle;
A3: if the first matching degree is not greater than a third preset value, the third characteristic point matching pair is a first inner point; if the first matching degree is greater than or equal to a third preset value, the third feature point matching pair is a first outlier;
A4: and (3) circulating the step A1 to the step A3 for K (K is more than or equal to 1) times to obtain F (F is more than or equal to 1) first inner points.
Optionally, the step S4 includes the following steps:
acquiring the number of the first interior points to obtain the number of the interior points;
establishing a epipolar geometry relation for each first internal point according to the first internal points to obtain the epipolar geometry relations of the internal points;
fitting and calculating displacement in the x-axis direction and displacement in the y-axis direction according to the number of the internal points and the epipolar geometric relationships;
And calculating to obtain a second basic matrix according to the displacement in the x-axis direction and the displacement in the y-axis direction.
Optionally, the step S5 includes the following steps:
b1: randomly extracting a pair of fourth feature point matching pairs from the first feature point matching pairs;
b2: using the second basic matrix to obtain a second matching degree of the fourth characteristic point matching pair according to a epipolar geometry principle;
B3: if the second matching degree is not greater than a fourth preset value, the fourth characteristic point matching pair is a second inner point; if the second matching degree is larger than or equal to a fourth preset value, the fourth characteristic point matching pair is a second outlier;
b4: and (3) circulating the step B1 to the step B3 for G (G is more than or equal to 1) times to obtain H (H is more than or equal to 1) second inner points.
In addition, in order to achieve the above object, the present invention also provides an outlier removing apparatus, comprising: the device comprises a triaxial rotation amount calculation module, a first basic matrix fitting module, a first interior point identification module, a second basic matrix fitting module, a second interior point identification module, a cyclic identification second interior point module and a maximum second interior point acquisition module, wherein:
The triaxial rotation amount calculating module is used for executing step S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
The first basic matrix fitting module is configured to execute step S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
The first interior point identification module is configured to execute step S3: identifying a first inner point and a first outlier in the first feature point matching pair by using the first basic matrix through a epipolar geometry principle;
the second basic matrix fitting module is configured to execute step S4: obtaining a second basic matrix according to the first interior point fitting;
the second interior point identification module is configured to execute step S5: identifying a second inner point and a second outlier in the first feature point matching pair by using the second basic matrix through a epipolar geometry principle;
The loop identifies a second interior point module for executing step S6: cycling the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points;
The maximum second interior point module is acquired and is used for executing step S7: and acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
In addition, to achieve the above object, the present invention also proposes a vehicle comprising: a memory, a processor, and an outlier removal program stored on the memory and executable on the processor, the outlier removal program configured to implement the steps of the outlier removal method as described above.
Furthermore, to achieve the above object, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the outlier removal method as described above.
The technical scheme provided by the invention adopts the following steps of S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix; s2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs is randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation quantity and the second characteristic point matching pair; s3: identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix; s4: obtaining a second basic matrix according to the first interior point fitting; s5: identifying a second interior point and a second outlier in the first feature point matching pair using a second basis matrix; s6: cycling the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points; s7: and acquiring one second interior point with the largest number of feature point matching pairs from the M second interior points as a target point. By the embodiment of the invention, the mismatching point pairs can be effectively and rapidly identified. Therefore, under the condition that the vehicle does not pitch rotation or roll rotation, the outlier is removed by adopting a single-point method, the mismatching point pairs can be effectively identified in an accelerating way, the extracted characteristic point matching pairs are few, and the calculation cost is low; the method can effectively cope with sideslip and rolling of the automobile during running, ensures the advantages of the single-point method, widens the application scene of the single-point method and makes the algorithm more robust.
Drawings
Fig. 1 is a schematic flow chart of an outlier removing method provided by the present invention.
Fig. 2 is another flow chart of an outlier removing method according to the present invention.
Fig. 3 is a schematic flow chart of a method for passing epipolar geometry inner points and outliers according to the present invention.
Fig. 4 is a schematic flow chart of a method for passing epipolar geometry internal points and outliers according to the present invention. .
FIG. 5 is a block diagram of an embodiment of an outlier removing apparatus according to the present invention.
FIG. 6 is a schematic diagram of a vehicle architecture of a hardware operating environment in which embodiments of the present invention are directed.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
In order to make the technical problems, technical schemes and beneficial effects to be solved more clear and obvious, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the particular embodiments described herein are illustrative only and are not limiting upon the invention.
In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present invention, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.
Referring to fig. 1, the present invention provides an outlier removing method, which includes:
And S1, calculating the triaxial rotation amount in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix.
S2, acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; and the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair.
And step S3, identifying a first inner point and a first outlier in the first feature point matching pair by using the first basic matrix.
And S4, fitting according to the first inner points to obtain a second basic matrix.
And S5, identifying a second inner point and a second outlier in the first characteristic point matching pair by using the second basic matrix.
And S6, circulating the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points.
And S7, acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
According to the outlier removing method, triaxial rotation quantity in a time interval between a first image frame and a second image frame of a shooting device is calculated according to triaxial angular speed, triaxial acceleration and a rotation matrix; acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs is randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation quantity and the second characteristic point matching pair; identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix; obtaining a second basic matrix according to the first interior point fitting; identifying a second interior point and a second outlier in the first feature point matching pair using a second basis matrix; cycling the steps for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points; and acquiring one second interior point with the largest number of feature point matching pairs from the M second interior points as a target point. Therefore, under the condition that the vehicle does not pitch rotation or roll rotation, the outlier is removed by adopting a single-point method, the mismatching point pairs can be effectively identified in an accelerating way, the extracted characteristic point matching pairs are few, and the calculation cost is low; the method can effectively cope with sideslip and rolling of the automobile during running, ensures the advantages of the single-point method, widens the application scene of the single-point method and makes the algorithm more robust.
Thus, as an embodiment, as shown in fig. 2, before the step S1, the method may further include:
and S0, judging whether the vehicle has pitching rotation and rolling rotation.
In this embodiment, if the determination result is no, that is, the vehicle does not pitch rotation and roll rotation, step S1 to step S7 are executed, and a single-point random sampling coincidence algorithm is adopted to obtain an interior point; if the judgment result is yes, namely that the vehicle has pitching rotation or rolling rotation, the steps S1 to S7 are not executed, the step S8 is directly executed, and a two-point random sampling consistency algorithm is executed to obtain the inner points.
And S8, executing a two-point random sampling coincidence algorithm to obtain the inner points.
In this embodiment, when the vehicle rotates in pitch, the "single point method" of the present invention cannot work normally, and the "single point method" needs to be returned to the "two point method". Therefore, it is desirable to determine whether the vehicle is turning in pitch and roll before the single point random sample consensus algorithm is performed to obtain the interior points. In the specific steps of the "two-point method", the description of the embodiment is omitted herein because it is the prior art.
As a specific embodiment, the step S0 specifically includes:
And calculating pitching rotation variable quantity r 1 and rolling rotation variable quantity r 2 in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix. Specifically, the vehicle pitching rotation variable quantity r 1 and the rolling rotation variable quantity r 2 in the time interval of the first image frame and the second image frame of the shooting device are calculated through the gyroscope measurement value of the IMU and the rotation matrix of the IMU to the shooting device.
Wherein the calculation method of r 1 and r 2 is shown in the following formula, ω 1 represents the reading of the IMU on the pitch axis in the time period from t s to t s+1, and ω 2 represents the reading on the yaw axis. The angle is calculated by a plurality of methods, and the following formula is a calculation method:
Then, the magnitude relation between the absolute value of the pitch rotation variation r 1 and the first preset value c 1 is further determined, wherein the first preset value c 1 is a preset parameter, for example, 0.5 degrees, and the specific magnitude is not limited herein. If |r 1|<c1, namely the absolute value of the pitching rotation variation is not larger than a first preset value, confirming that the vehicle does not have pitching rotation; and if |r 1|≥c1, namely the absolute value of the pitching rotation variation is larger than or equal to a first preset value, confirming that the vehicle has pitching rotation.
And further determining a magnitude relation between the absolute value of the roll rotation variation r 2 and the second preset value c 2, where the second preset value c 2 is a preset parameter, for example, 0.6 degrees, and the specific magnitude is not limited herein. If |r 2|<c2, namely the absolute value of the roll rotation variation is not larger than a second preset value, confirming that the vehicle does not have roll rotation; and if |r 2|≥c2, namely the absolute value of the roll rotation variation is larger than or equal to a second preset value, confirming that the vehicle has roll rotation.
In this way, under the condition that the vehicle does not have pitching rotation and rolling rotation, the steps S1 to S7 can be executed, and the interior points are obtained by adopting a single-point random sampling coincidence algorithm; and if the vehicle has pitching rotation or rolling rotation, directly entering into step S8 without executing step S1 to step S7, and executing a two-point random sampling coincidence algorithm to acquire the inner points.
Without pitch and roll rotation of the vehicle:
as a specific embodiment, the step S1 specifically includes:
And calculating the triaxial rotation quantity R in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix.
The reading of the meter IMU gyroscope is ω= [ ω 1,ω2,ω3 ], where ω 1 is the pitch axis reading of the IMU gyroscope, ω 2 is the roll axis reading of the IMU gyroscope, and ω 3 is the heading axis reading of the IMU gyroscope. The rotation matrix (calibrated parameters in advance at factory) from the IMU to the camera isThe calculation formula of the triaxial rotation amount R in the time interval from the first image frame to the second image frame of the camera is:
Where Δt represents the time interval of two frames of images.
In this embodiment, the specific calculation manner of the triaxial rotation amount R in the time interval between the first image frame and the second image frame of the photographing device can be calculated by using the gyroscope measurement value of the IMU and the rotation matrix from the IMU to the photographing device as in the "two-point method" in the prior art, and the embodiment is not described herein again.
As a specific embodiment, the step S2 specifically includes:
Acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; and the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair. Specifically, the rotation amount from the vehicle coordinate system to the camera coordinate system can be obtained; acquiring the motion displacement of a vehicle in a time interval of the first image frame and the second image frame; then randomly extracting a pair of feature point matching pairs from the first feature point matching pairs; a first basis matrix is obtained according to the epipolar geometry principle.
In this embodiment, after the vehicle coordinate system and the camera coordinate system are established, the rotation amount R 1 from the vehicle coordinate system to the camera coordinate system is obtained (the automatic driving automobile is required to be calibrated when leaving the factory, and the calibrated parameter includes the R 1); then, the motion displacement t' of the vehicle in the time interval of the first image frame and the second image frame is acquired, wherein the formula is as follows:
t'＝[t0' t1' 0]T (1)
where T 0 'represents the displacement in the x direction, T 1' represents the displacement in the y direction, and T represents the vector transposition.
For t' = [ t 0' t1' 0]T ] in equation (1) above, this is the displacement of the vehicle between two adjacent image frames in the vehicle coordinate system (defined as the vehicle right direction being the x-axis, the vehicle forward direction being the y-axis, and the vehicle above being the z-axis). In a planar scene (where the planar scene includes a sloping surface that slopes up and down), the vehicle can only have its motion displacement in the x-axis and y-axis directions of the vehicle coordinate system within the time corresponding to the two adjacent image frames, and the motion is 0 for the z-axis of the vehicle coordinate system. That is, in t ' = [ t 0' t1' 0]T, t 0 ' represents the displacement in the x-direction, t 1 ' represents the displacement in the y-direction, and as for 0, represents the displacement in the z-axis direction. As for the expression T denotes vector transposition, the motion displacement T' is a column vector.
In the existing "two-point method", we can determine that the displacement of the vehicle in the adjacent frame time of the camera is based on the dynamics of the vehicle (the characteristic that the vehicle is unlikely to fly in normal circumstances)Where t '0 represents the amount of translation of the car body to the right in the adjacent frame time, and t' 1 represents the amount of translation of the car body to the front in the adjacent frame time. And 0 represents the amount of upward translation of the car body in the adjacent frame time (obviously, the car cannot fly up, so the upward translation is 0).
Then, the motion of the camera in two adjacent frames is expressed by a known external parameter (the gesture of the vehicle to the rotation gesture of the camera) R 1 as t= [ t 0 t1 t2]＝R1 t', wherein t represents the translation amount of the vehicle in the time interval between the first image frame and the second image frame, the translation amount is represented by taking the camera coordinate system of the first image frame as an origin, t 0 represents the x direction, t 1 represents the y direction, and t 2 represents the z direction.
Thus, only 2 variables (2 variables illustrate only 1 pending degree of freedom, since the scale occupies 1 degree of freedom) in R and t of the solution base matrix E, 1 point can be sampled to estimate the base matrix E.
For the first image and the second image, a first characteristic point can be acquired from the first image, and a second characteristic point can be acquired from a second image adjacent to the first image; the first feature point and the second feature point form a first feature point matching pair.
Therefore, a pair of second feature point matching pairs can be randomly extracted from the first feature point matching pairs (assuming m point pairs), and a first base matrix is obtained by fitting the triaxial rotation amount and the second feature point matching pairs.
Specifically, a pair of second feature point matching pairs may be randomly extracted from the first feature point matching pairs, for example, P 1,j and P 2,j, and then the epipolar geometry of the first base matrix is obtained as follows:
P1,j T.[R1t']×R.P2,j＝0 (2)
Wherein R is the triaxial rotation amount in step S1.
Then substituting t 0 '=1 into formula (1) to solve t 1', and substituting t 1 'into formula (2) to solve t'; when t' and R are known, a first basis matrix E 1＝t′× R can be found.
As a specific embodiment, the step S3 specifically includes:
Referring to fig. 3 together, the present invention further provides a method for identifying a first interior point and a first outlier in the first feature point matching pair by using the first base matrix according to the epipolar geometry principle, which specifically includes the following steps:
And A1, randomly extracting a pair of third characteristic point matching pairs from the first characteristic point matching pairs.
In this embodiment, a pair of third feature point matching pairs, such as P 1,q and P 2,q, may be randomly extracted from the first feature point matching pairs (assuming m point pairs).
And A2, acquiring a first matching degree of the third feature point matching pair according to a epipolar geometry principle by using the first basic matrix.
In this embodiment, after a pair of third feature point matching pairs, such as P 1,q and P 2,q, is randomly extracted, the following formula may be executed to continuously obtain the first matching degree of the third feature point matching pair through epipolar geometry recognition:
P1,q T·[R1t′]×·R·P2,q＝S1 (3)
Wherein S 1 represents the first degree of matching.
Step A3, if the first matching degree is not greater than a third preset value, the third feature point matching pair is a first inner point; and if the first matching degree is greater than or equal to a third preset value, the third feature point matching pair is a first outlier.
In this embodiment, the third preset value c 3 is a preset parameter, which can be set empirically by adjusting the parameter, and the specific size is not limited in this embodiment. If |s 1|<c3, that is, the absolute value of the first matching degree S 1 is not greater than the third preset value, the third feature point matching pair P 1,q and P 2,q are the first inner point, that is, the correct matching point pair; if |s 1|≥c3, that is, the absolute value of the first matching degree is greater than or equal to the third preset value, the third feature point matching pair P 1,q and P 2,q are the first outlier, that is, the mismatching point pair.
And A4, circulating the steps A1 to A3 for K (K is more than or equal to 1) times to obtain F (F is more than or equal to 1) first inner points.
In this embodiment, since the above steps are only performed for the pair of third feature point matching pairs P 1,q and P 2,q that are randomly extracted, and the first feature point matching pair (assuming that m point pairs) includes other point pairs besides P 1,q and P 2,q, it is necessary to cycle K (K is 1 or more) from step A1 to step A3 to calculate all the first interior points included in the first feature point matching pair, and then F (F is 1 or more) first interior points can be obtained and marked as n 1,i interior points in the first feature point matching pair (assuming that m point pairs).
As a specific embodiment, the step S4 specifically includes:
Obtaining a second basic matrix according to the first interior point fitting, specifically: obtaining the number of the inner points by obtaining the number of the first inner points; then, establishing a epipolar geometry relation for each first internal point according to the first internal points to obtain the epipolar geometry relations of the internal points; fitting and calculating the displacement in the x-axis direction and the displacement in the y-axis direction according to the number of the internal points and the epipolar geometric relationships; and finally, calculating to obtain a second basic matrix according to the displacement in the x-axis direction and the displacement in the y-axis direction.
In this embodiment, after F (F is greater than or equal to 1) first inner points are obtained based on step A4, the number of inner points is F, and then a second basic matrix may be fitted based on n 1,i inner points, specifically, according to the first inner points, a epipolar geometry relation (2) is established for each first inner point, so as to obtain F epipolar geometry relations (2); and then fitting and calculating according to F epipolar geometric relationships (2) to obtain displacement t '0 in the x-axis direction and displacement t' 1 in the y-axis direction.
On the basis of obtaining the displacement amount t '0 in the x-axis direction and the displacement amount t' 1 in the y-axis direction, the second basis matrix E 2 can be calculated from the displacement amount in the x-axis direction and the displacement amount in the y-axis direction.
As a specific embodiment, the step S5 specifically includes:
Referring to fig. 4 together, the present invention further provides a method for identifying a second inlier and a second outlier in the first feature point matching pair by using the second basis matrix, wherein the steps are substantially the same as steps A1 to A4, and specifically includes the following steps:
and B1, randomly extracting a pair of fourth characteristic point matching pairs from the first characteristic point matching pairs.
In this embodiment, a pair of fourth feature point matching pairs, such as P 1,k and P 2,k, may be randomly extracted from the first feature point matching pairs (assuming m point pairs).
And B2, acquiring a second matching degree of the fourth characteristic point matching pair according to a epipolar geometry principle by using the second basic matrix.
In this embodiment, after a pair of fourth feature point matching pairs, such as P 1,k and P 2,k, is randomly extracted, the following formula may be executed to continuously obtain the second matching degree of the fourth feature point matching pair through epipolar geometry recognition:
P1,k T·[R1t′]×·R·P2,k＝S2 (4)
Wherein S 2 represents the second degree of matching.
Step B3, if the second matching degree is not greater than a fourth preset value, the fourth feature point matching pair is a second inner point; and if the second matching degree is greater than or equal to a fourth preset value, the fourth characteristic point matching pair is a second outlier.
In this embodiment, the fourth preset value c 4 is a preset parameter, which can be set empirically by adjusting the parameter, and the specific size is not limited in this embodiment. If |s 2|<c4, that is, the absolute value of the second matching degree S 2 is not greater than the fourth preset value, the fourth feature point matching pair P 1,k and P 2,k are second inner points, that is, correct matching point pairs; if |s 2|≥c4, that is, the absolute value of the second matching degree S 2 is greater than or equal to the fourth preset value, the fourth feature point matching pair P 1,k and P 2,k is the second outlier, that is, the mismatching point pair.
And B4, circulating the step B1 to the step B3 for G (G is more than or equal to 1) times to obtain H (H is more than or equal to 1) second inner points.
In this embodiment, since the above steps are only performed for the pair of fourth feature point matching pairs P 1,k and P 2,k that are randomly extracted, and the first feature point matching pair (assuming that m point pairs) includes other point pairs besides P 1,k and P 2,k, it is necessary to cycle G (G is 1 or more) from step B1 to step B3 to calculate all the second interior points included in the first feature point matching pair, and then H (H is 1 or more) can be obtained as n 2,i interior points in the first feature point matching pair (assuming that m point pairs).
As a specific embodiment, the step S6 specifically includes:
and (3) circulating the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points.
In this embodiment, the second inner points need to be identified circularly, i.e. steps S1 to S5 are repeated N (N is greater than or equal to 1) times, and it is understood that the number of times N can be manually set, for example, 10 times, and then M (M is greater than or equal to 1) second inner points such as N 2,0,n2,1,…,n2,M can be obtained finally.
As a specific embodiment, the step S7 specifically includes:
And acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
In this embodiment, after obtaining M (M is greater than or equal to 1) second inner points, for example, n 2,0,n2,1,…,n2,M, the largest second inner point needs to be obtained, that is, one second inner point with the largest number of feature point matching pairs is obtained from the M second inner points is the target point, that is, the marking result corresponding to max (n 2,0,n2,1,…,n2,M) is finally taken as the final recognition result.
Thus, compared with the single-point method of Davida Scaramuzza, the technical scheme of the invention can effectively cope with sideslip when the automobile runs, can also cope with the occurrence of small sudden rolling of the automobile body (such as the sudden pressing of small stones by the automobile tire), and can not generate trigonometric functions (the computational complexity is higher because of the introduction of the trigonometric function and the calculation of the inverse trigonometric function) in the method like Davida Scaramuzza. The method not only can ensure that all advantages of the single-point method are contained, but also can widen the application scene of the single-point method, so that the algorithm is more robust. And compared with the two-point scheme, the method has fewer extraction points, so the error is smaller, and the CPU overhead is lower.
In addition, an embodiment of the present invention further provides an outlier removing apparatus, referring to fig. 5, where the outlier removing apparatus includes: the three-axis rotation amount calculation module 501, the first basic matrix fitting module 502, the first interior point identification module 503, the second basic matrix fitting module 504, the second interior point identification module 505, the cyclic identification second interior point module 506, and the maximum second interior point acquisition module 507, wherein:
The triaxial rotation amount calculating module 501 is configured to execute step S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
A first basic matrix fitting module 502, configured to perform step S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
The first interior point identifying module 503 is configured to perform step S3: identifying a first inner point and a first outlier in the first feature point matching pair by using the first basic matrix through a epipolar geometry principle;
A second base matrix fitting module 504, configured to perform step S4: obtaining a second basic matrix according to the first interior point fitting;
the second interior point identifying module 505 is configured to execute step S5: identifying a second inner point and a second outlier in the first feature point matching pair by using the second basic matrix through a epipolar geometry principle;
The loop identifies a second interior point module 506 for executing step S6: cycling the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points;
the maximum second interior point obtaining module 507 is configured to execute step S7: and acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
In the embodiment of the invention, under the condition that the vehicle does not pitch rotation or roll rotation, the outlier is removed by adopting a single-point method, the mismatching point pairs can be effectively identified in an acceleration way, the extracted characteristic point matching pairs are few, and the calculation cost is low; the method can effectively cope with sideslip and rolling of the automobile during running, ensures the advantages of the single-point method, widens the application scene of the single-point method and makes the algorithm more robust.
It should be noted that each unit in the above apparatus may be used to implement each step in the above method, and achieve the corresponding technical effect, which is not described herein again.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a vehicle in a hardware running environment according to an embodiment of the present invention.
As shown in fig. 6, the vehicle may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include standard wired interfaces, wireless interfaces (e.g., WI-FI, 4G, 5G interfaces). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in fig. 6 is not limiting of the vehicle and may include more or fewer components than shown, or certain components may be combined, or a different arrangement of components.
As shown in fig. 6, an operating system, a network communication module, a user interface module, and an outlier removing program may be included in the memory 1005 as one type of computer storage medium.
In the vehicle shown in fig. 6, the network interface 1004 is mainly used for data communication with an external network; the user interface 1003 is mainly used for receiving an input instruction of a user; the vehicle invokes the outlier removal program stored in the memory 1005 by the processor 1001 and performs the following operations:
S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
s3: identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix;
S4: obtaining a second basic matrix according to the first interior point fitting;
s5: identifying a second interior point and a second outlier in the first feature point matching pair using the second basis matrix;
S6: cycling the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points;
s7: and acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
Optionally, the step S2 includes the following steps:
acquiring the rotation amount from a vehicle coordinate system to a camera coordinate system;
Acquiring the motion displacement of a vehicle in a time interval of the first image frame and the second image frame;
Randomly extracting a pair of feature point matching pairs from the first feature point matching pairs;
A first basis matrix is obtained according to the epipolar geometry principle.
Optionally, before step S1, the method further includes the following steps:
judging whether the vehicle has pitching rotation and rolling rotation;
if the vehicle has no pitch and roll rotations, steps S1 to S7 are performed.
Optionally, the determining whether the vehicle has pitch rotation and/or roll rotation includes the steps of:
Calculating the pitching rotation variable quantity and the rolling rotation variable quantity of the vehicle in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix;
If the absolute value of the pitching rotation variation is not larger than a first preset value, confirming that the vehicle does not have pitching rotation; if the absolute value of the pitching rotation variation is larger than or equal to a first preset value, confirming that the vehicle has pitching rotation;
if the absolute value of the roll rotation variation is not greater than a second preset value, confirming that the vehicle does not have roll rotation; and if the absolute value of the rolling rotation variation is larger than or equal to a second preset value, confirming that the vehicle has rolling rotation.
Optionally, the step S3 includes the following steps:
A1: randomly extracting a pair of third feature point matching pairs from the first feature point matching pairs;
A2: using the first basic matrix, and acquiring a first matching degree of the third characteristic point matching pair according to a epipolar geometry principle;
A3: if the first matching degree is not greater than a third preset value, the third characteristic point matching pair is a first inner point; if the first matching degree is greater than or equal to a third preset value, the third feature point matching pair is a first outlier;
A4: and (3) circulating the step A1 to the step A3 for K (K is more than or equal to 1) times to obtain F (F is more than or equal to 1) first inner points.
Optionally, the step S4 includes the following steps:
acquiring the number of the first interior points to obtain the number of the interior points;
establishing a epipolar geometry relation for each first internal point according to the first internal points to obtain the epipolar geometry relations of the internal points;
fitting and calculating displacement in the x-axis direction and displacement in the y-axis direction according to the number of the internal points and the epipolar geometric relationships;
And calculating to obtain a second basic matrix according to the displacement in the x-axis direction and the displacement in the y-axis direction.
Optionally, the step S5 includes the following steps:
b1: randomly extracting a pair of fourth feature point matching pairs from the first feature point matching pairs;
b2: using the second basic matrix to obtain a second matching degree of the fourth characteristic point matching pair according to a epipolar geometry principle;
B3: if the second matching degree is not greater than a fourth preset value, the fourth characteristic point matching pair is a second inner point; if the second matching degree is larger than or equal to a fourth preset value, the fourth characteristic point matching pair is a second outlier;
b4: and (3) circulating the step B1 to the step B3 for G (G is more than or equal to 1) times to obtain H (H is more than or equal to 1) second inner points.
In the embodiment of the invention, under the condition that the vehicle does not pitch rotation or roll rotation, the outlier is removed by adopting a single-point method, the mismatching point pairs can be effectively identified in an acceleration way, the extracted characteristic point matching pairs are few, and the calculation cost is low; the method can effectively cope with sideslip and rolling of the automobile during running, ensures the advantages of the single-point method, widens the application scene of the single-point method and makes the algorithm more robust.
In addition, the embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores an outlier removing program, and the outlier removing program realizes the following operations when being executed by a processor:
S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
s3: identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix;
S4: obtaining a second basic matrix according to the first interior point fitting;
s5: identifying a second interior point and a second outlier in the first feature point matching pair using the second basis matrix;
S6: cycling the steps S1 to S5 for N (N is more than or equal to 1) times to obtain M (M is more than or equal to 1) second inner points;
s7: and acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point.
Optionally, the step S2 includes the following steps:
acquiring the rotation amount from a vehicle coordinate system to a camera coordinate system;
Acquiring the motion displacement of a vehicle in a time interval of the first image frame and the second image frame;
Randomly extracting a pair of feature point matching pairs from the first feature point matching pairs;
A first basis matrix is obtained according to the epipolar geometry principle.
Optionally, before step S1, the method further includes the following steps:
judging whether the vehicle has pitching rotation and rolling rotation;
if the vehicle has no pitch and roll rotations, steps S1 to S7 are performed.
Optionally, the determining whether the vehicle has pitch rotation and/or roll rotation includes the steps of:
Calculating the pitching rotation variable quantity and the rolling rotation variable quantity of the vehicle in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix;
If the absolute value of the pitching rotation variation is not larger than a first preset value, confirming that the vehicle does not have pitching rotation; if the absolute value of the pitching rotation variation is larger than or equal to a first preset value, confirming that the vehicle has pitching rotation;
if the absolute value of the roll rotation variation is not greater than a second preset value, confirming that the vehicle does not have roll rotation; and if the absolute value of the rolling rotation variation is larger than or equal to a second preset value, confirming that the vehicle has rolling rotation.
Optionally, the step S3 includes the following steps:
A1: randomly extracting a pair of third feature point matching pairs from the first feature point matching pairs;
A2: using the first basic matrix, and acquiring a first matching degree of the third characteristic point matching pair according to a epipolar geometry principle;
A3: if the first matching degree is not greater than a third preset value, the third characteristic point matching pair is a first inner point; if the first matching degree is greater than or equal to a third preset value, the third feature point matching pair is a first outlier;
A4: and (3) circulating the step A1 to the step A3 for K (K is more than or equal to 1) times to obtain F (F is more than or equal to 1) first inner points.
Optionally, the step S4 includes the following steps:
acquiring the number of the first interior points to obtain the number of the interior points;
establishing a epipolar geometry relation for each first internal point according to the first internal points to obtain the epipolar geometry relations of the internal points;
fitting and calculating displacement in the x-axis direction and displacement in the y-axis direction according to the number of the internal points and the epipolar geometric relationships;
And calculating to obtain a second basic matrix according to the displacement in the x-axis direction and the displacement in the y-axis direction.
Optionally, the step S5 includes the following steps:
b1: randomly extracting a pair of fourth feature point matching pairs from the first feature point matching pairs;
b2: using the second basic matrix to obtain a second matching degree of the fourth characteristic point matching pair according to a epipolar geometry principle;
B3: if the second matching degree is not greater than a fourth preset value, the fourth characteristic point matching pair is a second inner point; if the second matching degree is larger than or equal to a fourth preset value, the fourth characteristic point matching pair is a second outlier;
b4: and (3) circulating the step B1 to the step B3 for G (G is more than or equal to 1) times to obtain H (H is more than or equal to 1) second inner points.
In the embodiment of the invention, under the condition that the vehicle does not pitch rotation or roll rotation, the outlier is removed by adopting a single-point method, the mismatching point pairs can be effectively identified in an acceleration way, the extracted characteristic point matching pairs are few, and the calculation cost is low; the method can effectively cope with sideslip and rolling of the automobile during running, ensures the advantages of the single-point method, widens the application scene of the single-point method and makes the algorithm more robust.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, a controller, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (8)
1. A method of outlier removal, the method comprising:
S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
s3: identifying a first interior point and a first outlier in the first feature point matching pair using the first basis matrix;
S4: obtaining a second basic matrix according to the first interior point fitting;
s5: identifying a second interior point and a second outlier in the first feature point matching pair using the second basis matrix;
s6: cycling the steps S1 to S5 for N times, wherein N is greater than or equal to 1, and M second inner points are obtained, and M is greater than or equal to 1;
s7: acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point;
Wherein, the step S2 includes the following steps:
acquiring the rotation amount from a vehicle coordinate system to a camera coordinate system;
Acquiring the motion displacement of a vehicle in a time interval of the first image frame and the second image frame;
Randomly extracting a pair of feature point matching pairs from the first feature point matching pairs;
Obtaining a first basic matrix according to the epipolar geometry principle;
Wherein, the step S4 includes the following steps:
acquiring the number of the first interior points to obtain the number of the interior points;
establishing a epipolar geometry relation for each first internal point according to the first internal points to obtain the epipolar geometry relations of the internal points;
fitting and calculating displacement in the x-axis direction and displacement in the y-axis direction according to the number of the internal points and the epipolar geometric relationships;
And calculating to obtain a second basic matrix according to the displacement in the x-axis direction and the displacement in the y-axis direction.
2. The method according to claim 1, characterized in that prior to step S1, it further comprises the steps of:
judging whether the vehicle has pitching rotation and rolling rotation;
if the vehicle has no pitch and roll rotations, steps S1 to S7 are performed.
3. The method of claim 2, wherein said determining whether the vehicle has pitch and/or roll rotation comprises the steps of:
Calculating the pitching rotation variable quantity and the rolling rotation variable quantity of the vehicle in the time interval of the first image frame and the second image frame of the shooting device according to the triaxial angular speed, the triaxial acceleration and the rotation matrix;
If the absolute value of the pitching rotation variation is not larger than a first preset value, confirming that the vehicle does not have pitching rotation; if the absolute value of the pitching rotation variation is larger than or equal to a first preset value, confirming that the vehicle has pitching rotation;
if the absolute value of the roll rotation variation is not greater than a second preset value, confirming that the vehicle does not have roll rotation; and if the absolute value of the roll rotation variation is larger than or equal to a second preset value, confirming that the vehicle has roll rotation.
4. The method according to claim 1, characterized in that said step S3 comprises the steps of:
A1: randomly extracting a pair of third feature point matching pairs from the first feature point matching pairs;
A2: using the first basic matrix, and acquiring a first matching degree of the third characteristic point matching pair according to a epipolar geometry principle;
A3: if the first matching degree is not greater than a third preset value, the third characteristic point matching pair is a first inner point; if the first matching degree is greater than or equal to a third preset value, the third feature point matching pair is a first outlier;
A4: and (3) circulating the steps A1 to A3 for K times, wherein K is greater than or equal to 1, and F first inner points are obtained, and F is greater than or equal to 1.
5. The method according to claim 1, characterized in that said step S5 comprises the steps of:
b1: randomly extracting a pair of fourth feature point matching pairs from the first feature point matching pairs;
b2: using the second basic matrix to obtain a second matching degree of the fourth characteristic point matching pair according to a epipolar geometry principle;
B3: if the second matching degree is not greater than a fourth preset value, the fourth characteristic point matching pair is a second inner point; if the second matching degree is larger than or equal to a fourth preset value, the fourth characteristic point matching pair is a second outlier;
b4: and (3) circulating the steps B1 to B3 for G times, wherein G is greater than or equal to 1, and H second inner points are obtained, and H is greater than or equal to 1.
6. An outlier removing apparatus, comprising: the device comprises a triaxial rotation amount calculation module, a first basic matrix fitting module, a first interior point identification module, a second basic matrix fitting module, a second interior point identification module, a cyclic identification second interior point module and a maximum second interior point acquisition module, wherein:
The triaxial rotation amount calculating module is used for executing step S1: calculating triaxial rotation amounts in a time interval between a first image frame and a second image frame of the photographing device according to the triaxial angular velocity, the triaxial acceleration and the rotation matrix;
The first basic matrix fitting module is configured to execute step S2: acquiring a first characteristic point from a first image, and acquiring a second characteristic point from a second image adjacent to the first image; the first characteristic points and the second characteristic points form a first characteristic point matching pair, a pair of second characteristic point matching pairs are randomly extracted from the first characteristic point matching pair, and a first basic matrix is obtained according to the triaxial rotation amount and the second characteristic point matching pair in a fitting mode;
The first interior point identification module is configured to execute step S3: identifying a first inner point and a first outlier in the first feature point matching pair by using the first basic matrix through a epipolar geometry principle;
the second basic matrix fitting module is configured to execute step S4: obtaining a second basic matrix according to the first interior point fitting;
the second interior point identification module is configured to execute step S5: identifying a second inner point and a second outlier in the first feature point matching pair by using the second basic matrix through a epipolar geometry principle;
The loop identifies a second interior point module for executing step S6: cycling the steps S1 to S5 for N times, wherein N is greater than or equal to 1, and M second inner points are obtained, and M is greater than or equal to 1;
the maximum second interior point module is acquired and is used for executing step S7: acquiring one second inner point with the largest number of feature point matching pairs from the M second inner points as a target point;
wherein, the step S2 includes: acquiring the rotation amount from a vehicle coordinate system to a camera coordinate system;
acquiring the motion displacement of a vehicle in a time interval of the first image frame and the second image frame; randomly extracting a pair of feature point matching pairs from the first feature point matching pairs; obtaining a first basic matrix according to the epipolar geometry principle;
Wherein, the step S4 includes: acquiring the number of the first interior points to obtain the number of the interior points; establishing a epipolar geometry relation for each first internal point according to the first internal points to obtain the epipolar geometry relations of the internal points; fitting and calculating displacement in the x-axis direction and displacement in the y-axis direction according to the number of the internal points and the epipolar geometric relationships; and calculating to obtain a second basic matrix according to the displacement in the x-axis direction and the displacement in the y-axis direction.
7. A vehicle, the vehicle comprising: memory, a processor and an outlier removal program stored on the memory and executable on the processor, the outlier removal program being configured to implement the steps of the outlier removal method according to any one of claims 1 to 5.
8. A computer readable storage medium, characterized in that a computer program is stored on the computer readable storage medium, which computer program, when being executed by a processor, implements the steps of the outlier removal method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210114108.8A CN114492652B (en) | 2022-01-30 | 2022-01-30 | Outlier removing method and device, vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210114108.8A CN114492652B (en) | 2022-01-30 | 2022-01-30 | Outlier removing method and device, vehicle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114492652A CN114492652A (en) | 2022-05-13 |
| CN114492652B true CN114492652B (en) | 2024-05-28 |
Family
ID=81477537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210114108.8A Active CN114492652B (en) | 2022-01-30 | 2022-01-30 | Outlier removing method and device, vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114492652B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110021065A (en) * | 2019-03-07 | 2019-07-16 | 杨晓春 | A kind of indoor environment method for reconstructing based on monocular camera |
| CN110335295A (en) * | 2019-06-06 | 2019-10-15 | 浙江大学 | A kind of plant point cloud acquisition registration and optimization method based on TOF camera |
| CN111540016A (en) * | 2020-04-27 | 2020-08-14 | 深圳南方德尔汽车电子有限公司 | Pose calculation method and device based on image feature matching, computer equipment and storage medium |
| CN113159197A (en) * | 2021-04-26 | 2021-07-23 | 北京华捷艾米科技有限公司 | Pure rotation motion state judgment method and device |
-
2022
- 2022-01-30 CN CN202210114108.8A patent/CN114492652B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110021065A (en) * | 2019-03-07 | 2019-07-16 | 杨晓春 | A kind of indoor environment method for reconstructing based on monocular camera |
| CN110335295A (en) * | 2019-06-06 | 2019-10-15 | 浙江大学 | A kind of plant point cloud acquisition registration and optimization method based on TOF camera |
| CN111540016A (en) * | 2020-04-27 | 2020-08-14 | 深圳南方德尔汽车电子有限公司 | Pose calculation method and device based on image feature matching, computer equipment and storage medium |
| CN113159197A (en) * | 2021-04-26 | 2021-07-23 | 北京华捷艾米科技有限公司 | Pure rotation motion state judgment method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114492652A (en) | 2022-05-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105021184B (en) | It is a kind of to be used for pose estimating system and method that vision under mobile platform warship navigation | |
| CN108932737B (en) | Vehicle-mounted camera pitch angle calibration method and device, electronic equipment and vehicle | |
| JP5804185B2 (en) | Moving object position / orientation estimation apparatus and moving object position / orientation estimation method | |
| CN112837352B (en) | Image-based data processing method, device and equipment, automobile and storage medium | |
| CN110211151B (en) | Method and device for tracking moving object | |
| CN111279354A (en) | Image processing method, apparatus and computer-readable storage medium | |
| CN111753605A (en) | Lane line positioning method and device, electronic equipment and readable medium | |
| CN111609868A (en) | Visual inertial odometer method based on improved optical flow method | |
| EP4386676A1 (en) | Method and apparatus for calibrating cameras and inertial measurement unit, and computer device | |
| CN115063480A (en) | Pose determination method and device, electronic equipment and readable storage medium | |
| CN108827287B (en) | Robust visual SLAM system in complex environment | |
| US8213684B2 (en) | Motion estimating device | |
| CN113639782A (en) | External parameter calibration method and device for vehicle-mounted sensor, equipment and medium | |
| CN114492652B (en) | Outlier removing method and device, vehicle and storage medium | |
| CN110827337B (en) | Method and device for determining posture of vehicle-mounted camera and electronic equipment | |
| US11468599B1 (en) | Monocular visual simultaneous localization and mapping data processing method apparatus, terminal, and readable storage medium | |
| CN111829552B (en) | Error correction method and device for visual inertial system | |
| CN114937250A (en) | Method and device for calculating relative pose of vehicle body, vehicle, equipment and storage medium | |
| CN110864685B (en) | Vehicle monocular vision wheeled type odometer positioning method based on loose coupling | |
| CN113670327A (en) | Visual inertial odometer initialization method, device, equipment and storage medium | |
| CN112037261A (en) | Method and device for removing dynamic features of image | |
| CN113034538A (en) | Pose tracking method and device of visual inertial navigation equipment and visual inertial navigation equipment | |
| CN114911225B (en) | Method and device for judging slip of double-wheel differential robot | |
| CN113608523B (en) | Monocular vision and inertia fusion based vehicle scene dynamic analysis method | |
| CN116252581B (en) | System and method for estimating vertical and pitching motion information of vehicle body under straight running working condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |