CN110044374A - A kind of method and odometer of the monocular vision measurement mileage based on characteristics of image - Google Patents
A kind of method and odometer of the monocular vision measurement mileage based on characteristics of image Download PDFInfo
- Publication number
- CN110044374A CN110044374A CN201810044762.XA CN201810044762A CN110044374A CN 110044374 A CN110044374 A CN 110044374A CN 201810044762 A CN201810044762 A CN 201810044762A CN 110044374 A CN110044374 A CN 110044374A
- Authority
- CN
- China
- Prior art keywords
- image
- camera
- point
- coordinate
- sin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C22/00—Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
It should include the following steps: that (1) demarcated camera based on the method and odometer of the monocular vision measurement mileage of characteristics of image, method the invention proposes a kind of；(2) the 2D characteristic point of the adjacent two field pictures in front and back is calculated along direction of advance；(3) the 2D characteristic point is matched, finds corresponding characteristic point in the two field pictures；(4) the 3D coordinate of corresponding characteristic point described in two field pictures is calculated, and the camera pose is calculated according to the 3D coordinate of the corresponding characteristic point and 2D coordinate, obtains the relative displacement of the camera；(5) subsequent frame is equally operated, all displacements that finally adds up obtain mileage.Mileage is measured using monocular vision, is compared to the method based on binocular vision, equipment is simple, and cost reduces；It compares based on sift, the method for Harris angle point, calculates characteristics of image speed faster, and there is rotation scale invariability, can handle in real time.
Description
Technical field
The present invention relates to a kind of technical field of image processing, in particular to a kind of monocular vision measurement based on characteristics of image
The method and odometer of mileage.
Background technique
During metro operation, the tables such as tunnel structure outlet percolating water or crack, peeling based on concrete material
Defect and tunnel cross-section deformation are seen, is all unavoidable defect phenomenon, and the long-run development of disease is to the safety in tunnel
Property causes irreversible negative effect.It therefore, is to ensure tunnel long-term safety to the maintenance of tunnel structure in metro operation
The necessary means of operation.The position control of sensor directly influences the validity of detection data acquisition in the detection process.Mesh
Before, the sensor position of most subway tunnel defect detections is all to carry out setting in advance to finish at this stage, for different tunnels
Road section environment cannot usually improve the validity of data by position adjustment, reduce the difficulty of software analysis.In recent years,
With computer technology, Theory of Automatic Control, embedded development, chip design and the rapid development of sensor technology allow tunnel
Road disease is detected automatically and is achieved, and extracts scene image on the detection vehicle of real time execution or image sequence is handled, mention
Take the validity feature of measured target, obtain extraterrestrial target real-time pose information, be subsequent tunnel defect image position with it is inner
Journey provides support.But due to the technical restriction of monocular-camera, it is desirable to which the three-dimensional coordinate information for obtaining mobile object is very
Difficult.There are three types of generating modes for monocular vision, and one is being generated by perspective geometry, with reference to end point, one is pass through mesh
Target is displaced to be formed, and the method for this generation is that restricted condition, such as camera is wanted to fix, and background is fixed, the speed of personage
Constant, then the movement speed of target is faster, he is closer to camera.There are also one is pass through focal length.By different focal length to same
The blur effect of one scene camera shooting measures.This method effect for the entire image of generation is not also fine, but
It is numerical value but than calibrated.Binocular vision is by parallax effect.This effect is the main reason for being capable of forming three-dimensional stereopsis.
Monocular finds parallax effect also mainly by by finding object of reference to generate three-dimensional depth information at present.Monocular estimation
The problem of pose is a three-dimensional scene structure needs through the mobile triangle geometrical relationship to constitute character pair point of interframe.
After triangle geometrical relationship is established, the three-dimensional coordinate of pose and characteristic point solves simultaneously, this is classical three-dimensional scenic knot
Structure problem.Therefore there is no first have chicken still first to have the problem of egg.The solution of three-dimensional scene structure has very much, most can simply lead to
Estimation essential matrix is crossed, the rotation R for obtaining camera and displacement T are then decomposed.In binocular stereo vision, due to base
Line is fixed and known, thus can directly trigonometric ratio obtain characteristic point three-dimensional coordinate.Then the motion information of interframe
It is exactly the kinematic parameter fitting between two heap three-dimensional points；The shortcomings that binocular is, due to baseline be it is fixed, simultaneously because carrier ruler
Very little limitation, usually will not be very wide.Therefore the precision that trigonometric ratio is rebuild generally is not likely to very high.
Therefore, it is necessary to develop a kind of Method for Calculate Mileage based on monocular vision, compare based on sift, Harris
The method of angle point, survey calculation characteristics of image speed faster, and have rotation scale invariability, being capable of real time processed images.
Summary of the invention
The mileage measurement method based on monocular vision that the technical problem to be solved in the present invention is to provide a kind of, equipment is simple,
Cost reduces；It compares based on sift, the method for Harris angle point, survey calculation characteristics of image speed faster, and has rotation
Scale invariability, the measurement method for the monocular vision mileage based on characteristics of image that can be handled in real time.
In order to solve the above-mentioned technical problem, the technical solution adopted by the present invention is that: should monocular vision based on characteristics of image
The method for measuring mileage, specifically comprises the following steps:
(1) camera is demarcated, obtains the inside and outside parameter of the camera；
(2) the 2D characteristic point of the adjacent two field pictures in front and back is calculated along direction of advance；
(3) the 2D characteristic point is matched, finds corresponding characteristic point in the two field pictures；
(4) the 3D coordinate of corresponding characteristic point described in two field pictures is calculated, and according to the 3D of the corresponding characteristic point
Coordinate calculates the camera pose with 2D coordinate, obtains the relative displacement of the camera；
(5) step (1)~(4) successively are repeated to subsequent frame, calculates the position of camera opposite former frame when shooting each frame
It moves, all displacements that finally adds up obtain mileage.
By adopting the above technical scheme, mileage is measured by using monocular vision, is compared to based on binocular vision
Method, equipment is simple, cost reduce；It compares based on sift, HarrisThe method of angle point calculates characteristics of image speed more
Fastly, and there is rotation scale invariability, can handles in real time.
The present invention further improvement lies in that, step (1) includes the following steps:
1-1, according to national forest park in Xiaokeng, obtain the conversion relation of image coordinate system, camera coordinates system and world coordinate system；
1-2, by shooting the gridiron pattern scaling board under multiple different perspectivess, and extract the angle point on scaling board image, root
The pixel coordinate and physical coordinates of angle point are obtained, according to gridiron pattern size so as to find out the homography matrix H of all scaling board images；
1-3, inside and outside parameter is solved；
1-4, minimum projection error is solved by Levenberg-Marquardt algorithm, optimize camera inside and outside parameter.
Preferably, 2D characteristic point, that is, orb characteristic point described in step (2) calculates the orb characteristic point of front and back two field pictures
Specifically comprise the following steps: that constructing image pyramid extracts key point at every layer, then according to brief according to fast algorithm
Algorithm, the selected point pair around the key point generate description, further according to key point and gray scale matter by comparing pixel value
The angle of the heart is sub to adjust description, so that description has rotational invariance, it is sub to finally obtain orb description.
2D Feature Points Matching described in step (3) specifically comprises the following steps:
3-1, k-d tree is established for the feature point set in image, i.e., selection has the dimension k of maximum variance in data set；
Then select the value in k dimension for the characteristic point of intermediate value m be division node；By being divided into less than m of the value on dimension k
Value on dimension k is divided into right subspace greater than m by left subspace；It is carried out respectively in left subspace and right subspace
State operation, until cannot it is subdivided until, obtain k-d tree；
3-2, characteristic matching lookup is carried out with bbf searching algorithm: i.e. since the root node of k-d tree, carrying out binary search,
By the node in query path according to being respectively ranked up at a distance from query point；When being recalled, from the tree of highest priority
Node starts, and when all nodes, which all pass through, to be checked or limit beyond runing time, will be used as apart from shortest point nearest
Adjacent matching characteristic point.
Step (4) specifically comprises the following steps:
4-1, it is arrived according to the pixel size and physical size of image using the image upper left corner as coordinate origin with image taking
Region be plane establish coordinate system, obtain the 3D coordinate of the characteristic point；
4-2, according to 3D coordinate in previous frame image of camera internal reference, the characteristic point and this feature point latter
2D coordinate in frame image, using coordinate transformation relationship, find out camera after the picture is taken a frame when pose, and then find out camera and exist
Shoot the displacement between former frame and latter frame position.
The conversion relation of image coordinate system, camera coordinates system and world coordinate system is specifically in the step 1-1:
(1.11) under world coordinate system, the coordinate of some point is [Xw, Yw, Zw], by under camera coordinates system, the angle
Point coordinate is [Xc, Yc, Zc], by rotating the correspondent transform relationship with translation, Wherein R is spin matrix, and T is the displacement of two coordinate origins, then has
(1.12) point is after camera imaging, and point is [x, y] in the coordinate system that image physical size indicates, according to similar
Triangle relation hasWherein f is the focal length of camera, i.e.,
(1.13) shown in the relationship such as formula (3) of image pixel dimensions coordinate system and image physical size coordinate system, which exists
The coordinate system that image pixel dimensions indicate is [u, v], then has corresponding relationship:Wherein (u0, v0) it is image slices
Plain center, dxPhysical size for a pixel in x-axis direction, dyFor a pixel y-axis direction physical size to get
(1.14) in summary formula (1), formula (2), formula (3) relationship, can obtain:
(1.15) consider that degree of bias parameter C is added, finally have
(1.16) because gridiron pattern scaling board is plane, Z is setw=0, if A indicates camera matrix,r1, r2, r3For the column vector of R, t is translation column vector, then formula (5) can be written as
Homography matrix H is solved, shoots the gridiron pattern scaling board under multiple different perspectivess, and extract on scaling board image
Angle point.Tessellated size is it is known that so the pixel coordinate of angle point can be obtained with physical coordinates again.Pass through least square
Method, can be in the hope of homography matrix H, the H=[h of all scaling board images1 h2 h3], according to formula (6), λ is enabled to indicate some often
Number, available [h1 h2 h3]=λ A [r1 r2t]； (7)
If α, beta, gamma is respectively x-axis, y-axis, the rotation angle in z-axis direction, then spin matrix It is availableAndTo obtain the final product | | r1| |=(cos γ cos β+sin γ
sinα sinβ)2+(-sinγ cosβ+cosγ sinα sinβ)2+(cosα sinβ)2=1, and | | r2| |=(sin γ
cosα)2+(cosγ cosα)2+(-sinα)2=1, so | | r1| |=| | r2| |=1. (8)
Calculate r1·r2=(cos γ cos β+sin γ sin α sin β) (sin γ cos α)+(- sin γ cos β+cos
γ sin α sin β) (cos γ cos α)+(cos α sin β) (- sin α)=0, (9)
It is available according to above-mentioned formula (7), formula (8), formula (9):
That is h1 TA-TA-1h2=0； (10)
Up to h1 TA-TA-1h1=h2 TA-TA-1h2。 (11)
Equation group is established according to above-mentioned formula (10), formula (11), by several groups of homography matrix value bands obtained in step (1.2)
Enter into equation group, can solve to obtain internal reference matrix A；
It enablesIf hi=[hi1, hi2, hi3]T, then haveWherein b
=[B11, B12, B22, B13, B23, B33]T, vij=[hi1hj1, hi1hj2+ hi2hj1, hi2hj2, hi3hj1+hi1hj3, hi3hj2+hi2hj3,
hi3hj3]T；So above-mentioned formula (10), formula (11) can be written asBring all homography matrixes into
Value, solves b, then solves each element value and outer ginseng in internal reference matrix A again.
The present invention also provides a kind of monocular vision odometer based on characteristics of image, using above based on the list of characteristics of image
The method of mesh vision measurement mileage carries out mileage calculation.
Compared with prior art, of the invention have the advantages that is compared to the method based on binocular vision, if
Standby simple, cost reduces；It comparing based on sift, the method for Harris angle point measures and calculates characteristics of image speed faster, and
With rotation scale invariability, can handle in real time.
Detailed description of the invention
Fig. 1 is the national forest park in Xiaokeng figure for the method that the monocular vision of the invention based on characteristics of image measures mileage；
Fig. 2 is adjacent two frame in front and back of the shooting for the method that the monocular vision of the invention based on characteristics of image measures mileage
Schematic diagram.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention figure, technical solution in the embodiment of the present invention carries out clear
Chu, complete description.
Embodiment 1: it should be applied based on the method for the monocular vision measurement mileage of characteristics of image in vehicle-mounted Tunnel testing equipment
In, specifically comprise the following steps:
(1) camera is demarcated first, obtains the parameter of camera；
(2) before and after the direction of advance of vehicle successively calculates two frames 2D characteristic point；
(3) the 2D characteristic point is matched, finds corresponding characteristic point；
(4) the 3D coordinate of characteristic point is calculated, and posture is calculated according to the 3D coordinate of characteristic point and 2D coordinate, is obtained opposite
Displacement；
(5) same method successively is taken to the subsequent frame measured, calculates camera opposite former frame when shooting each frame
Displacement, all displacements that finally adds up obtain mileage；
As shown in Figure 1, using gridiron pattern calibration algorithm in the step (1), camera is demarcated, obtains camera internal reference
Specifically includes the following steps:
(1.1) it according to national forest park in Xiaokeng, is closed by the transformation under image coordinate system, camera coordinates system, world coordinate system
System has:
(1.11) under world coordinate system, the coordinate of some point is [Xw, Yw, Zw], by under camera coordinates system, the angle
Point coordinate is [Xc, Yc, Zc], by rotating the correspondent transform relationship with translation, Wherein R is spin matrix, and T is the displacement of two coordinate origins, then has
(1.12) point is after camera imaging, and point is [x, y] in the coordinate system that image physical size indicates, according to similar
Triangle relation hasWherein f is the focal length of camera, i.e.,
(1.13) shown in the relationship such as formula (3) of image pixel dimensions coordinate system and image physical size coordinate system, which exists
The coordinate system that image pixel dimensions indicate is [u, v], then has corresponding relationship:Wherein (u0, v0) it is image slices
Plain center, dxPhysical size for a pixel in x-axis direction, dyFor a pixel y-axis direction physical size to get
(1.14) in summary formula (1), formula (2), formula (3) relationship, can obtain:
(1.15) consider that degree of bias parameter C is added, finally have
(1.16) because gridiron pattern scaling board is plane, Z is setw=0, if A indicates camera matrix,r1, r2, r3For the column vector of R, t is translation column vector, then formula (5) can be written as
(1.2) homography matrix H is solved, shoots the gridiron pattern scaling board under multiple different perspectivess, and extract scaling board figure
As upper angle point.Tessellated size is it is known that so the pixel coordinate of angle point can be obtained with physical coordinates again.Pass through minimum
Square law, can be in the hope of the homography matrix H of all scaling board images.
(1.3) homography matrix H=[h1 h2 h3], according to formula (6), λ is enabled to indicate some constant, available [h1 h2
h3]=λ A [r1 r2t]； (7)
If α, beta, gamma is respectively x-axis, y-axis, the rotation angle in z-axis direction, then spin matrix It is availableAndTo obtain the final product | | r1| |=(cos γ cos β+sin γ
sinα sinβ)2+(-sinγ cosβ+cosγ sinαs inβ)2+(cosα sinβ)2=1, and | | r2| |=(sin γ
cosα)2+(cosγ cosα)2+(-sinα)2=1, so | | r1| |=| | r2| |=1. (8)
Calculate r1·r2=(cos γ cos β+sin γ sin α sin β) (sin γ cos α)+(- sin γ cos β+cos
γ sin α sin β) (cos γ cos α)+(cos α sin β) (- sin α)=0, (9)
It is available according to above-mentioned formula (7), formula (8), formula (9):
Up to h1 TA-TA-1h1=h2 TA-TA-1h2。 (11)
(1.4) it solves inside and outside parameter: equation group being established according to above-mentioned formula (10), formula (11), will be obtained in step (1.2)
Several groups of homography matrix values be brought into equation group, can solve to obtain internal reference matrix A；
It enablesIf hi=[hi1, hi2, hi3]T, then haveWherein b
=[B11, B12, B22, B13, B23, B33]T, vij=[hi1hj1, hi1hj2+ hi2hj1, hi2hj2, hi3hj1+hi1hj3, hi3hj2+hi2hj3,
hi3hj3]T；So above-mentioned formula (10), formula (11) can be written asBring all homography matrixes into
Value, solves b, then solves each element value and outer ginseng in internal reference matrix A again；
(1.5) it is solved by Levenberg-Marquardt algorithm and minimizes projection error, Lai Youhua camera internal reference and outer
Ginseng；According to fast algorithm in the step 2), the pixel bigger with the difference value of the pixel in peripheral region is extracted
As key point；According to brief algorithm, the selected point pair around key point generates description by comparing pixel value.
Embodiment 2: should be specifically comprised the following steps: based on the method for the monocular vision measurement mileage of characteristics of image
1) according to gridiron pattern calibration algorithm, camera is demarcated, obtains camera internal reference；
2) calculate characteristics of image to two frame of front and back: construction image pyramid first extracts and peripheral region on every layer
The bigger pixel of the difference value of interior pixel is as key point；The selected point pair around key point, by comparing pixel
Value generates description；Description is adjusted according to the angle of key point and gray scale mass center, so that description has invariable rotary
Property；Finally obtain description of characteristics of image；
3) characteristic point on two frame of front and back is matched, obtains corresponding characteristic point: for the feature point set on image
Establish k-d tree: selection has the dimension k of maximum variance in data set；Then select the value in k dimension for the spy of intermediate value m
Sign point is division node；Division by the value on dimension k less than m obtains left subspace, by the value on dimension k greater than m's
It is divided into right subspace；Carry out aforesaid operations in left subspace and right subspace respectively, until cannot it is subdivided until, obtain k-
D tree；Characteristic matching lookup is carried out with bbf searching algorithm: since the root node of k-d tree, binary search is carried out, by query path
On node according to being respectively ranked up at a distance from query point；When being recalled, since the high tree node of priority, work as institute
When some nodes are all limited by inspection or beyond runing time, matched using the best result being currently found as arest neighbors special
Sign point；
4) the 3D coordinate of matched characteristic point is calculated: according to the pixel size and physical size of an image, with an image left side
Upper angle be coordinate origin, using image taking to region establish coordinate system, the 3D coordinate of available characteristic point as plane；
5) according to the 3D coordinate of characteristic point and 2D coordinate, calculate camera after the picture is taken a frame when opposite former frame move away from
From；
6) pose of camera under all frames is successively calculated, displacement is obtained, obtains mileage.
As shown in Figure 1, using gridiron pattern calibration algorithm in the step 1), camera is demarcated, obtains camera internal reference
Specifically includes the following steps:
(1.1) it according to national forest park in Xiaokeng, is closed by the transformation under image coordinate system, camera coordinates system, world coordinate system
System has:
(1.11) under world coordinate system, the coordinate of some point is [Xw, Yw, Zw], by under camera coordinates system, the angle
Point coordinate is [Xc, Yc, Zc], by rotating the correspondent transform relationship with translation, Wherein R is spin matrix, and T is the displacement of two coordinate origins, then has
(1.12) point is after camera imaging, and point is [x, y] in the coordinate system that image physical size indicates, according to similar
Triangle relation hasWherein f is the focal length of camera, i.e.,
(1.13) shown in the relationship such as formula (3) of image pixel dimensions coordinate system and image physical size coordinate system, which exists
The coordinate system that image pixel dimensions indicate is [u, v], then has corresponding relationship:Wherein (u0, v0) it is image slices
Plain center, dxPhysical size for a pixel in x-axis direction, dyFor a pixel y-axis direction physical size to get
(1.14) in summary formula (1), formula (2), formula (3) relationship, can obtain:
(1.15) consider that degree of bias parameter C is added, finally have
(1.16) because gridiron pattern scaling board is plane, Z is setw=0, if A indicates camera matrix,
r1, r2, r3For the column vector of R, t is translation column vector, then formula (5) can be written as
(1.2) homography matrix H is solved, shoots the gridiron pattern scaling board under multiple different perspectivess, and extract scaling board figure
As upper angle point.Tessellated size is it is known that so the pixel coordinate of angle point can be obtained with physical coordinates again.Pass through minimum
Square law, can be in the hope of the homography matrix H of all scaling board images.
(1.3) homography matrix H=[h1 h2 h3], according to formula (6), λ is enabled to indicate some constant, available [h1 h2
h3]=λ A [r1 r2t]； (7)
If α, beta, gamma is respectively x-axis, y-axis, the rotation angle in z-axis direction, then spin matrix It is availableAndTo obtain the final product | | r1| |=(cos γ cos β+sin γ
sinα sinβ)2+(-sinγ cosβ+cosγ sinα sinβ)2+(cosα sinβ)2=1, and | | r2| |=(sin γ
cosα)2+(cosγ cosα)2+(-sinα)2=1, so | | r1| |=| | r2| |=1. (8)
Calculate r1·r2=(cos γ cos β+sin γ sin α sin β) (sin γ cos α)+(- sin γ cos β+cos
γ sin α sin β) (cos γ cos α)+(cos α sin β) (- sin α)=0, (9)
It is available according to above-mentioned formula (7), formula (8), formula (9):
Up to h1 TA-TA-1h1=h2 TA-TA-1h2。 (11)
(1.4) it solves inside and outside parameter: equation group being established according to above-mentioned formula (10), formula (11), will be obtained in step (1.2)
Several groups of homography matrix values be brought into equation group, can solve to obtain internal reference matrix A；
It enablesIf hi=[hi1, hi2, hi3]T, then haveWherein b
=[B11, B12, B22, B13, B23, B33]T, vij=[hi1hj1, hi1hj2+ hi2hj1, hi2hj2, hi3hj1+hi1hj3, hi3hj2+hi2hj3,
hi3hj3]T；So above-mentioned formula (10), formula
(11) it can be written asThe value for bringing all homography matrixes into solves b, then asks again
Solve each element value and outer ginseng in internal reference matrix A；
(1.5) it is solved by Levenberg-Marquardt algorithm and minimizes projection error, Lai Youhua camera internal reference and outer
Ginseng；According to fast algorithm in the step 2), the pixel bigger with the difference value of the pixel in peripheral region is extracted
As key point；According to brief algorithm, the selected point pair around key point generates description by comparing pixel value；It is described
Step 5), specifically includes the following steps: the camera internal reference obtained using step 1), characteristic point 3D coordinate obtained in step 4),
And 2D pixel coordinate of the characteristic point in a later frame, can find out camera after the picture is taken a frame when pose, it is therein displacement point
Amount indicates displacement of the camera between shooting former frame and latter frame position；It is in the step 6) specifically includes the following steps: right
The image that direction of advance takes be repeated in carry out step 2) to step 5) operation, successively by camera before and after shooting two frames
When displacement component add up, obtain mileage.
Embodiment 3: should be specifically comprised the following steps: based on the method for the monocular vision measurement mileage of characteristics of image
1) according to gridiron pattern calibration algorithm, camera is demarcated, obtains camera internal reference；
2) calculate characteristics of image to two frame of front and back: construction image pyramid first extracts and peripheral region on every layer
The bigger pixel of the difference value of interior pixel is as key point；The selected point pair around key point, by comparing pixel
Value generates description；Description is adjusted according to the angle of key point and gray scale mass center, so that description has invariable rotary
Property；Finally obtain description of characteristics of image；
3) characteristic point on two frame of front and back is matched, obtains corresponding characteristic point: for the feature point set on image
Establish k-d tree: selection has the dimension k of maximum variance in data set；Then select the value in k dimension for the spy of intermediate value m
Sign point is division node；Division by the value on dimension k less than m obtains left subspace, by the value on dimension k greater than m's
It is divided into right subspace；Carry out aforesaid operations in left subspace and right subspace respectively, until cannot it is subdivided until, obtain k-
D tree；Characteristic matching lookup is carried out with bbf searching algorithm: since the root node of k-d tree, binary search is carried out, by query path
On node according to being respectively ranked up at a distance from query point；When being recalled, since the high tree node of priority, work as institute
When some nodes are all limited by inspection or beyond runing time, matched using the best result being currently found as arest neighbors special
Sign point；
4) the 3D coordinate of matched characteristic point is calculated: according to the pixel size and physical size of an image, with an image left side
Upper angle be coordinate origin, using image taking to region establish coordinate system, the 3D coordinate of available characteristic point as plane；
5) according to the 3D coordinate of characteristic point and 2D coordinate, calculate camera after the picture is taken a frame when opposite former frame move away from
From；
6) pose of camera under all frames is successively calculated, displacement is obtained, obtains mileage.
As shown in Figure 1, using gridiron pattern calibration algorithm in the step 1), camera is demarcated, obtains camera internal reference
Specifically includes the following steps:
(1.1) it according to national forest park in Xiaokeng, is closed by the transformation under image coordinate system, camera coordinates system, world coordinate system
System has:
(1.11) under world coordinate system, the coordinate of some point is [Xw, Yw, Zw], by under camera coordinates system, the angle
Point coordinate is [Xc, Yc, Zc], by rotating the correspondent transform relationship with translation, Wherein R is spin matrix, and T is the displacement of two coordinate origins, then has
(1.12) point is after camera imaging, and point is [x, y] in the coordinate system that image physical size indicates, according to similar
Triangle relation hasWherein f is the focal length of camera, i.e.,
(1.13) shown in the relationship such as formula (3) of image pixel dimensions coordinate system and image physical size coordinate system, which exists
The coordinate system that image pixel dimensions indicate is [u, v], then has corresponding relationship:Wherein (u0, v0) it is image slices
Plain center, dxPhysical size for a pixel in x-axis direction, dyFor a pixel y-axis direction physical size to get
(1.14) in summary formula (1), formula (2), formula (3) relationship, can obtain:
(1.15) consider that degree of bias parameter C is added, finally have
(1.16) because gridiron pattern scaling board is plane, Z is setw=0, if A indicates camera matrix,
r1, r2, r3For the column vector of R, t is translation column vector, then formula (5) can be written as
(1.2) homography matrix H is solved；The gridiron pattern scaling board under multiple different perspectivess is shot, and extracts scaling board figure
As upper angle point, and tessellated size passes through minimum it is known that so the pixel coordinate of angle point can be obtained with physical coordinates
Square law, can be in the hope of the homography matrix H of all scaling board images；
(1.3) homography matrix H=[h1 h2 h3], according to formula (6), λ is enabled to indicate some constant, available [h1 h2
h3]=λ A [r1 r2t]； (7)
If α, beta, gamma is respectively x-axis, y-axis, the rotation angle in z-axis direction, then spin matrix It is availableAndTo obtain the final product | | r1| |=(cos γ cos β+sin γ
sinα sinβ)2+(-sinγ cosβ+cosγ sinα sinβ)2+(cosα sinβ)2=1, and | | r2| |=(sin γ
cosα)2+(cosγ cosα)2+(-sinα)2=1, so | | r1| |=| | r2| |=1 (8)
Calculate r1·r2=(cos γ cos β+sin γ sin α sin β) (sin γ cos α)+(- sin γ cos β+cos
γ sin α sin β) (cos γ cos α)+(cos α sin β) (- sin α)=0, (9)
It is available according to above-mentioned formula (7), formula (8), formula (9):
Up to h1 TA-TA-1h1=h2 TA-TA-1h2 (11)
(1.4) it solves inside and outside parameter: equation group being established according to above-mentioned formula (10), formula (11), will be obtained in step (1.2)
Several groups of homography matrix values be brought into equation group, can solve to obtain internal reference matrix A；
It enablesIf hi=[hi1, hi2, hi3]T, then haveWherein b
=[B11, B12, B22, B13, B23, B33]T, vij=[hi1hj1, hi1hj2+ hi2hj1, hi2hj2, hi3hj1+hi1hj3, hi3hj2+hi2hj3,
hi3hj3]T；So above-mentioned formula (10), formula (11) can be written asBring all homography matrixes into
Value, solves b, then solves each element value and outer ginseng in internal reference matrix A again；
(1.5) it is solved by Levenberg-Marquardt algorithm and minimizes projection error, Lai Youhua camera internal reference and outer
Ginseng；According to fast algorithm in the step 2), the pixel bigger with the difference value of the pixel in peripheral region is extracted
As key point；According to brief algorithm, the selected point pair around key point generates description by comparing pixel value；It is described
Step 5), specifically includes the following steps: the camera internal reference obtained using step 1), characteristic point 3D coordinate obtained in step 4),
And 2D pixel coordinate of the characteristic point in a later frame, can find out camera after the picture is taken a frame when pose, it is therein displacement point
Amount indicates displacement of the camera between shooting former frame and latter frame position；It is in the step 6) specifically includes the following steps: right
The image that direction of advance takes be repeated in carry out step 2) to step 5) operation, successively by camera before and after shooting two frames
When displacement component add up, obtain mileage；The method of the monocular vision measurement mileage based on characteristics of image is used for tunnel
The monocular vision odometer of characteristics of image in detection；With feature testing result the most significant in target in the step (3)
As the primary condition of characteristic matching, determine target encoded surface visual in visual field, and as original state, start into
Row characteristic matching.
Embodiment 4
A kind of monocular vision odometer based on characteristics of image carries in the vehicle-mounted detection platform of Tunnel testing
Program used the method in Examples 1 to 3.
The following table is six parameter of pose of monocular vision measurement camera:
Six parameter of pose of 1 monocular vision of table measurement camera
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to restrict the invention, all in essence of the invention
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should all be included in the protection scope of the present invention.
Claims (9)
1. a kind of method of the monocular vision measurement mileage based on characteristics of image, which is characterized in that specifically comprise the following steps:
(1) camera is demarcated, obtains the inside and outside parameter of the camera；
(2) the 2D characteristic point of the adjacent two field pictures in front and back is calculated along direction of advance；
(3) the 2D characteristic point is matched, finds corresponding characteristic point in the two field pictures；
(4) the 3D coordinate of corresponding characteristic point described in two field pictures is calculated, and according to the 3D coordinate of the corresponding characteristic point
The camera pose is calculated with 2D coordinate, obtains the relative displacement of the camera；
(5) step (1)~(4) successively are repeated to subsequent frame, calculates the displacement of camera opposite former frame when shooting each frame, most
All displacements that adds up afterwards obtain mileage.
2. the method for the monocular vision measurement mileage according to claim 1 based on characteristics of image, which is characterized in that step
(1) specifically comprise the following steps:
1-1, according to national forest park in Xiaokeng, obtain the conversion relation of image coordinate system, camera coordinates system and world coordinate system；
1-2, by shooting the gridiron pattern scaling board under multiple different perspectivess, and the angle point on scaling board image is extracted, according to chess
Disk lattice size obtains the pixel coordinate and physical coordinates of angle point, so as to find out the homography matrix H of all scaling board images；
1-3, inside and outside parameter is solved；
1-4, minimum projection error is solved by Levenberg-Marquardt algorithm, optimize camera inside and outside parameter.
3. the method for the monocular vision measurement mileage according to claim 2 based on characteristics of image, which is characterized in that step
(2) characteristic point of 2D described in, that is, orb characteristic point, the orb characteristic point for calculating front and back two field pictures specifically comprise the following steps: to construct
Image pyramid extracts key point at every layer according to fast algorithm, then according to brief algorithm, around the key point
Selected point pair generates description by comparing pixel value, and description is adjusted further according to the angle of key point and gray scale mass center,
So that description has rotational invariance, orb description is finally obtained.
4. the method for the monocular vision measurement mileage according to claim 3 based on characteristics of image, which is characterized in that step
(3) the 2D Feature Points Matching described in specifically comprises the following steps:
3-1, k-d tree is established for the feature point set in image, i.e., selection has the dimension k of maximum variance in data set；Then
Select the value in k dimension for the characteristic point of intermediate value m be division node；Value on dimension k is divided into left son sky less than m
Between, the value on dimension k is divided into right subspace greater than m；Aforesaid operations are carried out in left subspace and right subspace respectively,
Until cannot it is subdivided until, obtain k-d tree；
3-2, characteristic matching lookup is carried out with bbf searching algorithm: i.e. since the root node of k-d tree, carrying out binary search, will look into
The node on path is ask according to being respectively ranked up at a distance from query point；When being recalled, from the tree node of highest priority
Start, it, will be apart from shortest point as arest neighbors when all nodes, which all pass through, to be checked or limit beyond runing time
With characteristic point.
5. the method for the monocular vision measurement mileage according to claim 4 based on characteristics of image, which is characterized in that step
(4) specifically comprise the following steps:
4-1, pixel size and physical size according to image, using the image upper left corner as coordinate origin, the area that is arrived with image taking
Domain is that plane establishes coordinate system, obtains the 3D coordinate of the characteristic point；
4-2, according to 3D coordinate in previous frame image of camera internal reference, the characteristic point and this feature point in a later frame figure
2D coordinate as in, using coordinate transformation relationship, find out camera after the picture is taken a frame when pose, and then find out camera and shooting
Displacement between former frame and latter frame position.
6. the method for the monocular vision measurement mileage according to claim 2 based on characteristics of image, which is characterized in that described
The conversion relation of image coordinate system, camera coordinates system and world coordinate system is specifically in step 1-1:
(1.11) under world coordinate system, the coordinate of some point is [Xw, Yw, Zw], under camera coordinates system, which is [Xc,
Yc, Zc], by rotating the correspondent transform relationship with translation,
Wherein R is spin matrix, and T is the displacement of two coordinate origins, then has
(1.12) point is after camera imaging, and point is [x, y] in the coordinate system that image physical size indicates, according to similar triangle
Relationship hasWherein f is the focal length of camera, i.e.,
(1.13) coordinate system that this indicates in image pixel dimensions is set as [u, v], then image pixel dimensions coordinate system and figure
As the relationship of physical size coordinate system:Wherein (u0, v0) it is image pixel center, dxIt is a pixel in x-axis
The physical size in direction, dyFor a pixel y-axis direction physical size to get
(1.14) in summary formula (1), formula (2), formula (3) relationship, can obtain:
(1.15) consider that degree of bias parameter C is added, finally have
(1.16) because gridiron pattern scaling board is plane, Z is setw=0, if A indicates camera matrix,
r1, r2, r3For the column vector of R, t is translation column vector, then formula (5) is written as
7. the method for the monocular vision measurement mileage according to claim 6 based on characteristics of image, which is characterized in that described
Homography matrix H=[h1h2h3], according to formula (6), enables λ indicate constant, obtain [h1h2h3]=λ A [r1r2t]； (7)
If α, beta, gamma is respectively x-axis, y-axis, the rotation angle in z-axis direction, then spin matrix It obtainsAndTo obtain the final product | | r1| |=(cos γ cos β+sin γ
sinαsinβ)2+(-sinγcosβ+cosγsinαsinβ)2+(cosαsinβ)2=1, and | | r2| |=(sin γ cos α)2+
(cosγcosα)2+(-sinα)2=1,
So | | r1| |=| | r2| |=1 (8)
Calculate r1·r2=(cos γ cos β+sin γ sin α sin β) (sin γ cos α)+(- sin γ cos β+cos γ sin α sin
β) (cos γ cos α)+(cos α sin β) (- sin α)=0, (9)
It is available according to above-mentioned formula (7), formula (8), formula (9):
I.e.
Up to h1 TA-TA-1h1=h2 TA-TA-1h2 (11)。
8. the method for the monocular vision measurement mileage according to claim 7 based on characteristics of image, which is characterized in that described
The inside and outside parameter detailed process for solving camera is: equation group is established according to formula (10), formula (11), it will be obtained in step (1-2)
Several groups of homography matrix values are updated in equation group, can solve to obtain internal reference matrix A；
It enablesIf hi=[hi1, hi2, hi3]T, then haveWherein b=
[B11, B12, B22, B13, B23, B33]T, vij=[hi1hj1, hi1hj2+hi2hj1, hi2hj2, hi3hj1+hi1hj3, hi3hj2+hi2hj3,
hi3hj3]T；So above-mentioned formula (10), formula (11) can be written asThe value of all homography matrixes is substituted into,
B is solved, then solves each element value and outer ginseng in internal reference matrix A again.
9. a kind of monocular vision odometer based on characteristics of image, it is characterised in that: the monocular vision odometer uses right
It is required that the method for 1~8 described in any item monocular vision measurement mileages based on characteristics of image carries out mileage calculation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810044762.XA CN110044374B (en) | 2018-01-17 | 2018-01-17 | Image feature-based monocular vision mileage measurement method and odometer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810044762.XA CN110044374B (en) | 2018-01-17 | 2018-01-17 | Image feature-based monocular vision mileage measurement method and odometer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110044374A true CN110044374A (en) | 2019-07-23 |
| CN110044374B CN110044374B (en) | 2022-12-09 |
Family
ID=67273048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810044762.XA Active CN110044374B (en) | 2018-01-17 | 2018-01-17 | Image feature-based monocular vision mileage measurement method and odometer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110044374B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110687929A (en) * | 2019-10-10 | 2020-01-14 | 辽宁科技大学 | Aircraft three-dimensional space target searching system based on monocular vision and motor imagery |
| CN110929567A (en) * | 2019-10-17 | 2020-03-27 | 北京全路通信信号研究设计院集团有限公司 | Monocular camera monitoring scene-based target position and speed measuring method and system |
| CN111922510A (en) * | 2020-09-24 | 2020-11-13 | 武汉华工激光工程有限责任公司 | Laser visual processing method and system |
| CN111990314A (en) * | 2020-08-25 | 2020-11-27 | 中国水产科学研究院渔业机械仪器研究所 | System and method for quantitative observation of fish behaviors |
| CN112066876A (en) * | 2020-08-27 | 2020-12-11 | 武汉大学 | Method for rapidly measuring object size by using mobile phone |
| CN112449176A (en) * | 2019-09-03 | 2021-03-05 | 浙江舜宇智能光学技术有限公司 | Test method and test system of lifting type camera device |
| CN112798812A (en) * | 2020-12-30 | 2021-05-14 | 中山联合汽车技术有限公司 | Target speed measuring method based on monocular vision |
| CN113223163A (en) * | 2021-04-28 | 2021-08-06 | Oppo广东移动通信有限公司 | Point cloud map construction method and device, equipment and storage medium |
| CN113223007A (en) * | 2021-06-28 | 2021-08-06 | 浙江华睿科技股份有限公司 | Visual odometer implementation method and device and electronic equipment |
| CN114764005A (en) * | 2021-03-11 | 2022-07-19 | 深圳市科卫泰实业发展有限公司 | Monocular vision odometer method for unmanned aerial vehicle |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663767A (en) * | 2012-05-08 | 2012-09-12 | 北京信息科技大学 | Method for calibrating and optimizing camera parameters of vision measuring system |
| US20150314443A1 (en) * | 2014-04-30 | 2015-11-05 | Shenzhen Mercury Optoelectronics Research Institute | Visual-based obstacle detection method and apparatus for mobile robot |
| CN105354273A (en) * | 2015-10-29 | 2016-02-24 | 浙江高速信息工程技术有限公司 | Method for fast retrieving high-similarity image of highway fee evasion vehicle |
| CN106920259A (en) * | 2017-02-28 | 2017-07-04 | 武汉工程大学 | A kind of localization method and system |
| CN106952299A (en) * | 2017-03-14 | 2017-07-14 | 大连理工大学 | A kind of 3 d light fields Implementation Technology suitable for Intelligent mobile equipment |
| CN107580175A (en) * | 2017-07-26 | 2018-01-12 | 济南中维世纪科技有限公司 | A kind of method of single-lens panoramic mosaic |
-
2018
- 2018-01-17 CN CN201810044762.XA patent/CN110044374B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663767A (en) * | 2012-05-08 | 2012-09-12 | 北京信息科技大学 | Method for calibrating and optimizing camera parameters of vision measuring system |
| US20150314443A1 (en) * | 2014-04-30 | 2015-11-05 | Shenzhen Mercury Optoelectronics Research Institute | Visual-based obstacle detection method and apparatus for mobile robot |
| CN105354273A (en) * | 2015-10-29 | 2016-02-24 | 浙江高速信息工程技术有限公司 | Method for fast retrieving high-similarity image of highway fee evasion vehicle |
| CN106920259A (en) * | 2017-02-28 | 2017-07-04 | 武汉工程大学 | A kind of localization method and system |
| CN106952299A (en) * | 2017-03-14 | 2017-07-14 | 大连理工大学 | A kind of 3 d light fields Implementation Technology suitable for Intelligent mobile equipment |
| CN107580175A (en) * | 2017-07-26 | 2018-01-12 | 济南中维世纪科技有限公司 | A kind of method of single-lens panoramic mosaic |
Non-Patent Citations (1)
| Title |
|---|
| 高成: "基于单目视觉的三维信息重构的研究与实现", 《中国优秀硕士学位论文（电子期刊）信息科技辑》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112449176A (en) * | 2019-09-03 | 2021-03-05 | 浙江舜宇智能光学技术有限公司 | Test method and test system of lifting type camera device |
| CN110687929B (en) * | 2019-10-10 | 2022-08-12 | 辽宁科技大学 | Aircraft three-dimensional space target searching system based on monocular vision and motor imagery |
| CN110687929A (en) * | 2019-10-10 | 2020-01-14 | 辽宁科技大学 | Aircraft three-dimensional space target searching system based on monocular vision and motor imagery |
| CN110929567A (en) * | 2019-10-17 | 2020-03-27 | 北京全路通信信号研究设计院集团有限公司 | Monocular camera monitoring scene-based target position and speed measuring method and system |
| CN110929567B (en) * | 2019-10-17 | 2022-09-27 | 北京全路通信信号研究设计院集团有限公司 | Monocular camera monitoring scene-based target position and speed measuring method and system |
| CN111990314A (en) * | 2020-08-25 | 2020-11-27 | 中国水产科学研究院渔业机械仪器研究所 | System and method for quantitative observation of fish behaviors |
| CN112066876A (en) * | 2020-08-27 | 2020-12-11 | 武汉大学 | Method for rapidly measuring object size by using mobile phone |
| CN111922510A (en) * | 2020-09-24 | 2020-11-13 | 武汉华工激光工程有限责任公司 | Laser visual processing method and system |
| CN112798812A (en) * | 2020-12-30 | 2021-05-14 | 中山联合汽车技术有限公司 | Target speed measuring method based on monocular vision |
| CN112798812B (en) * | 2020-12-30 | 2023-09-26 | 中山联合汽车技术有限公司 | Target speed measuring method based on monocular vision |
| CN114764005A (en) * | 2021-03-11 | 2022-07-19 | 深圳市科卫泰实业发展有限公司 | Monocular vision odometer method for unmanned aerial vehicle |
| CN113223163A (en) * | 2021-04-28 | 2021-08-06 | Oppo广东移动通信有限公司 | Point cloud map construction method and device, equipment and storage medium |
| CN113223007A (en) * | 2021-06-28 | 2021-08-06 | 浙江华睿科技股份有限公司 | Visual odometer implementation method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110044374B (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110044374A (en) | A kind of method and odometer of the monocular vision measurement mileage based on characteristics of image | |
| CN109360240B (en) | Small unmanned aerial vehicle positioning method based on binocular vision | |
| CN106651942B (en) | Three-dimensional rotating detection and rotary shaft localization method based on characteristic point | |
| CN104930985B (en) | Binocular vision 3 D topography measurement method based on space-time restriction | |
| CN103759670B (en) | A kind of object dimensional information getting method based on numeral up short | |
| CN106780619B (en) | Human body size measuring method based on Kinect depth camera | |
| CN109993793B (en) | Visual positioning method and device | |
| CN107154014B (en) | Real-time color and depth panoramic image splicing method | |
| CN109272537A (en) | A kind of panorama point cloud registration method based on structure light | |
| CN109523595A (en) | A kind of architectural engineering straight line corner angle spacing vision measuring method | |
| CN109859272A (en) | A kind of auto-focusing binocular camera scaling method and device | |
| CN110319772A (en) | Visual large-span distance measurement method based on unmanned aerial vehicle | |
| Nagy et al. | Online targetless end-to-end camera-LiDAR self-calibration | |
| CN104504691B (en) | Camera position and posture measuring method on basis of low-rank textures | |
| CN110230979A (en) | A kind of solid target and its demarcating three-dimensional colourful digital system method | |
| CN107845096B (en) | Image-based planet three-dimensional information measuring method | |
| CN113313659A (en) | High-precision image splicing method under multi-machine cooperative constraint | |
| CN116563377A (en) | Mars rock measurement method based on hemispherical projection model | |
| CN113012238B (en) | Method for quick calibration and data fusion of multi-depth camera | |
| CN106570908A (en) | Stereoscopic vision apparatus for testing scattering area of mother-son flyer | |
| CN103260008B (en) | A kind of image position is to the projection conversion method of physical location | |
| CN108257184B (en) | Camera attitude measurement method based on square lattice cooperative target | |
| Remondino | 3D reconstruction of static human body with a digital camera | |
| Zhang et al. | A stereo photogrammetry system for model position and attitude measurement in hypersonic wind tunnel testing | |
| Skuratovskyi et al. | Outdoor mapping framework: from images to 3d model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200703 Address after: No.5014-112, 5 / F, No.36, Haidian Street, Haidian District, Beijing 100080 Applicant after: Kuanyan (Beijing) Technology Development Co.,Ltd. Address before: Huaxi Securities Building, No. 9 Yuhuatai East Road, Yuhua District of Nanjing City, Jiangsu province 210012 Room 203 Applicant before: NANJING HUOYANHOU INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |