CN110375732A - Monocular camera pose measurement method based on inertial measurement unit and point line characteristics - Google Patents
Monocular camera pose measurement method based on inertial measurement unit and point line characteristics Download PDFInfo
- Publication number
- CN110375732A CN110375732A CN201910659311.1A CN201910659311A CN110375732A CN 110375732 A CN110375732 A CN 110375732A CN 201910659311 A CN201910659311 A CN 201910659311A CN 110375732 A CN110375732 A CN 110375732A
- Authority
- CN
- China
- Prior art keywords
- feature
- line
- point
- frame image
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
The relative pose of the monocular camera is measured by using the point-line characteristics in a plane scene based on an inertial measurement unit and a shooting measurement technology. The measuring system is composed of an inertial measuring unit and a camera, wherein the inertial measuring unit is used for providing a pitch angle and a roll angle for the monocular camera in real time, and the camera estimates a yaw angle and a translation vector according to the point line characteristics. The invention firstly utilizes the angle data output by the inertial measurement unit to correct the coordinate system of the camera to the direct downward view, and then matches the point-line characteristics in the image, thereby solving the relative pose of the camera. The method makes full use of visual information in the scene, applies the linear characteristics to pose estimation of the monocular camera, is suitable for the fields of autonomous navigation, automatic driving, augmented reality and the like, and has important theoretical research significance and wide application prospect.
Description
Technical field
This application involves videographic measurment technical fields, combine more particularly to a kind of Inertial Measurement Unit and dotted line feature
Monocular camera pose measuring method.
Background technique
Vision positioning technology is a kind of new technique developed rapidly in recent years, the application in navigator fix
Increasingly it is valued by people.Although existing vision guided navigation location algorithm has very much, humanoid robot, intelligence are being carried
Under some the special environments such as fork truck, high-precision determines that relative pose still suffers from certain challenge.It is solved using point feature single
Mesh camera pose is the method generallyd use now, but under the scene that texture is unobvious, angle point is few, it is then difficult to only rely on point feature
To obtain ideal result.Therefore in addition to point feature, how bonding wire Feature-solving pose is an important research side for research
To.The existing pose estimation based on line feature is mainly applied in binocular or multi-vision visual, with less in monocular.
Summary of the invention
Based on this, it is necessary to be difficult to carry out pose measurement in the case where scene textural characteristics are few for current monocular camera
The problem of, a kind of Inertial Measurement Unit is provided and monocular camera pose measuring method, device, computer that dotted line feature combines are set
Standby and storage medium.
A kind of monocular camera pose measuring method of Inertial Measurement Unit and the combination of dotted line feature, which comprises
Correction matrix is obtained using the roll angle and pitch angle information of Inertial Measurement Unit measurement, image rectification can be positive
Lower visible image, that is, the Z-direction of camera coordinates system is consistent with gravity direction after correcting；
The fisrt feature point and second feature point that same reference point is indicated in first frame image and the second frame image are obtained, with
And obtain the fisrt feature line and second feature line that same reference line is indicated in first frame image and the second frame image；
It is constrained according to the homography of vision algorithm dotted line feature, obtains the fisrt feature point and second feature point
The first constraint equation and obtain the second constraint equation of the fisrt feature line and the second feature line；
According to first constraint equation and the second constraint equation, homography matrix is obtained, homography matrix is further
It decomposes, the relative pose relationship after being corrected between image pair；
According to the relative pose relationship between image pair after the correction, conversion obtains the monocular before camera coordinates system correction
Camera pose measurement result.
The rotation between camera coordinates system and Inertial Measurement Unit coordinate system is demarcated in advance in one of the embodiments,
Torque battle array.According to the Inertial Measurement Unit with respect to camera spin matrix and Inertial Measurement Unit output pitch angle and
The corresponding spin matrix of roll angle, can be obtained correction matrix.After camera coordinates system correction, pose estimation problem can be effectively reduced
Freedom degree, the movement of 6DOF between camera can be reduced to 3DOF, that is, include the rotation angle of 1 freedom degree
With the translation vector of a 2DOF.
In one of the embodiments, further include: it can detect the point feature in image in real time using ORB algorithm,
And the matching of point feature can be rapidly completed according to ORB key point and description sub-information, finally obtain the fisrt feature point matched
With second feature point.
In one of the embodiments, further include: the point of the straight line and straight line neighborhood that are detected according to LSD algorithm is special
Sign, can construct the characteristic of straight line neighborhood, and then can acquire the similarity of two straight lines.It, can structure to any 2 frame image
A similarity matrix is built out, maximum two straight lines of similarity are final matching line segments as a result, final obtain in matrix
The fisrt feature line and second feature line matched.
In one of the embodiments, further include: closed according to the matching of the fisrt feature point and the second feature point
System, establishes the first constraint equation of the fisrt feature point and second feature point；According to the fisrt feature line and described
The matching relationship of second feature line establishes the second constraint equation of the fisrt feature line and the second feature line.
In one of the embodiments, further include: combine first constraint equation and second constraint equation, i.e.,
Obtain the pose measurement equation based on Joint of Line and Dot feature；By solving equation and further converting, monocular camera position can be obtained
Appearance measurement result.
A kind of monocular camera pose measuring apparatus of Inertial Measurement Unit and the combination of dotted line feature, described device include:
Inertia measuring module, for receiving the measurement result for the Inertial Measurement Unit being connected with monocular camera, according to described
Measurement result obtains correction matrix, and first frame image and the second frame image rectification are positive lower visible image；
Dotted line matching module, for the characteristic point and characteristic curve in detection image, and it is special between the dotted line different frame image
Sign is matched, and the point feature matched and line feature are obtained；
Module is established in constraint, for being constrained according to the homography of vision algorithm dotted line feature, is obtained based on point feature
First constraint equation and the second constraint equation based on line feature；
Pose measurement module is used for according to first constraint equation and the second constraint equation, image pair after being corrected
Between relative pose relationship；By further converting, the monocular camera pose measurement result before obtaining camera coordinates system correction.
A kind of computer equipment, including memory and processor, the memory are stored with computer program, the processing
Device performs the steps of when executing the computer program
The measurement result for receiving the Inertial Measurement Unit being connected with monocular camera is corrected according to the measurement result
The first frame plane of delineation and the second frame plane of delineation are corrected the lower apparent direction that is positive by matrix；
According to the algorithm of dotted line characteristic matching, first frame image and characteristic point and characteristic curve in the second frame image are detected,
Complete the matching of dotted line feature；
It is constrained according to the homography of vision algorithm dotted line feature, obtains the first constraint equation and the base based on point feature
In the second constraint equation of line feature；
According to first constraint equation and the second constraint equation, corresponding homography square between image pair after being corrected
Battle array, homography matrix is decomposed to obtain spin matrix and translation vector；
According to the relative pose relationship between image pair after the correction, conversion obtains the monocular before camera coordinates system correction
Camera pose measurement result.
A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor
It is performed the steps of when row
The measurement result for receiving the Inertial Measurement Unit being connected with monocular camera is corrected according to the measurement result
The first frame plane of delineation and the second frame plane of delineation are corrected the lower apparent direction that is positive by matrix；
According to the algorithm of dotted line characteristic matching, the characteristic point and feature obtained in first frame image and the second frame image is extracted
Line completes the matching of dotted line feature；
It is constrained according to the homography of vision algorithm dotted line feature, obtains the first constraint equation and the base based on point feature
In the second constraint equation of line feature；
According to first constraint equation and the second constraint equation, corresponding homography square between image pair after being corrected
Battle array, homography matrix is decomposed to obtain spin matrix and translation vector；
According to the relative pose relationship between image pair after the correction, conversion obtains the monocular before camera coordinates system correction
Camera pose measurement result.
Monocular camera pose measuring method, the device, computer equipment that above-mentioned Inertial Measurement Unit and dotted line feature combine
And storage medium, camera calibration is positive the direction of lower view using the measurement result of Inertial Measurement Unit, it is possible to reduce unknown number
Number is handled, by obtaining point feature and line feature in any two field pictures, according to point feature in two images and line feature
Matching relationship, construct the constraint equation based on homography principle, solved to obtain homography matrix according to constraint equation, will singly answered
Property the available correction of matrix decomposition after spin matrix and translation vector, final conversion obtains the list before camera coordinates system correction
The measurement result of mesh camera pose.
Detailed description of the invention
Fig. 1 is answering for the monocular camera pose measuring method that Inertial Measurement Unit and dotted line feature combine in one embodiment
With scene figure；
Fig. 2 is the original for the monocular camera pose measuring method that Inertial Measurement Unit and dotted line feature combine in one embodiment
Manage schematic diagram；
Fig. 3 is the stream for the monocular camera pose measuring method that Inertial Measurement Unit and dotted line feature combine in one embodiment
Journey schematic diagram；
Fig. 4 is camera calibration schematic diagram in one embodiment；
Fig. 5 is the knot for the monocular camera pose measuring apparatus that Inertial Measurement Unit and dotted line feature combine in one embodiment
Structure block diagram；
Fig. 6 is the internal structure chart of computer equipment in one embodiment.
Specific embodiment
It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood
The application is further elaborated.It should be appreciated that specific embodiment described herein is only to explain the application, and do not have to
In restriction the application.
The monocular camera pose measuring method that Inertial Measurement Unit and dotted line feature provided by the present application combines, can apply
In application environment as shown in Figure 1.Measuring system is made of measuring unit 102 and server 104, passes through network between them
It is communicated.Measuring unit 102 is made of 1 Inertial Measurement Unit and 1 camera, constitutes and working principle is as shown in Figure 2.
In one embodiment, as shown in figure 3, providing the monocular phase of a kind of Inertial Measurement Unit and the combination of dotted line feature
Seat in the plane gesture measuring method is applied to be illustrated for the server in Fig. 1 in this way, comprising the following steps:
Step 302, the roll angle and pitch angle information for receiving the Inertial Measurement Unit measurement that is connected with monocular camera, according to survey
Information is measured, correction matrix is obtained, the first frame plane of delineation and the second frame plane of delineation are corrected into the lower apparent direction that is positive.
As shown in figure 4, original camera is corrected into the direction on just lower view ground by camera calibration.If reference point is
Pw, the coordinate of fisrt feature point is Pc1, the coordinate of second feature point is Pc2, spin matrix peace of the monocular camera relative to plane
The amount of shifting to is Rwci、twci(i=1,2), then Pw、Pc1And Pc2Between relationship meet:
Pw=Rwc1Pc1+twc1
Pw=Rwc2Pc2+twc2
If Inertial Measurement Unit is with respect to the spin matrix of world coordinate systemWhereinTo bow
Elevation angle spin matrix and roll angle spin matrix, it is accurately known.Relative rotation square of the monocular camera relative to Inertial Measurement Unit
Battle array is denoted as Rcalib, accurately known.Since the yaw angular accuracy that Inertial Measurement Unit is surveyed is not high,It is set as unknown, then camera is opposite
The spin matrix of world coordinate system are as follows:
It is available according to above-mentioned two formula:
It is hereby achieved that following equation:
Correction matrix isThe plane of delineation can be corrected the direction for the lower view that is positive by correction matrix.
Step 304, according to the algorithm of dotted line characteristic matching, detect characteristic point in first frame image and the second frame image and
Characteristic curve, and complete the matching of dotted line feature.
First frame image and the second frame image can be the image selected in the multiple image that monocular camera is continuously taken pictures,
It can be the truncated picture from the video file that monocular camera images.When carrying out pose measurement using image, it is ensured that selection
2 frame images have public view field, i.e., reference point having the same or reference line；
Fisrt feature point is the point in first frame image, and second feature point is the point in the second frame image, specifically, first
Characteristic point and second feature point are monocular camera respectively in the corresponding point of different location shooting reference point, similarly, fisrt feature line
With the line that second feature line is respectively in first frame image and the second frame image, specifically, fisrt feature line and second feature line
It is monocular camera respectively in the corresponding line of different location shooting reference line.In this real-time step, without pre-setting reference
Point and reference line, can match in first frame image and the second frame image and obtain accurate fisrt feature point and second feature
Point, and matching obtain accurate fisrt feature line and second feature line.
ORB algorithm can detect the point feature in image in real time, and matching a little can be rapidly completed.ORB feature by
Key point and the sub- two parts composition of description.Wherein, key point has the advantages that scale invariability and rotational invariance；Description is adopted
With binary form, point feature can be significantly improved and detect matched speed,
For line feature, we are detected and are matched to straight line using LSD algorithm and matching line segments algorithm.Wherein, LSD
Detection line feature has the advantages that speed is fast, easy to use, and the straight line extracted has principal direction, when carrying out line match
It can exclude some error hidings.According to the point feature of the straight line and straight line neighborhood that detect, the spy of straight line neighborhood can be constructed
Number is levied, and then calculates the similarity of two straight lines.In this way, a similarity matrix, square can be constructed to any 2 frame image
Maximum two straight lines of similarity are final matching line segments result in battle array.
Step 306, it is constrained according to the homography of vision algorithm dotted line feature, obtains the first constraint side based on point feature
Journey and the second constraint equation based on line feature.
Since first frame image and the second frame image are the different perspectives to the shooting of plane dotted line feature, first frame
Image and the second frame image meet homography, can establish the first constraint equation according to fisrt feature point and second feature point.Together
Reason, according to the constraint of fisrt feature line and second feature line, can establish the second constraint equation.
Specifically, enabling the coordinate of fisrt feature point is qi=[xi,yi,wi]T, the coordinate of second feature point is qj=[xj,yj,
wj]T, according to homography, meet following constraint:
λqj=Hqi
Homography matrix H's is defined as:
Wherein,Indicate spin matrix,Indicate that translation vector, d indicate camera to the vertical range of plane, n expression
Normal vector, when camera image plane correction is positive lower apparent direction, only yaw angle is unknown, and enabling yaw angle is θ, then spin matrixIt can indicate are as follows:
It is available after spin matrix expression formula to be substituted into the defined formula of homography matrix:
Wherein,
Therefore, it according to the form of H, can be denoted as:
Peer-to-peer λ qj=HqiThe left multiplication cross q in both endsj, it is available:
qj×Hqi=0
It is available after homography matrix is substituted into:
After being arranged, available first constraint equation are as follows:
In another embodiment, in first frame image, the equation of fisrt feature line is ax+by+1=0, is expressed as square
Formation formula, it may be assumed that
Simplified, available mtxi=0, mt=[a b 1].
Similarly, in the second frame image, the equation of second feature line is cx+dy+1=0, pass through available ntxj=
0, nt=[c d 1].
According to homography, xiAnd xjMeet xj=HxiConstraint.Equation both ends premultiplication is with nt, it can obtain:
λntxj=ntHxi
ntxj=λ-1ntHxi
By ntxj=0 it is found that the left end of above formula is 0, therefore right end is also 0, it may be assumed that
λ-1ntHxi=0
By mtxi=0 can derive:
λ-1ntH=mt。
The two sides of above formula are converted, are obtained:
λ-1HtN=m
HtN=λ m
To eliminate scale factor, the left multiplication cross in the both ends of above formula [m]×, it can obtain:
[m]×HtN=0
It is worth noting that the homography matrix H in this formula is different from the homography matrix H in above formula, meet H=
KHK-1, therefore above formula can further be converted, obtained according to the inequality:
HtKtN=λ Ktm
Wherein, K is enabledtN=n ', KtM=m ', then can be by above formula abbreviation at HtN '=λ m '.And then it can obtain:
[m′]×HtN '=0
Enable n '=[n1 n2 1]T, m '=[m1 m2 1]T, above formula is substituted into, the second constraint side based on linear feature can be obtained
Journey are as follows:
Step 308, according to the first constraint equation and the second constraint equation, corresponding list is answered between image pair after being corrected
Property matrix, homography matrix is decomposed to obtain spin matrix and translation vector.
When calculating homography matrix, the first constraint equation and the second constraint equation can be combined, can be obtained:
The freedom degree of problem to be solved is 3, and the order of above formula equation group is 4, therefore can solve H.Using following formula,
And then spin matrix can be decompositedAnd translation vector
Step 310, according to the relative pose relationship between image pair after correction, before conversion obtains camera coordinates system correction
Monocular camera pose measurement result.
Above-mentionedFor the spin matrix and translation vector between 2 frame cameras after correction, it to be also converted into school
Result before just.
That is R and t is the measurement result of final monocular camera pose.
It should be understood that although each step in the flow chart of Fig. 3 is successively shown according to the instruction of arrow, this
A little steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly state otherwise herein, these steps
It executes there is no the limitation of stringent sequence, these steps can execute in other order.Moreover, at least part in Fig. 3
Step may include that perhaps these sub-steps of multiple stages or stage are executed in synchronization to multiple sub-steps
It completes, but can execute at different times, the execution sequence in these sub-steps or stage, which is also not necessarily, successively to be carried out,
But it can be executed in turn or alternately at least part of the sub-step or stage of other steps or other steps.
In one embodiment, as shown in figure 5, providing the monocular phase of a kind of Inertial Measurement Unit and the combination of dotted line feature
Machine pose measuring apparatus, comprising: module 506 and pose measurement are established in inertia measuring module 502, dotted line extraction module 504, constraint
Module 508, in which:
Inertia measuring module 502, for receiving the measurement result for the Inertial Measurement Unit being connected with monocular camera, according to institute
Measurement result is stated, correction matrix is obtained, first frame image and the second frame image rectification are positive lower visible image；
Dotted line extraction module 504, for detecting characteristic point and characteristic curve in first frame image and the second frame image, and it is right
Dotted line feature between different frame image is matched；
Module 506 is established in constraint, for being constrained according to the homography of vision algorithm dotted line feature, is obtained based on point feature
The first constraint equation and the second constraint equation based on line feature；
Pose measurement module 508 is used for according to first constraint equation and the second constraint equation, image after being corrected
Relative pose relationship between；By further converting, the monocular camera pose measurement knot before obtaining camera coordinates system correction
Fruit.
Specific restriction about Inertial Measurement Unit and the monocular camera pose measuring apparatus of dotted line feature combination can be joined
The restriction of the monocular camera pose measuring method combined in seeing above for Inertial Measurement Unit and dotted line feature, it is no longer superfluous herein
It states.The modules in monocular camera pose measuring apparatus that above-mentioned Inertial Measurement Unit and dotted line feature combine can whole or portion
Divide and is realized by software, hardware and combinations thereof.Above-mentioned each module can be embedded in the form of hardware or independently of computer equipment
In processor in, can also be stored in a software form in the memory in computer equipment, in order to processor calling hold
The corresponding operation of the above modules of row.
In one embodiment, a kind of computer equipment is provided, which can be server, internal junction
Composition can be as shown in Figure 6.The computer equipment include by system bus connect processor, memory, network interface and
Database.Wherein, the processor of the computer equipment is for providing calculating and control ability.The memory packet of the computer equipment
Include non-volatile memory medium, built-in storage.The non-volatile memory medium is stored with operating system, computer program and data
Library.The built-in storage provides environment for the operation of operating system and computer program in non-volatile memory medium.The calculating
The database of machine equipment is for storing measurement data.The network interface of the computer equipment is used to pass through network with external terminal
Connection communication.The monocular camera of Inertial Measurement Unit and the combination of dotted line feature is realized when the computer program is executed by processor
Pose measuring method.
It will be understood by those skilled in the art that structure shown in Fig. 6, only part relevant to application scheme is tied
The block diagram of structure does not constitute the restriction for the computer equipment being applied thereon to application scheme, specific computer equipment
It may include perhaps combining certain components or with different component layouts than more or fewer components as shown in the figure.
In one embodiment, a kind of computer equipment, including memory and processor are provided, which is stored with
Computer program, the processor realize that Inertial Measurement Unit and dotted line feature combine in above-described embodiment when executing computer program
Monocular camera pose measuring method the step of.
In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated
Machine program realizes above-mentioned Inertial Measurement Unit and the monocular camera pose measuring method that dotted line feature combines when being executed by processor
The step of.
It is that can pass through meter those of ordinary skill in the art will appreciate that realizing all or part of the process in the above method
Calculation machine program is completed to instruct relevant hardware, and the computer program, which can be stored in non-volatile computer and can be read, to be deposited
In storage media, the computer program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the application institute
Any reference used in each embodiment provided to memory, storage, database or other media may each comprise non-easy
The property lost and/or volatile memory.Nonvolatile memory may include read-only memory (ROM), programming ROM (PROM), electricity
Programming ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include that arbitrary access is deposited
Reservoir (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, such as static
RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhanced SDRAM
(ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) directly RAM (RDRAM), straight
Connect memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..
Each technical characteristic of above embodiments can be combined arbitrarily, for simplicity of description, not to above-described embodiment
In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance
Shield all should be considered as described in this specification.
The several embodiments of the application above described embodiment only expresses, the description thereof is more specific and detailed, but simultaneously
It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art
It says, without departing from the concept of this application, various modifications and improvements can be made, these belong to the protection of the application
Range.Therefore, the scope of protection shall be subject to the appended claims for the application patent.
Claims (9)
1. the monocular camera pose measuring method that a kind of Inertial Measurement Unit and dotted line feature combine, which comprises
The roll angle and pitch angle information that the Inertial Measurement Unit being connected with monocular camera provides are received, correction matrix is acquired, it will
First frame image and the second frame image rectification are positive lower apparent direction；
Obtaining in first frame image and the second frame image indicates the fisrt feature point and second feature point of same reference point, and obtains
The fisrt feature line and second feature line for indicating same reference line in first frame image and the second frame image are taken, realizes dotted line
The matching of feature；
It is constrained according to the homography of vision algorithm dotted line feature, obtains the of the fisrt feature point and second feature point
One constraint equation and the second constraint equation for obtaining the fisrt feature line and the second feature line；
According to first constraint equation and the second constraint equation, homography matrix is calculated, homography matrix is further decomposed,
Spin matrix and translation vector after being corrected between image pair；
According to the relative pose relationship between image pair after the correction, further conversion obtains the list before camera coordinates system correction
Mesh camera pose measurement result.
2. the method according to claim 1, wherein having demarcated camera coordinates system and Inertial Measurement Unit seat in advance
Spin matrix between mark system；The pitch angle and roll angle exported according to the spin matrix and Inertial Measurement Unit is corresponding
Spin matrix, obtain correction matrix.
3. the method according to claim 1, wherein first frame image and the are obtained and matched according to ORB algorithm
The fisrt feature point and second feature point of same reference point are indicated in two frame images.
4. the method according to claim 1, wherein according to LSD detection algorithm and the similarity matrix constructed
It is detected and is matched and obtain the fisrt feature line for indicating same reference line in first frame image and the second frame image and the
Two characteristic curves.
5. method according to claim 1 to 4, which is characterized in that according to the fisrt feature point with it is described
The matching relationship of second feature point establishes the first constraint equation of the fisrt feature point and second feature point；According to institute
The matching relationship for stating fisrt feature line and the second feature line establishes the of the fisrt feature line and the second feature line
Two constraint equations.
6. according to the described in any item methods of claim 5, which is characterized in that about by first constraint equation and described second
Shu Fangcheng joint, obtains the constraint equation of Joint of Line and Dot feature position-pose measurement；By solving equation, image after being corrected
Relative pose relationship between；By further converting, the monocular camera pose measurement knot before obtaining camera coordinates system correction
Fruit.
7. the monocular camera pose measuring apparatus that a kind of Inertial Measurement Unit and dotted line feature combine, which is characterized in that the dress
It sets and includes:
Inertia measuring module, for receiving the measurement result for the Inertial Measurement Unit being connected with monocular camera, according to the measurement
As a result, obtain correction matrix, first frame image and the second frame image rectification are positive lower visible image；
Dotted line matching module, for obtaining characteristic point and characteristic curve in image, and between the dotted line feature different frame image into
Row matching；
Module is established in constraint, for constraining according to the homography of vision algorithm dotted line feature, obtains first based on point feature
Constraint equation and the second constraint equation based on line feature；
Pose measurement module is used for according to first constraint equation and the second constraint equation, after being corrected between image pair
Relative pose relationship, and then obtain camera coordinates system correction before monocular camera pose measurement result.
8. a kind of computer equipment, including memory and processor, the memory are stored with computer program, feature exists
In the step of processor realizes any one of claims 1 to 6 the method when executing the computer program.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program quilt
The step of processor realizes method described in any one of claims 1 to 6 when executing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910659311.1A CN110375732A (en) | 2019-07-22 | 2019-07-22 | Monocular camera pose measurement method based on inertial measurement unit and point line characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910659311.1A CN110375732A (en) | 2019-07-22 | 2019-07-22 | Monocular camera pose measurement method based on inertial measurement unit and point line characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110375732A true CN110375732A (en) | 2019-10-25 |
Family
ID=68254517
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910659311.1A Pending CN110375732A (en) | 2019-07-22 | 2019-07-22 | Monocular camera pose measurement method based on inertial measurement unit and point line characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110375732A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111504312A (en) * | 2020-07-02 | 2020-08-07 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle pose estimation method based on visual inertial polarized light fusion |
| CN111791235A (en) * | 2020-07-17 | 2020-10-20 | 浙江大学 | Robot multi-camera visual inertia point-line characteristic positioning method and device |
| CN112639883A (en) * | 2020-03-17 | 2021-04-09 | 华为技术有限公司 | Relative attitude calibration method and related device |
| CN113496503A (en) * | 2020-03-18 | 2021-10-12 | 广州极飞科技股份有限公司 | Point cloud data generation and real-time display method, device, equipment and medium |
| CN113538477A (en) * | 2020-04-14 | 2021-10-22 | 北京达佳互联信息技术有限公司 | Plane pose acquisition method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060146136A1 (en) * | 2004-12-21 | 2006-07-06 | Seong-Ik Cho | Apparatus for correcting position and attitude information of camera and method thereof |
| CN106909877A (en) * | 2016-12-13 | 2017-06-30 | 浙江大学 | A kind of vision based on dotted line comprehensive characteristics builds figure and localization method simultaneously |
| CN107871327A (en) * | 2017-10-23 | 2018-04-03 | 武汉大学 | The monocular camera pose estimation of feature based dotted line and optimization method and system |
| CN109493385A (en) * | 2018-10-08 | 2019-03-19 | 上海大学 | Autonomic positioning method in a kind of mobile robot room of combination scene point line feature |
| CN109752003A (en) * | 2018-12-26 | 2019-05-14 | 浙江大学 | A kind of robot vision inertia dotted line characteristic positioning method and device |
-
2019
- 2019-07-22 CN CN201910659311.1A patent/CN110375732A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060146136A1 (en) * | 2004-12-21 | 2006-07-06 | Seong-Ik Cho | Apparatus for correcting position and attitude information of camera and method thereof |
| CN106909877A (en) * | 2016-12-13 | 2017-06-30 | 浙江大学 | A kind of vision based on dotted line comprehensive characteristics builds figure and localization method simultaneously |
| CN107871327A (en) * | 2017-10-23 | 2018-04-03 | 武汉大学 | The monocular camera pose estimation of feature based dotted line and optimization method and system |
| CN109493385A (en) * | 2018-10-08 | 2019-03-19 | 上海大学 | Autonomic positioning method in a kind of mobile robot room of combination scene point line feature |
| CN109752003A (en) * | 2018-12-26 | 2019-05-14 | 浙江大学 | A kind of robot vision inertia dotted line characteristic positioning method and device |
Non-Patent Citations (1)
| Title |
|---|
| OLIVIER SAURER ET AL.: "Homography Based Egomotion Estimation with a Common Direction", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112639883A (en) * | 2020-03-17 | 2021-04-09 | 华为技术有限公司 | Relative attitude calibration method and related device |
| CN112639883B (en) * | 2020-03-17 | 2021-11-19 | 华为技术有限公司 | Relative attitude calibration method and related device |
| CN113496503A (en) * | 2020-03-18 | 2021-10-12 | 广州极飞科技股份有限公司 | Point cloud data generation and real-time display method, device, equipment and medium |
| CN113496503B (en) * | 2020-03-18 | 2022-11-08 | 广州极飞科技股份有限公司 | Point cloud data generation and real-time display method, device, equipment and medium |
| CN113538477A (en) * | 2020-04-14 | 2021-10-22 | 北京达佳互联信息技术有限公司 | Plane pose acquisition method and device, electronic equipment and storage medium |
| CN113538477B (en) * | 2020-04-14 | 2023-08-29 | 北京达佳互联信息技术有限公司 | Method and device for acquiring plane pose, electronic equipment and storage medium |
| CN111504312A (en) * | 2020-07-02 | 2020-08-07 | 中国人民解放军国防科技大学 | Unmanned aerial vehicle pose estimation method based on visual inertial polarized light fusion |
| CN111791235A (en) * | 2020-07-17 | 2020-10-20 | 浙江大学 | Robot multi-camera visual inertia point-line characteristic positioning method and device |
| CN111791235B (en) * | 2020-07-17 | 2021-07-02 | 浙江大学 | Robot multi-camera visual inertia point-line characteristic positioning method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110375732A (en) | Monocular camera pose measurement method based on inertial measurement unit and point line characteristics | |
| CN107747941B (en) | Binocular vision positioning method, device and system | |
| CN107564061B (en) | Binocular vision mileage calculation method based on image gradient joint optimization | |
| EP3028252B1 (en) | Rolling sequential bundle adjustment | |
| CN108534782B (en) | Binocular vision system-based landmark map vehicle instant positioning method | |
| Kazik et al. | Real-time 6d stereo visual odometry with non-overlapping fields of view | |
| Lébraly et al. | Fast calibration of embedded non-overlapping cameras | |
| CN107193279A (en) | Robot localization and map structuring system based on monocular vision and IMU information | |
| CN107560603B (en) | Unmanned aerial vehicle oblique photography measurement system and measurement method | |
| CN110084832A (en) | Correcting method, device, system, equipment and the storage medium of camera pose | |
| WO2011118282A1 (en) | Server using world coordinate system database and terminal | |
| CN111754579B (en) | Method and device for determining external parameters of multi-view camera | |
| US20090141043A1 (en) | Image mosaicing apparatus for mitigating curling effect | |
| Frahm et al. | Pose estimation for multi-camera systems | |
| Honegger et al. | Embedded real-time multi-baseline stereo | |
| CN111127524A (en) | Method, system and device for tracking trajectory and reconstructing three-dimensional image | |
| US20160105590A1 (en) | Method and device for determining movement between successive video images | |
| WO2023005457A1 (en) | Pose calculation method and apparatus, electronic device, and readable storage medium | |
| US20230298344A1 (en) | Method and device for determining an environment map by a server using motion and orientation data | |
| AliAkbarpour et al. | Parallax-tolerant aerial image georegistration and efficient camera pose refinement—without piecewise homographies | |
| CN105324792A (en) | Method for estimating the angular deviation of a mobile element relative to a reference direction | |
| CN107749069A (en) | Image processing method, electronic equipment and image processing system | |
| Zhao et al. | RTSfM: Real-time structure from motion for mosaicing and DSM mapping of sequential aerial images with low overlap | |
| CN111383282B (en) | Pose information determining method and device | |
| CN116007609A (en) | Positioning method and computing system for fusion of multispectral image and inertial navigation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191025 |