CN108665499A - A kind of low coverage aircraft pose measuring method based on parallax method - Google Patents
A kind of low coverage aircraft pose measuring method based on parallax method Download PDFInfo
- Publication number
- CN108665499A CN108665499A CN201810420236.9A CN201810420236A CN108665499A CN 108665499 A CN108665499 A CN 108665499A CN 201810420236 A CN201810420236 A CN 201810420236A CN 108665499 A CN108665499 A CN 108665499A
- Authority
- CN
- China
- Prior art keywords
- aircraft
- plane
- dimensional
- dimensional image
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/77—Determining position or orientation of objects or cameras using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of, and the low coverage aircraft pose measuring method based on parallax method can be used for the pose resolving of refueled aircraft during air refuelling for aircraft position positioning under the conditions of the low coverage of high-altitude and posture solution, including：The inside and outside calibration joined of parallel two cameras and image rectification；After obtaining aircraft initial alignment region by detecting and tracking algorithm, Stereo matching is carried out, solves three-dimensional point cloud；Several three-dimensional feature point sets are formed, the plane equation of three-dimensional point set is solved in conjunction with three-dimensional point cloud by the methods of edge feature class indication two dimensional character point.After wiping out background point and noise spot, the shape feature of plane equation combination aircraft solves the spatial attitude of aircraft；This method is using characteristics of image in such a way that shape characteristic is combined, it can effectively realize low coverage aircraft pose measurement, reduce the time of algorithm for stereo matching consuming, and without increasing cooperation mark aboard, have the characteristics that fast and effective, realization is convenient, highly practical, algorithm is convenient for transplanting, is easily used in actual items.
Description
Technical field
This method is related to aircraft pose measuring method, and in particular to one kind being based on parallax method, and two dimensional image and three-dimensional point
The aircraft pose measuring method that cloud is combined.
Background technology
Parallax method is one of more common algorithm of stereoscopic vision.Parallax method is simulation human vision principle in fact, uses meter
The method of the passive perceived distance of calculation machine.An object from two or more points, obtains the image under different visual angles, root
According to the matching relationship of pixel between image, the offset between pixel is calculated by principle of triangulation to obtain the three-dimensional of object
Information.
During the respective pixel of parallax method is matched, since disparity search range is not easy to determine, matching is made easily to occur
Mistake.Inappropriate disparity search range is easy that Stereo Matching Algorithm is made to be absorbed in local minimum.Preset disparity search
Range is excessive or too small may all lead to matched mistake in parallax method.By taking maximum disparity as an example, if given maximum disparity
It is excessive, then it may increase error hiding.And consume more calculating times and memory headroom；If too small, then can not calculate correct
Parallax.Simultaneously in many applications, the disparity range of scene can not be known in advance, need artificial calibration and estimate.But
This method can not be suitable for actual stereo visual system.
After obtaining three-dimensional point cloud by disparity map, the real-time pose of aircraft is realized without quickly and effectively point cloud matching method
Clearing.Though the ICP (iteration closest approach method) that Besl and Mckay is proposed constantly is improved and is mended by development for many years
Fill, be greatly improved, and solve one of the registration problems main means based on free form curved surface, but there is still a need for compared with
Big calculation amount and multiple iterative process on calculating the time and do not have advantage.
Invention content
The technology of the present invention solves the problems, such as：Overcome the deficiencies of the prior art and provide a kind of low coverage aircraft position based on parallax method
Gesture measuring method can realize aircraft position that is more accurate and can guarantee real-time in the case of actual complex background environment
Appearance measures.
In order to achieve the above objectives, the technical solution of this method is realized in：
1, the low coverage aircraft pose measuring method based on parallax method, this method include：
A, it to two cameras being placed in parallel, defines using left camera photocentre as the camera coordinates system of coordinate axis origin.Pass through
Calibration obtains the external parameter (i.e. spin matrix and translation vector) of the inner parameter and two cameras of two cameras.Using inside and outside
The two dimensional image that two cameras of external parameter pair between parameter and camera take is corrected.
B, the upper thousand sheets (quantity can be adjusted according to actual demand, and the present invention is determined as three thousand sheets through overtesting) of shooting in advance is real
Border uses and the two dimensional image containing aircraft, marks out the two dimensional image for containing only aircraft region as positive sample collection, marks simultaneously
Go out the two dimensional image in non-aircraft region as negative sample collection.Plane prevention device is trained by positive and negative sample set.Use aircraft point
Class device detects aircraft region in the two dimensional image after the correction of two cameras respectively.The aircraft that will be detected in two dimensional image
Region substitutes into track algorithm, is obtained respectively in conjunction with detection algorithm and track algorithm into line trace to aircraft region in two dimensional image
The aircraft region obtained and algorithm respective weights obtain the aircraft initial alignment region in two magazine two dimensional images；
C, aircraft initial alignment region in two magazine two dimensional images is subjected to Stereo matching, obtains disparity map, it is right
Disparity map is solved to obtain the three-dimensional point cloud in aircraft initial alignment region in the two dimensional image of left camera；
D, aircraft initial alignment region carries out a screening in the two dimensional image of left camera, in conjunction with the three-dimensional point in step c
Cloud, obtain aeroplane nose and empennage institute in the planes the three-dimensional point set of key point with aircraft wing plane in key point three-dimensional
Point set solves the plane equation of each three-dimensional point set using RANSAC, plane and aircraft machine where obtaining aeroplane nose and empennage
The plane equation of wing plane；
E, according to the plane equation of plane and aircraft wing plane where the aeroplane nose and empennage acquired in step d, knot
Aircraft shape and three-dimensional data wiping out background point and noise spot are closed, obtains the three-dimensional point set near each plane, i.e., each plane
Interior point set, further iteration go out new plane equation, determine aircraft local coordinate system using two new plane equations, ask
Solve aircraft space pose.
2, to the reality of the calibration of inside and outside ginseng and the binocular image correction of the stereo visual system of two cameras composition in step a
It is existing that steps are as follows：
(1) target picture is shot, the angle point in target picture is extracted, calibrates the inner parameter of two cameras；
(2) according to the inner parameter and target picture of two cameras, external parameter (the i.e. spin moment of two cameras is calibrated
Battle array and translation vector)；
(3) it is carried out according to the two dimensional image that two cameras of external parameter pair of camera internal parameter and two cameras take
Correction.
3, tracking is detected to aircraft in step b, to obtain the aircraft initial alignment region of aircraft on 2d
Realization steps are as follows：
(1) in the two dimensional image shot in advance, the two dimensional image that extraction contains only aircraft region forms positive sample collection；It is free of
The two dimensional image of aircraft region is as negative sample collection.Aircraft rectangular characteristic is obtained by positive and negative sample set, trains plane prevention
Device；
(2) it uses plane prevention device in initial frame framing aircraft region, and is calculated using aircraft region initialization tracking
Method；
(3) in follow-up two-dimensional image sequence, it is separately operable detection algorithm and track algorithm, is flown what the two algorithms obtained
Machine region and algorithm weights carry out tactic fusion, obtain the aircraft initial alignment area in the two dimensional image that two cameras are shot
Domain.
4, Stereo matching is carried out to aircraft initial alignment region in the two dimensional image of two cameras shooting in step c, obtained
Disparity map, and solve the realization of three-dimensional point cloud steps are as follows：
(1) each pixel is all in aircraft initial alignment region in the two dimensional image that two cameras are shot in calculating step b
The matching degree for enclosing region calculates separately in the two dimensional image of two cameras shooting in aircraft initial alignment region around respective pixel
Matching degree between region obtains initial parallax figure；
(2) initial parallax figure is filtered, screens out Mismatching point, obtain disparity map；
(3) two dimension of left camera can be calculated in conjunction with disparity map by being based on stereoscopic model and camera pinhole imaging system principle
The three-dimensional coordinate of each X-Y scheme picture point in image in aircraft initial alignment region forms the three-dimensional in aircraft initial alignment region
Point cloud.
5, the realization of the plane equation of plane and aircraft wing plane where solving aeroplane nose and empennage is walked in step d
It is rapid as follows：
(1) in two dimensional image in aircraft initial alignment region, in conjunction with profile and framework information to aircraft in the picture
Two-dimensional points carry out screening extraction, and the key feature points extracted are carried out classification marker.By these key feature points according to point
Cloth position, the key feature point set b being divided on key feature point set a and the aircraft wing on aeroplane nose and empennage；
(2) according to the one-to-one relationship of point in the point and disparity map of two dimensional image, this is filtered out from three-dimensional point cloud
The corresponding three-dimensional points of a little key feature points, form the three-dimensional on three-dimensional point set A and aircraft wing on aeroplane nose and empennage
Point set B；
(3) equation of plane, that is, solve aircraft where using RANSAC algorithms to solve two above three-dimensional point set respectively
Plane where head and empennage and the two planes of aircraft wing plane place plane equation.
6, in step e, by the three-dimensional point cloud obtained in step c according to the plane equation solved in step d to noise
Point and background dot are filtered out, and the plane side of the two planes of plane and aircraft wing plane where aeroplane nose and empennage is passed through
Journey can solve straight line where airframe.In conjunction with aircraft realistic model dimension scale and shape feature, solves aircraft and exist
Spatial pose under camera system coordinate system.By the mutual conversion of coordinate system, aircraft can be solved under earth coordinates
Spatial pose.
(1) using the plane equation solved in step d to the aircraft initial alignment in the two dimensional image that is solved in step c
The three-dimensional point cloud in region carries out three-dimensional point screening.It, can be with wiping out background point and noise spot by the distance of point to plane.According to winged
Plane and aircraft wing plane can determine straight line where airframe where chain-drive section and empennage.
(2) according to aircraft realistic model dimension scale and shape feature, aircraft is solved under camera system coordinate system
Spatial pose.
(3) by the Conversion Relations between coordinate system, spatial pose of the aircraft under earth coordinates is solved.
The advantages of the present invention over the prior art are that：The present invention only needs after demarcating two cameras, you can
The three-dimensional pose that aircraft can be completed using two cameras is measured.Realization process does not need cooperation aircraft, and calculating process is easily realized,
Without complicated set-up procedure and harsh realization environment, versatile, calculating speed is fast.Therefore the present invention is multiple suitable for high-altitude
In the case of miscellaneous background and closer distance, the measurement of aircraft three dimensions pose, such as during air refuelling, refueled aircraft is three-dimensional
The measurement of spatial pose.
Description of the drawings
Fig. 1 is that the present invention is based on the low coverage aircraft pose measuring method flow charts of parallax method；
Fig. 2 is parallax method principle schematic；
Fig. 3 is aircraft profile and skeleton schematic diagram；
Fig. 4 is aircraft local coordinate system and attitude angle；
Fig. 5 relation schematic diagrams between coordinate system in attitude measurement.
Specific implementation mode
The basic thought of the present invention：The prime area of aircraft is positioned in two cameras using detecting and tracking method.In conjunction with
The first localization region of two cameras using parallax method obtain aircraft three-dimensional point cloud, in conjunction with image characteristic point, three-dimensional point cloud information and
Aircraft pattern feature solves position and the posture of aircraft.
This method is described further with reference to real process.
As shown in Figure 1, low coverage aircraft pose measuring method of this method based on parallax method mainly includes the following steps that：
Step 11：Inside and outside ginseng calibration and image rectification are carried out to two camera systems.
Here, each camera of biocular systems is demarcated first, that is, solves the inner parameter of camera, two cameras
The calibration of vision system solves the external parameter between two cameras, article " A of the specific method for solving in Zhang Zhengyou
flexible new technique for camera calibration[R].Microsoft Corporation,NSR-
It is had a detailed description in TR-98-71,1998 ".
After the inside and outside parameter for obtaining two camera systems, the image that left and right camera obtains is corrected, it is corrected
Because caused by camera distortion situations such as anamorphose.
Step 12：It is positioned and is flown in the image that two cameras obtain by detection algorithm and track algorithm detecting and tracking algorithm
The prime area of machine.This region is the input content subsequently calculated, zoning is reduced for subsequent algorithm, so as to boosting algorithm
Calculating speed.
Here, it is necessary first to obtain the two dimensional image conduct that three thousand sheets (quantity can be according to practical adjustment) contain only aircraft region
The two dimensional image of positive sample and six thousand sheets (quantity can be according to practical adjustment) without aircraft region is as negative sample.By dividing
The rectangular characteristic in positive negative sample is analysed, the characteristic information that can distinguish aircraft is filtered out.By cascade mode by these features
Information combines, and cascade rectangular characteristic is enable accurately to identify aircraft region in the picture.
By aircraft region substitution nuclear phase pass filter tracking algorithm, to aircraft area in the two dimensional image that camera takes in left and right
Domain is into line trace, the aircraft region obtained respectively in conjunction with detection algorithm and track algorithm and algorithm respective weights, obtains at two
Aircraft initial alignment region in magazine two dimensional image.
Step 13：Oriented aircraft region in the image that left and right camera obtains, is carried out by SGBM Stereo Matching Algorithms
Stereo matching obtains disparity map and solves the three of aircraft initial alignment region in conjunction with the system inside and outside parameter that step 11 calibrates
Dimension point cloud.
Here, since image carries out overcorrect, by limit restraint, therefore matching is constrained in corresponding row and carries out, and subtracts
The possibility of erroneous matching is lacked.By each pixel and the restriction relation of the value of surrounding pixel, calculate in another piece image
The location of pixels of the value of middle corresponding row and its Least-cost, according to corresponding pixel to calculating the parallax of the point.Traversal is asked
Solution whole region is to solve the initial local disparity map of the area image.This initial local disparity map is filtered, mistake is screened out
Match point obtains disparity map；
Stereo Matching Algorithm can be mainly divided into following four step：
(1) it pre-processes：
Using horizontal Sobel operators, following processing is done to each point in aircraft initial alignment region in two dimensional image：
Sobel (x, y)=2 [P (x+1, y)-P (x-1, y)]+P (x+1, y-1)-P (x-1, y-1)+P (x+1, y+1)-P
(x-1,y+1) (1)
Wherein, Sobel (x, y) represents the horizontal Sobel operators of the point.P (x, y) represent currently calculate pixel (x,
Y) pixel value.
Each pixel of above-mentioned processing region is formed one new two by the calculating of mapping function with a function
Tie up image-region.
Mapping function is as follows：
Wherein, preFilterCap is a constant parameter, generally takes 63.P is aircraft initial alignment area in two dimensional image
The current pixel value for calculating point in domain, PNPixel corresponding with P on two dimensional image region after expression is calculated by mapping function
Value.
Pretreatment is actually the gradient information in aircraft initial alignment region in two dimensional image in order to obtain.Through the above pre- place
The new two dimensional image region formed behind the aircraft initial alignment region of reason saves, and is calculated for cost.
(2) cost calculates：
The cost of each pixel is calculated using absolute error and algorithm.
(3) global optimization：
Global optimization is done by the way of Dynamic Programming, that is, solves minimal cost path.To each of region point p,
Around p, 8 paths are provided with for interval with 45 °, pass through 8 path computing minimal cost path Lr(p,d)。
Wherein, P1, P2For Dynamic Programming parameter, it is adjusted according to practical service environment；Lr(p, d) is indicated along current
Direction (i.e. from left to right), minimal cost path when the current parallax value for calculating pixel p is d.Indicate edge
It and works as front direction (i.e. from left to right), minimum of the current parallax value for calculating pixel p less than minimal cost path when being k
Value.
(4) it post-processes：
In aircraft initial alignment region in the two dimensional image that left camera obtains after the completion of each pixel matching, the right side is utilized
Each pixel goes to match the two dimensional image that left camera obtains in aircraft initial alignment region in the two dimensional image that camera obtains
In aircraft initial alignment region in pixel.If pixel is differed by the parallax matched twice, this
Pixel is considered as invalid matching.
By the above process, the disparity map in the aircraft initial alignment region in two dimensional image can be calculated.
According to stereoscopic model and camera pinhole imaging system principle, the visible parallax method principle schematic of details, i.e., such as Fig. 2 institutes
Show：P is the point in space, P in Fig. 2lAnd PrIt is imaging points of the point P in the camera image plane of left and right, f is focal length, OlAnd OrIt is left
The optical center of right camera.OlAnd OrThe distance between be binocular spacing, P to OlAnd OrThe distance of line is actual range.Left and right two
The optical axis of camera is parallel.xrAnd xlIt is the distance of two imaging points range image left hand edge in the two camera image planes in left and right.
If two cameras have corrected that completion reaches that polar curve is parallel, and two optical axis directions are also parallel.Then parallax and object
The relational expression of depth is as follows：
Wherein, xrAnd xlIt is the distance of two imaging points range image left hand edge in the two camera image planes in left and right, b is a left side
Right camera photocentre OlAnd OrThe distance of line.Z is actual range of the P points to camera.
It can derive：
Wherein, xrAnd xlThe distance for the range image left hand edge that is two imaging points in the two camera image planes in left and right, d be at
Parallax between picture point, i.e. xl-xr, b is left and right camera photocentre OlAnd OrThe distance of line, f are the focal lengths of camera
It can get in conjunction with the disparity map in the aircraft initial alignment region solved in two dimensional image by above formula
The three-dimensional point cloud in aircraft initial alignment region in two dimensional image.
Step 14：Just localization region carries out two feature point set screenings to two dimensional image, in conjunction with three dimensional point cloud, acquisition pair
Answer two three-dimensional point sets.
According to the whole pattern of aircraft, the object of two planes composition can be reduced to, be aeroplane nose respectively with
The plane and aircraft wing plane of empennage composition.Therefore the posture of survey aircraft can pass through plane where aeroplane nose and empennage
It is obtained with the equation solution of aircraft wing plane the two planes.
In the initial alignment region of two dimensional image, region internal edge feature is obtained using Boundary extracting algorithm.It utilizes
Edge feature combination three-dimensional point cloud, obtain aircraft profile information (be aircraft schematic diagram see Fig. 3, Fig. 3, two hatched planars in figure
The respectively plane and aircraft wing plane of aeroplane nose and empennage composition, black border are aircraft profile information, intermediate line
The point marked emphatically for airframe information, and on framework information is the aircraft key feature points extracted), it forms aircraft and cuts
Shadow zone domain.Using skeletal extraction algorithm, in aircraft outline extracted region skeleton, and according to skeleton property, with binary tree structure
Topology rebuilding is carried out to skeleton.The skeleton branches method for reducing for adapting to Aircraft Targets framework characteristic is established, target main framing is completed
Extraction (as shown in Figure 3).
The shape feature of the main framing combination aircraft of aircraft, you can positioning aeroplane nose, empennage, the position of both wings in the picture
It sets.Aeroplane nose can be extracted in the picture in conjunction with edge feature has been extracted, and the key point on empennage and two wings is fixed
It is key feature points to justice, is marked and is classified.Feature point group on head and empennage is at key feature point set A, two
Feature point group on wing is at key feature point set B.
It, can be every in above-mentioned key feature point set from being extracted in three-dimensional point cloud according to the relationship of two dimensional image and disparity map
The corresponding three-dimensional point of a two-dimensional points, forms corresponding two three-dimensional feature point sets.
Step 15：Use the plane equation of each three-dimensional point set obtained in RANSAC methods solution procedure 14.
The three-dimensional point set A that RANSAC methods are extracted at step 14 respectively and iteration fixed number of times in B are used in the present invention,
Ideal model (i.e. plane equation) is obtained, as the plane equation where point set, that is, acquires aeroplane nose and empennage group
At plane and aircraft wing plane.
RANSAC is the abbreviation of " RANdom SAmple Consensus (random sampling is consistent) ".It can be from one group of packet
In observation data set containing " point not in the know ", the parameter of mathematical model is estimated by iterative manner.
The input of RANSAC algorithms is one group of observation data (three-dimensional point set A and B that step 14 solves), by selecting repeatedly
One in data group of random subset is selected to reach aircraft.The subset being selected is assumed to be intra-office point, and following methods is used in combination to carry out
Verification：
(1) an areal model is solved using intra-office point, i.e., all areal model parameters can fall into a trap from intra-office point
It obtains.
(2) data except the intra-office point in test observation data are gone with the areal model obtained in (1), that is, are calculated and removed office
Distance of the three-dimensional point apart from plane equation except interior point, if calculated distance is less than the threshold value δ of setting, then it is assumed that it
It is intra-office point.
(3) if thering is 50% point to be classified as in the observation data (three-dimensional point set A and B that are solved in step 14) of input
Intra-office point, it is considered that calculated areal model is just reasonable enough.
(4) then, it goes to remove Calculation Plane model again with all intra-office points of acquisition.
(5) finally, areal model is assessed by estimating the error rate of intra-office point and model.
Above procedure is repeatedly executed fixed number, otherwise the areal model generated every time because intra-office point is very little and by
Give up or is selected because of more preferable than existing areal model.It is final to solve aeroplane nose and empennage composition respectively
Plane and aircraft wing plane the two plane equations.
Step 16：By plane equation wiping out background and noise spot in three-dimensional point cloud, solved in conjunction with aircraft pattern feature
Aircraft space pose.
The 3 d pose of aircraft refers to attitude angle of the aircraft in the air relative to ground survey coordinate system in fact.Therefore it needs
The local coordinate system (coordinate system for being fixed on aircraft itself) from aircraft is carried out to camera coordinates system, then arrives ground survey coordinate system
Three conversion processes.And camera coordinates system can pass through the side such as photoelectricity longitude and latitude relative to the coordinate and angle of ground survey coordinate system
Formula, which solves, to be come.So as long as posture (the i.e. local coordinate of aircraft of the aircraft in camera coordinates system in sequential images is determined
System), the absolute pose of aircraft, which can solve, to be come.
The side of the two planes of the aeroplane nose obtained using step 15 with the plane and aircraft wing plane of empennage composition
Journey in the three-dimensional point cloud obtained in step 13, obtains the point near the two planes, i.e. point set in plane respectively.Simultaneously
Mutually orthogonal relationship is answered in conjunction with the two planes, filters out noise spot, further iteration obtains more accurate plane equation.Then
By two plane equations and aircraft true form size and three dimensional point cloud newly solved, the intersection of two planes is
The straight line where airframe.Airframe center can be positioned according to the position of aeroplane nose and aircraft actual size,
As aircraft local coordinate system origin, while it is X-axis to define both wings direction (direction of the line i.e. from left wing to right flank).
The direction (direction of the line i.e. from tail to head, in other words the longitudinal axis of fuselage) of plane nose meaning is Y-axis.Aircraft machine
Normal, that is, vertical direction of body plane (plane sat in other words where the floor of cabin) is Z axis.Therefore pass through origin and other two
Plane equation is that can determine the local coordinate system of aircraft, and then solve posture of the aircraft under camera coordinates system.
As shown in figure 5, establishing camera coordinates system OcxcyczcAnd aircraft local coordinate system Opxpypzp。
Camera coordinates system OcxcyczcThe optical center point of the left camera in stereo visual system is established, being horizontally directed to right camera is
OcxcAxis is O perpendicular to groundcycAxis is horizontally directed to immediately ahead of left camera be OczcAxis.
Wherein, aeroplane nose and the equation and aircraft wing plane equation simultaneous of empennage place plane：.
Wherein, A1, B1, C1, D1, A2, B2, C2, D2The two plane equation parameters solved for step 15.Pass through above-mentioned side
Straight line where Cheng Kezhi fuselages, the central point of aircraft also should be on this straight line.
The aircraft main framing information combination aircraft actual size obtained in space line equation (formula 6), step 14 solves
The aircraft central point (barycenter) gone out.Therefore, aircraft local coordinate system O is definedpxpypzp, the central point (barycenter) of aircraft is as winged
Machine local coordinate system origin Op, the direction (being directed toward right flank) for defining wing is OpxpThe direction of axis, head is OpypAxis, perpendicular to winged
Machine is OpzpAxis, this coordinate system are changed as aspect changes, and are required the posture of aircraft at this time, are just equivalent to
Find out this coordinate system.It is overlapped with camera coordinates system assuming that aircraft original state is aircraft local coordinate system, at this time initial attitude
It is 0 degree.
And in inertia system, mainly use pitch, yaw, roll these three Eulerian angles to describe aspect.pitch
It is rotated around X-axis, also referred to as pitching angle theta.Yaw is rotated around Z axis, and yaw angle is alsoRoll is rotated around Y-axis, is also cried
Roll angle γ.With Eulerian angles come describe the rotation of object not only need it is angled, it is also necessary to have rotational order, general rotational order
It is first yaw pitch again, then roll.Therefore it is exactly first to rotate about the z axis to be reacted in the reference axis of definition, is rotated further around X-axis, finally
It is rotated around Y-axis.
What the transformation of coordinate system was generally stated by direction cosine matrix, it is expressed as：
Wherein,It is camera coordinates system OcxcyczcTo aircraft local coordinate system OpxpypzpTransfer matrix.rpFor aircraft office
Three-dimensional coordinate under portion's coordinate system, rcFor the three-dimensional coordinate under camera coordinates system.
It is defined by Eulerian angles and cosine matrix defines it is found that direction cosine matrix can be generalized to the single rotation of three axis.
From camera coordinates system OcxcyczcRotate to intermediate conversion coordinate system O1x1y1z1(by camera coordinates system coordinate system around OczcAxis rotates
The conversion coordinate system obtained afterwards), it is rotated again to intermediate conversion coordinate system O2x2y2z2(intermediate conversion coordinate system one is sat around camera
Mark system OcxcThe conversion coordinate system obtained after axis rotation), finally rotate to aircraft local coordinate system Opxpypzp, rotated according to single
Sequence can obtain：
In conclusion can obtain：
Wherein,It is camera coordinates system OcxcyczcTo one O of intermediate conversion coordinate system1x1y1z1(by camera coordinates system coordinate
System is around OczcObtained conversion coordinate system after axis rotation) spin matrix,It is one O of intermediate conversion coordinate system1x1y1z1(by camera
Coordinate system is around OczcThe conversion coordinate system obtained after axis rotation) arrive two O of intermediate conversion coordinate system2x2y2z2(by intermediate conversion coordinate
It is one around OcxcObtained conversion coordinate system after axis rotation) spin matrix,It is two O of intermediate conversion coordinate system2x2y2z2It (will
Intermediate conversion coordinate system one is around camera coordinates system OcxcThe conversion coordinate system obtained after axis rotation) arrive aircraft local coordinate system
OpxpypzpSpin matrix.The angle that θ, γ turn over for the aforementioned correspondence of rotation three times, θ,Straight line where fuselage can be passed through
(formula 6) is solved with camera coordinates system and is obtained.γ can by the plane that straight line where fuselage is formed with camera coordinates system Z axis with
The angle of aircraft wing plane, which solves, to be obtained.
Above example is provided just for the sake of the description purpose of the present invention, and is not intended to limit the scope of the present invention.This
The range of invention is defined by the following claims.It does not depart from spirit and principles of the present invention and the various equivalent replacements made and repaiies
Change, should all cover within the scope of the present invention.
Claims (6)
1. a kind of low coverage aircraft pose measuring method based on parallax method, which is characterized in that include the following steps：
A, it to two cameras being placed in parallel, defines using any one camera photocentre in two cameras as the camera of coordinate axis origin
Coordinate system, by calibration obtain two cameras inner parameter and two cameras external parameter, i.e., spin matrix be translated towards
Amount, the two dimensional image taken using two cameras of external parameter pair between inside and outside parameter and camera are corrected；
B, the actual use of the upper thousand sheets of shooting in advance and the two dimensional image containing aircraft, mark out the X-Y scheme for containing only aircraft region
As being used as positive sample collection, while the two dimensional image in non-aircraft region is marked out as negative sample collection.It is trained by positive and negative sample set
Go out plane prevention device, using detecting aircraft region in plane prevention the device respectively two dimensional image after the correction of two cameras；
The aircraft region detected in two dimensional image is substituted into track algorithm, to aircraft region into line trace in two dimensional image, in conjunction with
The aircraft region and algorithm respective weights that detection algorithm and track algorithm obtain respectively are obtained in two magazine two dimensional images
Middle aircraft initial alignment region；
C, aircraft initial alignment region in two magazine two dimensional images is subjected to Stereo matching, disparity map is obtained, to parallax
Figure is solved to obtain the three-dimensional point cloud in aircraft initial alignment region in the two dimensional image for the camera that optical center is defined as origin；
D, it is defined as aircraft initial alignment region in the two dimensional image of the camera of origin in optical center and carries out a screening, in conjunction with step c
In three-dimensional point cloud, obtain aeroplane nose and empennage in the planes in the three-dimensional point set of key feature points and aircraft wing plane
The three-dimensional point set of key feature points solves the plane equation of each three-dimensional point set using RANSAC, obtains aeroplane nose and empennage
The plane equation of place plane and aircraft wing plane；
E, according to the plane equation of plane and aircraft wing plane where the aeroplane nose and empennage acquired in step d, in conjunction with winged
Machine shape and three-dimensional data wiping out background point and noise spot, obtain the three-dimensional point set near each plane, i.e., point in each plane
Point set, further iteration go out new plane equation, and aircraft local coordinate system is determined using two new plane equations, solve
Go out aircraft space pose.
2. the low coverage aircraft pose measuring method according to claim 1 based on parallax method, it is characterised in that：The step
In a, calibration obtain two cameras inner parameter and two cameras external parameter the step of it is as follows：
(1) target picture is shot, the angle point in target picture is extracted, calibrates the inner parameter of two cameras；
(2) according to the inner parameter and target picture of two cameras, calibrate the external parameter of two cameras, i.e., spin matrix and
Translation vector；
(3) two dimensional image taken according to two cameras of external parameter pair of camera internal parameter and two cameras carries out school
Just.
3. the low coverage aircraft pose measuring method according to claim 1 based on parallax method, it is characterised in that：In step b,
Steps are as follows for the realization in acquisition aircraft initial alignment region in two magazine two dimensional images：
(1) in the two dimensional image shot in advance, the two dimensional image that extraction contains only aircraft region forms positive sample collection；Without aircraft
The two dimensional image in region obtains aircraft rectangular characteristic as negative sample collection, by positive and negative sample set, trains plane prevention device；
(2) it uses plane prevention device in initial frame framing aircraft region, and track algorithm is initialized using aircraft region；
(3) in follow-up two-dimensional image sequence, it is separately operable detection algorithm and track algorithm, the aircraft area that the two algorithms are obtained
Domain and algorithm weights carry out tactic fusion, obtain the aircraft initial alignment region in the two dimensional image that two cameras are shot.
4. the aircraft pose measuring method according to claim 1 based on stereoscopic vision, it is characterised in that：It is right in step c
Disparity map is solved to obtain the three-dimensional point cloud in aircraft initial alignment region in the two dimensional image for the camera that optical center is defined as origin
Realization steps are as follows：
(1) in aircraft initial alignment region each pixel peripheral region in the two dimensional image that two cameras are shot in calculating step b
The matching degree in domain, calculates separately in the two dimensional image of two cameras shooting that in aircraft initial alignment region respective pixel peripheral region
Between matching degree, obtain initial parallax figure；
(2) initial parallax figure is filtered, screens out Mismatching point, obtain disparity map；
(3) it is based on stereoscopic model and camera pinhole imaging system principle, in conjunction with disparity map, in the two dimensional image for calculating left camera
The three-dimensional coordinate of each X-Y scheme picture point in aircraft initial alignment region forms the three-dimensional point cloud in aircraft initial alignment region.
5. the low coverage aircraft pose measuring method according to claim 1 based on parallax method, it is characterised in that：In step d,
To the realization of the plane equation of plane and aircraft wing plane where solving aeroplane nose and empennage, steps are as follows：
(1) in two dimensional image in aircraft initial alignment region, the two dimension in conjunction with profile and framework information to aircraft in the picture
Point carries out screening extraction, and the key feature points extracted are carried out classification marker, by these key feature points according to distribution position
It sets, the key feature point set b being divided on key feature point set a and the aircraft wing on aeroplane nose and empennage；
(2) according to the one-to-one relationship of point in the point and disparity map of two dimensional image, these passes are filtered out from three-dimensional point cloud
The corresponding three-dimensional points of key characteristic point form the three-dimensional point set on three-dimensional point set A and aircraft wing on aeroplane nose and empennage
Close B；
(3) equation of plane, that is, solve aeroplane nose where using RANSAC algorithms to solve two above three-dimensional point set respectively
And plane equation where plane where empennage and aircraft wing plane the two planes.
6. the aircraft pose measuring method according to claim 1 based on stereoscopic vision, it is characterised in that：Have in step e
Body is realized as follows：
(1) using each plane equation solved in step d to the aircraft initial alignment in the two dimensional image that is solved in step c
The three-dimensional point cloud in region carries out three-dimensional point screening, by the distance of point to plane, wiping out background point and noise spot；According to aircraft nose
Plane and aircraft wing plane can determine straight line where airframe where portion and empennage；
(2) according to aircraft realistic model dimension scale and shape feature, space of the aircraft under camera system coordinate system is solved
Pose；
(3) by the Conversion Relations between coordinate system, spatial pose of the aircraft under earth coordinates is solved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810420236.9A CN108665499B (en) | 2018-05-04 | 2018-05-04 | Near distance airplane pose measuring method based on parallax method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810420236.9A CN108665499B (en) | 2018-05-04 | 2018-05-04 | Near distance airplane pose measuring method based on parallax method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108665499A true CN108665499A (en) | 2018-10-16 |
| CN108665499B CN108665499B (en) | 2021-08-10 |
Family
ID=63781859
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810420236.9A Active CN108665499B (en) | 2018-05-04 | 2018-05-04 | Near distance airplane pose measuring method based on parallax method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108665499B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110246212A (en) * | 2019-05-05 | 2019-09-17 | 上海工程技术大学 | A kind of target three-dimensional rebuilding method based on self-supervisory study |
| CN111238441A (en) * | 2020-02-14 | 2020-06-05 | 天津时空经纬测控技术有限公司 | Angular deviation measuring method, angular deviation measuring device, and storage medium |
| CN111288956A (en) * | 2018-12-07 | 2020-06-16 | 顺丰科技有限公司 | Target attitude determination method, device, equipment and storage medium |
| CN111966953A (en) * | 2020-08-07 | 2020-11-20 | 四川泛华航空仪表电器有限公司 | Aircraft attitude angle calculation method based on coordinate transformation |
| CN112037282A (en) * | 2020-09-04 | 2020-12-04 | 北京航空航天大学 | Aircraft attitude estimation method and system based on key points and skeleton |
| CN112862862A (en) * | 2021-02-10 | 2021-05-28 | 中国飞行试验研究院 | Airplane autonomous oil receiving device based on artificial intelligence visual tracking and application method |
| CN114166137A (en) * | 2021-11-26 | 2022-03-11 | 沪东中华造船(集团)有限公司 | Intelligent detection system and method for ship-to-ship filling interval |
| CN114355957A (en) * | 2021-12-29 | 2022-04-15 | 深圳市镭神智能系统有限公司 | Unmanned aerial vehicle autonomous landing method and system and unmanned aerial vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100295948A1 (en) * | 2009-05-21 | 2010-11-25 | Vimicro Corporation | Method and device for camera calibration |
| CN102506757A (en) * | 2011-10-10 | 2012-06-20 | 南京航空航天大学 | Self-positioning method of binocular stereo measuring system in multiple-visual angle measurement |
| CN104317391A (en) * | 2014-09-24 | 2015-01-28 | 华中科技大学 | Stereoscopic vision-based three-dimensional palm posture recognition interactive method and system |
| CN106898022A (en) * | 2017-01-17 | 2017-06-27 | 徐渊 | A kind of hand-held quick three-dimensional scanning system and method |
| CN107945220A (en) * | 2017-11-30 | 2018-04-20 | 华中科技大学 | A kind of method for reconstructing based on binocular vision |
-
2018
- 2018-05-04 CN CN201810420236.9A patent/CN108665499B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100295948A1 (en) * | 2009-05-21 | 2010-11-25 | Vimicro Corporation | Method and device for camera calibration |
| CN102506757A (en) * | 2011-10-10 | 2012-06-20 | 南京航空航天大学 | Self-positioning method of binocular stereo measuring system in multiple-visual angle measurement |
| CN104317391A (en) * | 2014-09-24 | 2015-01-28 | 华中科技大学 | Stereoscopic vision-based three-dimensional palm posture recognition interactive method and system |
| CN106898022A (en) * | 2017-01-17 | 2017-06-27 | 徐渊 | A kind of hand-held quick three-dimensional scanning system and method |
| CN107945220A (en) * | 2017-11-30 | 2018-04-20 | 华中科技大学 | A kind of method for reconstructing based on binocular vision |
Non-Patent Citations (1)
| Title |
|---|
| 刘震: "立体视觉传感器的一种灵活靶标新标定方法", 《光学学报》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111288956A (en) * | 2018-12-07 | 2020-06-16 | 顺丰科技有限公司 | Target attitude determination method, device, equipment and storage medium |
| CN111288956B (en) * | 2018-12-07 | 2022-04-22 | 顺丰科技有限公司 | Target attitude determination method, device, equipment and storage medium |
| CN110246212A (en) * | 2019-05-05 | 2019-09-17 | 上海工程技术大学 | A kind of target three-dimensional rebuilding method based on self-supervisory study |
| CN110246212B (en) * | 2019-05-05 | 2023-02-07 | 上海工程技术大学 | Target three-dimensional reconstruction method based on self-supervision learning |
| CN111238441A (en) * | 2020-02-14 | 2020-06-05 | 天津时空经纬测控技术有限公司 | Angular deviation measuring method, angular deviation measuring device, and storage medium |
| CN111966953A (en) * | 2020-08-07 | 2020-11-20 | 四川泛华航空仪表电器有限公司 | Aircraft attitude angle calculation method based on coordinate transformation |
| CN112037282A (en) * | 2020-09-04 | 2020-12-04 | 北京航空航天大学 | Aircraft attitude estimation method and system based on key points and skeleton |
| CN112037282B (en) * | 2020-09-04 | 2021-06-15 | 北京航空航天大学 | Aircraft attitude estimation method and system based on key points and skeleton |
| CN112862862A (en) * | 2021-02-10 | 2021-05-28 | 中国飞行试验研究院 | Airplane autonomous oil receiving device based on artificial intelligence visual tracking and application method |
| CN112862862B (en) * | 2021-02-10 | 2023-11-17 | 中国飞行试验研究院 | Aircraft autonomous oil receiving device based on artificial intelligence visual tracking and application method |
| CN114166137A (en) * | 2021-11-26 | 2022-03-11 | 沪东中华造船(集团)有限公司 | Intelligent detection system and method for ship-to-ship filling interval |
| CN114355957A (en) * | 2021-12-29 | 2022-04-15 | 深圳市镭神智能系统有限公司 | Unmanned aerial vehicle autonomous landing method and system and unmanned aerial vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108665499B (en) | 2021-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108665499A (en) | A kind of low coverage aircraft pose measuring method based on parallax method | |
| CN105588563B (en) | Binocular camera and inertial navigation combined calibrating method in a kind of intelligent driving | |
| CN105021184B (en) | It is a kind of to be used for pose estimating system and method that vision under mobile platform warship navigation | |
| CN108955685B (en) | Refueling aircraft taper sleeve pose measuring method based on stereoscopic vision | |
| CN106774431A (en) | One kind mapping unmanned plane route planning method and device | |
| CN104215239B (en) | Guidance method using vision-based autonomous unmanned plane landing guidance device | |
| CN105222788B (en) | The automatic correcting method of the matched aircraft Route Offset error of feature based | |
| CN104240262B (en) | Calibration device and calibration method for outer parameters of camera for photogrammetry | |
| CN107316325A (en) | A kind of airborne laser point cloud based on image registration and Image registration fusion method | |
| CN107121125B (en) | A kind of communication base station antenna pose automatic detection device and method | |
| CN104517291B (en) | Pose measuring method based on target coaxial circles feature | |
| CN108898628A (en) | Three-dimensional vehicle object's pose estimation method, system, terminal and storage medium based on monocular | |
| CN106408601B (en) | A kind of binocular fusion localization method and device based on GPS | |
| CN106989744A (en) | A kind of rotor wing unmanned aerial vehicle autonomic positioning method for merging onboard multi-sensor | |
| CN110033489A (en) | A kind of appraisal procedure, device and the equipment of vehicle location accuracy | |
| CN108845335A (en) | Unmanned aerial vehicle ground target positioning method based on image and navigation information | |
| CN104200086A (en) | Wide-baseline visible light camera pose estimation method | |
| WO2007133620A2 (en) | System and architecture for automatic image registration | |
| CN109724586B (en) | Spacecraft relative pose measurement method integrating depth map and point cloud | |
| CN104484870B (en) | Verify Plane location method | |
| CN109685855A (en) | A kind of camera calibration optimization method under road cloud monitor supervision platform | |
| CN106920276A (en) | A kind of three-dimensional rebuilding method and system | |
| CN106969723A (en) | High speed dynamic object key point method for three-dimensional measurement based on low speed camera array | |
| CN108917753A (en) | Method is determined based on the position of aircraft of structure from motion | |
| CN108180888A (en) | A kind of distance detection method based on rotating pick-up head |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |