CN105812769A - High-precision parallax tracker based on phase correlation - Google Patents
High-precision parallax tracker based on phase correlation Download PDFInfo
- Publication number
- CN105812769A CN105812769A CN201610205700.3A CN201610205700A CN105812769A CN 105812769 A CN105812769 A CN 105812769A CN 201610205700 A CN201610205700 A CN 201610205700A CN 105812769 A CN105812769 A CN 105812769A
- Authority
- CN
- China
- Prior art keywords
- parallax
- image
- tracker
- phase place
- high accuracy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 15
- 238000001228 spectrum Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims abstract description 5
- 238000005070 sampling Methods 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 6
- 208000003164 Diplopia Diseases 0.000 claims description 4
- 208000029444 double vision Diseases 0.000 claims description 4
- 230000000977 initiatory effect Effects 0.000 claims description 2
- 239000007787 solid Substances 0.000 claims description 2
- 238000005192 partition Methods 0.000 claims 3
- 230000002596 correlated effect Effects 0.000 claims 2
- 238000012952 Resampling Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 claims 1
- 230000008030 elimination Effects 0.000 claims 1
- 238000003379 elimination reaction Methods 0.000 claims 1
- 239000000758 substrate Substances 0.000 claims 1
- 238000005516 engineering process Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
-
- G06T5/80—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/128—Adjusting depth or disparity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Abstract
The invention provides a parallax tracker based on phase correlation (PC), relates to an unmanned aerial vehicle continuous image parallax estimation method based on a PC tracker, and belongs to the field of computer vision and aviation image processing. Aiming at the mismatching problem caused by the influence of big terrain drop, shadow and river regions in the unmanned aerial vehicle low altitude image parallax estimation based on the PC algorithm, the invention provides a high-precision parallax tracker on the basis of the PC matching method. The tracker comprises the following operations: firstly, dividing an image into a plurality of layers from coarse to fine; secondly, solving an initial parallax image and a cross-power spectrum Fourier inversion peak value through the adoption of a big window PC algorithm on the first layer; and meanwhile, setting a credibility threshold value according to the cross-power spectrum Fourier inversion peak value; thirdly, guiding a small window PC algorithm to iteratively track the credible parallax according to the credibility threshold value and the initial parallax image until the convergence; and finally acquiring a detailed parallax image.
Description
Technical field
The present invention relates to a kind of Disparity estimation, particularly relate to that a kind of landform drop is big, the high accuracy parallax estimation method of river and the impact of weak texture region based on phase place relevant can eliminating, belong to computer vision and remote sensing image process field.
Background technology
Disparity estimation is the committed step in computer vision and remote sensing image process field, is also one of emerging hot research direction, relates to multiple field in potential application, such as topographic survey, unmanned vehicle and robot navigation, living things feature recognition and Medical Image Processing etc..Along with popularizing of armarium, photographic sensor and unmanned plane product, this technology has attracted concern and the research of increasing research worker both at home and abroad.Disparity estimation is based on the 3 D visual image treatment technology of binocular image.Binocular can be carried out dense Stereo Matching as each pixel of centering by it, and solves sub-pixed mapping level parallax information by high-precision fitting algorithm.
Along with developing rapidly of the technology of taking photo by plane, computer technology and bionical theories of vision research, the high accuracy matching algorithm relevant based on phase place have also been obtained significant progress.Owing to it possesses, matching speed is fast, illumination robustness strong and the advantage of sub-pixed mapping matching precision so that it is be quickly generalized to medical image processing, 360 degree of fields such as vision and three-dimensional reconstruction.It is theoretical that the basic theories of phase place related algorithm is based on Fourier transformation translation invariance: in spatial domain, the relative translation parameter of two width images is just equal with the phase difference value of two width images in Fourier frequency territory.Therefore, utilize this theoretical, in Fourier frequency territory, by normalization crosspower spectrum formula, two width images are carried out correlation calculations, and solve sub-pixed mapping level translation parameters by two-dimentional approximating method.But, traditional phase correlation method is frequently subjected to the impact of stationary window, thus causing error hiding, and then cannot effectively processing some, to comprise landform drop bigger, the image of weak texture and river region, even if certain methods before has been attempt to by solving this problem based on the layering framework of stationary window, but DeGrain, there is no any robustness particularly with weak texture and river region.
Summary of the invention
The technical problem that present invention mainly solves is to provide a kind of parallax tracker relevant based on phase place, to eliminate the error hiding impact that stationary window phase place related algorithm is caused by landform drop, weak texture and river region, improve the reliability without texture and river region disparity estimation while promoting Local treatment precision.
The solution of the present invention is: utilize layering framework that image is divided into multilamellar；Then utilize multiwindow phase correlation method ground from coarse to fine estimating disparity figure, eliminate the stationary window impact on phase place related algorithm.Meanwhile, at initiation layer according to crosspower spectrum Fourier inversion peak-settings believability threshold, estimate to start to eliminate the impact of weak texture and river region from the second layer, promote the credibility of the parallax value in this region.
The step that the present invention realizes such scheme is as follows:
1. utilize polar curve correction algorithm to binocular solid picture to carrying out polar curve rectification: first, utilize the sparse matching algorithm of surf to determine the match point of stereogram；Second, utilize double-vision geometry F matrix restriction relation to obtain limit；3rd, limit is rotated on axis, and is projected into infinite point；Finally, it is achieved the polar curve of stereo pairs is alignd；
2. step 1 correct as to basis on, image is carried out layered shaping.Concrete hierarchy formulas is as follows: fH=f/tl.Wherein, fHIt is layered image, tlBeing the sampling interval, l is layering number of times, and f is original image；
3. on the stereogram basis that step 2 layered approach obtains, first with big window phase place related algorithm to correcting as to carrying out disparity computation and credibility setting.Its concrete formula is as follows: F (U)=G (U) EXP (UD)；C=EXP(UD)；U=(u,v)T, D=(dx,dy)T.Wherein, F and G represents the Fourier transformation domain matrix of stereogram, and C represents normalization crosspower spectrum, and U represents Fourier frequency domain coordinate, and D represents translation parameters.Secondly, C is carried out inverse Fourier transform, it is thus achieved that with the D sinc function being translation parameters and peak value thereof.According to sinc peak of function, set believability threshold.Its concrete formula is as follows: Th=Max (F-1(C))/P；Wherein, Th is believability threshold, F-1Being Fourier inversion, P is constant.Finally, the single peak characteristic according to sinc function, in conjunction with the characteristic that two-dimensional Gaussian function can be represented by one-dimensional Gaussian function on different One Dimensional Projections, we use one-dimensional Gaussian function that sinc function carries out high accuracy matching.Concrete fit approach is as follows: 1) take eight fields of peak value, the matrix of composition 3*3；2) will add up respectively along matrix x and y direction, form one-dimensional data；3) one-dimensional Gaussian function is used to be fitted: H (x)=a*EXP ((x-b)2/c2).Wherein, b is Gaussian function central point, and a and c is at the zoom factor of vertical and horizontal directions；
4. using the match window in the initial value guiding target image of last layer acquirement, phase place related algorithm follows the trail of the best parallax of every bit layer by layer.Meanwhile, when following the trail of for every layer, reducing window size, its concrete formula is: Wl=W0*b1-l.Wherein, W represents that window value, b are base (being typically set to 2), and l is the number of plies.When every point tracking, according to confidence value threshold value, it is judged that follow the trail of credibility, if credible, continue to follow the trail of, if insincere, then stop following the trail of.
Claims (8)
1. the high accuracy parallax tracker being correlated with based on phase place, it is characterised in that comprise the steps of:
1) utilize disparity correspondence algorithm and double-vision geometry epipolar-line constraint, correct unmanned plane stereo-picture；
2) utilize the step-length method of sampling that stereo pairs is carried out hierarchical partition；
3) big window phase correlation method is utilized at initiation layer, initial parallax and believability threshold to be solved；
4) utilize initial parallax value to guide wicket phase correlation method to refine disparity map iteratively, meanwhile, according to believability threshold, the parallax value of each picture point is carried out Credibility judgement, eliminate the impact of the weak texture of unmanned plane image and river region.
2. the one according to right 1 is based on phase place relevant high accuracy parallax tracker, it is characterised in that utilize double-vision geometry epipolar-line constraint that stereo-picture is carried out resampling rectification:
1) match point of Surf algorithm detection binocular image (such as unmanned plane continuous image) is utilized；
2) utilize double-vision geometry epipolar-line constraint relation to obtain limit, and rotate it to, on axis, be subsequently projected to infinite point, finally obtain the binocular image of polar curve alignment.
3. a kind of high accuracy parallax tracker relevant based on phase place according to right 1, it is characterised in that the hierarchical partition to binocular image:
1) by the step-length method of sampling, stereogram is carried out hierarchical partition；Its concrete formula is as follows:
Wherein, f represents that image function, l represent that sampling interval, t represent sampling level；
2) sampling interval changes along with the change of layering；Its concrete formula is as follows:
Wherein, b is substrate, l0Initial samples interval.
4. a kind of high accuracy parallax tracker relevant based on phase place according to right 1, it is characterised in that each picture point is carried out highly reliable parallax and solves；Its concrete formula is as follows:
Wherein: F and G is the fourier function of binocular solid picture pair respectively, U is Fourier frequency coordinate, and D is translation parameters, and R represents real number.
5. a kind of high accuracy parallax tracker relevant based on phase place according to right 1, it is characterised in that the image that disparity map is solved by the insincere region of the elimination utilizing believability threshold (such as weak texture and river region)；Its concrete formula is as follows:
Wherein, Th represents believability threshold, F-1Representing Fourier inversion function, C represents crosspower spectrum function, and P is parameter factors.
6. a kind of high accuracy parallax tracker relevant based on phase place according to right 1, it is characterised in that the control of window size, during aerial images continuous in Processing for removing unmanned plane, due to the error hiding problem that landform drop causes more greatly；Its concrete formula is as follows:
Wherein, W represents window size, t representational level, and b is base.
7. the high accuracy parallax tracker that a kind of phase place according to right 1 is correlated with, it is characterised in that follow the trail of the parallax value of each picture point iteratively；Its concrete formula is as follows:
Wherein, M represents tracking parallax, and d represents every layer of parallax solved, and X represents that image coordinate, k represent tracking number of times, and R represents real number.
8. a kind of parallax tracker relevant based on phase place according to right 1, it is characterised in that utilize the parallax value of back to guide the window progressively reduced to follow the trail of the parallax value of each picture point iteratively as priori value；Simultaneously in whole iteration cycle process, before each picture point carries out parallax tracking, it is carried out Credibility judgement, if credible continuation is followed the trail of, if insincere, stop this parallax and follow the trail of；High-quality disparity map is obtained finally by iterative refinement procedure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610205700.3A CN105812769B (en) | 2016-04-06 | 2016-04-06 | Based on the related high-precision parallax tracker of phase |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610205700.3A CN105812769B (en) | 2016-04-06 | 2016-04-06 | Based on the related high-precision parallax tracker of phase |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105812769A true CN105812769A (en) | 2016-07-27 |
| CN105812769B CN105812769B (en) | 2018-04-03 |
Family
ID=56460437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610205700.3A Active CN105812769B (en) | 2016-04-06 | 2016-04-06 | Based on the related high-precision parallax tracker of phase |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105812769B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106408531A (en) * | 2016-09-09 | 2017-02-15 | 四川大学 | GPU acceleration-based hierarchical adaptive three-dimensional reconstruction method |
| CN109792484A (en) * | 2016-09-23 | 2019-05-21 | 高通股份有限公司 | Image procossing in unmanned automated spacecraft |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002257543A (en) * | 2001-03-05 | 2002-09-11 | National Aerospace Laboratory Of Japan Mext | High precision stereo vision using continuous frame image |
| CN102999913A (en) * | 2012-11-29 | 2013-03-27 | 清华大学深圳研究生院 | Local three-dimensional matching method based on credible point spreading |
| CN103226821A (en) * | 2013-04-27 | 2013-07-31 | 山西大学 | Stereo matching method based on disparity map pixel classification correction optimization |
| CN104166987A (en) * | 2014-07-08 | 2014-11-26 | 哈尔滨师范大学 | Parallax estimation method based on improved adaptive weighted summation and belief propagation |
| CN104376593A (en) * | 2014-11-25 | 2015-02-25 | 四川大学 | Three-dimensional image reconstruction method based on multi-window phase correlation |
| CN104463958A (en) * | 2014-11-25 | 2015-03-25 | 四川大学 | Three-dimensional super-resolution method based on disparity map fusing |
| CN104574342A (en) * | 2013-10-14 | 2015-04-29 | 株式会社理光 | Noise identification method and noise identification device for parallax depth image |
-
2016
- 2016-04-06 CN CN201610205700.3A patent/CN105812769B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002257543A (en) * | 2001-03-05 | 2002-09-11 | National Aerospace Laboratory Of Japan Mext | High precision stereo vision using continuous frame image |
| CN102999913A (en) * | 2012-11-29 | 2013-03-27 | 清华大学深圳研究生院 | Local three-dimensional matching method based on credible point spreading |
| CN103226821A (en) * | 2013-04-27 | 2013-07-31 | 山西大学 | Stereo matching method based on disparity map pixel classification correction optimization |
| CN104574342A (en) * | 2013-10-14 | 2015-04-29 | 株式会社理光 | Noise identification method and noise identification device for parallax depth image |
| CN104166987A (en) * | 2014-07-08 | 2014-11-26 | 哈尔滨师范大学 | Parallax estimation method based on improved adaptive weighted summation and belief propagation |
| CN104376593A (en) * | 2014-11-25 | 2015-02-25 | 四川大学 | Three-dimensional image reconstruction method based on multi-window phase correlation |
| CN104463958A (en) * | 2014-11-25 | 2015-03-25 | 四川大学 | Three-dimensional super-resolution method based on disparity map fusing |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106408531A (en) * | 2016-09-09 | 2017-02-15 | 四川大学 | GPU acceleration-based hierarchical adaptive three-dimensional reconstruction method |
| CN109792484A (en) * | 2016-09-23 | 2019-05-21 | 高通股份有限公司 | Image procossing in unmanned automated spacecraft |
| US10917561B2 (en) | 2016-09-23 | 2021-02-09 | Qualcomm Incorporated | Image processing in an unmanned autonomous vehicle |
| CN109792484B (en) * | 2016-09-23 | 2021-03-09 | 高通股份有限公司 | Image processing in unmanned autonomous aircraft |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105812769B (en) | 2018-04-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fan et al. | Road surface 3D reconstruction based on dense subpixel disparity map estimation | |
| CN112435325B (en) | VI-SLAM and depth estimation network-based unmanned aerial vehicle scene density reconstruction method | |
| CN107564061B (en) | Binocular vision mileage calculation method based on image gradient joint optimization | |
| CN104281148A (en) | Mobile robot autonomous navigation method based on binocular stereoscopic vision | |
| CN104197928A (en) | Multi-camera collaboration-based method for detecting, positioning and tracking unmanned aerial vehicle | |
| CN108917753B (en) | Aircraft position determination method based on motion recovery structure | |
| CN112734839B (en) | Monocular vision SLAM initialization method for improving robustness | |
| Yue et al. | 3D point clouds data super resolution-aided LiDAR odometry for vehicular positioning in urban canyons | |
| Li et al. | Hierarchical and adaptive phase correlation for precise disparity estimation of UAV images | |
| Zhao et al. | Review of slam techniques for autonomous underwater vehicles | |
| CN113887624A (en) | Improved feature stereo matching method based on binocular vision | |
| Liu et al. | Dense stereo matching strategy for oblique images that considers the plane directions in urban areas | |
| CN105812769A (en) | High-precision parallax tracker based on phase correlation | |
| Zhao et al. | A review of 3D reconstruction from high-resolution urban satellite images | |
| Sergiyenko et al. | Machine vision sensors | |
| Hoang et al. | Motion estimation based on two corresponding points and angular deviation optimization | |
| Le Besnerais et al. | Dense height map estimation from oblique aerial image sequences | |
| Liu et al. | Accurate real-time visual SLAM combining building models and GPS for mobile robot | |
| CN108090930A (en) | Barrier vision detection system and method based on binocular solid camera | |
| CN112446915A (en) | Picture-establishing method and device based on image group | |
| Wang et al. | A research on stereo matching algorithm for underwater image | |
| CN116151320A (en) | Visual odometer method and device for resisting dynamic target interference | |
| Cao et al. | Template matching based on convolution neural network for UAV visual localization | |
| CN104376593B (en) | Based on the related three-dimensional reconstruction method of multiwindow phase place | |
| Lee | Accuracy improvement of the ICP DEM matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |