CN104318566A - Novel multi-image plumb line track matching method capable of returning multiple elevation values - Google Patents

Abstract

The invention discloses a novel multi-image plumb line track matching method capable of returning multiple elevation values. The method comprises the following steps of: confirming m to-be-matched points of a ground space element (X0, Y0) on a benchmark image by using an image imaging model; matching multiple images restrained by space information to m to-be-matched points; calculating space three-dimensional coordinates (Xi, Yi, Zi) corresponding to the to-be-matched points and an absolute difference (deltaXi and deltaYi) between each coordinate (Xi, Yi) and the space element (X0, Y0) by adopting a multi-image adjustment bundle method, and deciding the returned elevation values according to whether the absolute difference is less than a set threshold. In the multi-image plumb line track matching method, multi-image matching and accuracy test of matching result are together fused to effectively eliminate wrong matching, so that the method can solve the disadvantage that only one elevation value can be returned by the traditional ground image matching method.

Description

The novel multi-view images pedal line path matching method of multiple height value can be returned

Technical field

The invention belongs to digital photogrammetry, Geographic Information System and computer vision field, relate to and determining and the aspects such as error hiding result elimination based on the matching constraint strategy in the multi-view images matching process of object space, hunting zone.

Background technology

Digital photogrammetry technology is from two-dimensional digital Extraction of Image and the important means upgrading the geological information of Three dimensional Targets, radiation information and semantic information, play an important role in the development of the national economy and social development, be widely used in the fields such as base surveying, survey of territorial resources exploitation and digital city modeling.Image Matching, as the gordian technique of in digital photogrammetry, through the whole flow process of photogrammetric data process, is the study hotspot in the field such as Photogrammetry and Remote Sensing, computer vision always.In recent years, due to widely using of digital sensor, obtain large degree of overlapping and look digitized video more and more easily more, an object space point can image on 15 even more images simultaneously.Day by day increasing and the continuous growth of three-dimensional spatial information application demand of mass remote sensing image data, also more and more urgent to the full-automation of digital photogrammetry technology, high-level efficiency and high reliability request.How reliably, exactly mate out by the corresponding image points on multi-view images, become the photogrammetric problem demanding prompt solution of Contemporary Digital, the degree that image matching problem solves finally determines the automaticity of digital photogrammetry.

The key issue of Image Matching technology be automatically set up different images picture point between corresponding relation.Image Matching Algorithm can be divided into stereopsis to mate and multi-view images coupling by the image quantity participating in coupling, can be divided into based on the Image Matching of image space primitive and the coupling based on object space primitive by Matching unit.Image Matching Algorithm based on image space primitive can be divided into three major types again: based on the Image Matching of gray scale, the Image Matching of feature based and understanding based on image and the Image Matching of decipher.Be the widely used method in current photogrammetric field based on the coupling of gray scale and feature-based matching, and also there is very high reliability the open region that, texture information level and smooth for image mesorelief enriches; Based on image understanding and fitting through of decipher, semantic description is carried out to image object, thus reach the object of coupling, but with regard to current research situation, it still there is a lot of insurmountable practical problems, seldom uses in photogrammetric field.

But the object of Image Matching is the geological information of extraction object, determines its locus.Thus based on the image matching method of image space primitive after the parallax obtaining left and right image, space intersection also to be utilized to resolve the 3 d space coordinate (X, Y, Z) of its corresponding object point, and then set up digital surface model.And certain interpolating method also may can be used when setting up digital surface model, object space precision of information is reduced more or less.Therefore, it is possible to directly determine that the image matching method based on object space of body surface space of points three-dimensional coordinate obtains research, these methods are also referred to as " ground element image coupling ".Based in the Image Matching of object space, be known as the topocentric planimetric coordinates (X, Y) for the treatment of a primitive, only need to determine its elevation Z.The existing image matching method based on object space mainly adopts following strategy when determining image space corresponding image points: according to topocentric minimum and maximum elevation scope, with object space elevation Z for search benchmark, from minimum height value, each elevation increases Δ Z, determines the object space height value Z of to be matched some object space Searching point at object space _i=Z _min+ i × Δ Z, i=1,2,3 ..., n, n are searching times, thus obtain three-dimensional coordinate (X, Y, the Z of object space Searching point _i); Then, object space Searching point is projected to the Searching point each search image obtaining image space, and then complete the calculating of similarity between point to be matched and Searching point, and the Z selecting maximum similarity corresponding _ias topocentric height value.

But the size of existing ground element image matching process Δ Z value in search procedure is difficult to accurately determine, cannot ensure the Z of Searching point _icertain for ground point; Δ Z value is excessive, then can miss correct candidate point; Δ Z value is too small, the calculating costed a lot of money again and search time.In addition, existing method can only return a height value at each ground point place, this has the vertical atural object (such as building facade, electric pole etc.) of multiple height value for same planimetric position, obviously can not get the result reasonably reflecting object height journey distribution situation practically.And existing method lacks enough checkout procedures for the whether accurate of matching result, the matching similarity only with image space is difficult to the accuracy ensureing matching result.

Summary of the invention

The object of the invention is to the deficiency that can only return a height value, matching result shortage validation verification existed for existing ground element image matching process, propose a kind of novel multi-view images pedal line path matching method returning multiple height value.

The novel multi-view images pedal line path matching method that can return multiple height value comprises the steps:

Step one, utilizes the imaging model of image, according to multi-view images outer orientation parameter, and the planimetric coordinates (X of the ground object space primitive of input ₀, Y ₀), the object space maximum elevation Z in region to be matched _maxwith minimum elevation Z _min, determine the image space ranks number of m the point to be matched that object space primitive is corresponding in reference images;

Step 2, to the m in reference images point to be matched, carries out the multi-view images coupling that object space is information constrained, to obtain the ranks number of the corresponding image points of each point to be matched on other search image;

Step 3, according to the same place result on multi-view images, utilizes many picture bundle adjustments, calculates m the object space three-dimensional coordinate (X corresponding to point to be matched _i, Y _i, Z _i), i=1,2 ..., m, then the coordinate (X calculating each point to be matched _i, Y _i) with input object space primitive coordinate (X ₀, Y ₀) between poor absolute value delta X _i, Δ Y _i, and according to Δ X _i, Δ Y _ithe threshold value whether being less than setting decides the height value of returned object space primitive.

Wherein, the detailed process of described step one is:

(1) input has the n width multi-view images (Aeronautics and Astronautics or up short image) of elements of exterior orientation, and the planimetric coordinates (X of ground object space primitive ₀, Y ₀), the object space maximum elevation Z in region to be matched _maxwith minimum elevation Z _min;

(2) utilize the imaging model of image, according to elements of exterior orientation, the planimetric coordinates of ground object space primitive, the minimum and maximum object space elevation of input of reference images, calculate the image space ranks number of m the candidate to be matched point of object space primitive in reference images.

The detailed process of described step 2 is:

(1) for each to be matched some q in reference images _i(i=1,2 ..., m), utilize it as the minimum and maximum object space elevation of planimetric coordinates, input, according to the imaging model of image, calculate to be matched some q _iobject point is at the object space three-dimensional coordinate of the highs and lows of the object space region of search accordingly;

(2) according to video imaging model, by the highs and lows of the object space region of search toward n-1 width search image S ₁..., S _j..., S _n-1on project, obtain the image space ranks number of search core line of the same name two end points to be matched some same place place on each width search image;

(3) according to the image space ranks number of two end points of corresponding epipolar line on each search image, the straight-line equation h of picture plane coker line is determined ^{' j}=k ^{' j}× l ^{' j}+ b ^{' j}, and the interval range [S_l of candidate point range l' of the same name ^j, E_l ^j], j=1,2 ..., n-1, and with burst length E_l ^j-S_l ^ja maximum width search image is main search image, and remaining n-2 width image is secondary search image;

(4) in the image space region of search main search image, take out each pixel one by one as the current candidate corresponding image points on this image, first utilize double image forward intersection method, calculate to be matched some q _iwith the object space three-dimensional coordinate of this candidate point institute intersection culture point, and this three-dimensional coordinate is projected on remaining n-2 width pair search image, obtain n-2 candidate's corresponding image points on secondary search image, again together with the current candidate corresponding image points on main search image, thus obtain often group n-1 candidate's corresponding image points on each pair search image; Then, the comprehensive matching Likelihood Computation method based on RGB color characteristic and SIFT feature is recycled, to be matched some q on Calculation Basis image _ioften organize many pictures comprehensive matching similarity of n-1 candidate's same place, and with that group candidate point corresponding to maximum similarity, as many n-1 corresponding image points q' obtained as matching process that object space is information constrained ₁, q' ₂..., q' _n-1.

The detailed process of described step 3 is:

(1) for each to be matched some q in reference images _i, and n-1 corresponding image points q' on each width search image ₁, q' ₂..., q' _n-1, according to the picture planimetric coordinates of n picture point on each self imaging and the elements of exterior orientation of n width image, utilize many picture bundle adjustments, calculate to be matched some q _iobject space three-dimensional coordinate (the X of corresponding culture point _i, Y _i, Z _i);

(2) each to be matched some q is calculated _icorresponding object space planimetric coordinates (X _i, Y _i) with input object space primitive coordinate (X ₀, Y ₀) between poor absolute value delta X _i, Δ Y _iif, Δ X _iwith Δ Y _iall be less than or equal to the threshold value of setting, then think to be matched some p _imanyly meet object coordinates coherence request, by q as matching result _iand corresponding image points returns, by Z as one group of image space matching result of ground primitive _ian object space height value as ground primitive returns; If Δ X _ior Δ Y _ibe greater than the threshold value of setting, then think to be matched some q _iand many picture matching results are invalid, abandon this group result.

Multi-view images pedal line path matching method of the present invention is a kind of PARALLEL MATCHING method, and each topocentric matching process is independent of each other, and is very beneficial for the efficient Rapid matching of millet cake in large quantities.Multi-view images matching process of the present invention merges the image space information employing topocentric object space information and multi-view images, by the information constrained match search scope of topocentric object space, but matching process completes in image space again, make the sure picture point corresponding through ground point of the candidate search of image space point, ensure that the time efficiency of coupling and the validity of Searching point; In addition, matching result is added to the consistency checking of object space information, can effectively eliminate error hiding result, improve based on the reliability of the multi-view images matching result of object space, solve tradition based on also can only obtain the problems such as a height value and validity thereof cannot be verified in the image matching method of object space at vertical atural object place.

Accompanying drawing explanation

Fig. 1 is the method frame figure of the embodiment of the present invention;

Fig. 2 is that the ground candidate of object space primitive in the reference images point to be matched of the embodiment of the present invention determines schematic diagram;

Fig. 3 is the multi-view images coupling schematic diagram information constrained based on object space of the embodiment of the present invention;

Fig. 4 is the corresponding image point position of an actual measurement ground object space primitive on three width aviation images of the embodiment of the present invention, wherein, a () is the image point position of ground primitive in reference images, b () is the image point position of ground primitive on the first width search image, (c) is the image point position of ground primitive on the second width search image;

Fig. 5 (a), (b), (c) are the image space result utilizing traditional multi-view images pedal line path matching method to return for Fig. 4 (a), (b), (c) respectively;

Fig. 6 (a), (b), (c) are the image space result utilizing the present invention's novel multi-view images pedal line path matching method to return for Fig. 4 (a), (b), (c) respectively.

Embodiment

Below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in further detail.

The present invention, according to the image outer orientation parameter of the imaging model of image and input and ground object space information, first determines the multiple to be matched point of ground object space primitive in reference images; Again the information constrained multi-view images coupling of object space is carried out to these points to be matched, to make full use of the redundant information of many images, improve the reliability of match measure calculating and ensure that candidate matches result is affirmed through picture point corresponding to ground primitive; Then, many picture bundle adjustments are carried out to multi-view images matching result, to calculate object space three-dimensional coordinate corresponding to each candidate matches result, and the planimetric coordinates of the ground primitive of the planimetric coordinates in this coordinate and input is carried out the checking of object space consistency on messaging, effectively to eliminate the error hiding result in candidate matches result, improve the accuracy of multi-view images matching result.

As shown in Figure 1, the novel multi-view images pedal line path matching method that can return multiple height value comprises three parts: (1) determines the multiple candidates of ground object space primitive in reference images point to be matched; (2) the information constrained multi-view images coupling of object space is carried out to point to be matched in reference images; (3) how as the object coordinates consistency checking of matching result.Concrete implementation step is:

The first step: determine the multiple candidates of ground object space primitive in reference images point to be matched.

Determine the schematic diagram of the multiple candidates of ground object space primitive in reference images point to be matched as shown in Figure 2, its idiographic flow is as follows:

(1) (Aeronautics and Astronautics or up short image, wherein, 1 width is reference images I to input the n width multi-view images with elements of exterior orientation ₀, remaining is n-1 width search image S ₁..., S _j..., S _n-1), and the planimetric coordinates (X of ground object space primitive ₀, Y ₀), the object space maximum elevation Z in region to be matched _maxwith minimum elevation Z _min;

(2) utilize the imaging model of image, according to elements of exterior orientation, the planimetric coordinates of ground object space primitive, the minimum and maximum object space elevation of input of reference images, calculate the image space ranks number of m the candidate to be matched point of object space primitive in reference images.

For the in-line model of aviation image, if the planimetric coordinates of the ground object space primitive P of input is (X ₀, Y ₀), reference images I ₀elements of exterior orientation be object space maximum elevation Z _maxwith minimum elevation Z _min, then ground point is at the peak P of the object space region of search _maxwith minimum point P _minthree-dimensional coordinate respectively (X ₀, Y ₀, Z _max), (X ₀, Y ₀, Z _min); By P _maxand P _minin reference images, carry out back projection respectively, obtain two corresponding picture point p _maxand p _mintwo-dimensional image planimetric coordinates respectively (x ₁, y ₁), (x ₂, y ₂).The formula being calculated image space two-dimensional image planimetric coordinates (x, y) by object space three-dimensional coordinate (X, Y, Z) is as follows:

$\left\{\begin{array}{c}x=-f\frac{a_1^{I_0}(X-X_{I_0})+b_1^{I_0}(Y-Y_{I_0})+c_1^{I_0}(Z-X_{I_0})}{a_3^{I_0}(X-X_{I_0})+b_3^{I_0}(Y-Y_{I_0})+c_3^{I_0}(Z-X_{I_0})}\\ y=-f\frac{a_2^{I_0}(X-X_{I_0})+b_2^{I_0}(Y-Y_{I_0})+c_2^{I_0}(Z-X_{I_0})}{a_3^{I_0}(X-X_{I_0})+b_3^{I_0}(Y-Y_{I_0})+c_3^{I_0}(Z-X_{I_0})}\end{array}\right.---\left(1\right)$

In formula, by image I ₀foreign side's parallactic angle element nine direction cosine in the rotation matrix determined, f is the focal length of filming image camera.

Again according to image I ₀principal point ranks number (h ₀, l ₀), pixel dimension μ, following formula can be utilized to put p _maxand p _mintwo-dimensional image planimetric coordinates (x ₁, y ₁), (x ₂, y ₂) be converted to image ranks coordinate (h respectively ₁, l ₁), (h ₂, l ₂).

$h=h_0-\frac{y}{\mathrm{\μ}};l=l_0+\frac{x}{\mathrm{\μ}}---\left(2\right)$

Therefore, the m of ground primitive in reference images candidate point to be matched is located in the straight line p of picture plane _maxp _minon, i-th (i=1,2 ..., m) candidate to be matched some q _iranks number (h _i, l _i) by two kinds of situations calculating below:

If 1. abs (l ₁– l ₂) >=abs (h ₁– h ₂), straight line p is described _maxp _minclose with line direction, then l _max=max (l ₁, l ₂), l _min=min (l ₁, l ₂), function max () and min () represents respectively and takes from minimum value in variable and maximal value, m=l _max– l _min+ 1, l _ifrom l _minto l _maxvalue one by one, h _ibe calculated as follows:

$\left\{\begin{array}{c}{h}_i=k\×l_i+b\\ k=\frac{h_2-h_1}{l_2-l_1},b=h_1-k\×l_1\end{array}\right.---\left(3\right)$

If 2. abs (l ₁– l ₂) <abs (h ₁– h ₂), straight line p is described _maxp _minclose with column direction, then h _max=max (h ₁, h ₂), h _min=min (h ₁, h ₂), m=h _max– h _min+ 1, h _ifrom h _minto h _maxvalue one by one, l _ibe calculated as follows:

$\left\{\begin{array}{c}{l}_i=k\&times;h_i+b\\ k=\frac{l_2-l_1}{h_2-h_1},b=l_1-k\&times;h_1\end{array}\right.---\left(4\right)$

Second step: the information constrained multi-view images coupling of object space is carried out to point to be matched in reference images.

To be matched some q in reference images _iinformation constrained based on object space multi-view images coupling schematic diagram as shown in Figure 3, its process is:

(1) for each to be matched some q in reference images _i(i=1,2 ..., m), utilize its as planimetric coordinates and input minimum and maximum object space elevation, according to the imaging model of image, calculate to be matched some q _iobject point is at the object space three-dimensional coordinate of the highs and lows of the object space region of search accordingly;

Suppose to be matched some q _iat reference images I ₀image space ranks number be (h _i, l _i), first according to formula (2) conversion imaging planimetric coordinates (x _i, y _i); Again according to the elements of exterior orientation of reference images and the object space maximum elevation Z of input _maxwith minimum elevation Z _min, try to achieve a q by formula (5) _ithe peak Q of the region of search on object space photography light _maxwith minimum point Q _minobject space planimetric coordinates (X _max, Y _max), (X _min, Y _min) (with a Q _maxbe calculated as example).

$\left\{\begin{array}{c}{X}_{\max }=X_{I_0}+(Z_{\max }-Z_{I_0})\frac{a_1^{I_0}{x}_i+a_2^{I_0}{y}_i-a_3^{I_0}f}{c_1^{I_0}{x}_i+c_2^{I_0}{y}_i-c_3^{I_0}f}\\ Y_{\max }=Y_{I_0}+(Z_{\max }-Z_{I_0})\frac{b_1^{I_0}{x}_i+b_2^{I_0}{y}_i-b_3^{I_0}f}{c_1^{I_0}{x}_i+c_2^{I_0}{y}_i-c_3^{I_0}f}\end{array}\right.---\left(5\right)$

In formula, deng the implication of character and the same in formula (1).

(2) according to video imaging model, by the highs and lows of the object space region of search toward n-1 width search image S ₁..., S _j..., S _n-1on project, obtain the image space ranks number of search core line of the same name two end points to be matched some same place place on each width search image;

To search for image for S _jexample, to be matched some q _ithe image space ranks number of search core line of the same name two end points at the same place place on this image calculate by mode below:

First, the imaging model according to formula (1), by the peak Q of the to be matched some object space region of search _maxwith minimum point Q _min(the search of the correlation parameter now, in the imaging model image S that projects is carried out on this image _jelements of exterior orientation calculate), obtain corresponding two picture point q ^j _maxand q ^j _minpicture planimetric coordinates then, according to formula (2), will as planimetric coordinates convert image space ranks number to

(3) according to the image space ranks number of two end points of corresponding epipolar line on each search image, the straight-line equation h of picture plane coker line is determined ^{' j}=k ^{' j}× l ^{' j}+ b ^{' j}, and the interval range [S_l of candidate point range l' of the same name ^j, E_l ^j], j=1,2 ..., n-1, and with burst length E_l ^j-S_l ^ja maximum width search image is main search image, and remaining n-2 width image is secondary search image;

By to be matched some q _iat search image S _jupper search core line of the same name two end points q ^j _maxand q ^j _minimage space ranks number candidate's corresponding image points row number on corresponding epipolar line can be obtained interval scope [S_l ^j, E_l ^j] (wherein, t=1,2 ..., E_l ^j-S_l ^j, and calculate straight-line equation (the i.e. line number of candidate's same place of corresponding epipolar line by formula (6)+1) with row number between calculated relationship):

$\left\{\begin{array}{c}{h}_t^{\&prime;j}=k^{\&prime;j}\&times;l_t^{\&prime;j}+b^{\&prime;j}\\ k^{\&prime;j}=\frac{h_2^{\&prime;j}-h_1^{\&prime;j}}{l_2^{\&prime;j}-l_1^{\&prime;j}},b^{\&prime;j}=h_1^{\&prime;j}-k^{\&prime;j}{l}_1^{\&prime;j}\end{array}\right.---\left(6\right)$

(4) in the image space region of search main search image, take out each pixel one by one as the current candidate corresponding image points on this image, first utilize double image forward intersection method, calculate to be matched some q _iwith the object space three-dimensional coordinate of this candidate point institute intersection culture point, and this three-dimensional coordinate is projected on remaining n-2 width pair search image, obtain n-2 candidate's corresponding image points on secondary search image, again together with the current candidate corresponding image points on main search image, thus obtain often group n-1 candidate's corresponding image points on each pair search image; Then, the comprehensive matching Likelihood Computation method based on RGB color characteristic and SIFT feature is recycled, to be matched some q on Calculation Basis image _ioften organize many pictures comprehensive matching similarity of n-1 candidate's same place, and with that group candidate point corresponding to maximum similarity, as many n-1 corresponding image points q' obtained as matching process that object space is information constrained ₁, q' ₂..., q' _n-1.

Suppose with S ₁for main search image, remainder are n-2 pair search image, determine to be matched some q _imanyly as same place process be:

First, from main search image S ₁on search core line on take out any one candidate's same place q' _{1, t}(its row number span be [S_l ¹, E_l ¹], corresponding line number calculate by formula (6), t=1,2 ..., E_l ¹-S_l ¹, and will q' be put according to formula (2)+1) _{1, t}ranks number conversion imaging planimetric coordinates recycling double image forward intersection method (formula (7)), calculates to be matched some q _iwith candidate point q' _{1, t}candidate's culture point Q of institute's intersection _ithree-dimensional coordinate (X _i, Y _i, Z _i).

$\left\{\begin{array}{c}{X}_i=X_{I_0}+N_1{X}_i^{I_0}\\ Y_i=(Y_{I_0}+N_1{Y_i}^{I_0}+Y_{S_1}+N_2{Y_t}^{S_1})/2\\ Z_i=Z_{I_0}+N_1{Z}_i^{I_0}\end{array}\right.---\left(7\right)$

In formula, $\left\{\begin{array}{c}{N}_1=\frac{B_X{Z}_t^{S_1}-B_Z{X}_t^{S_1}}{X_i^{I_0}{Z}_t^{S_1}-X_t^{S_1}{Z}_i^{I_0}};N_2=\frac{B_X{Z}_i^{I_0}-B_Z{X}_i^{I_0}}{X_i^{I_0}{Z}_t^{S_1}-X_t^{S_1}{Z}_i^{I_0}}\\ B_X=X_{S_1}-X_{I_0};B_Z=Z_{S_1}-Z_{I_0}\\ {[X_i^{I_0},{Y_i}^{I_0},Z_i^{I_0}]}^T=R^{I_0}{[x_i,y_i,f]}^T;{[X_t^{S_1},{{Y_t}^{S_1},Z}_t^{S_1}]}^T=R^{S_1}{[{x^{\&prime;}}_t^1,{y^{\&prime;}}_t^1,f]}^T\end{array}\right.,$ for by image I ₀, image S ₁the rotation matrix that calculates of foreign side parallactic angle element.

Secondly, utilize formula (1), will Q be put _iproject toward remaining n-2 width pair search image, obtain search image S _j(j=2 ..., n-1) and upper candidate's same place q' _j,tpicture planimetric coordinates and utilize formula (2) to be converted into image space row number again according to image S _jcorresponding epipolar line equation (6) calculate corresponding image space line number thus obtain and main search image candidate same place q' _{1, t}n-2 candidate's same place q' on other corresponding secondary search image _j,t, and then form to be matched some q _it group n-1 candidate's same place q' _{1, t}, q' _{2, t}..., q' _j,t..., q' _{n-1, t}.

Again, first utilize the double image comprehensive matching Likelihood Computation method based on color characteristic and SIFT feature, calculate to be matched some q respectively _iwith the double image match measure of the t group n-1 candidate point on each width search image the mean value getting n-1 double image match measure is again estimated as comprehensive matching as t group n-1 candidate's same place and the many of point to be matched ${{\mathrm{\&rho;}}^m}_t=({{\mathrm{\&rho;}}^s}_{1,t}+{{\mathrm{\&rho;}}^s}_{2,t}+...+{\mathrm{\&rho;}}_{j,t}^s+...+{{\mathrm{\&rho;}}^s}_{n-1,t})/(n-1).$

Finally, the maximum max{ ρ of many picture Synthetic Measurements is got ^m _t| t=1,2 ..., E_l ¹-S_l ¹that group candidate point of+1}, as many to be matched some q obtained as matching process that object space is information constrained _in-1 corresponding image points q' on n-1 width search image ₁, q' ₂..., q' _n-1.

For reference images I ₀on to be matched some q _iwith search image S _jon certain candidate's same place q' _j,t, the double image comprehensive matching Likelihood Computation method based on grey color characteristic and SIFT feature between 2 is as follows:

First, respectively with a q _iwith a q' _j,tcentered by, at image I ₀go up and image S _jon get imaging window W, W that two sizes are N × (N generally gets odd number) ', calculate the gray scale correlation coefficient ρ in red, green, blue three gray channel between two windows respectively ^r, ρ ^g, ρ ^b(computing formula such as formula (8), for red channel), and the mean value getting three gray scale related coefficients is as the similarity measure ρ based on color characteristic ^c=(ρ ^r+ ρ ^g+ ρ ^b)/3.

${\mathrm{\&rho;}}^R=\frac{\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{f}_R(i,j){f^{\&prime;}}_R(i,j)-\frac{\left(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{f}_R(i,j)\right)\left(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{f^{\&prime;}}_R(i,j)\right)}{N^2}}{\sqrt{(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left(f_R(i,j)\right)}^2-\frac{{\left(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{f}_R(i,j)\right)}^2}{N^2})(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left({f^{\&prime;}}_R(i,j)\right)}^2-\frac{{\left(\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{N}{\mathrm{\&Sigma;}}}{f^{\&prime;}}_R(i,j)\right)}^2}{N^2})}}---\left(8\right)$

In formula, f _r(i, j), f' _r(i, j) represent respectively imaging window W, W ' in the gray-scale value of the i-th row jth row pixel in red channel.

Secondly, respectively with q _iwith a q' _j,tcentered by, at image I ₀go up and image S _jon get two sizes be 16 × 16 imaging window W, W ', each element value g (i in window, j), g'(i, j) get mean value g (i, the j)=(f of the gray-scale value of respective pixel in red green, blue three gray channel respectively _r(i, j)+f _g(i, j)+f _b(i, j))/3, g'(i, j)=(f' _r(i, j)+f' _g(i, j)+f' _b(i, j))/3, recycling SIFT feature describing method calculates SIFT feature vector V, V of 128 dimensions of point to be matched and Searching point ' (multiscale space not relating to the image in SIFT feature computing method builds, extreme point detects and characteristic point position such as to determine at the step), and calculates the SIFT feature similarity ρ between two imaging windows by formula (9) ^s;

${\mathrm{\&rho;}}^S=\frac{\underset{k=1}{\overset{128}{\mathrm{\&Sigma;}}}(V_k\&times;{V^{,}}_k)}{\sqrt{\underset{k=1}{\overset{128}{\mathrm{\&Sigma;}}}{V}_k^2}\&times;\sqrt{\underset{i=1}{\overset{128}{\mathrm{\&Sigma;}}}{V^{,}}_k^2}}---\left(9\right)$

Finally, the similarity measure ρ based on color characteristic is got ^cwith based on SIFT feature similarity ρ ^smean value, estimate ρ as the double image comprehensive matching between point to be matched and candidate's same place ^s=(ρ ^c+ ρ ^s)/2.

3rd step: the object coordinates consistency checking of many picture matching results.

(1) for each to be matched some q in reference images _i, and n-1 corresponding image points q' on each width search image ₁, q' ₂..., q' _n-1, according to the picture planimetric coordinates of n picture point on each self imaging and the elements of exterior orientation of n width image, utilize many picture bundle adjustments, calculate to be matched some q _iobject space three-dimensional coordinate (the X of corresponding culture point _i, Y _i, Z _i);

Many is with the picture point on every width image as bundle adjustment, the light beam that object point and photo centre are formed accordingly is the elementary cell of adjustment, with the imaging model of image (as the collinearity condition equation model of aviation image, the rational function model of space flight image) as the basic equation of adjustment, in every width image what treat that solution asks the unknown picture point of coordinate is observed reading as planimetric coordinates, list file names with the error equation of all picpointed coordinates participated on coupling image, the three dimensional space coordinate of six elements of exterior orientation of every width image and picture point to be asked object point is accordingly resolved with least square method.When the elements of exterior orientation of image is known, then bundle adjustment is used for resolving the object space three-dimensional coordinate of point to be located.For reference images I ₀on certain to be matched some q _i, depending on it as planimetric coordinates (x _i, y _i) be observed reading, at its elements of exterior orientation be when known, following picture planimetric coordinates (x can be listed _i, y _i) with corresponding object space three-dimensional coordinate (X _i, Y _i, Z _i) between error equation (co-colouration effect for aviation image):

$\left\{\begin{array}{c}{v}_{x_i}^{I_0}=-a_{11}^{I_0}{\mathrm{dX}}_i-a_{12}^{I_0}{\mathrm{dY}}_i-a_{13}^{I_0}{\mathrm{dZ}}_i-(x_i-x_i^0)\\ v_{y_i}^{I_0}=-a_{21}^{I_0}{\mathrm{dX}}_i-a_{22}^{I_0}{\mathrm{dY}}_i-a_{23}^{I_0}{\mathrm{dZ}}_i-(y_i-y_i^0)\end{array}\right.---\left(10\right)$

In formula,

$\left\{\begin{array}{c}{a}_{11}^{I_0}=(a_1^{I_0}f+a_3^{I_0}{x}_i)/\stackrel{\&OverBar;}{Z},a_{12}^{I_0}=(b_1^{I_0}f+b_3^{I_0}{x}_i)/\stackrel{\&OverBar;}{Z},a_{13}^{I_0}=(c_1^{I_0}f+c_3^{I_0}{x}_i)/\stackrel{\&OverBar;}{Z}\\ a_{21}^{I_0}=(a_2^{I_0}f+a_3^{I_0}{x}_i)/\stackrel{\&OverBar;}{Z},a_{22}^{I_0}=(b_2^{I_0}f+b_3^{I_0}{x}_i)/\stackrel{\&OverBar;}{Z},a_{23}^{I_0}=(c_2^{I_0}f+c_3^{I_0}{x}_i)/\stackrel{\&OverBar;}{Z}\\ \stackrel{\&OverBar;}{Z}=a_3^{I_0}(X_i^0-X_{I_0})+b_3^{I_0}({Y_i}^0-Y_{I_0})+c_3^{I_0}(Z_i^0-Z_{I_0})\end{array}\right.$

deng the implication of character and the same in formula (1); (X _i ⁰, Y _i ⁰, Z _i ⁰) be culture point three-dimensional coordinate (X to be asked _i, Y _i, Z _i) approximate value (can according to a q _iwith arbitrary width search image S _jupper same place q' _j, utilize the double image forward intersection method shown in formula (7) to calculate), (dX _i, dY _i, dZ _i) be the adjustment correction number of approximate three-dimensional coordinate, (x _i ⁰, y _i ⁰) be approximate three-dimensional coordinate is brought into the some q that formula (1) calculates _iat I ₀on approximate picture planimetric coordinates.

Therefore, for be matched some q _in-1 corresponding image points set { q' on n-1 width search image ₁, q' ₂..., q' _n-1, the error equation of 2 (n-1) individual same place can be listed by formula (10), add to be matched some q _i2 error equations in reference images, obtain 2n error equation of n picture point altogether, then many error equations resolved as bundle adjustment three-dimensional coordinate are as follows:

V＝BdX-L (11)

In formula, $V={\left[\begin{array}{ccccccc}{v}_{x_i}^{I_0}& v_{y_i}^{I_0}& v_{x^{\&prime;}}^{S_1}& v_{y^{\&prime;}}^{S_1}& K& v_{x^{\&prime;}}^{S_{n-1}}& v_{y^{\&prime;}}^{S_{n-1}}\end{array}\right]}^T,$ dX＝[dX _i dY _i dZ _i] ^T

$B=\left[\begin{array}{ccc}-a_{11}^{I_0}& -a_{12}^{I_0}& -a_{13}^{I_0}\\ -a_{21}^{I_0}& -a_{22}^{I_0}& -a_{23}^{I_0}\\ -a_{11}^{S_1}& -a_{12}^{S_1}& -a_{13}^{S_1}\\ -a_{21}^{S_1}& -a_{22}^{S_1}& -a_{23}^{S_1}\\ M& M& M\\ -a_{11}^{S_k}& -a_{12}^{S_k}& -s_{13}^{S_k}\\ -a_{21}^{S_k}& -a_{22}^{S_k}& -a_{23}^{S_k}\end{array}\right],L=\left[\begin{array}{c}{x}_i-x_i^0\\ y_i-y_i^0\\ {x^{\&prime;}}_{S_1}-{x^{\&prime;}}_{S_1}^0\\ {y^{\&prime;}}_{S_1}-{y^{\&prime;}}_{S_1}^0\\ M\\ {x^{\&prime;}}_{S_{n-1}}-{x^{\&prime;}}_{S_{n-1}}^0\\ {y^{\&prime;}}_{S_{n-1}}-{y^{\&prime;}}_{S_{n-1}}^0\end{array}\right]$

Utilize the principle of least square, the correction that can solve approximate coordinates according to error equation is: dX=(B ^tb) ^-1(B ^tl); And then obtain to be matched some q _ithe three-dimensional coordinate of object point is accordingly: X _i=X _i ⁰+ dX _i, Y _i=Y _i ⁰+ dY _i, Z _i=Z _i ⁰+ dZ _i.

(2) each to be matched some q is calculated _icorresponding object space planimetric coordinates (X _i, Y _i) with input object space primitive coordinate (X ₀, Y ₀) between poor absolute value delta X _i, Δ Y _iif, Δ X _iwith Δ Y _iall be less than or equal to the threshold value of setting, then think to be matched some p _imanyly meet object coordinates coherence request, by q as matching result _iand corresponding image points returns, by Z as one group of image space matching result of ground primitive _ian object space height value as ground primitive returns; If Δ X _ior Δ Y _ibe greater than the threshold value of setting, then think to be matched some q _iand many picture matching results are invalid, abandon this group result.

Through the multi-view images coupling based on object space and how as bundle adjustment, ground object space primitive (X can be obtained ₀, Y ₀) m candidate to be matched some q of correspondence in reference images _ithree-dimensional coordinate (X _i, Y _i, Z _i).But this m point to be matched differs, and to establish a capital be the picture point that ground primitive is corresponding.If primitive place, ground only has a height value (culture point as ground in plane), then it also only has a corresponding picture point in reference images; If there is multiple height value (as the vertically atural object such as building facade, electric pole) at primitive place, ground, then it has multiple corresponding picture point in reference images.Therefore, certainly some points are had to be other topocentric picture points in m the candidate point to be matched of ground primitive, this can utilize between the object space three-dimensional coordinate of candidate's point to be matched and the object coordinates of ground primitive whether meet coherence request, confirm which point to be matched is correct, concrete object coordinates consistency check method is as follows:

First, i-th candidate to be matched some q is calculated _iobject coordinates and ground primitive object coordinates between poor absolute value delta X _i=abs (X _i– X ₀), Δ Y _i=abs (Y _i– Y ₀).Secondly, by Δ X _i, Δ Y _icompare with given poor threshold value T (the ground space resolution of an image pixel can be set to), if | Δ X _i| <=T and | Δ Y _i| <=T, then think Z _ibe correct, put it in the set of returned object space height value, and by be matched some q _iand n-1 corresponding image points q' on each width search image ₁, q' ₂..., q' _n-1returned image space results set is put into as one group of image space matching result; If | Δ X _i| >T or | Δ Y _i| >T, then think Z _i, and the image space matching result (q of correspondence _iand n-1 corresponding image points q' ₁, q' ₂..., q' _n-1) be all wrong, abandon this group result.

Therefore, the novel multi-view images pedal line path matching method returning multiple height value of the present invention is under the constraint of object coordinates consistency checking, defer to the principle of " put qualitying before quantity ", one or more object space height value can be returned, also may a height value also not return.Accompanying drawing 4 illustrates the correspondence position of an actual measurement ground object space primitive on three width aviation images, this ground primitive is a point of building facade, its object coordinates is (397211.3781,3555317.2179), wherein, leftmost image is reference images, and image that is middle and the right is search image, image shows corresponding image point position with crosshair shape, and the straight line of the horizontal direction on middle and the right image represents corresponding epipolar line; Figure 5 shows and the image space result that traditional multi-view images pedal line path matching method returns is carried out to three width aviation images in accompanying drawing 4; Accompanying drawing 6 illustrates and carries out to three width aviation images in accompanying drawing 4 the image space result that returns based on the present invention's novel multi-view images pedal line path matching method.In Fig. 5, Fig. 6, with adding the image space matching result that the shape representation image of crosshair returns in circle, in addition, 19.213 seconds classic method used times, 1.809 seconds novel method used times of the present invention.

As can be seen from the results, traditional pedal line path matching method of looking not only consuming time longer more, an image space matching result and object space height value can only be returned, and this rreturn value is also not necessarily correct, as can be seen from the result of Fig. 5, the image space matching result that classic method returns obviously departs from the actual position of ground primitive on image.And NEW Pb vertical line path matching method of the present invention not only consuming time be only 1/10th of classic method, and, multiple height value can be returned, and owing to carrying out object coordinates consistency checking to matching result, a lot of incorrect matching result can be eliminated.

Claims (4)

1. can return the novel multi-view images pedal line path matching method of multiple height value, it is characterized in that, comprise the steps:

Step one, utilizes the imaging model of image, according to multi-view images outer orientation parameter, and the planimetric coordinates (X of the ground object space primitive of input ₀, Y ₀), the object space maximum elevation Z in region to be matched _maxwith minimum elevation Z _min, determine the image space ranks number of m the point to be matched that object space primitive is corresponding in reference images;

Step 2, to the m in reference images point to be matched, carries out the multi-view images coupling that object space is information constrained, to obtain the ranks number of the corresponding image points of each point to be matched on other search image;

Step 3, according to the same place result on multi-view images, utilizes many picture bundle adjustments, calculates m the object space three-dimensional coordinate (X corresponding to point to be matched _i, Y _i, Z _i), i=1,2 ..., m, then the coordinate (X calculating each point to be matched _i, Y _i) with input object space primitive coordinate (X ₀, Y ₀) between poor absolute value delta X _i, Δ Y _i, and according to Δ X _i, Δ Y _ithe threshold value whether being less than setting decides the height value of returned object space primitive.

2. the novel multi-view images pedal line path matching method returning multiple height value according to claim 1, it is characterized in that, the detailed process of described step one is:

(1) input has the n width multi-view images of elements of exterior orientation, comprises Aeronautics and Astronautics or up short image, and the planimetric coordinates (X of ground object space primitive ₀, Y ₀), the object space maximum elevation Z in region to be matched _maxwith minimum elevation Z _min;

(2) utilize the imaging model of image, according to elements of exterior orientation, the planimetric coordinates of ground object space primitive, the minimum and maximum object space elevation of input of reference images, calculate the image space ranks number of m the candidate to be matched point of object space primitive in reference images.

3. the novel multi-view images pedal line path matching method returning multiple height value according to claim 2, it is characterized in that, the detailed process of described step 2 is:

(1) for each to be matched some q in reference images _i, i=1,2 ..., m, utilizes it as the minimum and maximum object space elevation of planimetric coordinates, input, according to the imaging model of image, calculates to be matched some q _iobject point is at the object space three-dimensional coordinate of the highs and lows of the object space region of search accordingly;

(2) according to video imaging model, by the highs and lows of the object space region of search toward n-1 width search image S ₁..., S _j..., S _n-1on project, obtain the image space ranks number of search core line of the same name two end points to be matched some same place place on each width search image;

(3) according to the image space ranks number of two end points of corresponding epipolar line on each search image, the straight-line equation h of picture plane coker line is determined ^{' j}=k ^{' j}× l ^{' j}+ b ^{' j}, and the interval range [S_l of candidate point range l' of the same name ^j, E_l ^j], j=1,2 ..., n-1, and with burst length E_l ^j-S_l ^ja maximum width search image is main search image, and remaining n-2 width image is secondary search image;

(4) in the image space region of search main search image, take out each pixel one by one as the current candidate corresponding image points on this image, first utilize double image forward intersection method, calculate to be matched some q _iwith the object space three-dimensional coordinate of this candidate point institute intersection culture point, and this three-dimensional coordinate is projected on remaining n-2 width pair search image, obtain n-2 candidate's corresponding image points on secondary search image, again together with the current candidate corresponding image points on main search image, thus obtain often group n-1 candidate's corresponding image points on each pair search image; Then, the comprehensive matching Likelihood Computation method based on RGB color characteristic and SIFT feature is recycled, to be matched some q on Calculation Basis image _ioften organize many pictures comprehensive matching similarity of n-1 candidate's same place, and with that group candidate point corresponding to maximum similarity, as many n-1 corresponding image points q' obtained as matching process that object space is information constrained ₁, q' ₂..., q' _n-1.

4., according to the novel multi-view images pedal line path matching method returning multiple height value one of claims 1 to 3 Suo Shu, it is characterized in that, the detailed process of described step 3 is:

(1) for each to be matched some q in reference images _i, and n-1 corresponding image points q' on each width search image ₁, q' ₂..., q' _n-1, according to the picture planimetric coordinates of n picture point on each self imaging and the elements of exterior orientation of n width image, utilize many picture bundle adjustments, calculate to be matched some q _iobject space three-dimensional coordinate (the X of corresponding culture point _i, Y _i, Z _i);

(2) each to be matched some q is calculated _icorresponding object space planimetric coordinates (X _i, Y _i) with input object space primitive coordinate (X ₀, Y ₀) between poor absolute value delta X _i, Δ Y _iif, Δ X _iwith Δ Y _iall be less than or equal to the threshold value of setting, then think to be matched some q _imanyly meet object coordinates coherence request, by q as matching result _iand corresponding image points returns, by Z as one group of image space matching result of ground primitive _ian object space height value as ground primitive returns; If Δ X _ior Δ Y _ibe greater than the threshold value of setting, then think to be matched some q _iand many picture matching results are invalid, abandon this group result.