CN108052986B - Least square matching method based on multiple channels - Google Patents

Least square matching method based on multiple channels Download PDF

Info

Publication number
CN108052986B
CN108052986B CN201711475582.9A CN201711475582A CN108052986B CN 108052986 B CN108052986 B CN 108052986B CN 201711475582 A CN201711475582 A CN 201711475582A CN 108052986 B CN108052986 B CN 108052986B
Authority
CN
China
Prior art keywords
image
matching
point
window
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201711475582.9A
Other languages
Chinese (zh)
Other versions
CN108052986A (en
Inventor
杨楠
张延波
杨艳玲
邵云明
马杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Heilongjiang Longfei Aviation Photography Co ltd
Original Assignee
Heilongjiang Longfei Aviation Photography Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Heilongjiang Longfei Aviation Photography Co ltd filed Critical Heilongjiang Longfei Aviation Photography Co ltd
Priority to CN201711475582.9A priority Critical patent/CN108052986B/en
Publication of CN108052986A publication Critical patent/CN108052986A/en
Application granted granted Critical
Publication of CN108052986B publication Critical patent/CN108052986B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Other Investigation Or Analysis Of Materials By Electrical Means (AREA)

Abstract

The invention relates to a least square matching method based on multiple channels, which is mainly used for matching optimization of a multi-channel remote sensing image and comprises the following steps: firstly, selecting an image window with a fixed size by taking an initial coarse matching point as a center; and then, establishing an error equation based on the brightness values of the corresponding pixels in the image window in the corresponding channels, performing iterative optimization of radiation distortion and geometric distortion correction parameters, and searching for the optimal matching point. Compared with the original matching method based on the gray level image, the method can reduce the interference of color difference on the matching result and improve the accuracy and stability of remote sensing image matching.

Description

Least square matching method based on multiple channels
Technical Field
The invention belongs to the technical field of remote sensing image processing image matching, and relates to a least square matching method based on multiple channels.
Background
Least square Image Matching (Least square Image Matching) is a high-precision Matching algorithm based on Image gray scale proposed by professor Ackermann of Stuttgart university in Germany, and because the Matching precision of the algorithm can reach the sub-pixel level, the Least square Image Matching is widely applied to the fields of digital ground model generation, space-three encryption, face recognition and the like. To date, least squares matching and its associated algorithms remain one of the most effective solutions to the field of photogrammetry and remote sensing to handle many matching tasks.
As a gray-based image matching algorithm, the input data of the least-squares matching algorithm is a gray image. With the improvement of photography and information storage and communication technologies, color and multiband image acquisition becomes easier. In the process of multi-channel image matching, the traditional matching method is to convert multi-channel images into gray images for matching, and the advantages of multi-channel information of color and multi-band images in the process of image matching are not reflected.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides the method for introducing the multi-channel color information into least square matching so as to achieve the purposes of improving the image matching precision and improving the stability of the matching method.
The technical scheme of the invention is a least square matching method based on multiple channels, which comprises the following steps:
step 1, initially matching forward intersection to obtain an object space three-dimensional point coordinate P (X)c,Yc,Zc)；
Step 2, establishing an error equation based on the matching points and the gray values of the pixels in the surrounding image window in different channels, searching the optimal matching point position through iterative optimization of radiation distortion and geometric distortion correction values, and realizing the following steps,
step 2.1, the image point (x) to be matched on the reference image is used0c,y0c) Taking an image window with the size of mu x mu pixels from the reference image, and calculating the pixel coordinate (x) of each pixel in the window0,y0) Wherein mu is a preset value;
step 2.2, with three-dimensional points (X)c,Yc,Zc) Establishing an object space bin P for the center;
step 2.3, calculating object point coordinates (X, Y, Z) of the mu X mu image points in the image window acquired in the step 2.2 projected on the object square surface element;
step 2.4, calculating the coordinates (X) of image points of the object space bin object points (X, Y, Z) projected on the search imagei,yi) Wherein i is the ith search image;
step 2.5, respectively calculating corresponding image points (x)0,y0) And (x)i,yi) Gray value g at each channel0(x0j,y0j)、gi(x ij,y ij) Wherein j is the jth wave band channel of the image;
step 2.6, establishing an error equation according to collinearity equation constraint and a least square matching principle;
step 2.7, calculating an error equation, and solving a radiation distortion and geometric distortion correction value;
step 2.8, stopping iteration and taking the center points of the reference image and each search image window as the best matching point when all correction values are smaller than a preset correction value threshold; otherwise, calculating a correction value to correct the distortion parameter, repeating the steps from 2.2 to 2.7, calculating the iteration times by the value +1, and judging that the matching fails when the iteration times are greater than the preset time threshold and all the correction values are smaller than the preset correction value threshold.
The technical scheme of the invention is that each channel in the multi-channel image is regarded as a gray image to respectively establish a least square error equation and obtain an image matching result. The information content of the image is multiplied compared with the traditional gray-scale image. According to the method, the multi-channel component of the image is introduced into the least square matching error equation, so that the interference of color difference on the matching result can be reduced, the contribution of brightness difference of different channels of neighborhood pixels is increased, and the matching precision and stability are improved.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention.
FIG. 2 is a diagram illustrating the contrast of each channel component of a gray scale image and a color RGB image according to an embodiment of the present invention.
Detailed Description
For a better understanding of the technical solutions of the present invention, the following detailed description of the present invention is made with reference to the accompanying drawings.
The technical scheme of the invention can adopt a computer software technology to realize an automatic operation process. The embodiment of the invention matches two images of RGB colored unmanned aerial vehicles in urban areas, and referring to FIG. 1, the flow of the embodiment of the invention comprises the following steps:
step 1, initially matching forward intersection to obtain object space three-dimensional point coordinates.
In the embodiment, image related parameters are firstly input, and a three-dimensional point coordinate P (X) of a matching point corresponding to an object space is calculated by adopting a forward intersection method (formula (1))c,Yc,Zc)。
Figure BDA0001532762300000021
Wherein (X)s0,Ys0,Zs0) And (X)si,Ysi,Zsi) The coordinates, lambda, of the projection centers of the reference image and the search image in the object space coordinate system0And λiIs a projection coefficient, (u)0c,v0c,w0c) And (u)ic,v ic,w ic) Are respectively the image points (x) to be matched0c,y0c) And (x)ic,yic) Image space auxiliary coordinates.
Step 2, establishing error equations based on the matching points and the gray values of the pixels in the surrounding image window in different channels, and performing iterative optimization of radiation distortion and geometric distortion correction values, wherein the threshold is set to be 10 in the embodiment-5I.e. when the correction value of each parameter solved is less than 10-5And stopping iteration, otherwise, correcting the relevant parameters by adopting the obtained correction value and continuing the iteration. And recording the iteration number, and when the iteration number exceeds 300 times, failing the iteration and invalidating the point.
The matching optimization process in step 2 of the embodiment is specifically realized by the following steps:
step 2.1, with reference to the image point (x)0c,y0c) Taking out mu x mu pixels at intervals of one pixel size as a matching image window, and recording as R (x)0,y0)；
Step 2.2, with three-dimensional point P (X)c,Yc,Zc) Establishing an initial object side bin P for the normal vector for the starting point (a, b, c), the object side bin equation being:
a(X-Xc)+b(Y-Yc)+c(Z-Zc)＝0 (2)
here, the normal vector (a, b, c) can be represented by its direction angle (α, β) (the initial value is set to (0 °,90 °) in the embodiment), that is:
Figure BDA0001532762300000031
step 2.3, the object space bin equation and the projection equation (formula (4)) are combined, and the image window R (x) is calculated0,y0) The coordinates of the object point projected by the image point in (1) onto the object side element are marked as P (X, Y, Z);
Figure BDA0001532762300000032
wherein λ is a projection coefficient; (u)0,v0,w0) Is R (x)0,y0) Coordinates of the image point in (1) in the auxiliary coordinate system of the image space;
step 2.4, calculating coordinates of image points of the object points P (X, Y, Z) on the object side bin projected on the search image according to a collinear condition equation (formula (5)) to obtain a search image window S (X)i,yi)；
Figure BDA0001532762300000033
Wherein f isiSearching the image main distance;
Figure BDA0001532762300000034
rotating the matrix for searching the image;
step 2.5, resampling is carried out by adopting a bilinear interpolation method, and reference image windows R (x) are respectively calculated0,y0) And searching S (x) in the image windowi,yi) Pixel point (x)0,y0) And (x)i,yi) Gray value g in three channels of RGB0r(x0,y0,z0r))、gir(xi,yi,zir)，g0g(x0,y0,z0g)、gig(xi,yi,zig) And g0b(x0,y0,z0b)、gib(xi,yi,zib)；
Step 2.6, establishing a least square error equation pixel by pixel in the matched image window, wherein because the image adopted in the embodiment is an RGB true color image, three error equations can be established for each pixel:
Figure BDA0001532762300000041
wherein v isr、vg、vbRespectively projection error, h0i、h1iTo the radiation distortion factor, dh0iAnd dh1iAre respectively h0iAnd h1iCorrection value of (dx)i,dyi) A geometric distortion correction value;
step 2.7, substituting the formulas (1) to (5) into a formula (6), calculating an error equation, and solving a radiation distortion and geometric distortion correction value;
step 2.8, the distortion parameters are optimized in an iterative manner, a threshold value of the correction value and a threshold value of times can be preset according to the precision requirement during specific implementation, and the threshold value of the correction value is set to be 10 in the embodiment-5I.e. when all correction values are less than 10-5Stopping iteration, if the correction value does not meet the threshold requirement, repeating the iteration times of +1 and the steps of 2.2-2.7;
and recording the result after matching as a txt document, wherein the document content comprises: the method comprises the steps of initial matching point three-dimensional point coordinates, corrected three-dimensional point coordinates, reference image point coordinates, search image point coordinates, matching iteration times, whether iteration is successful or not and the like.
The effectiveness of the invention was verified by simulation experiments as follows:
the simulation experiment adopts two city area true color unmanned aerial vehicle images with overlapped courses, accurate inner and outer orientation elements are obtained, the overlapping degree between the adjacent images is more than 80%, and the size of the standard image is 3888 multiplied by 2592. The experimental image window size is 7 × 7, 9 × 9, 11 × 11, 13 × 13, 15 × 15, 17 × 17, 19 × 19 pixels.
Evaluation indexes are as follows: and evaluating the matching success rate and the matching precision.
(1) Matching success rate: and statistical comparison adopts a multichannel-based least square matching method and a traditional least square matching method to respectively optimize the successful matching probability of 58015 initial image pairs which are uniformly distributed in the overlapped part of the two images.
(2) Matching precision: and counting and comparing the relative elevation precision of the three-dimensional points obtained by front intersection of the matching points, wherein the smaller the error in the relative elevation is, the higher the matching precision is.
And (3) simulation results: according to the evaluation indexes, the experimental data of the simulation experiment are shown in the following table:
table 1: statistical table of experimental results
Figure BDA0001532762300000042
Figure BDA0001532762300000051
As can be seen from the experimental results in table 1, as the image window is gradually increased, the higher the matching success rate is, the smaller the error is. This is because the more pixels that participate in matching, the more the amount of information available in the image window, and thus the better the matching accuracy and stability. The invention utilizes the RGB three color channels of the experimental image to obtain the information amount which is three times of that of the gray image, and the matching result is superior to the traditional least square matching method based on the gray image.
In summary, the invention has the following advantages:
(1) and constructing a least square error equation by using the multi-channel information of the image to increase the contribution of the brightness difference of different channels of the neighborhood pixels and reduce the interference of the color difference on the matching result.
(2) The method enlarges the information content of the pixels in the image matching window and improves the accuracy and the stability of image matching.
The foregoing is a more detailed description of the invention in connection with remotely sensed images and preferred embodiments and it is not intended that the practice of the invention be limited to these descriptions. It will be understood by those skilled in the art that various changes in detail may be effected therein without departing from the scope of the invention as defined by the appended claims.

Claims (2)

1. A least square matching method based on multiple channels is characterized by comprising the following steps:
step 1, knowing the exterior orientation elements of the image, collectingCalculating the coordinate P (X) of the three-dimensional point of the object space corresponding to the initial matching point by using the formula (1) of the forward intersection methodc,Yc,Zc)；
Figure FDA0002927619990000011
Wherein (X)s0,Ys0,Zs0) And (X)si,Ysi,Zsi) The coordinates, lambda, of the projection centers of the reference image and the search image in the object space coordinate system0And λiIs a projection coefficient, (u)0c,v0c,w0c) And (u)ic,vic,wic) Are respectively the image points (x) to be matched0c,y0c) And (x)ic,yic) (ii) image space auxiliary coordinates;
step 2, establishing an error equation based on the matching points and the gray values of the pixels in the surrounding image window in different channels, searching the optimal matching point position through iterative optimization of radiation distortion and geometric distortion correction values, and realizing the following steps,
step 2.1, the image point (x) to be matched on the reference image is used0c,y0c) Taking a pixel as a sampling interval, taking an image window with the size of mu x mu pixels from the reference image, and recording the image window as R (x)0,y0)；
Step 2.2, with three-dimensional point P (X)c,Yc,Zc) Establishing an initial object side bin P for a normal vector for a starting point (a, b, c), wherein direction angles (alpha, beta) of the normal vector (a, b, c) are preset values;
step 2.3, calculate the image window R (x)0,y0) The coordinates of the object point projected by each pixel on the object side surface element are marked as P (X, Y, Z);
step 2.4, calculating coordinates of image points of the object points P (X, Y, Z) on the object side bin projected on the search image according to a collinear condition equation to obtain a search image window S (X)i,yi)；
Step 2.5, calculating reference image windows R (x) respectively0,y0) And searching the image window S (x)i,yi) Middle corresponding pixel point (x)0,y0) And (x)i,yi) Gray value g in each band channel0k(x0,y0,z0k) And g) andik(xi,yi,zik) Wherein k is 1,2, …, n is channel number, n is image channel number;
step 2.6, establishing a least square error equation by pixel and wave band channels in the matched image window as follows,
Figure FDA0002927619990000021
where v is the projection error, subscript 1<k<n is a channel number, vkFor projection error, h0i、h1iTo the radiation distortion factor, dh0iAnd dh1iAre respectively h0iAnd h1iCorrection value of (dx)i,dyi) A geometric distortion correction value;
step 2.7, calculating an error equation, and solving a radiation distortion and geometric distortion correction value;
step 2.8, carrying out iterative optimization on distortion parameters, and stopping iteration when all correction values are smaller than a preset correction value threshold value, and taking the central points of the reference image and each search image window as optimal matching points; otherwise, calculating a correction value to correct the distortion parameter, repeating the steps from 2.2 to 2.7, calculating the iteration times by the value +1, and judging that the matching fails when the iteration times are greater than the preset time threshold and all the correction values are smaller than the preset correction value threshold.
2. The multi-pass based least squares matching method of claim 1, wherein: in step 2.5, a bilinear interpolation formula (3) is adopted to respectively calculate the gray value of the pixel in the matched image window in each wave band channel
gk(x，y，zk)＝(1-i)(1-j)·gk(x-i，y-j，zk)+j(1-i)·gk(x-i，y-j+1，zk)+i(1-j)·gk(x-i+1，y-j，zk)+ij·gk(x-i+1，y-j+1，zk) (3)
Wherein, gk(x,y,zk) Is the gray value of the image point (x, y) on the k-th channel, i and j are the fractional parts of the horizontal and vertical coordinates x and y of the image point, respectively.
CN201711475582.9A 2017-12-29 2017-12-29 Least square matching method based on multiple channels Expired - Fee Related CN108052986B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711475582.9A CN108052986B (en) 2017-12-29 2017-12-29 Least square matching method based on multiple channels

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711475582.9A CN108052986B (en) 2017-12-29 2017-12-29 Least square matching method based on multiple channels

Publications (2)

Publication Number Publication Date
CN108052986A CN108052986A (en) 2018-05-18
CN108052986B true CN108052986B (en) 2021-05-18

Family

ID=62129366

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711475582.9A Expired - Fee Related CN108052986B (en) 2017-12-29 2017-12-29 Least square matching method based on multiple channels

Country Status (1)

Country Link
CN (1) CN108052986B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112288030A (en) * 2020-11-10 2021-01-29 深圳市数字城市工程研究中心 Least square matching method based on adaptive object space element
CN116597184B (en) * 2023-07-11 2023-09-22 中国人民解放军63921部队 Least square image matching method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101604018A (en) * 2009-07-24 2009-12-16 中国测绘科学研究院 High-definition remote sensing image data disposal route and system thereof
CN103218783A (en) * 2013-04-17 2013-07-24 国家测绘地理信息局卫星测绘应用中心 Fast geometric correction method for satellite remote sensing image and based on control point image database
CN103473782A (en) * 2013-09-24 2013-12-25 测绘遥感信息工程国家重点实验室深圳研发中心 Least-square matching method based on object space vertical double-face element
CN105825543A (en) * 2016-03-31 2016-08-03 武汉大学 Multi-view dense point cloud generation method and system based on low-altitude remote sensing images

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101604018A (en) * 2009-07-24 2009-12-16 中国测绘科学研究院 High-definition remote sensing image data disposal route and system thereof
CN103218783A (en) * 2013-04-17 2013-07-24 国家测绘地理信息局卫星测绘应用中心 Fast geometric correction method for satellite remote sensing image and based on control point image database
CN103473782A (en) * 2013-09-24 2013-12-25 测绘遥感信息工程国家重点实验室深圳研发中心 Least-square matching method based on object space vertical double-face element
CN105825543A (en) * 2016-03-31 2016-08-03 武汉大学 Multi-view dense point cloud generation method and system based on low-altitude remote sensing images

Also Published As

Publication number Publication date
CN108052986A (en) 2018-05-18

Similar Documents

Publication Publication Date Title
WO2020151212A1 (en) Calibration method for extrinsic camera parameter of on-board camera system, and calibration system
CN102750537B (en) Automatic registering method of high accuracy images
CN109523585B (en) Multisource remote sensing image feature matching method based on direction phase consistency
US8577139B2 (en) Method of orthoimage color correction using multiple aerial images
CN110660105B (en) Calibration parameter optimization method and device for panoramic looking-around system
CN107492080B (en) Calibration-free convenient monocular head image radial distortion correction method
CN111508029A (en) Satellite-borne segmented linear array CCD optical camera overall geometric calibration method and system
CN108759788B (en) Unmanned aerial vehicle image positioning and attitude determining method and unmanned aerial vehicle
CN107330927B (en) Airborne visible light image positioning method
CN112270698B (en) Non-rigid geometric registration method based on nearest curved surface
CN108052986B (en) Least square matching method based on multiple channels
CN104346833A (en) Vehicle restructing algorithm based on monocular vision
CN110738693B (en) Multi-angle image registration method for ground-based imaging radar
CN111583330B (en) Multi-scale space-time Markov remote sensing image sub-pixel positioning method and system
CN110660099B (en) Rational function model fitting method for remote sensing image processing based on neural network
CN115049535A (en) Method for obtaining effective area of fisheye lens and finely correcting image
CN108917722B (en) Vegetation coverage degree calculation method and device
CN110322476B (en) Target tracking method for improving STC and SURF feature joint optimization
CN117011561A (en) Image matching optimization method and system based on geometric constraint and convolutional neural network
CN113592744B (en) Geometric fine correction method suitable for high-resolution remote sensing image
CN113379648B (en) High-resolution seven-number and resource three-number stereoscopic image joint adjustment method
CN105184327A (en) Vertex trisection strategy-based remote sensing image feature point matching method
CN107705244A (en) A kind of edge fit correcting method suitable for big several remote sensing images of region
CN115330619A (en) Local geometric fine correction method suitable for high-resolution remote sensing image
CN114565653A (en) Heterogeneous remote sensing image matching method with rotation change and scale difference

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20210518

Termination date: 20211229