CN116109848A - SAR image feature matching method and device with unchanged radiation geometry - Google Patents
SAR image feature matching method and device with unchanged radiation geometry Download PDFInfo
- Publication number
- CN116109848A CN116109848A CN202211733718.2A CN202211733718A CN116109848A CN 116109848 A CN116109848 A CN 116109848A CN 202211733718 A CN202211733718 A CN 202211733718A CN 116109848 A CN116109848 A CN 116109848A
- Authority
- CN
- China
- Prior art keywords
- sar image
- rpc
- point
- radiation
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005855 radiation Effects 0.000 title claims abstract description 109
- 238000000034 method Methods 0.000 title claims abstract description 66
- 230000009466 transformation Effects 0.000 claims abstract description 20
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 5
- ARPUHYJMCVWYCZ-UHFFFAOYSA-N ciprofloxacin hydrochloride hydrate Chemical compound O.Cl.C12=CC(N3CCNCC3)=C(F)C=C2C(=O)C(C(=O)O)=CN1C1CC1 ARPUHYJMCVWYCZ-UHFFFAOYSA-N 0.000 claims description 4
- 238000011524 similarity measure Methods 0.000 claims description 4
- 238000001228 spectrum Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 2
- 238000000638 solvent extraction Methods 0.000 claims 1
- 238000003384 imaging method Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 101100173587 Schizosaccharomyces pombe (strain 972 / ATCC 24843) fft3 gene Proteins 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/753—Transform-based matching, e.g. Hough transform
Abstract
The disclosure provides a SAR image feature matching method and device with unchanged radiation geometry, wherein the method comprises the following steps: acquiring radiation consistent features of the first SAR image and radiation consistent features of the second SAR image and salient points in the corresponding images; determining a point to be matched and an RPC consistent direction template according to the coordinates of the salient points, the RPC image positioning parameters of the first SAR image and the RPC image positioning parameters of the second SAR image; according to the RPC consistent direction template, a first matching point corresponding to the salient point in the second SAR image and a coordinate offset of the first matching point are obtained, and the geometric error of the RPC image positioning parameter of the second SAR image is corrected; re-acquiring a second matching point corresponding to the salient point and a second matching point coordinate offset in the second SAR image based on the corrected RPC image positioning parameter of the second SAR image; and carrying out fitting treatment on the salient points and the second matching points to obtain an affine transformation model and matching point coordinates meeting the affine transformation model.
Description
Technical Field
The disclosure relates to the technical field of SAR image processing, in particular to a SAR image feature matching method and device with unchanged radiation geometry.
Background
Image matching, which aims at aligning two or more images acquired at different times, in different modes, or under different imaging conditions, is a prerequisite for many remote sensing applications, such as image fusion, change detection, three-dimensional reconstruction. SAR (Synthetic Aperture Radar ) images have serious speckle noise, overlay mask, perspective shrinkage and other geometric deformations in the images due to side-looking coherent imaging mechanisms. The existing SAR image matching method mainly comprises a feature-based method, a region-based method and a deep learning-based method.
The feature-based method finds feature correspondence across images by manually designing matching features and matching strategies. However, since the size of the SAR image is usually large, searching for the corresponding point in the whole image is very time-consuming, and the accuracy of the search is low. Whereas the region-based approach matches the intensity template through a similarity measure, the intensity template is sensitive to radiation differences and cannot handle geometric deformations.
Disclosure of Invention
The present disclosure aims to solve, at least to some extent, one of the technical problems in the related art.
For this reason, a first aspect of the present disclosure proposes a method for matching SAR image features with unchanged radiation geometry, including:
acquiring a radiation consistency characteristic of a first SAR image and a radiation consistency characteristic of a second SAR image based on a sparse ratio characteristic detector, and acquiring salient points in the first SAR image based on the radiation consistency characteristic of the first SAR image;
determining a point to be matched in the second SAR image and an RPC consistent direction template according to the coordinates of the salient points, the RPC image positioning parameters of the first SAR image and the RPC image positioning parameters of the second SAR image;
acquiring a first matching point and a first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the point to be matched, the radiation consistent feature of the first SAR image, the radiation consistent feature of the second SAR image and the RPC consistent direction template;
correcting the geometric error of the RPC image positioning parameter of the second SAR image according to the first matching point coordinate offset, and re-determining the point to be matched and the RPC consistent direction template based on the corrected RPC image positioning parameter of the second SAR image, the coordinate of the salient point and the RPC image positioning parameter of the first SAR image, and acquiring the second matching point corresponding to the salient point and the second matching point coordinate offset in the second SAR image;
and carrying out fitting processing on the salient points and the second matching points to obtain an affine transformation model and matching point coordinates meeting the affine transformation model.
A second aspect of the present disclosure proposes a device for matching SAR image features with unchanged radiation geometry, comprising:
the first acquisition module is used for acquiring the radiation consistency characteristic of the first SAR image and the radiation consistency characteristic of the second SAR image based on the sparse ratio characteristic detector, and acquiring the salient points in the first SAR image based on the radiation consistency characteristic of the first SAR image;
the determining module is used for determining a point to be matched in the second SAR image and an RPC consistent direction template according to the coordinates of the salient points, the RPC image positioning parameters of the first SAR image and the RPC image positioning parameters of the second SAR image;
the first matching module is used for acquiring a first matching point and a first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the point to be matched, the radiation consistent characteristic of the first SAR image, the radiation consistent characteristic of the second SAR image and the RPC consistent direction template;
the second matching module is used for correcting the geometric error of the RPC image positioning parameter of the second SAR image according to the coordinate offset of the first matching point, and re-determining the point to be matched and the RPC consistent direction template based on the corrected RPC image positioning parameter of the second SAR image, the coordinate of the salient point and the RPC image positioning parameter of the first SAR image, and acquiring the coordinate offset of a second matching point and a second matching point corresponding to the salient point in the second SAR image;
and the second acquisition module is used for carrying out fitting processing on the salient points and the second matching points so as to obtain an affine transformation model and matching point coordinates meeting the affine transformation model.
A third aspect of the present disclosure proposes an electronic device comprising: a processor; a memory for storing the processor-executable instructions; wherein the instructions are executable by the processor to enable the processor to perform the method of the first aspect.
A fourth aspect of the present disclosure proposes a non-transitory computer readable storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the method of the preceding first aspect.
According to the SAR image feature matching method with unchanged radiation geometry, the radiation and geometric properties of the SAR images are considered, the radiation consistent features of the first SAR image and the second SAR image are obtained, matching invariance under different imaging conditions is guaranteed, and the SAR image matching method is applicable to SAR image matching under different imaging conditions. By constructing the RPC consistent direction template, the matching windows of the first SAR image and the second SAR image have more common areas, so that a more accurate and robust matching result is obtained. In addition, the geometric errors among the images are corrected by the aid of the double-stage matching algorithm, so that the number of matching points can be increased while matching accuracy is ensured, uniformly distributed matching points with a large number and high accuracy are obtained to the maximum extent, and the process is simple and easy to realize in an engineering mode.
Additional aspects and advantages of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.
Drawings
The foregoing and/or additional aspects and advantages of the present disclosure will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a flow chart of a method for matching SAR image features with unchanged radiation geometry according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a method for acquiring a radiation consistent feature of a first SAR image according to an embodiment of the present disclosure;
fig. 3 is a flowchart of a method for obtaining a first matching point corresponding to a salient point and a coordinate offset of the first matching point in a second SAR image according to an embodiment of the present disclosure;
fig. 4 is a block diagram of a SAR image feature matching device with unchanged radiation geometry according to an embodiment of the present disclosure;
fig. 5 is a block diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present disclosure and are not to be construed as limiting the present disclosure.
The disclosure provides a SAR image feature matching method and device with unchanged radiation geometry. Specifically, the following describes a method and an apparatus for matching SAR image features with unchanged radiation geometry according to an embodiment of the present disclosure with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method for matching SAR image features with unchanged radiation geometry according to an embodiment of the present disclosure. It should be noted that the method for matching SAR image features with unchanged radiation geometry may be applied to the device for matching SAR image features with unchanged radiation geometry according to the embodiments of the present disclosure, and the device for matching SAR image features with unchanged radiation geometry may be configured on an electronic device. As shown in fig. 1, the method for matching the characteristics of the SAR image with unchanged radiation geometry comprises the following steps:
It should be noted that the first SAR image may be understood as a reference SAR image, and the second SAR image may be understood as a SAR image to be matched.
It should also be noted that in some embodiments of the present disclosure, the sparse ratio feature detector is composed of a multi-directional multi-scale multiplicative weighting filter and a deep sparse convolution, while feature construction is performed on the SAR image to construct features with radiation consistency and high singularities.
Alternatively, taking the first SAR image as an example, in some embodiments of the present disclosure, the radiation consistent feature of the first SAR image may be uniformly segmented. The radiation consistent features of the first SAR image are uniformly segmented, for example, according to a block size of 1024 x 1024. And extracting FAST corner points from the radiation consistent features in each block, and taking T FAST corner points with maximum response as salient points in the first SAR image. Wherein T is a positive integer.
As an example, assuming that the cumulative consistent eigenvalue of a certain block to be processed is set to Ip, a preset threshold t is set, and a discretized Bresenham circle with a radius equal to 3 pixels centered on the point is considered, and there are 16 pixels on the boundary of the circle, if there are n consecutive points on the circle with the size of 16 pixels, the eigenvalues of the points are both larger than ip+t or smaller than Ip-t, then the point P is a corner point. The value of n is used as FAST corner response, assuming that the decision threshold is set to 9. And selecting 100 FAST corner points with maximum corner response in the block as a salient point set of the block. Alternatively, the corner extraction of the plurality of blocks may be accelerated by CPU multi-core parallel processing.
As a possible implementation manner, the longitude and latitude coordinates corresponding to the salient point may be solved by back projection according to the coordinates of the salient point, the RPC image positioning parameter of the first SAR image, and the DEM (digital elevation model), as expressed in formula (1).
lat，lon＝RPC 1-forward (x，y，h) (1)
Wherein lat, lon are latitude and longitude, respectively, (x, y) are salient point coordinates, h is altitude, RPC 1 Fitting positioning parameters for polynomials of the first SAR image, RPC 1-forward The RPC for the first SAR image is forward transformed (from image pixel coordinates to latitude and longitude). According to the latitude and longitude projected into the second SAR image, the coordinates (x) of the point to be matched on the first SAR image can be obtained 0 ，y 0 ) As expressed in equation (2).
x 0 ，y 0 ＝RPC 2-backward (lat，lon，h) (2)
By setting different elevation values (h-h 0 ，h+h 0 ) And RPC image positioning parameters of the SAR image, and solving the image space coordinate (x) corresponding to the longitude and latitude coordinate in the SAR image by orthographic projection 1 ，y 1 )，(x 2 ，y 2 ) As expressed by formula (3) and formula (4). The corresponding matching point of the salient point in the SAR image should be theoretically located at (x) 1 ，y 1 )，(x 2 ，y 2 ) Therefore, compared with the traditional square template, the RPC consistent direction template can acquire a larger common area so as to obtain a more accurate and robust matching result.
x 1 ，y 1 ＝RPC 2-backward (lat，lon，h-h 0 ) (3)
x 2 ，y 2 ＝RPC 2-backward (lat，lon，h+h 0 ) (4)
Wherein, RPC 2 Fitting positioning parameters for polynomials of the second SAR image, RPC 2-backward The RPC of the SAR image is transformed in reverse (from longitude and latitude to image coordinates).
According to the image space coordinates (x 1 ，y 1 )，(x 2 ，y 2 ) The direction β of the RPC consistent direction template is determined as expressed in equation (5). And further determining an RPC consistent direction template according to the obtained direction and the preset size.
β＝atan(y 2 -y 1 /x 2 -x 1 ) (5)
And step 103, acquiring a first matching point and a first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the point to be matched, the radiation consistent characteristic of the first SAR image, the radiation consistent characteristic of the second SAR image and the RPC consistent direction template.
It should be noted that the point to be matched is obtained by projecting longitude and latitude into the SAR image. Because the RPC image positioning parameters of the SAR image have deviation, a certain initial positioning error exists in the point to be matched, and therefore the coordinate offset of the first matching point needs to be acquired through the RPC consistent direction template to eliminate the error. In some embodiments of the present disclosure, the salient points and the points to be matched may be used as centers, and the windows to be matched in the first SAR image and the second SAR image may be selected according to the RPC consistent direction template direction β and a preset size. As an example, the preset size may be 200×50. And carrying out high-dimensional feature correlation calculation on the radiation consistent features in the windows to be matched in the first SAR image and the second SAR image, and obtaining the coordinate offset of the first matching point corresponding to the salient point in the second SAR image. And determining the first matching point according to the first matching point coordinate offset and the salient point coordinate.
And 104, correcting the geometric error of the RPC image positioning parameter of the first SAR image according to the coordinate offset of the first matching point, and re-determining a point to be matched and an RPC consistent direction template based on the corrected RPC image positioning parameter of the second SAR image, the coordinate of the salient point and the RPC image positioning parameter of the first SAR image, and acquiring a second matching point corresponding to the salient point and the coordinate offset of the second matching point in the second SAR image.
In some embodiments of the present disclosure, an average offset may be obtained from all first matching point coordinate offsets (Δx and Δy), as expressed in equation (6).
Wherein n is the number of first matching points.
And correcting the geometric error of the RPC image positioning parameter of the second SAR image according to the average offset, such as the expression of the formula (7) and the formula (8).
RPC′ 2-xoffset ＝RPC 2-xoffset +Dx (7)
RPC′ 2-yoffset ＝RPC 2-yoffset +Dy (8)
Wherein, RPC 2-xoffset And RPC 2-yoffset For the compensation coefficient of the image coordinates in the RPC image positioning parameter of the second SAR image, RPC' 1-xoffset With RPC' 1-yoffset And positioning parameters for the RPC image of the corrected first SAR image. And after correcting the RPC image positioning parameters of the second SAR image, repeating the step 102, and re-determining the points to be matched, the RPC consistent direction template and the second matching points corresponding to the salient points in the second SAR image.
And 105, fitting the salient points and the second matching points to obtain an affine transformation model and matching point coordinates meeting the affine transformation model.
As can be seen from step 104, a plurality of point pairs (each point pair including one salient point and a second matching point corresponding to the salient point) are obtained, as expressed in formula (9).
Where Nc is the number of pairs of points, (x) i ，y i ) As the i-th salient point of the image,
is the second matching point corresponding to the i-th salient point.
In some embodiments of the present disclosure, affine transformation models may be iteratively estimated among the acquired plurality of point pairs, and mismatching points are culled. As one example, the number of iterations, the error threshold, and the maximum point set number may be preset. 4 point pairs are arbitrarily selected from the plurality of point pairs, an affine transformation model is fitted using least squares approximation, and the residual of each of the remaining point pairs is calculated. Points when the residual is less than a given threshold are recorded as an intermediate set. If the number of intermediate sets is greater than the maximum number of point sets, the maximum number of point sets is then set as the number of intermediate sets. And continuing the iterative process until the maximum point set number reaches the maximum or the iterative times reaches the upper limit, taking the affine transformation model corresponding to the maximum point set as the affine transformation model finally output, and outputting all matching points conforming to the final affine transformation model.
According to the SAR image feature matching method with unchanged radiation geometry, the radiation and geometric properties of the SAR images are considered, the radiation consistent features of the first SAR image and the second SAR image are obtained, matching invariance under different imaging conditions is guaranteed, and the SAR image matching method is applicable to SAR image matching under different imaging conditions. By constructing the RPC consistent direction template, the matching windows of the first SAR image and the second SAR image have more common areas, so that a more accurate and robust matching result is obtained. In addition, the geometric errors among the images are corrected by the aid of the double-stage matching algorithm, so that the number of matching points can be increased while matching accuracy is ensured, uniformly distributed matching points with a large number and high accuracy are obtained to the maximum extent, and the process is simple and easy to realize in an engineering mode.
Fig. 2 is a flowchart of a method for acquiring a radiation consistent feature of a first SAR image according to an embodiment of the present disclosure. As shown in fig. 2, the method may include, but is not limited to, the following steps.
In some embodiments of the present disclosure, convolution operations in each scale and each direction may be performed by using a multi-direction multi-scale multiplicative weighting filter and a SAR image, to obtain multi-scale multi-direction high-dimensional feature values of each pixel, and then calculate to obtain feature amplitudes of each pixel.
As one example, a convolution window of a multi-directional multi-scale multiplicative weighting filter may be constructed first, as expressed by equations (10) and (11).
Wherein (x, y) is the image coordinates, F sθ-1 (x, y) is the upper part of the convolution window, F sθ-2 (x, y) is the lower part of the convolution window, s is the scale, θ is the direction parameter, and w is the frequency parameter. The convolution window and the first SAR image are subjected to convolution operation to obtain convolution responses such as the expression (12) and the expression (13).
FT sθ-1 (x，y)＝F sθ-1 (x，y)*I(x，y) (12)
FT sθ-2 (x，y)＝F sθ-2 (x，y)*I(x，y) (13)
Wherein I (x, y) is the magnitude of the image, which is a convolution operation. The convolution response is used for carrying out logarithmic ratio operation to obtain pixel-by-pixel characteristics of the first SAR image in the theta direction, and the first pixel-by-pixel high-dimensional characteristic SAR of the first SAR image can be obtained by accumulating multi-directional dimensions sθ (x, y) as represented by formula (14).
SAR sθ (x，y)＝log(FT sθ-1 (x，y)/FT sθ-2 -(x，y)) (14)
In some embodiments of the present disclosure, the first pixel-by-pixel high-dimensional feature may be used as an initial feature, and a pre-designed depth sparse convolution may be input to obtain a second pixel-by-pixel high-dimensional feature SARF of the first SAR image sθ (x, y) as expressed by equation (15) to enhance the first pixel-by-pixel high-dimensional feature to improve the singularity and correlation of the high-dimensional feature.
SARF sθ (x，y)＝SAR sθ (x，y)*conv(K，d) (15)
The depth sparse convolution uses Gaussian kernel as convolution weight, wherein K is the size of the convolution template, d is the multiple of sparse sampling, and conv (K, d) has a dimension of 1 XN θ ×K×K，N θ Is the number of directions. As an example, a typical convolution template of k=3, d=1 is
Correspondingly, a typical convolution template of k=3, d=2 is
It can be seen that sparse convolution can expand the receptive field by adjusting different sampling multiples, thereby avoiding pooling operations to reduce the resolution of the features.
And 203, performing multidimensional normalization processing on the second pixel-by-pixel high-dimensional characteristic of the first SAR image to obtain a radiation consistent characteristic of the first SAR image.
In some embodiments of the present disclosure, the dimension is N θ XH X W, where N θ The number of directions, H, W is the size of the processed image. In order to improve the radiation invariance of the features, the features are normalized in three dimensions simultaneously, so as to obtain the radiation consistent feature SARFN of the first SAR image, as shown in a formula (16), so as to eliminate the radiation difference.
SARFN＝Normalize NHW (SARF sθ (x,y)) (16)
It should be noted that, the method for acquiring the radiation consistent feature of the second SAR image is the same as the method for acquiring the radiation consistent feature of the first SAR image, and will not be described herein. By implementing the embodiment of the disclosure, the sparse ratio feature detector is adopted to acquire the radiation consistent feature of the first SAR image and the radiation consistent feature of the second SAR image, so that the SAR image features with unchanged radiation under different imaging conditions can be acquired, and the matching invariance under different imaging conditions is ensured. Therefore, the SAR image matching method is applicable to SAR image matching of different imaging conditions, and provides a basis for subsequent feature matching.
Fig. 3 is a flowchart of a method for obtaining a first matching point corresponding to a salient point and a coordinate offset of the first matching point in a second SAR image according to an embodiment of the present disclosure. As shown in fig. 3, the method may include, but is not limited to, the following steps.
That is, the salient point in the first SAR image is taken as the center, and the first window to be matched in the first SAR image is determined according to the direction of the RPC consistent direction template and the preset size.
In some embodiments of the present disclosure, for the calculation of the translational offset in the spatial domain, it may be converted to a phase change in the frequency domain based on the nature of the fourier transform. Based on the characteristics, three-dimensional Fourier transformation can be performed on the radiation consistent features in the first window to be matched and the radiation consistent features in the second window to be matched, and a normalized cross power spectrum is obtained, as represented by a formula (17).
The SARFN1 is a radiation consistent characteristic in a first window to be matched, the SARFN2 is a radiation consistent characteristic in a second window to be matched, the fft3 is three-dimensional Fourier transform, the x is conjugate complex signal, and the Deltax and Deltay are matched coordinate offsets to be calculated. And carrying out inverse Fourier transform on the normalized cross power spectrum based on a correlation method of a spatial domain to obtain a similarity measure, as represented by a formula (18).
Q(x,y)＝fft3 -1 Q(u,v,w)＝δ(x-Δx,y-Δy) (18)
Where (x, y) is the coordinates of the salient point. Ideally, i.e., a dirac delta function centered on Δx, Δy. Therefore, the translational displacement can obtain the coordinate offsets deltax and deltay of the first matching points according to the similarity measurement peak value, and determine the first matching point corresponding to the salient point in the second SAR image according to the coordinate offsets of the first matching points. Thus, the first matching point coordinates are determined according to the first matching point coordinate offset and the salient point coordinates.
It should be noted that, in step 104, based on the corrected RPC image positioning parameter of the second SAR image, the coordinates of the salient point, and the RPC image positioning parameter of the first SAR image, the method for redefining the point to be matched and the RPC consistent direction template, and obtaining the second matching point and the coordinate offset of the second matching point corresponding to the salient point in the second SAR image is the same as the method for obtaining the first matching point and the coordinate offset of the first matching point in the embodiment, which is not described herein. By implementing the embodiment of the disclosure, the first matching point corresponding to the salient point and the coordinate offset of the first matching point can be determined in the second SAR image according to the RPC consistent direction template, so that the geometric error of the RPC image positioning parameter of the first SAR image is corrected, and the matching accuracy is improved.
The SAR image feature matching method with unchanged radiation geometry improves matching accuracy and obtains more matching points, and the processing effect of the method is verified through a processing example of actual data. And the test adopts a bunching mode SAR image and an ultra-fine stripe mode SAR image which are high-resolution three-number different imaging modes and different imaging angles to carry out a matching test. The sampling interval of the bunching mode SAR image is 1 meter, and the sampling interval of the ultra-fine stripe mode SAR image is 3 meters. The quantity and the precision of the matching points obtained by the SAR image feature matching method with unchanged radiation geometry are far higher than those obtained by the SIFT feature matching method.
The SIFT feature matching method utilizes the DoG operator to extract SAR image features, but because the noise model of the SAR is multiplicative noise, the DoG operator is easily affected by noise and radiation change, and particularly, a plurality of inconsistent features are easily extracted in a homogeneous region. In addition, the SIFT feature matching method cannot well process geometric errors such as perspective shrinkage, overlay masking and the like caused by SAR image imaging characteristics, so that the final matching result accuracy is affected. The method disclosed by the invention starts from the radiation and geometric properties of SAR, firstly, the consistent characteristics of the radiation invariance are extracted from SAR images, the invariance of matching under different imaging conditions is ensured, then, the accurate matching points are obtained by utilizing the high-dimensional characteristic correlation, the geometric errors in the SAR images are corrected, and finally, the higher matching precision and the larger matching quantity are obtained by matching again.
Fig. 4 is a block diagram of a device for matching SAR image features with unchanged radiation geometry according to an embodiment of the present disclosure. As shown in fig. 4, the SAR image feature matching device with unchanged radiation geometry includes a first acquisition module 401, a determination module 402, a first matching module 403, a second matching module 404, and a second acquisition module 405. Wherein, the liquid crystal display device comprises a liquid crystal display device,
the first obtaining module 401 is configured to obtain a radiation consistent feature of the first SAR image and a radiation consistent feature of the second SAR image based on the sparse ratio feature detector, and obtain a salient point in the first SAR image based on the radiation consistent feature of the first SAR image.
The determining module 402 is configured to determine a point to be matched in the second SAR image and an RPC consistent direction template according to coordinates of the salient point, the RPC image positioning parameter of the first SAR image, and the RPC image positioning parameter of the second SAR image.
The first matching module 403 is configured to obtain a first matching point and a first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the point to be matched, the radiation consistent feature of the first SAR image, the radiation consistent feature of the second SAR image, and the RPC consistent direction template.
The second matching module 404 is configured to correct a geometric error of an RPC image positioning parameter of the second SAR image according to the first matching point coordinate offset, and redetermine a point to be matched and an RPC consistent direction template based on the corrected RPC image positioning parameter of the second SAR image, the coordinate of the salient point, and the RPC image positioning parameter of the first SAR image, and obtain a second matching point corresponding to the salient point and a second matching point coordinate offset in the second SAR image.
The second obtaining module 405 is configured to perform a fitting process on the salient point and the second matching point, so as to obtain an affine transformation model and matching point coordinates that satisfy the affine transformation model.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
According to the SAR image feature matching device with unchanged radiation geometry, the radiation consistency features of the first SAR image and the second SAR image are obtained in consideration of the radiation and the geometric properties of the SAR image, so that the matching invariance under different imaging conditions is ensured, and the SAR image feature matching device can be suitable for SAR image matching under different imaging conditions. By constructing the RPC consistent direction template, the matching windows of the first SAR image and the second SAR image have more common areas, so that a more accurate and robust matching result is obtained. In addition, the geometric errors among the images are corrected by the aid of the double-stage matching algorithm, so that the number of matching points can be increased while matching accuracy is ensured, uniformly distributed matching points with a large number and high accuracy are obtained to the maximum extent, and the process is simple and easy to realize in an engineering mode.
In order to implement the above embodiments, the present disclosure further provides an electronic device. Fig. 5 is a block diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 5, the electronic device 500 may include a memory 501, a processor 502, and a computer program 503 stored in the memory 501 and executable on the processor 502, and when the processor 502 executes the computer program 503, the method for matching SAR image features with unchanged radiation geometry according to any of the above embodiments of the present disclosure is performed.
In order to implement the above embodiments, the disclosure further proposes a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method for matching SAR image features with unchanged radiation geometry according to any of the above embodiments of the disclosure.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present disclosure in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present disclosure.
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
Furthermore, each functional unit in the embodiments of the present disclosure may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. Although embodiments of the present disclosure have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the present disclosure, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the present disclosure.
Claims (10)
1. The SAR image feature matching method with unchanged radiation geometry is characterized by comprising the following steps:
acquiring a radiation consistency characteristic of a first SAR image and a radiation consistency characteristic of a second SAR image based on a sparse ratio characteristic detector, and acquiring salient points in the first SAR image based on the radiation consistency characteristic of the first SAR image;
determining a point to be matched in the second SAR image and an RPC consistent direction template according to the coordinates of the salient points, the RPC image positioning parameters of the first SAR image and the RPC image positioning parameters of the second SAR image;
acquiring a first matching point and a first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the point to be matched, the radiation consistent feature of the first SAR image, the radiation consistent feature of the second SAR image and the RPC consistent direction template;
correcting the geometric error of the RPC image positioning parameter of the second SAR image according to the first matching point coordinate offset, and re-determining the point to be matched and the RPC consistent direction template based on the corrected RPC image positioning parameter of the second SAR image, the coordinate of the salient point and the RPC image positioning parameter of the first SAR image, and acquiring the second matching point corresponding to the salient point and the second matching point coordinate offset in the second SAR image;
and carrying out fitting processing on the salient points and the second matching points to obtain an affine transformation model and matching point coordinates meeting the affine transformation model.
2. The method of claim 1, wherein acquiring the radiance-consistent feature of the first SAR image based on the sparse ratio feature detector comprises:
performing convolution operation based on a multidirectional multiscale multiplicative weighting filter on the first SAR image to obtain a first pixel-by-pixel high-dimensional characteristic of the first SAR image;
performing deep sparse convolution on the first pixel-by-pixel high-dimensional characteristic of the first SAR image to obtain a second pixel-by-pixel high-dimensional characteristic of the first SAR image;
and carrying out multidimensional normalization processing on the second pixel-by-pixel high-dimensional characteristic of the first SAR image to obtain a radiation consistent characteristic of the first SAR image.
3. The method of claim 2, wherein the formula for deep sparse convolution of pixel-wise high-dimensional features of the first SAR image is expressed as follows:
SARF sθ (x,y)＝AR sθ (x,y)*onv(K,d)
the depth sparse convolution adopts Gaussian kernel as convolution weight, wherein K is the size of a convolution template, d is the multiple of sparse sampling, and the dimension of conv (K, d) is 1 XN θ ×K×K，N θ Is the number of directions.
4. The method of claim 1, wherein the acquiring salient points in the first SAR image based on the consistent radiation characteristics of the first SAR image comprises:
uniformly partitioning the radiation consistent characteristics of the first SAR image;
extracting T FAST corner points with maximum response from each block to serve as salient points in the first SAR image; wherein T is a positive integer.
5. The method of claim 1, wherein the determining the matching point and the RPC consistent direction template in the second SAR image according to the coordinates of the salient point, the RPC image positioning parameter of the first SAR image, and the RPC image positioning parameter of the second SAR image comprises:
determining longitude and latitude coordinates corresponding to the salient points based on the coordinates of the salient points and RPC image positioning parameters of the first SAR image;
determining a point to be matched in the second SAR image according to the longitude and latitude coordinates and the RPC image positioning parameters of the second SAR image;
acquiring image space coordinates corresponding to longitude and latitude coordinates in the second SAR image according to the longitude and latitude coordinates corresponding to the salient points and RPC image positioning parameters of the second SAR image;
and determining the direction of the RPC consistent direction template according to the image side coordinates, and determining the RPC consistent direction template according to a preset size and the direction.
6. The method of claim 1, wherein the obtaining the first matching point and the first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the to-be-matched point, the radiation coincidence feature of the first SAR image, the radiation coincidence feature of the second SAR image, and the RPC coincidence direction template comprises:
taking a salient point in the first SAR image as a center, determining a first window to be matched in the first SAR image according to the RPC consistent direction template, and acquiring a radiation consistent characteristic in the first window to be matched according to the radiation consistent characteristic of the first SAR image;
taking a point to be matched in the second SAR image as a center, determining a second window to be matched in the second SAR image according to the RPC consistent direction template, and acquiring a radiation consistent characteristic in the second window to be matched according to the radiation consistent characteristic of the second SAR image;
and acquiring a first matching point corresponding to the salient point in the second SAR image and the coordinate offset of the first matching point according to the consistent radiation characteristic in the first window to be matched and the consistent radiation characteristic in the second window to be matched.
7. The method of claim 6, wherein the obtaining the first matching point and the first matching point coordinate offset corresponding to the salient point in the second SAR image according to the consistent radiation characteristic in the first window to be matched and the consistent radiation characteristic in the second window to be matched comprises:
performing three-dimensional Fourier transform on the radiation consistent features in the first window to be matched and the radiation consistent features in the second window to be matched to obtain a normalized cross power spectrum;
performing inverse Fourier transform on the normalized cross power spectrum to obtain similarity measure;
and acquiring the coordinate offset of the first matching point according to the peak value of the similarity measure, and determining a first matching point corresponding to the salient point in the second SAR image according to the coordinate offset of the first matching point.
8. A radiation geometry invariant SAR image feature matching device, comprising:
the first acquisition module is used for acquiring the radiation consistency characteristic of the first SAR image and the radiation consistency characteristic of the second SAR image based on the sparse ratio characteristic detector, and acquiring the salient points in the first SAR image based on the radiation consistency characteristic of the first SAR image;
the determining module is used for determining a point to be matched in the second SAR image and an RPC consistent direction template according to the coordinates of the salient points, the RPC image positioning parameters of the first SAR image and the RPC image positioning parameters of the second SAR image;
the first matching module is used for acquiring a first matching point and a first matching point coordinate offset corresponding to the salient point in the second SAR image according to the salient point, the point to be matched, the radiation consistent characteristic of the first SAR image, the radiation consistent characteristic of the second SAR image and the RPC consistent direction template;
the second matching module is used for correcting the geometric error of the RPC image positioning parameter of the second SAR image according to the coordinate offset of the first matching point, and re-determining the point to be matched and the RPC consistent direction template based on the corrected RPC image positioning parameter of the second SAR image, the coordinate of the salient point and the RPC image positioning parameter of the first SAR image, and acquiring the coordinate offset of a second matching point and a second matching point corresponding to the salient point in the second SAR image;
and the second acquisition module is used for carrying out fitting processing on the salient points and the second matching points so as to obtain an affine transformation model and matching point coordinates meeting the affine transformation model.
9. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions; wherein the instructions are executable by the processor to enable the processor to perform the method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211733718.2A CN116109848B (en) | 2022-12-30 | 2022-12-30 | SAR image feature matching method and device with unchanged radiation geometry |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211733718.2A CN116109848B (en) | 2022-12-30 | 2022-12-30 | SAR image feature matching method and device with unchanged radiation geometry |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116109848A true CN116109848A (en) | 2023-05-12 |
| CN116109848B CN116109848B (en) | 2023-11-24 |
Family
ID=86259197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211733718.2A Active CN116109848B (en) | 2022-12-30 | 2022-12-30 | SAR image feature matching method and device with unchanged radiation geometry |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116109848B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102213762A (en) * | 2011-04-12 | 2011-10-12 | 中交第二公路勘察设计研究院有限公司 | Method for automatically matching multisource space-borne SAR (Synthetic Aperture Radar) images based on RFM (Rational Function Model) |
| US9245201B1 (en) * | 2013-03-15 | 2016-01-26 | Excelis Inc. | Method and system for automatic registration of images |
| CN107341781A (en) * | 2017-07-20 | 2017-11-10 | 中国科学院电子学研究所 | Based on the SAR image correcting methods for improving the matching of phase equalization characteristic vector base map |
| CN109671110A (en) * | 2018-12-29 | 2019-04-23 | 西南交通大学 | A kind of wide Baseline Images characteristic point matching method in city of local geometry constraint |
| CN112270320A (en) * | 2020-11-16 | 2021-01-26 | 云南电网有限责任公司昆明供电局 | Power transmission line tower coordinate calibration method based on satellite image correction |
-
2022
- 2022-12-30 CN CN202211733718.2A patent/CN116109848B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102213762A (en) * | 2011-04-12 | 2011-10-12 | 中交第二公路勘察设计研究院有限公司 | Method for automatically matching multisource space-borne SAR (Synthetic Aperture Radar) images based on RFM (Rational Function Model) |
| US9245201B1 (en) * | 2013-03-15 | 2016-01-26 | Excelis Inc. | Method and system for automatic registration of images |
| CN107341781A (en) * | 2017-07-20 | 2017-11-10 | 中国科学院电子学研究所 | Based on the SAR image correcting methods for improving the matching of phase equalization characteristic vector base map |
| CN109671110A (en) * | 2018-12-29 | 2019-04-23 | 西南交通大学 | A kind of wide Baseline Images characteristic point matching method in city of local geometry constraint |
| CN112270320A (en) * | 2020-11-16 | 2021-01-26 | 云南电网有限责任公司昆明供电局 | Power transmission line tower coordinate calibration method based on satellite image correction |
Non-Patent Citations (4)
| Title |
|---|
| FAN ZHONGLI 等: "Exploiting High Geopositioning Accuracy of SAR Data to Obtain Accurate Geometric Orientation of Optical Satellite Images", 《REMOTE SENSING》, vol. 13, no. 17, pages 1 - 30 * |
| LI XIN 等: "SARPointNet: An Automated Feature Learning Framework for Spaceborne SAR Image Registration", 《IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING》, vol. 15, pages 6371 - 6381 * |
| WANG ZHENGBIN 等: "A Fast Registration Method for Optical and SAR Images Based on SRAWG Feature Description", 《REMOTE SENSING》, vol. 14, no. 19, pages 1 - 25 * |
| 樊仲藜 等: "SAR影像和光学影像梯度方向加权的快速匹配方法", 《测绘学报》, vol. 50, no. 10, pages 1390 - 1403 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116109848B (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10445616B2 (en) | Enhanced phase correlation for image registration | |
| Goshtasby | Image registration: Principles, tools and methods | |
| US11633146B2 (en) | Automated co-registration of prostate MRI data | |
| US7912321B1 (en) | Image registration with uncertainty analysis | |
| Inglada et al. | On the possibility of automatic multisensor image registration | |
| US6219462B1 (en) | Method and apparatus for performing global image alignment using any local match measure | |
| CN108051810B (en) | InSAR distributed scatterer phase optimization method | |
| US20120105868A1 (en) | Measurement device and measurement method | |
| CN109949349B (en) | Multi-mode three-dimensional image registration and fusion display method | |
| KR20140109439A (en) | Image registration method and system robust to noise | |
| Uss et al. | A precise lower bound on image subpixel registration accuracy | |
| CN108701360B (en) | Image processing system and method | |
| CN116109848B (en) | SAR image feature matching method and device with unchanged radiation geometry | |
| CN109959933B (en) | Multi-baseline circular synthetic aperture radar imaging method based on compressed sensing | |
| CN111126508A (en) | Hopc-based improved heterogeneous image matching method | |
| CN116468760A (en) | Multi-source remote sensing image registration method based on anisotropic diffusion description | |
| Sadeq | Using total probability in image template matching. | |
| Lohani et al. | An evaluation of intensity augmented ICP for terrestrial LiDAR data registration | |
| Yu et al. | SAR pixelwise registration via multiscale coherent point drift with iterative residual map minimization | |
| Uss et al. | Efficient rotation-scaling-translation parameter estimation based on the fractal image model | |
| Ye et al. | Precise disparity estimation for narrow baseline stereo based on multiscale superpixels and phase correlation | |
| Permuter et al. | Estimating the orientation of planar surfaces: Algorithms and bounds | |
| Hassen et al. | Multi-sensor image registration based-on local phase coherence | |
| Sroubek et al. | Variational Bayesian image reconstruction with an uncertainty model for measurement localization | |
| US20240029288A1 (en) | Image processing apparatus, image processing method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |