CN112734816A - Heterogeneous image registration method based on CSS-Delaunay - Google Patents
Heterogeneous image registration method based on CSS-Delaunay Download PDFInfo
- Publication number
- CN112734816A CN112734816A CN202110042319.0A CN202110042319A CN112734816A CN 112734816 A CN112734816 A CN 112734816A CN 202110042319 A CN202110042319 A CN 202110042319A CN 112734816 A CN112734816 A CN 112734816A
- Authority
- CN
- China
- Prior art keywords
- image
- point
- points
- edge
- heterogeneous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 50
- 238000001514 detection method Methods 0.000 claims abstract description 11
- 230000011218 segmentation Effects 0.000 claims abstract description 3
- 239000011159 matrix material Substances 0.000 claims description 16
- 230000000877 morphologic effect Effects 0.000 claims description 12
- 230000003287 optical effect Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 238000003708 edge detection Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 5
- 238000005259 measurement Methods 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 4
- 230000001629 suppression Effects 0.000 claims description 4
- 230000009977 dual effect Effects 0.000 claims description 2
- 238000005530 etching Methods 0.000 claims description 2
- 230000000717 retained effect Effects 0.000 claims description 2
- 230000009191 jumping Effects 0.000 claims 1
- 238000005457 optimization Methods 0.000 claims 1
- 238000001228 spectrum Methods 0.000 claims 1
- 238000013519 translation Methods 0.000 claims 1
- 238000005728 strengthening Methods 0.000 abstract 1
- 238000004088 simulation Methods 0.000 description 9
- 230000010339 dilation Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000003628 erosive effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000007797 corrosion Effects 0.000 description 4
- 238000005260 corrosion Methods 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 230000008961 swelling Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20036—Morphological image processing
Abstract
The invention discloses a heterogeneous image registration method based on CSS-Delaunay, which comprises the following steps: clear target information is obtained by using an FCM clustering segmentation algorithm, and then outline characteristics are extracted from the target information. Complete contour information of a target is extracted by using a Canny algorithm, so that stable homonymous contour points in a heterogeneous image can be extracted more favorably; secondly, combining a CSS corner detection algorithm and a Delaunay algorithm to extract valuable contour feature points, reflecting the relative position relation between the contour feature points of the target by utilizing the uniqueness of a Delaunay triangular grid, and strengthening the association between the contour feature points to obtain a final registration result.
Description
Technical Field
The invention relates to the technical field of heterogeneous image registration, in particular to a heterogeneous image registration method based on CSS-Delaunay.
Background
The image registration technology is an important component in the field of image processing, and is widely applied in military and civil fields such as projectile body positioning, aviation guidance, computer vision, mode recognition, remote sensing technology, medicine, climate and the like. The registration technology of the heterogeneous images can make up the defects of the single sensor image in the image registration technology, and is a research hotspot in the field of the current image registration technology. Due to the existence of non-linear gray scale difference between different source images, the image registration method based on gray scale information is not suitable for the field of different source image registration. The feature information is a higher-level description of the image information and can stably exist in the heterogeneous images, so that the image registration method based on the feature information is commonly used for matching among the heterogeneous images and also becomes a main direction for research in the field at home and abroad.
In order to obtain a heterogeneous image registration result with strong robustness and high accuracy, stable feature information needs to be extracted. The most basic feature information in an image is a point feature and a contour feature. The point features have wide application in homologous image matching and stereo scene matching. In the heterogeneous images, due to the different imaging characteristics of the two images, the extracted feature point sets generally have larger differences, which may cause the failure of the point feature matching algorithm. The classic SIFT (scale invariant feature transform) algorithm is sensitive to noise in the feature extraction process, and correlation among feature points in an image is ignored in the feature description process, so that the SIFT algorithm is unstable in performance in registration of a heterogeneous image. Although the time complexity of the registration algorithm is reduced by the Harris operator and the SURF (accelerated robust feature) operator, the common features of the heterogeneous image pair are still difficult to extract, so that the robustness of the heterogeneous image registration method based on the point features is not strong, and a stable registration effect is difficult to obtain.
Disclosure of Invention
Aiming at the problems of the traditional point feature algorithm in the registration of the heterogeneous images, the invention aims to provide a CSS-Delaunay-based heterogeneous image registration method, and the Canny algorithm is used for extracting complete contour information of a target, so that the stable homonymous contour points in the heterogeneous images can be extracted more conveniently; secondly, the CSS algorithm and the Delaunay algorithm are combined, the association between the contour feature points is strengthened, and the robustness of the registration of the heterogeneous images is improved.
In order to achieve the purpose, the invention is realized by adopting the following technical scheme.
The CSS-Delaunay-based heterogeneous image registration method comprises the following steps of:
step 1, acquiring an optical image and an SAR image of the same area; respectively carrying out block-based weight probability filtering (PPB) on the optical image and the SAR image to obtain a smoothed optical image and the smoothed SAR image, namely a smoothed heterogeneous image pair;
step 2, clustering and segmenting the smoothed heterogeneous image pair by adopting a fuzzy c-means clustering algorithm to obtain a corresponding target and background separated binary image pair;
step 3, performing morphological opening operation processing on the binary image pair, filling the hole of the target area of the binary image pair, and obtaining a heterogeneous binary image pair with a continuous target area;
step 4, respectively extracting the edge contour of the target area of the heterogeneous binary image pair by adopting a Canny edge detection algorithm to obtain the edge contour of the corresponding target area;
step 5, connecting the edge discontinuity points of the edge contour of the target area to form a heterogeneous binary image pair with a continuous edge contour target area; extracting contour points of a heterogeneous binary image pair with a continuous edge contour target region by adopting a curvature scale space corner detection algorithm to obtain target region contour points of each image;
and 6, constructing a Delaunay triangular grid on the contour point of the target area of each image to complete the registration of the two heterogeneous image pairs.
Compared with the prior art, the invention has the beneficial effects that:
the method comprises the steps of firstly obtaining clear target information by using an FCM (Fuzzy c-means) clustering segmentation algorithm, and then extracting contour features from the target information. Valuable characteristic points of the image are generally distributed at the edge of the image, complete contour information of a target is extracted by using a Canny algorithm, and stable homonymous contour points in a heterogeneous image can be extracted more conveniently; secondly, combining a CSS (Curvature Scale Space) corner detection algorithm with a Delaunay algorithm to extract valuable contour feature points, reflecting the relative position relationship between the contour feature points of the target by utilizing the uniqueness of a Delaunay triangular grid, and enhancing the association between the contour feature points to obtain a final registration result.
Drawings
The invention is described in further detail below with reference to the figures and specific embodiments.
FIG. 1 is a flow chart of the implementation of the method of the present invention
FIG. 2 is a graph showing the effect of swelling of the morphological treatment of an example of the present invention;
FIG. 3 is a graph showing the corrosion effect of the morphological treatment according to the embodiment of the present invention;
FIG. 4 is a heterogeneous image pair of simulation experiment data for an embodiment of the present invention; wherein (a) is an optical image and (b) is a SAR image;
FIG. 5 is a diagram of simulation result connections based on the conventional SIFT algorithm according to an embodiment of the present invention;
fig. 6 is a line graph of simulation results based on the method of the present invention according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to examples, but it will be understood by those skilled in the art that the following examples are only illustrative of the present invention and should not be construed as limiting the scope of the present invention.
Referring to fig. 1, the invention provides a heterogeneous image registration method based on CSS-Delaunay, which includes the following steps:
step 1, acquiring an optical image and an SAR image of the same region, and respectively performing block-based weighted probability filtering (PPB) on the optical image and the SAR image to obtain a smoothed optical image and the smoothed SAR image, namely a smoothed heterogeneous image pair;
the optical image is used as a reference image, and the SAR image is used as an image to be registered;
for a SAR image, it is assumed that none of the pixels in the image existCorrelation, maIs the pixel value at pixel M, defining the noise distribution present in the image as
Model in which naIs a pixel N in the real image1The pixel value of (b), then the filtering is equivalent to looking for the pair naBest estimated value of
The size of the similarity window in the PPB is generally 7 multiplied by 7; the search window is generally 21 × 21;
the PPB filtering gives the optimal image block similarity measurement under any noise distribution through probability theory derivation, so that the edge characteristics can be effectively maintained while additive noise and multiplicative noise are removed.
Step 2, clustering and segmenting the smoothed heterogeneous image pair by adopting a fuzzy c-means clustering algorithm to obtain a corresponding target and background separated binary image pair;
specifically, the FCM clustering algorithm continuously optimizes an objective function J of the intra-class distance sum of squares through an iterative methodmSo that JmAnd (4) minimizing. J. the design is a squaremIs defined as follows:
wherein X＝{x1，x2，...，xNDenotes a one-dimensional row vector, x, corresponding to the image matrixiIs the ith element of the one-dimensional row vector corresponding to the image matrix, N is the total number of elements of the vector X, c is the number of clusters, c is more than or equal to 2 and less than or equal to N, ujiDenotes xiDegree of membership to the jth cluster center, j ═ 1,2, …, c, m denotes the weight index of the function of degree of membership, vjIs the cluster center of class j, d (x)i，vj) Denotes xiTo the center of the cluster vjThe distance of (c). The algorithm comprises the following steps:
(2.1) setting values of c, m and epsilon;
(2.2) initializing membership degree matrix U at random(0)Matrix U(0)The size of (a) is c × N;
(2.3) setting the initial cycle number b to be 0;
(2.4) according to the current membership matrix U(b)C clustering centers are respectively calculated:
(2.5) calculating the membership degree matrix U of the next cycle(b+1)：
wherein ,dji＝d(xi，vj)；
(2.6) judging whether the membership degree matrix of two adjacent cycles meets max { U }(b)-U(b+1)If } < epsilon, if yes, the algorithm is ended, and U is output(b+1)Otherwise, let b be b +1, jump to step (2.4). Where ε is a very small constant.
Clustering each image through an FCM clustering algorithm, and then carrying out membership matrix U on the obtained images(b +1)And (4) performing normalization processing, wherein the pixel value of the target area is 1 and the pixel value of the background area is 0 in the image matrix. And segmenting target information and background information by clustering the images to obtain a clear binary image pair with the target and the background separated.
Step 3, performing morphological opening operation processing on the binary image pair, filling the hole of the target area of the binary image pair, and obtaining a heterogeneous binary image pair with a continuous target area;
specifically, the morphology is composed of a group of morphological algebra operators, and expansion, corrosion, opening and closing operations are basic operations of the mathematical morphology. The opening and closing operations are based on the dilation and erosion operations, and the structural elements are the core content of the dilation and erosion operations. Typically, the structural elements are much smaller than the image being processed. The shape of the structural elements is linear, rectangular, square, spherical, diamond and the like, and the structural elements in any shape can be defined by users. Constructing different structural elements to complete different image processing and obtain different processing results; morphological image processing is a process of moving a structural element in an image to be processed and performing a set operation on the structural element and the processed image.
Dilation and erosion are two basic morphological operations, and many morphological operations are based on dilation and erosion. Dilation generally is adding pixels to the object boundary in an image, so that the region expands outwards from the periphery; and the corrosion is to delete some elements of the object boundary, so that the area is reduced from the periphery inwards.
Respectively carrying out morphological opening operation of firstly expanding and then corroding on the two binary images obtained in the step 2; the method comprises the following specific steps:
the expansion operation is as follows:
the dilation of the structuring element B in the image A to be processed is noted
The operation, defined as:
wherein ,
representing the mapping of the structuring element B with respect to the origin, n represents the set intersection operation,
indicating an empty set.
The above equation shows that the process of expanding a with B is to move the mapping of the structuring element B with respect to the origin over the entire plane. Mapping of the structuring element B with respect to the origin if translated to x
If at least one non-zero element intersects A, the set of all such x points is the result of the expansion of the object A, and the effect is shown in FIG. 2:
the etching operation was as follows:
the corrosion of the structural element B to the image A to be processed is recorded as A theta B operation, and is defined as:
the above formula illustrates that the process of eroding the image a to be processed with the structural element B is to move B over the entire plane. If translated to x, the structural element B can be completely contained in the object a, i.e. the translated structural element B is superimposed on the background of the image a to be processed. The set of all such x points is the result of the erosion of object a, and the effect graph is shown in fig. 3.
Step 4, respectively extracting the edge contour of the target area of the heterogeneous binary image pair by adopting a Canny edge detection algorithm to obtain the edge contour of the corresponding target area;
specifically, the Canny edge detection algorithm is a multi-level detection algorithm, and Canny proposes three criteria of edge detection:
(1) edge detection with low error rate: the detection algorithm should accurately find as many edges in the image as possible, reducing missed and false detections as possible.
(2) Optimal positioning: the detected edge point should be located exactly at the center of the edge.
(3) Any edge in the image should be marked only once, while image noise should not create a false edge.
The specific detection steps of the Canny algorithm are as follows:
(4.1) Gaussian smoothing Filter
The process of denoising the image by using gaussian filtering is a process of performing weighted average on the image. Let f (x, y) be input data, fs(x, y) is the image after convolution smoothing. G (x, y, σ) represents a Gaussian function, σ is a scale factor, tableThe expression is as follows:
fs(x，y)＝f(x，y)*G(x，y，σ)
(4.2) gradient calculation
The Canny algorithm detects horizontal and vertical edges in an image by two operators. The gradient is calculated by convolution of the image, such as Roberts, Prewitt, Sobel, etc., the Sobel operator is generally selected to calculate the difference value between the x-axis and the y-axis of the two-dimensional image,
convolving two templates of a Sobel operator with an image to be processed respectively, wherein the two templates are difference values of a two-dimensional image obtained through calculation of the Sobel operator on an x axis and a y axis to obtain a difference value graph in the x direction and the y direction, and finally calculating a gradient amplitude G and a gradient direction theta of the point:
θ＝arctan(Gy/Gx)
wherein ,GxIs the gradient amplitude, G, of the pixel point in the x directionyThe gradient amplitude of the pixel point in the y direction is obtained;
(4.3) non-maximum suppression calculation
The non-maxima suppression algorithm for each pixel in the gradient image is: comparing the gradient amplitudes of the current pixel points along two adjacent pixel points in the positive gradient direction and the negative gradient direction respectively; and if the gradient amplitude of the current pixel point is larger than the gradient amplitudes of the other two pixel points, regarding the current pixel point as an edge, reserving the edge, and otherwise, removing the edge, traversing the whole image and obtaining a corresponding edge candidate point.
(4.4) Dual threshold calculation
And carrying out non-maximum suppression operation. Still, some pixels are generated due to noise and gray scale variation, so that the influence of these pixels on the edge detection result is eliminated. The Canny algorithm generally reduces false edges in the form of high and low thresholds.
Setting a high threshold T1And a low threshold T2If the edge candidate point is above the high threshold T1The point is retained and regarded as the strong edge of the image, if the edge candidate point is lower than the low threshold value T2Removing the solution; and if one pixel point in the eight neighborhoods is a strong edge point, the point is reserved as an edge pixel point, and if the other pixel point is not the strong edge point, the point is removed. T is1 and T2The relationship between the two is as follows:
T1＝2T2。
and (4) processing each image in the heterogeneous binary image pair with the continuous target area obtained in the step (3) through the steps (4.1) - (4.4), wherein the reserved pixel points of each image are the edge contour of the target area corresponding to the image.
Step 5, connecting the edge discontinuity points of the edge contour of the target area to form a heterogeneous binary image pair with a continuous edge contour target area; extracting contour points of a heterogeneous binary image pair with a continuous edge contour target region by adopting a curvature scale space corner detection algorithm to obtain target region contour points of each image;
firstly, filling discontinuous points in the edge profile obtained in the step 4 to enable the edge profile to be connected to form a complete profile curve; on the basis, extracting contour points by adopting a CSS (curvature scale space corner) algorithm; the CSS algorithm detects the corner points by adopting the highest scale and positions the corner points by utilizing the lower scale, and has better effects on enhancing the characteristic information and inhibiting the influence of noise on the characteristics.
The steps of the CSS corner detection algorithm are as follows:
(5.1) searching T-shaped intersection points on the complete contour curve and marking the T-shaped intersection points as T-shaped corner points;
(5.2) calculating the curvature of the point on each edge profile curve by using the highest scale;
(5.3) acquiring points of which the absolute curvature is a local maximum, and judging whether each local maximum point meets the following conditions: the absolute curvature value is larger than a global threshold value and is twice of an adjacent local minimum value, and if the absolute curvature value is larger than the global threshold value, the local maximum value point is used as a CSS candidate corner point;
(5.4) searching curvature maximum points in a neighborhood of a first-level scale lower than the local maximum points detected on a high scale, and repeating the steps to track the curvature maximum points to a lower scale until the curvature maximum points are at the lowest scale, so that the positions of the finally positioned curvature maximum points are obtained, and the positions are the real positions of the candidate corner points of the CSS;
(5.5) judging whether the distance between each T-shaped corner point and the CSS candidate corner point is smaller than eta, if so, removing the T-shaped corner point, and otherwise, keeping the T-shaped corner point; and the obtained CSS candidate corner points and the reserved T-shaped corner points are contour points of the corresponding image. η is a very small constant.
And 6, constructing a Delaunay triangular grid on the contour point of the target area of each image to complete the registration of the two heterogeneous image pairs.
Firstly, constructing a Delaunay triangular grid for the contour points of each image extracted in the step 5; constructing a Delaunay triangulation network through a triangulation network construction function carried by Matlab; selecting homonymous point pairs of the reference image and the SAR image to be matched, namely matching characteristic points in the two images;
and then, carrying out similarity measurement on the triangulation network constructed in the reference image and the image to be registered: carrying out similarity measurement on the characteristic point pairs in the two images;
assuming that Δ ABC and Δ a ' B ' C ' are two triangles which need to be determined to be similar, where point a and point a ', point B and point B ', and point C ' are corresponding point pairs respectively, the similarity of two angles ═ a (assuming that its value is a) and ═ a ' (assuming that its value is x) is:
wherein ,
ρ is aP/3, and P is generally 1/2. For a pair of triangles, the similarity of three corresponding internal angles of the pair of triangles is obtained through the formula, and then the average value of the similarity of the three internal angles of the pair of triangles is obtained as the similarity of the pair of triangles by the following formula:
I＝(Ia+Ib+Ic)/3
and finally, searching a triangle pair with the similarity larger than 0.75, and listing the triangle pair as a matching triangle pair, wherein the corresponding homonymous point pair is a registration point pair.
And registering the two images according to the obtained slice registration point, thereby completing the registration process of the different-source images.
Simulation experiment
The effectiveness of the invention is verified by simulation experiments as follows, with the simulation parameters shown in table 1:
(1) simulation conditions
And carrying out registration simulation analysis by utilizing the measured data of the SAR images and the visible light images, and comparing the measured data with the simulation result of the traditional SIFT algorithm. Table 1 shows registration data specific information.
TABLE 1 registration data specific information
Fig. 4 shows an SAR image and a visible light image in the experimental data, which shows that the quality of the two images is poor, and the SAR image is greatly interfered by noise. The traditional SIFT algorithm and the method provided by the invention are respectively adopted for registration, the result is shown in fig. 5 and fig. 6, and the comparison of fig. 5 and fig. 6 shows that the registration result cannot be obtained by the traditional SIFT algorithm in fig. 5, and the registration is invalid; in fig. 6, by using the method of the present invention, a correct registration result can be obtained.
The method can realize registration of two heterogeneous images with poor quality because the edge contour characteristic information of the images is taken as the basis, the edge contour characteristic is a stable characteristic in a heterogeneous source image pair and is less influenced by external factors; CSS corner features are extracted on the basis of the extracted edge features, and more accurate and more different source image corners can be extracted; and combining the CSS corner detection algorithm and the Delaunay algorithm, and obtaining a final registration result by using the geometric structure information of point distribution in the corner neighborhood.
Although the present invention has been described in detail in this specification with reference to specific embodiments and illustrative embodiments, it will be apparent to those skilled in the art that modifications and improvements can be made thereto based on the present invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.
Claims (7)
1. The method for registering the heterogeneous images based on the CSS-Delaunay is characterized by comprising the following steps of:
step 1, acquiring an optical image and an SAR image of the same area; respectively carrying out block-based weight probability filtering on the optical image and the SAR image to obtain a smoothed optical image and a smoothed SAR image, namely a smoothed heterologous image pair;
step 2, clustering and segmenting the smoothed heterogeneous image pair by adopting a fuzzy c-means clustering algorithm to obtain a corresponding target and background separated binary image pair;
step 3, performing morphological opening operation processing on the binary image pair, filling the hole of the target area of the binary image pair, and obtaining a heterogeneous binary image pair with a continuous target area;
step 4, respectively extracting the edge contour of the target area of the heterogeneous binary image pair by adopting a Canny edge detection algorithm to obtain the edge contour of the corresponding target area;
step 5, connecting the edge discontinuity points of the edge contour of the target area to form a heterogeneous binary image pair with a continuous edge contour target area; extracting contour points of a heterogeneous binary image pair with a continuous edge contour target region by adopting a curvature scale space corner detection algorithm to obtain target region contour points of each image;
and 6, constructing a Delaunay triangular grid on the contour point of the target area of each image to complete the registration of the two heterogeneous image pairs.
2. The CSS-Delaunay-based heterogeneous image registration method according to claim 1, wherein the clustering segmentation is performed on the smoothed heterogeneous image pairs by using a fuzzy c-means clustering algorithm, which is equivalent to: make the objective function JmAn optimization process of the minimized sum of squares of the intra-class distances;
wherein X ═ { X ═ X1,x2,...,xNDenotes a one-dimensional row vector, x, corresponding to the image matrixiIs the ith element of the one-dimensional row vector corresponding to the image matrix, N is the total number of elements of the vector X, c is the number of clusters, c is more than or equal to 2 and less than or equal to N, ujiDenotes xiDegree of membership to the jth cluster center, j ═ 1,2, …, c, m denotes the weight index of the function of degree of membership, vjIs the cluster center of class j, d (x)i,vj) Denotes xiTo the center of the cluster vjThe distance of (d);
the solving process is as follows:
(2.1) setting values of c, m and epsilon;
(2.2) initializing membership degree matrix U at random(0)Matrix U(0)The size of (a) is c × N;
(2.3) setting the initial cycle number b to be 0;
(2.4) according to the current membership matrix U(b)Respectively calculating c corresponding clustering centers:
(2.5) calculating the membership degree matrix U of the next cycle(b+1)：
wherein ,dji＝d(xi,vj)；
(2.6) judging whether the membership degree matrix of two adjacent cycles meets max { U }(b)-U(b+1)If } < epsilon, if yes, the algorithm is ended, and U is output(b+1)Otherwise, making b equal to b +1, and jumping to the step (2.4);
wherein ε is a constant.
3. The CSS-Delaunay-based heterogeneous image registration method of claim 2, wherein the morphological opening operation processing is performed on the binary image pair, specifically: performing morphological opening operation of firstly expanding and then corroding on the two binary images obtained in the step 2 respectively;
the expansion operation is specifically as follows: the process of expanding the image A to be processed by adopting the structural element B is to move the structural element B on the whole plane relative to the mapping of the origin; mapping of the structuring element B with respect to the origin if translated to x
The following conditions are met: if the X point intersects with A by at least one nonzero element, a set A' formed by all x points meeting the conditions is called a result after A is expanded;
the etching operation is specifically as follows: the process of corroding the image A' to be processed by adopting the structural element B is to move the structural element B on the whole plane; if the translation is carried out to the position x, the condition is met: the structural element B can be completely contained in the A ', namely the translated structural element B is superposed with the background of the image A ' to be processed, and the set consisting of all x points meeting the conditions is called the result after the A ' is corroded.
4. The CSS-Delaunay-based heterogeneous image registration method according to claim 1, wherein the extracting the edge contour of the target region of the heterogeneous binary image pair respectively by using a Canny edge detection algorithm specifically comprises:
(4.1) Gaussian smoothing Filter
Let f (x, y) be input data, fs(x, y) is the image after convolution smoothing, G (x, y, σ) represents a Gaussian function, then:
fs(x,y)＝f(x,y)*G(x,y,σ)
wherein σ is a scale factor;
(4.2) gradient calculation
Convolving the two templates of the Sobel operator with an image to be processed respectively to obtain a difference value graph in the x direction and the y direction, and calculating the gradient amplitude G and the gradient direction theta of the point:
θ＝arctan(Gy/Gx)
wherein ,GxIs the gradient amplitude, G, of the pixel point in the x directionyThe gradient amplitude of the pixel point in the y direction is obtained;
(4.3) non-maximum suppression calculation
Comparing the gradient amplitude of the current pixel point with two adjacent pixel points in the positive gradient direction and the negative gradient direction respectively; if the gradient amplitude of the current pixel point is larger than the gradient amplitudes of the other two pixel points, the current pixel point is regarded as an edge and is reserved, otherwise, the edge is removed, and the whole image is traversed to obtain a corresponding edge candidate point;
(4.4) Dual threshold calculation
Setting a high threshold T1And a low threshold T2If the edge candidate point is above the high threshold T1The point is retained and regarded as the strong edge of the image, if the edge candidate point is lower than the low threshold value T2Removing the solution; the edge candidate point between the high threshold and the low threshold is regarded as a weak edge, if one pixel point in the eight neighborhoods of the pixel points existing on the weak edge is a strong edge point, the edge candidate point is regarded as the weak edgeThe point is reserved as an edge pixel point, and if not, the point is removed; wherein, T1＝2T2；
And (4) processing each image in the heterogeneous binary image pair with the continuous target area obtained in the step (3) through the steps (4.1) - (4.4), wherein the reserved pixel points of each image are the edge contour of the target area corresponding to the image.
5. The CSS-Delaunay-based heterogeneous image registration method according to claim 1, wherein the extracting contour points of the heterogeneous binary image pair having the continuous edge contour target region by using a curvature scale space corner detection algorithm is specifically:
(5.1) searching T-shaped intersection points on the complete contour curve and marking the T-shaped intersection points as T-shaped corner points;
(5.2) calculating the curvature of the point on each edge profile curve by using the highest scale;
(5.3) acquiring points of which the absolute curvature is a local maximum, and judging whether each local maximum point meets the following conditions: the absolute curvature value is larger than a global threshold value and is twice of an adjacent local minimum value, and if the absolute curvature value is larger than the global threshold value, the local maximum value point is used as a CSS candidate corner point;
(5.4) searching curvature maximum points in a neighborhood of a first-level scale lower than the local maximum points detected on a high scale, and repeating the steps to track the curvature maximum points to a lower scale until the curvature maximum points are at the lowest scale, so that the positions of the finally positioned curvature maximum points are obtained, and the positions are the real positions of the candidate corner points of the CSS;
(5.5) judging whether the distance between each T-shaped corner point and the CSS candidate corner point is smaller than eta, if so, removing the T-shaped corner point, and otherwise, keeping the T-shaped corner point; the obtained CSS candidate corner points and the reserved T-shaped corner points are contour points of the corresponding image;
wherein η is a predetermined constant.
6. The CSS-Delaunay based heterologous image registration method according to claim 1, wherein the clutter spectrum restoration is performed by a compressed sensing algorithm, in particular:
firstly, constructing a Delaunay triangular grid for the contour points of each image extracted in the step 5; selecting homonymous point pairs of the two images, namely pairing the characteristic points in the two images;
then, carrying out similarity measurement on the triangulation networks constructed in the two images;
and finally, searching a triangle pair with the similarity larger than 0.75, and listing the triangle pair as a matching triangle pair, wherein the corresponding homonymous point pair is a registration point pair.
7. The CSS-Delaunay-based heterogeneous image registration method according to claim 6, wherein the similarity measurement is performed on the triangulation network constructed in the two images, specifically:
let Δ ABC and Δ a ' B ' C ' be two triangles which need to be judged whether to be similar, where point a and point a ', point B and point B ', and point C ' are corresponding point pairs respectively, then the similarity of the two angles a and a ' is:
wherein ,
p is aP/3, P is an empirical value, a is a value of < A, and x is a value of < A';
similarity I of & lt B and & lt B' is solved similarlybSimilarity I of < C > and < Cc(ii) a Then, the average value of the similarity of the three interior angles of the pair of triangles is calculated as the similarity I of the pair of triangles:
I＝(Ia+Ib+Ic)/3。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110042319.0A CN112734816B (en) | 2021-01-13 | 2021-01-13 | Heterologous image registration method based on CSS-Delaunay |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110042319.0A CN112734816B (en) | 2021-01-13 | 2021-01-13 | Heterologous image registration method based on CSS-Delaunay |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112734816A true CN112734816A (en) | 2021-04-30 |
| CN112734816B CN112734816B (en) | 2023-09-05 |
Family
ID=75591860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110042319.0A Active CN112734816B (en) | 2021-01-13 | 2021-01-13 | Heterologous image registration method based on CSS-Delaunay |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112734816B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113609898A (en) * | 2021-06-23 | 2021-11-05 | 国网山东省电力公司泗水县供电公司 | Power transmission line icing monitoring method and system based on SAR image |
| CN114943752A (en) * | 2022-05-31 | 2022-08-26 | 河南埃尔森智能科技有限公司 | Self-adaptive contour template identification and registration method based on curvature feature description |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012058902A1 (en) * | 2010-11-02 | 2012-05-10 | 中兴通讯股份有限公司 | Method and apparatus for combining panoramic image |
| US20130094750A1 (en) * | 2011-10-12 | 2013-04-18 | Tolga Tasdizen | Methods and systems for segmentation of cells for an automated differential counting system |
| CN104298990A (en) * | 2014-09-15 | 2015-01-21 | 西安电子科技大学 | Rapid graph matching and recognition method based on skeleton graphs |
| CN109409292A (en) * | 2018-10-26 | 2019-03-01 | 西安电子科技大学 | The heterologous image matching method extracted based on fining characteristic optimization |
| CN111145228A (en) * | 2019-12-23 | 2020-05-12 | 西安电子科技大学 | Heterogeneous image registration method based on local contour point and shape feature fusion |
| CN112017223A (en) * | 2020-09-11 | 2020-12-01 | 西安电子科技大学 | Heterologous image registration method based on improved SIFT-Delaunay |
-
2021
- 2021-01-13 CN CN202110042319.0A patent/CN112734816B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012058902A1 (en) * | 2010-11-02 | 2012-05-10 | 中兴通讯股份有限公司 | Method and apparatus for combining panoramic image |
| US20130094750A1 (en) * | 2011-10-12 | 2013-04-18 | Tolga Tasdizen | Methods and systems for segmentation of cells for an automated differential counting system |
| CN104298990A (en) * | 2014-09-15 | 2015-01-21 | 西安电子科技大学 | Rapid graph matching and recognition method based on skeleton graphs |
| CN109409292A (en) * | 2018-10-26 | 2019-03-01 | 西安电子科技大学 | The heterologous image matching method extracted based on fining characteristic optimization |
| CN111145228A (en) * | 2019-12-23 | 2020-05-12 | 西安电子科技大学 | Heterogeneous image registration method based on local contour point and shape feature fusion |
| CN112017223A (en) * | 2020-09-11 | 2020-12-01 | 西安电子科技大学 | Heterologous image registration method based on improved SIFT-Delaunay |
Non-Patent Citations (3)
| Title |
|---|
| 孙劲光;周勃;: "曲线约束Delaunay三角剖分及在地形构建中的应用", 计算机应用与软件, no. 12 * |
| 章为川;水鹏朗;徐国靖;: "边缘线上各向异性高斯核信息熵的角点检测", 西安电子科技大学学报, no. 04 * |
| 赵亚利;章为川;李云红;: "图像边缘轮廓自适应阈值的角点检测算法", 中国图象图形学报, no. 11 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113609898A (en) * | 2021-06-23 | 2021-11-05 | 国网山东省电力公司泗水县供电公司 | Power transmission line icing monitoring method and system based on SAR image |
| CN113609898B (en) * | 2021-06-23 | 2023-09-29 | 国网山东省电力公司泗水县供电公司 | SAR image-based power transmission line icing monitoring method and system |
| CN114943752A (en) * | 2022-05-31 | 2022-08-26 | 河南埃尔森智能科技有限公司 | Self-adaptive contour template identification and registration method based on curvature feature description |
| CN114943752B (en) * | 2022-05-31 | 2024-03-29 | 河南埃尔森智能科技有限公司 | Self-adaptive contour template identification registration method based on curvature feature description |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112734816B (en) | 2023-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110097093B (en) | Method for accurately matching heterogeneous images | |
| Tissainayagam et al. | Object tracking in image sequences using point features | |
| CN110349207B (en) | Visual positioning method in complex environment | |
| Tazir et al. | CICP: Cluster Iterative Closest Point for sparse–dense point cloud registration | |
| EP2079054B1 (en) | Detection of blobs in images | |
| CN109961506A (en) | A kind of fusion improves the local scene three-dimensional reconstruction method of Census figure | |
| CN106960449B (en) | Heterogeneous registration method based on multi-feature constraint | |
| CN112017223B (en) | Heterologous image registration method based on improved SIFT-Delaunay | |
| CN107169972B (en) | Non-cooperative target rapid contour tracking method | |
| CN110021029B (en) | Real-time dynamic registration method and storage medium suitable for RGBD-SLAM | |
| CN110084830B (en) | Video moving object detection and tracking method | |
| CN112734816A (en) | Heterogeneous image registration method based on CSS-Delaunay | |
| CN111783722B (en) | Lane line extraction method of laser point cloud and electronic equipment | |
| CN114067147B (en) | Ship target confirmation method based on local shape matching | |
| CN111145198A (en) | Non-cooperative target motion estimation method based on rapid corner detection | |
| CN117078726A (en) | Different spectrum image registration method based on edge extraction | |
| CN116429087A (en) | Visual SLAM method suitable for dynamic environment | |
| Priyadharshini et al. | Automatic registration of images with simulated rotation and translation estimation using HAIRIS | |
| CN113409334A (en) | Centroid-based structured light angle point detection method | |
| Du et al. | A high-precision vision-based mobile robot slope detection method in unknown environment | |
| Yi et al. | Automatic detection of cross-shaped targets for laser scan registration | |
| KR100540889B1 (en) | Tracking Method for The Moving Object using Basic Points | |
| Qu et al. | Contour detection based on SUSAN principle and surround suppression | |
| Zhang et al. | Adaptive segmentation of cervical smear image based on GVF Snake model | |
| Pramanik et al. | A Region-to-pixel based Multi-sensor Image Fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |