CN109448031B - Image registration method and system based on Gaussian field constraint and manifold regularization - Google Patents

Image registration method and system based on Gaussian field constraint and manifold regularization Download PDF

Info

Publication number
CN109448031B
CN109448031B CN201811140774.9A CN201811140774A CN109448031B CN 109448031 B CN109448031 B CN 109448031B CN 201811140774 A CN201811140774 A CN 201811140774A CN 109448031 B CN109448031 B CN 109448031B
Authority
CN
China
Prior art keywords
matrix
image
parameter
geometric transformation
objective function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811140774.9A
Other languages
Chinese (zh)
Other versions
CN109448031A (en
Inventor
马泳
黄珺
樊凡
马佳义
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201811140774.9A priority Critical patent/CN109448031B/en
Publication of CN109448031A publication Critical patent/CN109448031A/en
Application granted granted Critical
Publication of CN109448031B publication Critical patent/CN109448031B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides an image registration method and system based on Gaussian field constraint and manifold regularization, which are used for establishing correct matching by removing wrong matching in an initial matching point pair, and comprise the steps of establishing a non-rigid geometric transformation model corresponding to geometric transformation between images to be matched, establishing an objective function based on the Gaussian field constraint and the manifold regularization, and solving a parameter matrix by a deterministic annealing method; calculating a non-rigid geometric transformation model by using the model parameters, and judging the correctness of the initial matching point pair according to a threshold value; and finally, establishing a mapping relation between the two images by using the correct matching point set to realize image registration. The invention carries out modeling aiming at the condition that non-rigid transformation exists between images to be registered, establishes a robust estimator, greatly reduces the error rate of matching and still keeps good robustness even under the condition that a large number of error matches exist in primary matching.

Description

Image registration method and system based on Gaussian field constraint and manifold regularization
Technical Field
The invention relates to the technical field of image registration, in particular to an image registration technical scheme based on Gaussian field constraint and manifold regularization.
Background
The basic objective of image registration is to correspond the same parts, and to find a spatial transformation, so as to correspond the points corresponding to the same position in space in two images of the same scene obtained by different sensors at different time or view angles one by one, thereby achieving the purpose of information fusion.
Over the past decades, researchers have studied many approaches to address the image registration problem. These methods can be broadly divided into two categories: a region-based registration method and a feature-based registration method. The former searches for matching information by searching the similarity degree of original gray values in a certain area in two images; the latter uses descriptor similarity of local features or space geometric constraints to find matching point pairs, thereby establishing registration relationship. In cases with a small amount of significant detail, the grey values provide more information than the local shape and structure, so that the region-based approach has a better matching effect. However, the region-based method is computationally intensive and is not applicable in the case of image distortion and luminosity changes. On the contrary, the characteristic method has better robustness, can process images with complex distortion and is widely applied.
How to find corresponding matching points in the two images to form matching point pairs and ensure the correctness of the matching point pairs is the key of the image registration method.
The region-based registration method mainly comprises a correlation method, a Fourier method and a mutual information method. The main idea of the correlation method is to calculate the similarity of corresponding windows in two images, and then to take the pair with the greatest degree of similarity as a matching pair. However, the correlation method cannot be applied to a non-textured area with insignificant similarity and is computationally complex. The fourier method makes use of a fourier representation of the image in the frequency domain. Compared with the traditional correlation method, the method is more efficient in calculation and has good robustness to frequency-like noise. However, this approach has certain limitations in processing images with different spectral structures. The mutual information method, although it is good in matching effect, cannot obtain a global maximum value in the whole search space, and thus inevitably reduces its robustness.
In the feature-based registration method, a matching strategy divided into two steps is usually adopted first. In the first step, a group of initial matching point pairs is determined according to the similarity degree of the feature descriptors, wherein the initial matching point pairs comprise correct matching but inevitably contain a large number of error matching. And secondly, removing wrong matching through geometric constraint, and finally obtaining a correct matching point pair and a geometric parameter for transformation between the two images. Typical examples of such strategies include RANSAC methods (M.A. Fishler and R.C. bolts, "Random sample consensus: A parallel for model setting with application to image analysis and automatic card-graph," Commun.ACM, vol.24, No.6, pp.381-395, Jun.1981), ARHV methods (P.H.S.Torrer and A.Zisserman, "MLESAC: A.new sample estimate with application to image analysis," computer video. image Under-stand, 78, No.1, pp.138-156, Apr.2000) and VFC methods (J.Ma, J.Zo.J.Zo.J.J.Zo.J.Zo.sub.1, sample J.J.Zo.1, FIG. 1, J.Z.1, IEEE.1. sample parallel, FIG. 1, IEEE.1. sample consensus, and FIG. 1. balance, IEEE.1. balance, "int.j.comput.vis., vol.89, No.1, pp.1-17, aug.2010)," etc. are based on non-parametric models.
Although these methods have been successful in many fields, when the image contains a large amount of local distortion and the image content is complex, many wrong initial matching point pairs are obtained after the initial matching, and when the error rate exceeds a certain proportion, the methods cannot effectively remove the errors. Therefore, a matching method with a strong robustness to the initial matching error rate is needed.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides an image registration technical scheme based on Gaussian field constraint and manifold regularization.
In order to achieve the above object, the technical solution adopted by the present invention provides an image registration method based on gaussian field constraint and manifold regularization, comprising the following steps,
step 1, establishing a non-rigid geometric transformation model corresponding to geometric transformation between images to be matched, and realizing the following steps,
setting two images to be matched as an image a and an image b, wherein a known group of initial matching point pairs have a point set X (X) on the image a1,…,xN}TThe corresponding point set on the image b is Y ═ Y1,…,yN}TN is the number of the initial matching point pairs, aiming at the non-rigid geometric transformation between the images to be matched, a non-rigid geometric transformation model is established as follows,
Y＝f(X)＝X+ΓC
wherein f (X) represents a geometric transformation relation, the kernel function matrix gamma is an NxN matrix which represents a kernel function defined in a regenerative kernel Hilbert space, and the parameter matrix C is an Nx2 matrix which represents parameters of a non-rigid geometric transformation model;
step 2, according to the point set X ═ { X ═ X1,…,xN}TAnd Y ═ Y1,…,yN}TSolving the parameter matrix C, comprising the sub-steps of,
step 2.1, calculating a kernel function matrix gamma, the ith row and the jth column element
Figure BDA0001815751130000021
Wherein e is a mathematical constant and β is a preset coefficient;
step 2.2, initialize parameter matrix C ═ 0N×2And the interior point noise parameter σ ═ σ1，σ1Is a preset initial value of sigma;
step 2.3, establishing an objective function based on Gaussian field constraint and manifold regularization, and solving a parameter matrix C by a deterministic annealing method;
step 3, solving the transformation relation between the images by using the parameter matrix C obtained in the step 2, registering the two images, comprising the following substeps,
step 3.1, removing the error matching in the initial matching to obtain a correct matching point set, including calculating a geometric transformation relation f (X) according to the non-rigid geometric transformation model in the step 1, and for a certain point x in the image aiWhen satisfying | | yi-f(xi)||2At < t, xiAnd yiIs considered as a pair of correct matches, t is a fixed threshold of 1, when yi-f(xi)||2When t is more than or equal to xiAnd yiIs treated as a pair of false matches;
step 3.2, fitting quadratic transformation by a least square method by using a correct matching point set
Figure BDA0001815751130000031
In which the quadratic representation of the coordinates of the points
Figure BDA0001815751130000032
m＝(m1,m2)T，n1,n2Is the abscissa, ordinate, m, of a point n in an image a1,m2The horizontal and vertical coordinates of the corresponding matching point m in the image b are shown, and Q is a 2 multiplied by 6 transformation matrix; then, by the quadratic transformation, the image aTransformed and registered with image b.
Furthermore, step 2.3 comprises the following sub-steps,
step 2.3.1, let the current iteration number k equal to 1, calculate the graph laplacian L, where L is an N × N matrix with the ith row and the jth column of elements LijCan be calculated by the following formula,
Lij＝Dij-Wij
wherein the weight matrix WijThe calculation is as follows,
Figure BDA0001815751130000033
wherein, | | xi-xj||2Less than or equal to epsilon, epsilon is a preset threshold value, DijRepresents the ith row and jth column elements of the matrix D, and the matrix
Figure BDA0001815751130000034
Wherein
Figure BDA0001815751130000035
Representing the arrangement of elements in a diagonal matrix;
step 2.3.2, the objective function g (c) is updated, as shown in the following formula,
Figure BDA0001815751130000041
wherein x isiAnd yiPixel coordinate vectors of initial matching points on image a and image b, respectively, C is a parameter of the non-rigid geometric transformation model,. Γ i,. represents the ith line of the kernel function Γ defined in the reconstructed kernel Hilbert space,. lambda.1,λ2For the preset parameters, tr () represents the trace of the matrix, f is an N × 2 matrix, and f is (f (x)1),f(x2),…,f(xN) Wherein f (x)i) Extracting according to the geometric transformation relation in step 1, sigmakThe current noise coefficient of the interior point is obtained;
step 2.3.3, updating the objective functionGradient of gradient
Figure BDA0001815751130000042
As shown in the following formula:
Figure BDA0001815751130000043
step 2.3.4, optimizing the objective function G (C) by a quasi-Newton method, and inputting the objective function G (C) and the gradient of the objective function
Figure BDA0001815751130000044
Outputting the current value of the parameter matrix C as the corresponding parameter matrix C when the minimum value of the objective function G (C) is taken;
step 2.3.5, judging an iteration end condition, and when k is satisfied, determining k as kmaxEnd of time iteration, kmaxFor maximum number of iterations, otherwise annealing, updating interior point noise parameters, and ordering
Figure BDA0001815751130000045
Where γ is the annealing rate, k ═ k +1, return to step 2.3.2.
The invention also provides an image registration system based on Gaussian field constraint and manifold regularization, which comprises the following modules,
the model construction module is used for establishing a non-rigid geometric transformation model corresponding to the geometric transformation between the images to be matched, and is realized as follows,
setting two images to be matched as an image a and an image b, wherein a known group of initial matching point pairs have a point set X (X) on the image a1,…,xN}TThe corresponding point set on the image b is Y ═ Y1,…,yN}TN is the number of the initial matching point pairs, aiming at the non-rigid geometric transformation between the images to be matched, a non-rigid geometric transformation model is established as follows,
Y＝f(X)＝X+ΓC
wherein f (X) represents a geometric transformation relation, the kernel function matrix gamma is an NxN matrix which represents a kernel function defined in a regenerative kernel Hilbert space, and the parameter matrix C is an Nx2 matrix which represents parameters of a non-rigid geometric transformation model;
a parameter solving module for solving the parameter according to the point set X ═ { X ═ X1,…,xN}TAnd Y ═ Y1,…,yN}TSolving the parameter matrix C, comprising the following sub-modules,
kernel function matrix submodule for calculating kernel function matrix gamma, i row and j column elements
Figure BDA0001815751130000046
Wherein e is a mathematical constant and β is a preset coefficient;
an initialization parameter submodule for initializing the parameter matrix C to 0N×2And the interior point noise parameter σ ═ σ1，σ1Is a preset initial value of sigma;
the solution model parameter submodule is used for establishing an objective function based on Gaussian field constraint and manifold regularization and solving a parameter matrix C by a deterministic annealing method;
a registration module for solving the transformation relation between the images by using the parameter matrix C obtained by the parameter solving module and registering the two images, comprising the following sub-modules,
the mismatching removal submodule is used for removing the mismatching in the initial matching to obtain a correct matching point set, and comprises the steps of calculating a geometric transformation relation f (X) according to a non-rigid geometric transformation model in the model construction module, and calculating a certain point x in the image aiWhen satisfying | | yi-f(xi)||2At < t, xiAnd yiIs considered as a pair of correct matches, t is a fixed threshold of 1, when yi-f(xi)||2When t is more than or equal to xiAnd yiIs treated as a pair of false matches;
an image transformation submodule for fitting quadratic transformation by least square method using correct matching point set
Figure BDA0001815751130000051
In which the quadratic representation of the coordinates of the points
Figure BDA0001815751130000052
m＝(m1,m2)T，n1,n2Is the abscissa, ordinate, m, of a point n in an image a1,m2The horizontal and vertical coordinates of the corresponding matching point m in the image b are shown, and Q is a 2 multiplied by 6 transformation matrix; image a is then transformed and registered with image b by this quadratic transformation.
Furthermore, the solution model parameters submodule includes the following elements,
a graph laplacian L calculating unit, configured to calculate a graph laplacian L by setting the current iteration number k to 1, where L is an N × N matrix having an i-th row and a j-th column of elements LijCan be calculated by the following formula,
Lij＝Dij-Wij
wherein the weight matrix WijThe calculation is as follows,
Figure BDA0001815751130000053
wherein, | | xi-xj||2Less than or equal to epsilon, epsilon is a preset threshold value, DijRepresents the ith row and jth column elements of the matrix D, and the matrix
Figure BDA0001815751130000054
Wherein
Figure BDA0001815751130000055
Representing the arrangement of elements in a diagonal matrix;
an objective function updating unit for updating the objective function G (C) as shown in the following formula,
Figure BDA0001815751130000061
wherein x isiAnd yiPixel coordinate vectors of initial matching points on the image a and the image b respectively, C is a parameter of a non-rigid geometric transformation model, and gamma isiAnd, an ith line, λ, representing a kernel function Γ defined in a regenerative kernel Hilbert space1,λ2For the preset parameters, tr () represents the trace of the matrix, f is an N × 2 matrix, and f is (f (x)1),f(x2),…,f(xN) Wherein f (x)i) According to the geometric transformation relation extraction in the model building module, sigmakThe current noise coefficient of the interior point is obtained;
an objective function gradient update unit for updating the gradient of the objective function
Figure BDA0001815751130000062
As shown in the following formula:
Figure BDA0001815751130000063
an optimization objective function unit for optimizing an objective function G (C) by a quasi-Newton method, wherein the input is the objective function G (C) and the gradient of the objective function
Figure BDA0001815751130000064
Outputting the current value of the parameter matrix C as the corresponding parameter matrix C when the minimum value of the objective function G (C) is taken;
an iteration judgment unit for judging the iteration end condition when k is equal to kmaxEnd of time iteration, kmaxFor maximum number of iterations, otherwise annealing, updating interior point noise parameters, and ordering
Figure BDA0001815751130000065
Where γ is the annealing rate, and k is k +1, the objective function update unit is commanded to operate.
The invention has the following advantages:
1. the invention provides a non-rigid registration framework aiming at the problem of image registration. Compared with the commonly used parametric model, the framework can process the non-parametric model condition and has wider application range.
2. The invention constructs a model based on the Gaussian field criterion and the manifold regularization for optimization. The model considers space constraint and has good robustness, so that the model can be used for processing the condition containing a large number of error matching and can obtain good registration effect.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and examples.
The method provided by the invention firstly carries out mathematical modeling on image registration, then establishes correct matching by removing wrong matching in a series of preliminarily established matching point pairs, and then establishes an image mapping relation by using the correct matching to realize the image registration effect. The method utilizes an annealing algorithm and a quasi-Newton method to solve. Meanwhile, a constraint of Gaussian field and manifold regularization is constructed, and even under the condition that a large number of error matches exist in preliminary matching, good robustness is still kept.
Referring to fig. 1, the method provided by the embodiment of the present invention mainly includes 3 steps:
step 1, establishing a model corresponding to geometric transformation between images to be matched, and realizing the following,
marking two images to be matched as an image a and an image b, setting a known group of initial matching point pairs, and setting a point set on the image a as X ═ X1,…,xN}TThe corresponding point set on the image b is Y ═ Y1,…,yN}TAnd N is the number of the initial matching point pairs. For non-rigid geometric transformation between images to be matched, a transformation mathematical model is established as follows,
Y＝f(X)＝X+ΓC
wherein f (X) represents a geometric transformation relationship, matrix Γ is an N × N matrix representing a kernel function defined in a regenerative kernel Hilbert space, and the ith row and jth column elements of matrix Γ
Figure BDA0001815751130000071
Where e is a mathematical constant, xi、xjThe ith point and the jth point (position coordinate) in the image a in the initial matching point pair respectivelyMark), β is a preset coefficient, which can be specified in advance by those skilled in the art, and 0.01 is adopted in the embodiment; the matrix C is an Nx 2 matrix and represents parameters of the non-rigid geometric transformation model;
step 2, according to the point set X ═ { X ═ X1,…,xN}TAnd Y ═ Y1,…,yN}TSolving a parameter matrix C, wherein C ═ C1,c2,…,cN)TComprising the sub-steps of,
step 2.1, computing a kernel function matrix gamma, wherein the elements
Figure BDA0001815751130000072
Step 2.2, initialize parameter matrix C ═ 0N×2And the interior point noise parameter σ ═ σ1，σ1The preset initial value of σ can be pre-specified by those skilled in the art, and the embodiment adopts 0.3, 0N×2Is a full 0 matrix with the size of Nx 2;
step 2.3, establishing an objective function based on Gaussian field constraint and manifold regularization, and solving a parameter matrix C by a deterministic annealing method, comprising the following substeps,
step 2.3.1, let the current iteration number k equal to 1, calculate the graph laplacian L, where L is an N × N matrix with the ith row and the jth column of elements LijCan be calculated by the following formula,
Lij＝Dij-Wij
wherein the weight matrix WijThe calculation is as follows,
Figure BDA0001815751130000081
wherein, | | xi-xj||2ε is a predetermined threshold, and 0.05 is used in the examples. DijRepresents the ith row and jth column elements of the matrix D, and the matrix
Figure BDA0001815751130000082
Wherein
Figure BDA0001815751130000083
Representing the arrangement of elements in a diagonal matrix;
step 2.3.2, updating the target function G (C) as shown in the following formula:
Figure BDA0001815751130000084
wherein x isiAnd yiPixel coordinate vectors of initial matching points on the image a and the image b respectively, C is a parameter of a non-rigid geometric transformation model, and gamma isi,·Line i, λ, representing the kernel function Γ defined in the regenerative kernel Hilbert space1,λ2For the preset parameters, empirical values can be used, and the implementation can be specified by those skilled in the art in advance, where tr () represents the trace of the matrix, f is an N × 2 matrix, and f ═ is (f (x)1),f(x2),…,f(xN) Wherein f (x)i) The transformation function in step 1;
f(xi) Is xiCorresponding non-rigid geometric transformation, f (x)i) Can be considered as line i of the transformation function f (x) in step 1; sigmakIs the current inlier noise figure.
The first term of the expression provided by the invention corresponds to a Gaussian field criterion, the second term is a smooth constraint term, and the third term corresponds to manifold regularization. Each time step 2.3.2 is executed, the current objective function value is calculated using this equation.
Step 2.3.3, update the gradient of the objective function
Figure BDA0001815751130000085
As shown in the following formula:
Figure BDA0001815751130000086
step 2.3.4, optimizing the objective function G (C) by a quasi-Newton method, and inputting the objective function G (C) and the gradient of the objective function
Figure BDA0001815751130000087
Outputting the current value of the parameter matrix C as the corresponding parameter matrix C when the minimum value of the objective function G (C) is taken; the Quasi-Newton method (Quasi-Newton Methods) adopted in the embodiments is one of the most effective Methods for solving the nonlinear optimization problem, and is not repeated in the present invention for the prior art, and other Methods for solving the nonlinear optimization problem, such as a confidence domain method, may be adopted in specific implementation;
step 2.3.5, judging an iteration end condition, and when k is satisfied, determining k as kmaxEnd of time iteration, kmaxThe maximum number of iterations can be pre-specified by those skilled in the art, and the embodiment adopts 2, otherwise, annealing, updating the interior point noise parameter and enabling
Figure BDA0001815751130000091
Where γ is the annealing rate and is a certain value, which can be specified in advance by a person skilled in the art, and in the example, 0.93 is used, k is k +1, and the process returns to step 2.3.2;
step 3, solving the transformation relation between the images by using the parameter matrix C obtained in the step 2, registering the two images, comprising the following substeps,
step 3.1, removing the error matching in the initial matching to obtain a correct matching point set, including calculating a transformation function, namely a geometric transformation relation f (x), according to the transformation model in the step 1, and for a certain point x in the image aiWhen satisfying | | yi-f(xi)||2At < t, xiAnd yiIs considered a pair of correct matches, where yiAs the neutralization point x in the image biThe corresponding initial matching point, t is a fixed threshold value, which can be pre-specified by those skilled in the art, and 0.01 is adopted in the embodiment; when yi-f(xi)||2When t is more than or equal to xiAnd yiIs treated as a pair of false matches;
step 3.2, fitting a quadratic transformation by a least square method by using the correct matching point set
Figure BDA0001815751130000092
In which the coordinates of the points are represented twice
Figure BDA0001815751130000093
m＝(m1,m2)T，n1,n2Is the abscissa, ordinate, m, of a point n in an image a1,m2The horizontal and vertical coordinates of the corresponding matching point m in the image b are shown, and Q is a 2 multiplied by 6 transformation matrix; image a is then transformed and registered with image b by this transformation.
In specific implementation, the above processes can be automatically operated by adopting a software mode. The invention also provides a corresponding system in a modularized mode, and the embodiment of the invention also correspondingly provides an image registration system based on Gaussian field constraint and manifold regularization, which comprises the following modules,
the model construction module is used for establishing a non-rigid geometric transformation model corresponding to the geometric transformation between the images to be matched, and is realized as follows,
setting two images to be matched as an image a and an image b, wherein a known group of initial matching point pairs have a point set X (X) on the image a1,…,xN}TThe corresponding point set on the image b is Y ═ Y1,…,yN}TN is the number of the initial matching point pairs, aiming at the non-rigid geometric transformation between the images to be matched, a non-rigid geometric transformation model is established as follows,
Y＝f(X)＝X+ΓC
wherein f (X) represents a geometric transformation relation, the kernel function matrix gamma is an NxN matrix which represents a kernel function defined in a regenerative kernel Hilbert space, and the parameter matrix C is an Nx2 matrix which represents parameters of a non-rigid geometric transformation model;
a parameter solving module for solving the parameter according to the point set X ═ { X ═ X1,…,xN}TAnd Y ═ Y1,…,yN}TSolving the parameter matrix C, comprising the following sub-modules,
kernel function matrix submodule for calculating kernel function matrix gamma, i row and j column elements
Figure BDA0001815751130000101
Wherein e is a mathematical constant and β is a preset coefficient;
an initialization parameter submodule for initializing the parameter matrix C to 0N×2And the interior point noise parameter σ ═ σ1，σ1Is a preset initial value of sigma;
the solution model parameter submodule is used for establishing an objective function based on Gaussian field constraint and manifold regularization and solving a parameter matrix C by a deterministic annealing method;
a registration module for solving the transformation relation between the images by using the parameter matrix C obtained by the parameter solving module and registering the two images, comprising the following sub-modules,
the mismatching removal submodule is used for removing the mismatching in the initial matching to obtain a correct matching point set, and comprises the steps of calculating a geometric transformation relation f (X) according to a non-rigid geometric transformation model in the model construction module, and calculating a certain point x in the image aiWhen satisfying | | yi-f(xi)||2At < t, xiAnd yiIs considered as a pair of correct matches, t is a fixed threshold of 1, when yi-f(xi)||2When t is more than or equal to xiAnd yiIs treated as a pair of false matches;
an image transformation submodule for fitting quadratic transformation by least square method using correct matching point set
Figure BDA0001815751130000102
In which the quadratic representation of the coordinates of the points
Figure BDA0001815751130000103
m＝(m1,m2)T，n1,n2Is the abscissa, ordinate, m, of a point n in an image a1,m2The horizontal and vertical coordinates of the corresponding matching point m in the image b are shown, and Q is a 2 multiplied by 6 transformation matrix; image a is then transformed and registered with image b by this quadratic transformation.
Further, the solution model parameters submodule includes the following elements,
a graph laplacian L calculating unit, configured to calculate a graph laplacian L by setting the current iteration number k to 1, where L is an N × N matrix having an i-th row and a j-th column of elements LijCan be calculated by the following formula,
Lij＝Dij-Wij
wherein the weight matrix WijThe calculation is as follows,
Figure BDA0001815751130000104
wherein, | | xi-xj||2Less than or equal to epsilon, epsilon is a preset threshold value, DijRepresents the ith row and jth column elements of the matrix D, and the matrix
Figure BDA0001815751130000105
Wherein
Figure BDA0001815751130000106
Representing the arrangement of elements in a diagonal matrix;
an objective function updating unit for updating the objective function G (C) as shown in the following formula,
Figure BDA0001815751130000111
wherein x isiAnd yiPixel coordinate vectors of initial matching points on the image a and the image b respectively, C is a parameter of a non-rigid geometric transformation model, and gamma isi,·Line i, λ, representing the kernel function Γ defined in the regenerative kernel Hilbert space1,λ2For the preset parameters, tr () represents the trace of the matrix, f is an N × 2 matrix, and f is (f (x)1),f(x2),…,f(xN) Wherein f (x)i) According to the geometric transformation relation extraction in the model building module, sigmakThe current noise coefficient of the interior point is obtained;
an objective function gradient update unit for updating the gradient of the objective function
Figure BDA0001815751130000112
As shown in the following formula:
Figure BDA0001815751130000113
an optimization objective function unit for optimizing an objective function G (C) by a quasi-Newton method, wherein the input is the objective function G (C) and the gradient of the objective function
Figure BDA0001815751130000114
Outputting the current value of the parameter matrix C as the corresponding parameter matrix C when the minimum value of the objective function G (C) is taken;
an iteration judgment unit for judging the iteration end condition when k is equal to kmaxEnd of time iteration, kmaxFor maximum number of iterations, otherwise annealing, updating interior point noise parameters, and ordering
Figure BDA0001815751130000115
Where γ is the annealing rate, and k is k +1, the objective function update unit is commanded to operate.
And selecting RANSAC, ICF and GS methods to compare with the method for image matching. The comparison result is shown in the following table, wherein the time is the running time of the algorithm on each graph, and the accuracy refers to the proportion of correct matching point pairs in the matching point pairs given by the method finally; the leak rate refers to the proportion of the method for judging the correct matching point pairs as the wrong matching point pairs in the screening process. The method has the advantages of short time, highest accuracy and lowest leakage rate.
Method effect comparison table
Figure BDA0001815751130000116
Figure BDA0001815751130000121
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or scope of the invention as defined in the appended claims.

Claims (4)

1. An image registration method based on Gaussian field constraint and manifold regularization is characterized in that: comprises the following steps of (a) carrying out,
step 1, establishing a non-rigid geometric transformation model corresponding to geometric transformation between images to be matched, and realizing the following steps,
setting two images to be matched as an image a and an image b, wherein a known group of initial matching point pairs have a point set X (X) on the image a1,…,xN}TThe corresponding point set on the image b is Y ═ Y1,…,yN}TN is the number of the initial matching point pairs, aiming at the non-rigid geometric transformation between the images to be matched, a non-rigid geometric transformation model is established as follows,
Y＝f(X)＝X+ΓC
wherein f (X) represents a geometric transformation relation, the kernel function matrix gamma is an NxN matrix which represents a kernel function defined in a regenerative kernel Hilbert space, and the parameter matrix C is an Nx2 matrix which represents parameters of a non-rigid geometric transformation model;
step 2, according to the point set X ═ { X ═ X1,…,xN}TAnd Y ═ Y1,…,yN}TSolving the parameter matrix C, comprising the sub-steps of,
step 2.1, calculating a kernel function matrix gamma, the ith row and the jth column element
Figure FDA0002970659290000011
Wherein e is a mathematical constant and β is a preset coefficient;
step 2.2, initialize parameter matrix C ═ 0N×2And the interior point noise parameter σ ═ σ1，σ1Is a preset initial value of sigma;
step 2.3, establishing an objective function based on Gaussian field constraint and manifold regularization, and solving a parameter matrix C by a deterministic annealing method;
step 3, solving the transformation relation between the images by using the parameter matrix C obtained in the step 2, registering the two images, comprising the following substeps,
step 3.1, removing the error matching in the initial matching to obtain a correct matching point set, including calculating a geometric transformation relation f (X) according to the non-rigid geometric transformation model in the step 1, and for a certain point x in the image aiWhen satisfying | | yi-f(xi)||2At < t, xiAnd yiIs considered as a pair of correct matches, t is a fixed threshold of 1, when yi-f(xi)||2When t is more than or equal to xiAnd yiIs treated as a pair of false matches;
step 3.2, fitting quadratic transformation by a least square method by using a correct matching point set
Figure FDA0002970659290000012
In which the quadratic representation of the coordinates of the points
Figure FDA0002970659290000013
m＝(m1,m2)T，n1,n2Is the abscissa, ordinate, m, of a point n in an image a1,m2The horizontal and vertical coordinates of the corresponding matching point m in the image b are shown, and Q is a 2 multiplied by 6 transformation matrix; image a is then transformed and registered with image b by this quadratic transformation.
2. The image registration method based on Gaussian field constraint and manifold regularization according to claim 1, wherein: step 2.3 comprises the sub-steps of,
step 2.3.1, let the current iteration number k equal to 1, calculate the graph laplacian L, where L is an N × N matrix with the ith row and the jth column of elements LijCan be calculated by the following formula,
Lij＝Dij-Wij
wherein the weight matrix WijThe calculation is as follows,
Figure FDA0002970659290000021
wherein, | | xi-xj||2Less than or equal to epsilon, epsilon is a preset threshold value, DijRepresents the ith row and jth column elements of the matrix D, and the matrix
Figure FDA0002970659290000022
Wherein
Figure FDA0002970659290000023
Representing the arrangement of elements in a diagonal matrix;
step 2.3.2, the objective function g (c) is updated, as shown in the following formula,
Figure FDA0002970659290000024
wherein x isiAnd yiPixel coordinate vectors of initial matching points on the image a and the image b respectively, C is a parameter of a non-rigid geometric transformation model, and gamma isiAnd, an ith line, λ, representing a kernel function Γ defined in a regenerative kernel Hilbert space1,λ2For the preset parameters, tr () represents the trace of the matrix, f is an N × 2 matrix, and f is (f (x)1),f(x2),…,f(xN) Wherein f (x)i) Extracting according to the geometric transformation relation in step 1, sigmakThe current noise coefficient of the interior point is obtained;
step 2.3.3, update the gradient of the objective function
Figure FDA0002970659290000025
As shown in the following formula:
Figure FDA0002970659290000026
step 2.3.4, optimizing the objective function G (C) by a quasi-Newton method, and inputting the objective function G (C) and the gradient of the objective function
Figure FDA0002970659290000027
Outputting the current value of the parameter matrix C as the corresponding parameter matrix C when the minimum value of the objective function G (C) is taken;
step 2.3.5, judging an iteration end condition, and when k is satisfied, determining k as kmaxEnd of time iteration, kmaxFor maximum number of iterations, otherwise annealing, updating interior point noise parameters, and ordering
Figure FDA0002970659290000031
Where γ is the annealing rate, k ═ k +1, return to step 2.3.2.
3. An image registration system based on Gaussian field constraints and manifold regularization, characterized by: comprises the following modules which are used for realizing the functions of the system,
the model construction module is used for establishing a non-rigid geometric transformation model corresponding to the geometric transformation between the images to be matched, and is realized as follows,
setting two images to be matched as an image a and an image b, wherein a known group of initial matching point pairs have a point set X (X) on the image a1,…,xN}TThe corresponding point set on the image b is Y ═ Y1,…,yN}TN is the number of the initial matching point pairs, aiming at the non-rigid geometric transformation between the images to be matched, a non-rigid geometric transformation model is established as follows,
Y＝f(X)＝X+ΓC
wherein f (X) represents a geometric transformation relation, the kernel function matrix gamma is an NxN matrix which represents a kernel function defined in a regenerative kernel Hilbert space, and the parameter matrix C is an Nx2 matrix which represents parameters of a non-rigid geometric transformation model;
a parameter solving module for solving the parameter according to the point set X ═ { X ═ X1,…,xN}TAnd Y ═ Y1,…,yN}TSolving the parameter matrix C, comprising the following sub-modules,
kernel function matrix submodule for calculating kernel function matrix gamma, i row and j column elements
Figure FDA0002970659290000032
Wherein e is a mathematical constant and β is a preset coefficient;
an initialization parameter submodule for initializing the parameter matrix C to 0N×2And the interior point noise parameter σ ═ σ1，σ1Is a preset initial value of sigma;
the solution model parameter submodule is used for establishing an objective function based on Gaussian field constraint and manifold regularization and solving a parameter matrix C by a deterministic annealing method;
a registration module for solving the transformation relation between the images by using the parameter matrix C obtained by the parameter solving module and registering the two images, comprising the following sub-modules,
the mismatching removal submodule is used for removing the mismatching in the initial matching to obtain a correct matching point set, and comprises the steps of calculating a geometric transformation relation f (X) according to a non-rigid geometric transformation model in the model construction module, and calculating a certain point x in the image aiWhen satisfying | | yi-f(xi)||2At < t, xiAnd yiIs considered as a pair of correct matches, t is a fixed threshold of 1, when yi-f(xi)||2When t is more than or equal to xiAnd yiIs treated as a pair of false matches;
an image transformation submodule for fitting quadratic transformation by least square method using correct matching point set
Figure FDA0002970659290000033
In which the quadratic representation of the coordinates of the points
Figure FDA0002970659290000034
m＝(m1,m2)T，n1,n2As the abscissa and ordinate of the point n in the image a，m1,m2The horizontal and vertical coordinates of the corresponding matching point m in the image b are shown, and Q is a 2 multiplied by 6 transformation matrix; image a is then transformed and registered with image b by this quadratic transformation.
4. The image registration system based on Gaussian field constraints and manifold regularization according to claim 3, wherein: the solution model parameters submodule includes the following elements,
a graph laplacian L calculating unit, configured to calculate a graph laplacian L by setting the current iteration number k to 1, where L is an N × N matrix having an i-th row and a j-th column of elements LijCan be calculated by the following formula,
Lij＝Dij-Wij
wherein the weight matrix WijThe calculation is as follows,
Figure FDA0002970659290000041
wherein, | | xi-xj||2Less than or equal to epsilon, epsilon is a preset threshold value, DijRepresents the ith row and jth column elements of the matrix D, and the matrix
Figure FDA0002970659290000042
Wherein
Figure FDA0002970659290000043
Representing the arrangement of elements in a diagonal matrix;
an objective function updating unit for updating the objective function G (C) as shown in the following formula,
Figure FDA0002970659290000044
wherein x isiAnd yiPixel coordinate vectors of initial matching points on the image a and the image b respectively, C is a parameter of a non-rigid geometric transformation model, and gamma isiAnd, an ith line, λ, representing a kernel function Γ defined in a regenerative kernel Hilbert space1,λ2For the preset parameters, tr () represents the trace of the matrix, f is an N × 2 matrix, and f is (f (x)1),f(x2),…,f(xN) Wherein f (x)i) According to the geometric transformation relation extraction in the model building module, sigmakThe current noise coefficient of the interior point is obtained;
an objective function gradient update unit for updating the gradient of the objective function
Figure FDA0002970659290000045
As shown in the following formula:
Figure FDA0002970659290000046
an optimization objective function unit for optimizing an objective function G (C) by a quasi-Newton method, wherein the input is the objective function G (C) and the gradient of the objective function
Figure FDA0002970659290000047
Outputting the current value of the parameter matrix C as the corresponding parameter matrix C when the minimum value of the objective function G (C) is taken;
an iteration judgment unit for judging the iteration end condition when k is equal to kmaxEnd of time iteration, kmaxFor maximum number of iterations, otherwise annealing, updating interior point noise parameters, and ordering
Figure FDA0002970659290000051
Where γ is the annealing rate, and k is k +1, the objective function update unit is commanded to operate.
CN201811140774.9A 2018-09-28 2018-09-28 Image registration method and system based on Gaussian field constraint and manifold regularization Active CN109448031B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811140774.9A CN109448031B (en) 2018-09-28 2018-09-28 Image registration method and system based on Gaussian field constraint and manifold regularization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811140774.9A CN109448031B (en) 2018-09-28 2018-09-28 Image registration method and system based on Gaussian field constraint and manifold regularization

Publications (2)

Publication Number Publication Date
CN109448031A CN109448031A (en) 2019-03-08
CN109448031B true CN109448031B (en) 2021-04-16

Family

ID=65544723

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811140774.9A Active CN109448031B (en) 2018-09-28 2018-09-28 Image registration method and system based on Gaussian field constraint and manifold regularization

Country Status (1)

Country Link
CN (1) CN109448031B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667515B (en) * 2020-04-22 2023-11-14 北京理工大学 Vessel 3D/2D elastic registration method and device based on manifold regularization
CN113781559B (en) * 2021-08-31 2023-10-13 南京邮电大学 Robust abnormal matching point eliminating method and image indoor positioning method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609550A (en) * 2009-07-10 2009-12-23 南方医科大学 Method for registering images based on stream shape subspace
CN105469110A (en) * 2015-11-19 2016-04-06 武汉大学 Non-rigid transformation image characteristic matching method based on local linear transfer and system
CN105469415A (en) * 2015-12-28 2016-04-06 云南师范大学 Multi-view remote sensing image fusion method
CN105469112A (en) * 2015-11-19 2016-04-06 武汉大学 Image feature matching method and system based on local linear migration and rigid model

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8023732B2 (en) * 2006-07-26 2011-09-20 Siemens Aktiengesellschaft Accelerated image registration by means of parallel processors

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609550A (en) * 2009-07-10 2009-12-23 南方医科大学 Method for registering images based on stream shape subspace
CN105469110A (en) * 2015-11-19 2016-04-06 武汉大学 Non-rigid transformation image characteristic matching method based on local linear transfer and system
CN105469112A (en) * 2015-11-19 2016-04-06 武汉大学 Image feature matching method and system based on local linear migration and rigid model
CN105469415A (en) * 2015-12-28 2016-04-06 云南师范大学 Multi-view remote sensing image fusion method

Also Published As

Publication number Publication date
CN109448031A (en) 2019-03-08

Similar Documents

Publication Publication Date Title
CN109685152B (en) Image target detection method based on DC-SPP-YOLO
CN109285110B (en) Infrared visible light image registration method and system based on robust matching and transformation
CN105469110B (en) Non-rigid transformation Image Feature Matching method and system based on local linear migration
CN112581515B (en) Outdoor scene point cloud registration method based on graph neural network
CN111553939B (en) Image registration algorithm of multi-view camera
JP2010238226A (en) Method and system for tracking object
CN110136177B (en) Image registration method, device and storage medium
CN111709980A (en) Multi-scale image registration method and device based on deep learning
Du et al. New iterative closest point algorithm for isotropic scaling registration of point sets with noise
CN113177592B (en) Image segmentation method and device, computer equipment and storage medium
CN109448031B (en) Image registration method and system based on Gaussian field constraint and manifold regularization
CN105488754B (en) Image Feature Matching method and system based on local linear migration and affine transformation
CN109961435B (en) Brain image acquisition method, device, equipment and storage medium
CN114966576A (en) Radar external reference calibration method and device based on prior map and computer equipment
CN109785372B (en) Basic matrix robust estimation method based on soft decision optimization
CN113902828A (en) Construction method of indoor two-dimensional semantic map with corner as key feature
CN117541632A (en) Multi-mode image registration method based on feature enhancement and multi-scale correlation
CN111814884A (en) Target detection network model upgrading method based on deformable convolution
CN105469112B (en) Image Feature Matching method and system based on local linear migration and rigid model
CN110728296A (en) Two-step random sampling consistency method and system for accelerating feature point matching
CN116128919A (en) Multi-temporal image abnormal target detection method and system based on polar constraint
Chen et al. Mismatch removal for remote sensing images based on non-rigid transformation and local geometrical constraint
Kher Implementation of image registration for satellite images using mutual information and particle swarm optimization techniques
Wu et al. Spatio-temporal fish-eye image processing based on neural network
Li et al. Truncated signed distance function volume integration based on voxel-level optimization for 3D reconstruction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant