CN102005047A - Image registration system and method thereof - Google Patents

Abstract

The invention provides an image registration system and a method thereof. The method comprises the following steps of: firstly calculating scale invariant feature transform (SIFT transform) of a real-time image and a reference image to obtain feature points of the real-time image and the reference image; selecting a plurality of pairs of feature points with maximum similarity from the feature points of the real-time image and the reference image; carrying out matching verification of the plurality of pairs of feature points with maximum similarity selected from the feature points of the real-time image and the reference image and rejecting a mismatching point pair therein to obtain a plurality of pairs of feature points which are verified; according to the plurality of pairs of feature points which are verified, determining the parameter of an affine transformation function of the real-time image relative to the reference image; and carrying out radioactive transform of the real-time image according to the affine transformation parameter to obtain the registered image. The system and the method are difficult to influence and high in registration accuracy.

Description

Figure registration system and method thereof

[technical field]

Field of image recognition of the present invention, particularly figure registration system and method thereof about two width of cloth or the multiple image of Same Scene are spatially aimed at.

[background technology]

Image registration is that two width of cloth or the multiple image of Same Scene are spatially aimed at.It is widely used in numerous art of image analysis, as medical science, remote Sensing Image Analysis, image co-registration, image retrieval, Target Recognition etc.

The present the most frequently used method for registering images that is based on feature.(wherein the image as standard becomes reference diagram for registration two width of cloth images, the current image that obtains becomes real-time figure), need from image, extract feature, and with the corresponding relation of these features foundation, then by finding the solution the feature corresponding relation, schemed in real time and reference diagram between transformation relation, at last with real-time figure according to the geometric relationship unscented transformation of trying to achieve to needed form.

Existing image registration based on feature mainly is to use the angle point of Harris Corner Detection device extraction as registration features.The disadvantage of angle point feature is responsive to noise ratio, and along with the difference of image resolution ratio, angle point is easy to generate drift, and Harris Corner Detection operator is not that yardstick is constant.

[summary of the invention]

The object of the present invention is to provide a kind of not susceptible to and the high method for registering images of registration accuracy.

Another object of the present invention is to provide a kind of not susceptible to and the high figure registration system of registration accuracy.

For reaching aforementioned purpose, a kind of method for registering images of the present invention, it comprises:

Calculate the yardstick invariant features conversion (SIFT conversion) of real-time figure and reference diagram, schemed unique point in real time with reference diagram;

From the aforementioned unique point of aforementioned real-time figure and reference diagram, select the some of similarity maximum to unique point;

The some of similarity maximum in aforementioned real-time figure and the reference diagram are mated checking to unique point, and the mispairing point of rejecting wherein is right, some to unique point after being verified;

According to some after the aforementioned authentication unique point is determined the affined transformation function parameters that realtime graphic takes place for reference diagram;

Figure is in real time radiated image after conversion obtains registration according to aforementioned affine transformation parameter.

Further, each local feature that the conversion of described yardstick invariant features is obtained is a histogram, described select the similarity maximum from the aforementioned unique point of aforementioned real-time figure and reference diagram some calculate similarity between any two unique points to unique point by formula, and described formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B represents the total interval number of local histogram,

The expression vector

U element,

Be the local feature vectors of reference diagram correspondence,

The expression vector

U element,

For scheming corresponding local feature vectors in real time.

It is further, described that similarity maximum in aforementioned real-time figure and the reference diagram some are mated checking to unique point is to adopt the random sampling consistency algorithm to verify that it comprises:

Select two points from data acquisition randomly, these two points are determined straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select at random to repeat for several times, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

Further, the affined transformation function of described real-time figure and reference diagram is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point among the real-time figure, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

To use the method for least square can obtain parameter by the unique point substitution equation of matching characteristic point checking, (u, v), θ and s last, will scheme in real time around (convergent-divergent 1/s doubly can obtain the image behind the registration again for u, the v) anglec of rotation-θ.

For reaching aforementioned another purpose, a kind of figure registration system of the present invention, it comprises:

The image characteristic point extraction unit is used for obtaining respectively the real-time figure of needs aligning and the unique point of reference diagram image;

The Feature Points Matching unit, the unique point that is used for two width of cloth figure that the image characteristic point extraction unit is obtained is mated, and obtains the unique point of similarity maximum among two width of cloth figure;

Matching characteristic point checking unit is used for somely unique point being verified the mispairing point of rejecting wherein is right to what the Feature Points Matching unit obtained, some to unique point after being verified;

The affined transformation unit calculates the affine transformation parameters of figure and reference diagram in real time according to some after the checking to unique point, and the described parameter of foundation is radiated image after conversion obtains registration to real-time figure.

Further, described image characteristic point extraction unit is to adopt yardstick invariant features conversion (SIFT conversion) to obtain the unique point of real-time figure and reference diagram image.

Further, each local feature that the conversion of described yardstick invariant features is obtained is a histogram, and described Feature Points Matching unit calculates similarity between any two unique points by formula, obtains the unique point of similarity maximum among two width of cloth figure, and described formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B represents the total interval number of local histogram,

The expression vector U element,

Be the local feature vectors of reference diagram correspondence,

The expression vector U element,

For scheming corresponding local feature vectors in real time.

Further, described matching characteristic point verifies that the unit is to adopt the random sampling consistency algorithm to come the unique point that the Feature Points Matching unit obtains is verified that described algorithm comprises:

Select two points from data acquisition randomly, these two points are determined straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select at random to repeat for several times, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

Further, described affined transformation unit comprises the affined transformation function of real-time figure and reference diagram, and described function is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point among the real-time figure, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

Described affined transformation unit will use the method for least square can obtain parameter, (u by the unique point substitution equation of matching characteristic point checking, v), θ and s, last, will scheme in real time around (u, the v) anglec of rotation-θ, convergent-divergent 1/s doubly can obtain the image behind the registration again.

Compared with prior art, the present invention uses the SIFT conversion to extract image local feature, and the feature of extraction all remains unchanged to image zoom, rotation even affined transformation.The unique point of using the checking of random sampling coherence method to extract is right, and the point that has reduced mispairing is right, has improved the levels of precision of coupling.Use least square method to calculate affine transformation parameter, better than direct solving equation organizing, stability.Total system and method thereof be susceptible to and registration accuracy height not.

[description of drawings]

Fig. 1 is the process flow diagram of the method for image registration of the present invention.

Fig. 2 carries out the synoptic diagram that key point detects for using the SIFT conversion.

Fig. 3 is the key point feature synoptic diagram of SIFT conversion.

Fig. 4 is the structured flowchart of figure registration system of the present invention.

[embodiment]

Alleged herein " embodiment " or " embodiment " are meant special characteristic, structure or the characteristic that can be contained at least one implementation of the present invention.Different in this manual local " in one embodiment " that occur not are all to refer to same embodiment, neither be independent or optionally mutually exclusive with other embodiment embodiment.

Image registration is that two width of cloth or the multiple image of Same Scene are spatially aimed at, the present invention proposes a kind of method for registering images based on SIFT conversion (full name is Scale Invariant Feature Transformation, i.e. yardstick invariant features conversion).Method for registering images of the present invention is described for example with registration two width of cloth images (wherein the image as standard becomes reference diagram, and the current image that obtains becomes real-time figure) below.

See also shown in Figure 1, method for registering images of the present invention, it comprises the steps:

Step S101: calculate the SIFT conversion of real-time figure and reference diagram, schemed unique point in real time with reference diagram.

SIFT conversion (full name is Scale Invariant Feature Transformation, i.e. yardstick invariant features conversion) is present the most frequently used local feature extracting method.This conversion can extract the image local feature that image zoom, rotation even affined transformation are all remained unchanged.

The method of SIFT conversion comprises the steps:

(1) at first image is carried out the metric space extreme value and detect, with preliminary definite key point position and place yardstick.As shown in Figure 2, when detecting the yardstick spatial extrema, the pixel that is labeled as cross among the figure need the attendant of a stage actor draw together same yardstick around 9 * 2 pixels of neighborhood altogether 26 pixels compare, to guarantee all to detect local extremum at metric space and two dimensional image space.

(2) by fitting three-dimensional quadratic function accurately to determine the position and the yardstick of key point, remove the key point and the unsettled skirt response point (because the DoG operator can produce stronger skirt response) of low contrast simultaneously, to strengthen coupling stability, to improve noise resisting ability.

(3) utilize the gradient direction distribution character of key point neighborhood territory pixel to be each key point assigned direction parameter, make operator possess rotational invariance.

$m(x,y)=\sqrt{{(L(x+l,y)-L(x-l,y))}^2+{(L(x,y+1)-L(x,y)-1)}^2}$

θ(x，y)＝atan?2((L(x，y+1)-L(x，y-1))/(L(x+1，y)-L(x-1，y)))

Above-mentioned formula is that (x y) locates the mould value and the direction formula of gradient.Wherein the used yardstick of L is each key point yardstick at place separately.

(4) generate the SIFT proper vector.At first coordinate axis is rotated to be the direction of key point, to guarantee rotational invariance.As shown in Figure 3, next be that 8 * 8 window is got at the center with the key point.Calculate the gradient orientation histogram of 8 directions then on per 4 * 4 fritter, draw the accumulated value of each gradient direction, the compute gradient direction histogram is as the feature of each key point.

Step S102: the unique point that use characteristic point matching module treatment step 1 obtains, select the most similar l to unique point.

After the SIFT processing, obtained m unique point on the reference diagram

, corresponding local feature is

, and on the figure n unique point arranged in real time

, corresponding local feature is

Because each local feature that SIFT extracts is a kind of histogram, so we use following formula to calculate similarity between any two unique points:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))---\left(1\right)$

Wherein, B represents the total interval number of local histogram,

The expression vector

U element, The expression vector

U element.

We calculate with formula (1)

With

Similarity between any two, the l that selects the similarity maximum then is to unique point.

Step S103: use the l that obtains in the matching characteristic point checking module verification step 2 to unique point, the mispairing point of rejecting wherein is right, keeps L to unique point.

Consider the situation that may occur mispairing in the Feature Points Matching module, use l that random sampling consistency algorithm (RANSAC) verifies that the Feature Points Matching module obtains to unique point in one embodiment of the present of invention, the mispairing point of rejecting wherein is right.

Specific practice is as follows:

At first select two points from data acquisition randomly, these two points are determined straight line, and the point in this straight line certain distance scope is called the support of this straight line.Select at random to repeat for several times, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be.Point in the error distance scope that fits is called interior point, otherwise then is exterior point.All exterior points are removed, and point is as data in only keeping.After the processing of random sampling consistency algorithm, can obtain L to unique point.

Step S104: use the L among the step S103 that unique point is calculated the affined transformation coefficient of scheming reference diagram in real time, promptly the translation position (u, v), anglec of rotation θ and scaling s, then, current figure is centered on (u, the v) anglec of rotation-θ, convergent-divergent 1/s doubly obtains the image behind the registration again, output.

Scheme I in real time _tWith respect to reference to figure I ₀The affined transformation that takes place mainly comprises three kinds of translation, rotation and convergent-divergents.

Suppose to scheme in real time I _tWith respect to reference to figure I ₀The affined transformation that takes place is: (convergent-divergent s doubly for u, v) anglec of rotation θ around point.So I ₀Last arbitrfary point (x ₀, y ₀) pass through as obtaining (x behind the down conversion _t, y _t):

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]---\left(2\right)$

L that will be by matching characteristic point checking module uses the method for least square can obtain parameter to unique point substitution equation (2), (u, v), θ and s.At last, with us with I _tAround (convergent-divergent 1/s doubly obtains the image behind the registration, output again for u, the v) anglec of rotation-θ.

The present invention uses the SIFT conversion to extract image local feature, and the feature of extraction all remains unchanged to image zoom, rotation even affined transformation.The unique point of using the checking of random sampling coherence method to extract is right, and the point that has reduced mispairing is right, has improved the levels of precision of coupling.Use least square method to calculate affine transformation parameter, better than direct solving equation organizing, stability.

See also shown in Figure 4, the figure registration system 4 that two width of cloth or the multiple image of Same Scene are spatially aimed at of the present invention, it comprises: image characteristic point extraction unit 41, Feature Points Matching unit 42, matching characteristic point checking unit 43 and affined transformation unit 44.

Image characteristic point extraction unit 41 is used for obtaining respectively the image that two width of cloth need aim at, and (wherein the image as standard is a reference diagram, the current image that obtains is real-time figure) unique point, in one embodiment of the invention, described image characteristic point extraction unit adopts the SIFT conversion to obtain the unique point of reference diagram and real-time figure respectively.

Feature Points Matching unit 42 is used for the unique point of two width of cloth figure is mated.After the SIFT processing, on reference diagram, can obtain the several features point, also can obtain the several features point on the figure in real time.Because each local feature that SIFT extracts is a kind of histogram, the similarity among Feature Points Matching unit use formula calculating reference diagram and the real-time figure between any two unique points is selected the some to unique point of similarity maximum.

Matching characteristic point verifies that some that 43 pairs of Feature Points Matching unit, unit obtain verify unique point.Because may occur the situation of mispairing in the Feature Points Matching module, matching characteristic point checking unit uses random sampling consistency algorithm (RANSAC) to verify the unique point that the Feature Points Matching module is obtained, and the mispairing point of rejecting wherein is right.

Affined transformation unit 44 is used for real-time figure is carried out affined transformation.In real time the affined transformation that takes place with respect to reference diagram of figure mainly comprises three kinds of translation, rotation and convergent-divergents in practice.If in real time figure is that rotation or convergent-divergent have taken place with respect to reference diagram, the affined transformation unit carries out corresponding opposite spin or convergent-divergent with real-time figure so, makes itself and reference diagram registration.

Above-mentioned explanation has fully disclosed the specific embodiment of the present invention.It is pointed out that and be familiar with the scope that any change that the person skilled in art does the specific embodiment of the present invention does not all break away from claims of the present invention.Correspondingly, the scope of claim of the present invention also is not limited only to previous embodiment.

Claims (9)

1. method for registering images, it comprises:

Calculate the yardstick invariant features conversion (SIFT conversion) of real-time figure and reference diagram, schemed unique point in real time with reference diagram;

From the aforementioned unique point of aforementioned real-time figure and reference diagram, select the some of similarity maximum to unique point;

The some of similarity maximum in aforementioned real-time figure and the reference diagram are mated checking to unique point, and the mispairing point of rejecting wherein is right, some to unique point after being verified;

According to some after the aforementioned authentication unique point is determined the affined transformation function parameters that realtime graphic takes place for reference diagram, figure is in real time radiated image after conversion obtains registration according to aforementioned affine transformation parameter.

2. method for registering images as claimed in claim 1, it is characterized in that: each local feature that the conversion of described yardstick invariant features is obtained is a histogram, described select the similarity maximum from the aforementioned unique point of aforementioned real-time figure and reference diagram some calculate similarity between any two unique points to unique point by formula, and described formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B represents the total interval number of local histogram,

The expression vector

U element,

Be the local feature vectors of reference diagram correspondence,

The expression vector

U element,

For scheming corresponding local feature vectors in real time.

3. method for registering images as claimed in claim 1 is characterized in that: described similarity maximum in aforementioned real-time figure and the reference diagram some are mated checking to unique point is to adopt the random sampling consistency algorithm to verify that it comprises:

Select two points from data acquisition randomly, these two points are determined straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select at random to repeat for several times, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

4. method for registering images as claimed in claim 1 is characterized in that: the affined transformation function of described real-time figure and reference diagram is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate (x of the point among the real-time figure ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

To use the method for least square can obtain parameter by the unique point substitution equation of matching characteristic point checking, (u, v), θ and s last, will scheme in real time around (convergent-divergent 1/s doubly can obtain the image behind the registration again for u, the v) anglec of rotation-θ.

5. figure registration system, it comprises:

The image characteristic point extraction unit is used for obtaining respectively the real-time figure of needs aligning and the unique point of reference diagram image;

The Feature Points Matching unit, the unique point that is used for two width of cloth figure that the image characteristic point extraction unit is obtained is mated, and obtains the unique point of similarity maximum among two width of cloth figure;

Matching characteristic point checking unit is used for somely unique point being verified the mispairing point of rejecting wherein is right to what the Feature Points Matching unit obtained, some to unique point after being verified;

The affined transformation unit calculates the affine transformation parameters of figure and reference diagram in real time according to some after the checking to unique point, and the described parameter of foundation is radiated image after conversion obtains registration to real-time figure.

6. figure registration system as claimed in claim 5 is characterized in that: described image characteristic point extraction unit is to adopt yardstick invariant features conversion (SIFT conversion) to obtain the unique point of real-time figure and reference diagram image.

7. figure registration system as claimed in claim 6, it is characterized in that: each local feature that the conversion of described yardstick invariant features is obtained is a histogram, described Feature Points Matching unit calculates similarity between any two unique points by formula, obtain the unique point of similarity maximum among two width of cloth figure, described formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B represents the total interval number of local histogram, The expression vector

U element,

Be the local feature vectors of reference diagram correspondence,

The expression vector U element,

For scheming corresponding local feature vectors in real time.

8. figure registration system as claimed in claim 5 is characterized in that: described matching characteristic point checking unit is to adopt the random sampling consistency algorithm to come the unique point that the Feature Points Matching unit obtains is verified that described algorithm comprises:

Select two points from data acquisition randomly, these two points are determined straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select at random to repeat for several times, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

9. method for registering images as claimed in claim 5 is characterized in that: described affined transformation unit comprises the affined transformation function of real-time figure and reference diagram, and described function is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point among the real-time figure, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

Described affined transformation unit will use the method for least square can obtain parameter, (u by the unique point substitution equation of matching characteristic point checking, v), θ and s, last, will scheme in real time around (u, the v) anglec of rotation-θ, convergent-divergent 1/s doubly can obtain the image behind the registration again.

Images