CN102005047B - Image registration system and method thereof - Google Patents

Abstract

The invention provides an image registration system and a method thereof. The method comprises the following steps of: firstly calculating scale invariant feature transform (SIFT transform) of a real-time image and a reference image to obtain feature points of the real-time image and the reference image; selecting a plurality of pairs of feature points with maximum similarity from the feature points of the real-time image and the reference image; carrying out matching verification of the plurality of pairs of feature points with maximum similarity selected from the feature points of the real-time image and the reference image and rejecting a mismatching point pair therein to obtain a plurality of pairs of feature points which are verified; according to the plurality of pairs of feature points which are verified, determining the parameter of an affine transformation function of the real-time image relative to the reference image; and carrying out radioactive transform of the real-time image according to the affine transformation parameter to obtain the registered image. The system and the method are difficult to influence and high in registration accuracy.

Description

Figure registration system and method thereof

[technical field]

Field of image recognition of the present invention, particularly figure registration system and method thereof about two width of cloth or the multiple image of Same Scene are spatially aimed at.

[background technology]

Image registration is spatially to aim at two width of cloth or the multiple image of Same Scene.It is widely used in numerous art of image analysis, like medical science, remote Sensing Image Analysis, image co-registration, image retrieval, Target Recognition etc.

The present the most frequently used method for registering images that is based on characteristic.(wherein the image as standard becomes reference diagram for registration two width of cloth images; The current image that obtains becomes real-time figure); Need from image, extract characteristic, and these characteristics are set up corresponding relationship, then through finding the solution the characteristic corresponding relation; Schemed in real time and reference diagram between transformation relation, will scheme in real time at last according to the geometric relationship unscented transformation of trying to achieve to needed form.

Existing image registration based on characteristic mainly is to use the angle point of Harris Corner Detection device extraction as registration features.The disadvantage of angle point characteristic is responsive to noise ratio, and along with the difference of image resolution ratio, angle point is easy to generate drift, and Harris Corner Detection operator is not that yardstick is constant.

[summary of the invention]

The object of the present invention is to provide a kind of not susceptible to and the high method for registering images of registration accuracy.

Another object of the present invention is to provide a kind of not susceptible to and the high figure registration system of registration accuracy.

For reaching aforementioned purpose, a kind of method for registering images of the present invention, it comprises:

The unique point with reference diagram is schemed in the yardstick invariant features conversion (SIFT conversion) of calculating real-time figure and reference diagram in real time;

From the aforementioned unique point of aforementioned real-time figure and reference diagram, select the some of similarity maximum to unique point;

Maximum some of similarity in aforementioned real-time figure and the reference diagram are mated checking to unique point, and the mispairing point of rejecting wherein is right, some to unique point after obtaining verifying;

According to some after the aforementioned authentication unique point is confirmed the affined transformation function parameters that realtime graphic takes place for reference diagram;

With scheming to radiate the image after conversion obtains registration in real time according to aforementioned affine transformation parameter.

Further; Each local feature that the conversion of said yardstick invariant features is obtained is a histogram; Saidly from the aforementioned unique point of aforementioned real-time figure and reference diagram, select maximum some of similarity unique point is calculated the similarity between any two unique points through formula, said formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B representes the total interval number of local histogram; U element of expression vector ;

is the corresponding local feature vectors of reference diagram; U element of

expression vector

, are to scheme corresponding local feature vectors in real time.

It is further, said that maximum some of similarity in aforementioned real-time figure and the reference diagram are mated checking to unique point is to adopt the random sampling consistency algorithm to verify that it comprises:

From data acquisition, select two points randomly, these two points are confirmed straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select repetition for several times at random, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

Further, the affined transformation function of said real-time figure and reference diagram is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point in scheming in real time, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

To use the method for least square can obtain parameter through the unique point substitution equation of matched feature points checking, (u, v), θ and s last, will scheme in real time around (convergent-divergent 1/s doubly can obtain the image behind the registration again for u, the v) anglec of rotation-θ.

For reaching aforementioned another purpose, a kind of figure registration system of the present invention, it comprises:

The image characteristic point extraction unit is used for obtaining respectively the real-time figure of needs aligning and the unique point of reference diagram image;

The Feature Points Matching unit, the unique point of two width of cloth figure that are used for the image characteristic point extraction unit is obtained is mated, and obtains similarity biggest characteristic point among two width of cloth figure;

The matched feature points authentication unit is used for some unique point being verified that to what the Feature Points Matching unit obtained the mispairing point of rejecting wherein is right, some to unique point after obtaining verifying;

The affined transformation unit according to the some affine transformation parameters to unique point calculating real-time figure and reference diagram after the checking, radiates the image after conversion obtains registration according to said parameter to real-time figure.

Further, said image characteristic point extraction unit is to adopt yardstick invariant features conversion (SIFT conversion) to obtain the unique point of real-time figure and reference diagram image.

Further, each local feature that the conversion of said yardstick invariant features is obtained is a histogram, and said Feature Points Matching unit calculates the similarity between any two unique points through formula, obtains similarity biggest characteristic point among two width of cloth figure, and said formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B representes the total interval number of local histogram; U element of

expression vector

;

is the corresponding local feature vectors of reference diagram; U element of

expression vector

,

are to scheme corresponding local feature vectors in real time.

Further, said matched feature points authentication unit is to adopt the random sampling consistency algorithm to come the unique point that the Feature Points Matching unit obtains is verified that said algorithm comprises:

From data acquisition, select two points randomly, these two points are confirmed straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select repetition for several times at random, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

Further, said affined transformation unit comprises the affined transformation function of real-time figure and reference diagram, and said function is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point in scheming in real time, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

Said affined transformation unit will use the method for least square can obtain parameter, (u through the unique point substitution equation of matched feature points checking; V), θ and s, last, will scheme in real time around (u; The v) anglec of rotation-θ, convergent-divergent 1/s doubly can obtain the image behind the registration again.

Compared with prior art, the present invention uses the SIFT conversion to extract image local feature, and the characteristic of extraction all remains unchanged to image zoom, rotation even affined transformation.The unique point of using the checking of random sampling coherence method to extract is right, and the point that has reduced mispairing is right, has improved the levels of precision of coupling.Use least square method to calculate affine transformation parameter, better than direct solving equation organizing, stability.Total system and method thereof be susceptible to and registration accuracy height not.

[description of drawings]

Fig. 1 is the process flow diagram of the method for image registration of the present invention.

Fig. 2 carries out the synoptic diagram that key point detects for using the SIFT conversion.

Fig. 3 is the key point characteristic synoptic diagram of SIFT conversion.

Fig. 4 is the structured flowchart of figure registration system of the present invention.

[embodiment]

Alleged here " embodiment " or " embodiment " are meant special characteristic, structure or the characteristic that can be contained at least one implementation of the present invention.Different in this manual local " in one embodiment " that occur not are all to refer to same embodiment, neither be independent or optionally mutually exclusive with other embodiment embodiment.

Image registration is spatially to aim at two width of cloth or the multiple image of Same Scene; The present invention proposes a kind of method for registering images based on SIFT conversion (full name is Scale Invariant Feature Transformation, i.e. yardstick invariant features conversion).Method for registering images of the present invention is described for example with registration two width of cloth images (wherein the image as standard becomes reference diagram, and the current image that obtains becomes real-time figure) below.

See also shown in Figure 1, method for registering images of the present invention, it comprises the steps:

The unique point with reference diagram is schemed in the SIFT conversion of step S101: calculating real-time figure and reference diagram in real time.

SIFT conversion (full name is Scale Invariant Feature Transformation, i.e. yardstick invariant features conversion) is present the most frequently used local feature method for distilling.This conversion can extract the image local feature that image zoom, rotation even affined transformation are all remained unchanged.

The method of SIFT conversion comprises the steps:

(1) at first image is carried out the metric space extreme value and detect, confirm the key point position and belong to yardstick with preliminary.As shown in Figure 2, when detecting the yardstick spatial extrema, the pixel that is labeled as cross among the figure need the attendant of a stage actor draw together same yardstick around 9 * 2 pixels of neighborhood altogether 26 pixels compare, to guarantee all to detect local extremum at metric space and two dimensional image space.

(2) through fitting three-dimensional quadratic function accurately to confirm the position and the yardstick of key point; Remove the key point and the unsettled skirt response point (because the DoG operator can produce stronger skirt response) of low contrast simultaneously, to strengthen coupling stability, to improve noise resisting ability.

(3) utilize the gradient direction distribution character of key point neighborhood territory pixel to be each key point assigned direction parameter, make operator possess rotational invariance.

$m(x,y)=\sqrt{{(L(x+l,y)-L(x-l,y))}^2+{(L(x,y+1)-L(x,y)-1)}^2}$

θ(x，y)＝atan?2((L(x，y+1)-L(x，y-1))/(L(x+1，y)-L(x-1，y)))

Above-mentioned formula is that (x y) locates the mould value and the direction formula of gradient.The yardstick that belongs to separately for each key point of the used yardstick of L wherein.

(4) generate the SIFT proper vector.At first coordinate axis is rotated to be the direction of key point, to guarantee rotational invariance.As shown in Figure 3, next be that 8 * 8 window is got at the center with the key point.On per 4 * 4 fritter, calculate the gradient orientation histogram of 8 directions then, draw the accumulated value of each gradient direction, the compute gradient direction histogram is as the characteristic of each key point.

Step S102: the unique point that use characteristic point matching module treatment step 1 obtains, select the most similar l to unique point.

After the SIFT processing; Obtained m unique point

on the reference diagram; Corresponding local feature is

; And on the figure n unique point

being arranged in real time, corresponding local feature is

.Because each local feature that SIFT extracts is a kind of histogram, so we use following formula to calculate the similarity between any two unique points:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))---\left(1\right)$

Wherein, B representes the total interval number of local histogram; U element of

expression vector

, u element of

expression vector

.

We calculate

and

similarity between any two with formula (1), select the maximum l of similarity then to unique point.

Step S103: use the l that obtains in the matched feature points authentication module verification step 2 to unique point, the mispairing point of rejecting wherein is right, keeps L to unique point.

Consider the situation that possibly occur mispairing in the Feature Points Matching module, use l that random sampling consistency algorithm (RANSAC) verifies that the Feature Points Matching module obtains to unique point in one embodiment of the present of invention, the mispairing point of rejecting wherein is right.

Specific practice is following:

At first from data acquisition, select two points randomly, these two points are confirmed straight line, and the point in this straight line certain distance scope is called the support of this straight line.Select repetition for several times at random, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be.Point in the error distance scope that fits is called interior point, otherwise then is exterior point.All exterior points are removed, and point is as data in only keeping.After the processing of random sampling consistency algorithm, can obtain L to unique point.

Step S104: use L among the step S103 to the affined transformation coefficient of unique point calculating real-time figure to reference diagram, promptly translation location (u, v), anglec of rotation θ and scaling s; Then, current figure is centered on (u, the v) anglec of rotation-θ; Convergent-divergent 1/s doubly obtains the image behind the registration again, output.

Scheme I in real time _tWith respect to reference to figure I ₀The affined transformation that takes place mainly comprises three kinds of translation, rotation and convergent-divergents.

Suppose to scheme in real time I _tWith respect to reference to figure I ₀The affined transformation that takes place is: (convergent-divergent s doubly for u, v) anglec of rotation θ around point.So I ₀Last arbitrfary point (x ₀, y ₀) pass through as obtaining (x behind the down conversion _t, y _t):

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]---\left(2\right)$

L that will be through the matched feature points authentication module uses the method for least square can obtain parameter to unique point substitution equation (2), (u, v), θ and s.At last, with us with I _tAround (convergent-divergent 1/s doubly obtains the image behind the registration, output again for u, the v) anglec of rotation-θ.

The present invention uses the SIFT conversion to extract image local feature, and the characteristic of extraction all remains unchanged to image zoom, rotation even affined transformation.The unique point of using the checking of random sampling coherence method to extract is right, and the point that has reduced mispairing is right, has improved the levels of precision of coupling.Use least square method to calculate affine transformation parameter, better than direct solving equation organizing, stability.

See also shown in Figure 4; The figure registration system of spatially aiming at two width of cloth or the multiple image of Same Scene 4 of the present invention, it comprises: image characteristic point extraction unit 41, Feature Points Matching unit 42, matched feature points authentication unit 43 and affined transformation unit 44.

Image characteristic point extraction unit 41 is used for obtaining respectively the image that two width of cloth need aim at, and (wherein the image as standard is a reference diagram; The current image that obtains is real-time figure) unique point; In one embodiment of the invention, said image characteristic point extraction unit adopts the SIFT conversion to obtain reference diagram and the unique point of scheming in real time respectively.

Feature Points Matching unit 42 is used for the unique point of two width of cloth figure is mated.After the SIFT processing, on reference diagram, can obtain the several features point, also can obtain the several features point on the figure in real time.Because each local feature of SIFT extraction is a kind of histogram, the similarity among Feature Points Matching unit use formula calculating reference diagram and the real-time figure between any two unique points is selected the some to unique point of similarity maximum.

43 pairs of Feature Points Matching unit of matched feature points authentication unit obtain somely verifies unique point.Because possibly occur the situation of mispairing in the Feature Points Matching module, the matched feature points authentication unit uses random sampling consistency algorithm (RANSAC) to verify the unique point that the Feature Points Matching module is obtained, and the mispairing point of rejecting wherein is right.

Affined transformation unit 44 is used for real-time figure is carried out affined transformation.In real time the affined transformation that takes place with respect to reference diagram of figure mainly comprises three kinds of translation, rotation and convergent-divergents in reality.If in real time figure is that rotation or convergent-divergent have taken place with respect to reference diagram, the affined transformation unit will scheme to carry out corresponding opposite spin or convergent-divergent in real time so, make itself and reference diagram registration.

Above-mentioned explanation has fully disclosed embodiment of the present invention.It is pointed out that any change that technician's specific embodiments of the invention of being familiar with this field is done does not all break away from the scope of claims of the present invention.Correspondingly, the scope of claim of the present invention also is not limited only to previous embodiment.

Claims (6)

1. method for registering images, it comprises:

The unique point with reference diagram is schemed in the yardstick invariant features conversion (SIFT conversion) of calculating real-time figure and reference diagram in real time, and each local feature that the conversion of said yardstick invariant features is obtained is a histogram;

From the aforementioned unique point of aforementioned real-time figure and reference diagram, select the some to unique point of similarity maximum, calculate the similarity between any two unique points through formula, said formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B representes the total interval number of local histogram; U element of

expression vector

;

is the corresponding local feature vectors of reference diagram; U element of

expression vector

,

are to scheme corresponding local feature vectors in real time;

Maximum some of similarity in aforementioned real-time figure and the reference diagram are mated checking to unique point, and the mispairing point of rejecting wherein is right, some to unique point after obtaining verifying;

According to some after the aforementioned authentication unique point is confirmed the affined transformation function parameters that realtime graphic takes place for reference diagram, figure is in real time radiated the image after conversion obtains registration according to aforementioned affine transformation parameter.

2. method for registering images as claimed in claim 1 is characterized in that: said maximum some of similarity in aforementioned real-time figure and the reference diagram are mated checking to unique point is to adopt the random sampling consistency algorithm to verify that it comprises:

From data acquisition, select two points randomly, these two points are confirmed straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select repetition for several times at random, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

3. method for registering images as claimed in claim 1 is characterized in that: the affined transformation function of said real-time figure and reference diagram is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point in scheming in real time, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

To use the method for least square can obtain parameter through the unique point substitution equation of matched feature points checking, (u, v), θ and s last, will scheme in real time around (convergent-divergent 1/s doubly can obtain the image behind the registration again for u, the v) anglec of rotation-θ.

4. figure registration system, it comprises:

The image characteristic point extraction unit is used for obtaining the real-time figure of needs aligning and the unique point of reference diagram image respectively, and it adopts yardstick invariant features conversion (SIFT conversion) to obtain the unique point of real-time figure and reference diagram image;

The Feature Points Matching unit; The unique point of two width of cloth figure that are used for the image characteristic point extraction unit is obtained is mated; Obtain similarity biggest characteristic point among two width of cloth figure, each local feature that the conversion of said yardstick invariant features is obtained is a histogram, and said Feature Points Matching unit calculates the similarity between any two unique points through formula; Obtain similarity biggest characteristic point among two width of cloth figure, said formula is:

$s(p_i,q_j)=\underset{u=1}{\overset{B}{\mathrm{\Σ}}}\min ({\stackrel{\→}{f}}_i^p\left(u\right),{\stackrel{\→}{f}}_j^q\left(u\right))$

Wherein, B representes the total interval number of local histogram; U element of

expression vector

;

is the corresponding local feature vectors of reference diagram; U element of

expression vector

,

are to scheme corresponding local feature vectors in real time;

The matched feature points authentication unit is used for some unique point being verified that to what the Feature Points Matching unit obtained the mispairing point of rejecting wherein is right, some to unique point after obtaining verifying;

The affined transformation unit according to the some affine transformation parameters to unique point calculating real-time figure and reference diagram after the checking, radiates the image after conversion obtains registration according to said parameter to real-time figure.

5. figure registration system as claimed in claim 4 is characterized in that: said matched feature points authentication unit is to adopt the random sampling consistency algorithm to come the unique point that the Feature Points Matching unit obtains is verified that said algorithm comprises:

From data acquisition, select two points randomly, these two points are confirmed straight line, and the point in this straight line certain distance scope is called the support of this straight line;

Select repetition for several times at random, it is fitting of sample point set that the straight line with maximum support feature set is confirmed to be; Point in the error distance scope that fits is called interior point, otherwise then is exterior point; All exterior point is removed, in only keeping point as data, after handling through the random sampling consistency algorithm, some after can obtaining to verify to unique point.

6. figure registration system as claimed in claim 4 is characterized in that: said affined transformation unit comprises the affined transformation function of real-time figure and reference diagram, and said function is:

$\left[\begin{array}{c}{x}_t\\ y_t\end{array}\right]=s\left[\begin{array}{cc}\cos \mathrm{\θ}& \sin \mathrm{\θ}\\ -\sin \mathrm{\θ}& \cos \mathrm{\θ}\end{array}\right]\left[\begin{array}{c}{x}_0-u\\ y_0-v\end{array}\right]+\left[\begin{array}{c}\mathrm{su}\\ \mathrm{sv}\end{array}\right]$

(x wherein _t, y _t) be the coordinate of the point in scheming in real time, (x ₀, y ₀) be the coordinate of the point in the reference diagram, reference diagram with respect to real-time figure be around point (u, v) anglec of rotation θ, convergent-divergent s are doubly;

Said affined transformation unit will use the method for least square can obtain parameter, (u through the unique point substitution equation of matched feature points checking; V), θ and s, last, will scheme in real time around (u; The v) anglec of rotation-θ, convergent-divergent 1/s doubly can obtain the image behind the registration again.

Images