CN1251142C - Multi-source image registering method on the basis of contour under rigid body transformation - Google Patents

Abstract

The present invention relates to a multi-source image registering method on the basis of a contour under the transformation of a rigid body. After the input multi-source image respectively has noise pretreatment and filtering pretreatment, the multi-source image has edge detection, every two detected edges are linked, and a closed contour or an open contour is obtained. The closed contours are matched by moment invariant, and the open contours are matched by the method of chain code representation. Then, the open contours are changed into closed contour pairs by connecting two end points of each open contour in the open contours by straight lines. At last, the centroid and the minimum rotary inertia shaft of the closed contours in all the closed contour pairs are obtained, the rotation quantity and the translation quantity between two images to be registered are obtained, and thereafter, the registering is realized. The present invention avoids registering error brought by the adoption of control points composed of the centroid of the closed contours or angle points of the open contours in the existing registering method. The present invention can be used for civil or military fields such as remote sensing, safe monitoring, multiple-source airborne image combination by machine carrier, image tracking, etc.

Description

Under the rigid body translation based on the multi-source image method for registering of profile

Technical field:

The present invention relates under a kind of rigid body translation multi-source image method for registering based on profile, be used to solve multi-source image registration problems, all be widely used at civil area such as remote sensing, safety monitoring and military fields such as the fusion of airborne multi-source image, image tracking with rigid body distortion.

Background technology:

Image registration is the process that two width of cloth images or the multiple image to the Same Scene of taking from different time, different sensors or different visual angles mates.Piece image and another width of cloth image are carried out image registration, need piece image is carried out a series of coordinate transform, these conversion can be divided into rigid body translation, affined transformation, projective transformation and nonlinear transformation.If the distance of the point-to-point transmission in first width of cloth image still remains unchanged after in transforming to second width of cloth image, then this conversion is called rigid body translation, that is to say only to have rotation and displacement between two images subject to registration.Method for registering images under the rigid body translation at present commonly used can be summed up as method for registering based on the gray scale method for registering images of simple crosscorrelation, mutual information (for example based on), based on the method for registering of the frequency domain method for registering images of fast fourier transform, wavelet transformation, pyramid transform (for example based on), based on the method for registering of feature (as edge, angle point, profile or target etc.) etc.

The multi-source image registration is to two width of cloth images of taking from the different sensors Same Scene or process that multiple image mates.The different source images that different sensors obtains have very big-difference on the intensity profile characteristic of pixel.The general image registration algorithm based on half-tone information (as image registration algorithm), can not well be applied in the multi-source image registration based on the image registration algorithm of frequency domain based on mutual information or simple crosscorrelation.The method for registering images of using always in the multi-source image registration that is based on feature.Although between the intensity profile characteristic of multi-source image pixel very big-difference is arranged, some apparent contours of object all can be kept in two width of cloth images preferably, and these contour features can be used as with reference to carrying out image registration.At present the image registration algorithm based on profile generally is to adopt the barycenter of closed contour to carry out registration as the reference mark, but the number of closed contour is less even do not possess closed profile in the real image, occurs maximum profiles of still opening in the image.Other has some algorithms is that the barycenter of closed contour and the angle point of opening profile are carried out registration as the reference mark, but because the difference of multi-source sensor imaging character, the contour shape that same object became is also incomplete same, adopts the angle point of holding profile can influence registration accuracy as the reference mark.

Summary of the invention:

The objective of the invention is to deficiency at the prior art existence, multi-source image method for registering based on profile is provided under a kind of rigid body translation, avoid generally based in the method for registering of profile owing to adopt the barycenter of closed contour or open the registration error that the angle point of profile brings as the reference mark, the less and registration accuracy height of operand.

For achieving the above object, in the technical scheme of the present invention, at first the input multi-source image being carried out noise filtering respectively handles, to suppress the influence that picture noise separately extracts profile to the full extent, again pretreated multi-source image is carried out rim detection, edge pixel after detecting is separately linked, can obtain profile closed or that open.In order to utilize the contour feature after the detection to carry out registration, must carry out outline to the profile after all detect in two width of cloth images, it is right to find out the identical coupling profile with one-to-one relationship of shape between two width of cloth image outlines after extract, and comprises that closed contour is to open profile right.Using the square invariant for closed contour mates, mate for the method for opening profile application chain representation, it is right to being converted into closed contour to open profile then, ask for the direction of the barycenter and the minimum rotation axis of inertia of all closed contour centering closed contours at last, and utilize the right barycenter of all closed contours and the minimum rotation axis of inertia (being major axis) to carry out registration as feature, obtain two rotation amount and translational movements between the image subject to registration, realize registration.

Method of the present invention comprises following concrete steps:

1, image pre-service and profile extract: at first adopt the method for medium filtering that two width of cloth of original input are treated

The registration multi-source image carries out the noise filtering pre-service respectively, makes an uproar to suppress separately image to the full extent

The influence that sound extracts profile is carried out profile to pretreated image again and is extracted, and profile extracts bag

Draw together rim detection and edge link.Adopt the Canny operator that pretreated multi-source image is carried out

The Canny operator edge detection, according to the spatial relation of edge pixel with adjacent edge pixel

Couple together the profile of compositing area, comprise closed profile and the profile of opening.Calculate for improving registration

The speed of method only keeps in the image length greater than the profile of a certain threshold value.

2, outline: carry out registration in order to utilize the contour feature after the extraction, must be to institute in two width of cloth images

There is the profile after the extraction to carry out outline.The purpose of outline is exactly to extract from above-mentioned profile

After two width of cloth image outlines between find out the identical coupling profile of shape with one-to-one relationship

Right, comprise that closed contour is to open profile right.Use normalized square invariant for closed contour

Mate, if the distance between the normalized square invariant of two closed contours is about to less than a certain threshold value

It is right that this two closed contour is labeled as the closed contour of a coupling, uses profile chain code table for opening profile

The method of showing is mated.

The profile of 3, rejecting mistake coupling is right: according to the relation of the position between the coupling profile that previous step obtained,

The profile of rejecting wherein issuable mistake coupling is right.The right distance of definition closed contour is closed wheel

Distance between wide centering two closed contour barycenter, definition open the right distance of profile for open profile on institute

The average of distance between corresponding point arranged.Ask for right the opening of all closed contours apart from summation and all

Profile right apart from summation, both additions, and divided by profile to number (comprise closed contour to

Open profile to), thereby obtain profile to mean distance, if the right distance of certain profile is right with profile

The absolute value of the difference of mean distance surpasses some threshold values, is about to this profile to thinking what mistake was mated

Profile is right, and with its rejecting.

4, will mate to open profile right to being converted into closed profile: method is to open the profile of profile centering

Thereby two end points connect the profile that forms sealing with straight-line segment.

5, ask for the barycenter of all closed contour centering closed contours and the deflection of the minimum rotation axis of inertia: head

Be expressed as enclosed region after earlier the closed contour zone being filled, ask for the barycenter of this enclosed region then

With the direction of the minimum rotation axis of inertia direction as the barycenter and the moment of inertia axle of this closed contour

The angle.

6, ask for registration parameter: the barycenter of all closed contour centering closed contours of asking for according to previous step and

Relation between the moment of inertia axle is asked for registration parameter.Suppose the image subject to registration of two original inputs

In first width of cloth image be reference picture, only second width of cloth image is carried out changes in coordinates in the registration process.

At first the closed contour of the coupling in the image subject to registration of two original inputs is used to minimum rotation

The angle of amount axle is series arrangement by size, remove several minimums with several maximums, ask for all

The residue closed contour is to the mean value of the angle of the minimum rotation axis of inertia, this mean value be two former

Rotation amount between the image subject to registration of beginning input.According to this rotation amount second width of cloth image is advanced then

The right center-of-mass coordinate poor (horizontal direction or vertical direction) of rotation back closed contour is asked in the row rotation

And sort by size, remove several minimums with maximum, ask for the average of all residue coordinate differences

Value, this mean value are the translational movement (horizontal direction between the image subject to registration of two original inputs

Or vertical direction).Registration parameter is institute's rotation amount of asking and translational movement (comprises horizontal direction and vertical

Nogata to).Second width of cloth image for the treatment of in the registering images according to registration parameter carries out coordinate transform,

Finish image registration.

The present invention carries out registration by adopting the right barycenter of all closed contours and the minimum rotation axis of inertia as feature, avoided generally based in the method for registering of profile owing to adopt the barycenter of closed contour or open the registration error that the angle point of profile brings as the reference mark, operand is little, the registration accuracy height.

The present invention can be applicable to civilian or military domain such as remote sensing, safety monitoring, the fusion of airborne multi-source image, image tracking.

Description of drawings:

Fig. 1 is based on the multi-source image method for registering process flow diagram of profile under the rigid body translation of the present invention.

The subject to registration image of Fig. 2 for adopting in the embodiment of the invention.

Wherein: Fig. 2 (a) is a reference picture I-infrared image, and Fig. 2 (b) is I '-visible images.

Fig. 3 extracts the result for profile.

Wherein: the profile of Fig. 3 (a) for extracting in the infrared image, the profile of Fig. 3 (b) for extracting in the visible images.

Fig. 4 is the outline result.

Wherein: Fig. 4 (a) is the profile after the coupling in the infrared image, and Fig. 4 (b) is the profile after the coupling in the visible images.

Fig. 5 is the barycenter and the minimum rotation axis of inertia of closed contour.

Wherein: Fig. 5 (a) is the barycenter and the minimum rotation axis of inertia of closed contour in the infrared image, and Fig. 5 (b) is the barycenter and the minimum rotation axis of inertia of closed contour in the visible images.

Fig. 6 is a registration results.

Wherein: Fig. 6 (a) is reference picture-infrared image, and Fig. 6 (b) is the visible images behind the registration.

Embodiment:

In order to understand technical scheme of the present invention better, embodiments of the present invention are further described below in conjunction with accompanying drawing.

1, image pre-service and profile extract

In the multi-source image all in various degree have a noise pollution, these noises extract the profile of image and constitute very big interference.Because medium filtering can be rejected the noise in the image preferably and can be kept the edge preferably, at first adopt median filter method that the multi-source image of importing is carried out pre-service respectively among the present invention.

Profile extracts and comprises rim detection and edge link.At first adopt the Canny operator that pretreated input picture is carried out rim detection, according to the spatial relation of edge pixel adjacent edge pixel is coupled together the profile (comprising closed profile and the profile of opening) of compositing area then.Be to improve the speed of registration Algorithm, only keep in the image length greater than the profile of a certain threshold value.

2, outline

The purpose of outline is exactly to mate profile one to one from finding out identical the having of shape between two width of cloth image outlines after extracting.Adopt the square invariant to mate for closed contour; Adopt the method for chain representation to mate for opening profile.

(1) coupling of closed contour

Closed contour is corresponding to the border in zone, and the pixel in the zone can be filled up by filling algorithm.Matching problem for closed contour has adopted invariant moments.At first closed contour is carried out the zone filling and obtain enclosed region.If represent that any two closed contour A and the pairing enclosed region of B are RA and RB, seven square invariants that propose according to M.K.Hu (1996) calculate the first six the square invariant in its corresponding seven square invariants, be designated as respectively  ' _IA, i=1,2 ..., 6} and  ' _IB, i=1,2 ..., 6}, carry out normalization to six invariant moments functional expressions then:

${\mathrm{\φ}}_i^{\′}=\frac{{\mathrm{\φ}}_i}{\sqrt{\underset{i=1}{\overset{6}{\mathrm{\Σ}}}{\mathrm{\φ}}_i^2}},---\left(1\right)$

Calculate then

$d_{\mathrm{AB}}=\sqrt{\underset{i=1}{\overset{6}{\mathrm{\Σ}}}{[{\mathrm{\φ}}_{\mathrm{iA}}^{\′}-{\mathrm{\φ}}_{\mathrm{iB}}^{\′}]}^2},---\left(2\right)$

If d _ABLess than some threshold value d, think that promptly these two closed contour A and B are right for the closed contour of coupling.

(2) open the coupling of profile

Used the method for the improved chain representation of H.Li (1995) proposition for the coupling of opening profile.The 8 adjoint point chain codes of supposing a profile are expressed as { a ₁, a ₂, a ₃..., a _n, the chain representation after the improvement is earlier to former 8 adjoint point chain code { a ₁, a ₂, a ₃..., a _nCarry out as down conversion (wherein mod represents to ask modulo operation):

Then at { b ₁, b ₂, b ₃..., b _nThereby the basis on adopt Gauss smoothly to obtain final chain representation { c to it ₁, c ₂, c ₃..., c _n.

Suppose that any two improvement chain representation of opening profile A and B to be matched are respectively { a ₁, a ₂, a ₃..., a _NAAnd { b ₁, b ₂, b ₃..., b _NB; from two open appoint respectively on the profile get length be n (n generally greater than open profile A and B minimum length 2/3) segment α and β; wherein α originates in k the marginal point of profile A, and β originates in the 1st marginal point of profile B, and then the matching degree between α and the β is defined as:

$D_{\mathrm{kl}}^n=\frac{1}{n}\underset{j=0}{\overset{n-1}{\mathrm{\Σ}}}\cos \frac{\mathrm{\π}}{4}(a_{k+j}^{\′}-b_{l+j}^{\′})---\left(4\right)$

Wherein $a_{k+l}^{\&prime;}=a_{k+i}-\frac{1}{n}\underset{j=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{a}_{k+j},0\&le;i<n,b_{k+l}^{\&prime;}=b_{k+1}-\frac{1}{n}\underset{j=0}{\overset{n-1}{\mathrm{\&Sigma;}}}{b}_{k+j},0\&le;i<n,$ Find sheet then

Optimum matching when disconnected length is n is right $(K_n,L_n)=\arg \underset{k,l}{\max }\left(D_{\mathrm{kl}}^n\right),$ Its matching degree is expressed as D _KnLn ⁿSuppose that the matching degree threshold value is decided to be D, then can try to achieve $N=\max \left\{n\right|D_{K_n{L}_n}^n>D\},$ If N is an empty set, then explanation is opened profile A and B is unmatched, and length is N on profile A and the B otherwise just can try to achieve out, and the start edge point is respectively K _NAnd L _NThe coupling profile right.

The profile of 3, rejecting mistake coupling is right

Through the coupling between the closed contour and after opening coupling between the profile, suppose to obtain the L bar and mate profile (A _i, B _i), i=1,2 ..., L, it is right that this L bar coupling profile can not be got rid of the profile that miss coupling because of shape is quite similar to the inside, if profiles that these mistakes are mated are not to getting rid of the precision that will have a strong impact on the registration parameter that subsequent process asks for.

For the coupling closed contour to (A _i, B _i), i=1,2 ..., N obtains its barycenter (CA after its zone can being filled _i, CB _i), and definition

D(A _i，B _i)＝d(CA _i，CB _i)， (5)

D (CA wherein _i, CB _i) be barycenter CA in the two dimensional surface _i, CB _iThe Euclidean distance of point-to-point transmission.

Open profile to (A for what mate _i, B _i), i=1,2 ..., M, length is L _i, A wherein _iBe the ordered list { PA of its coboundary point coordinate ₁ ⁱ, PA ₂ ⁱ, PA ₃ ⁱ..., PA _Li ⁱ, B _iBe { PB ₁ ⁱ, PB ₂ ⁱ, PB ₃ ⁱ..., PB _Li ⁱDefinition

$D(A_i,B_i)=\frac{1}{L_i}\underset{p=1}{\overset{L_i}{\mathrm{\&Sigma;}}}d(P{A}_p^i,{\mathrm{PB}}_p^i)---\left(6\right)$

D (PA wherein _p ⁱ, PB _p ⁱ) be PA on the two dimensional surface _p ⁱ, PB _p ⁱEuclidean distance between 2.

$\stackrel{\&OverBar;}{D}=\frac{1}{M+N}\underset{i=1}{\overset{M+N}{\mathrm{\&Sigma;}}}D(A_i,B_i)---\left(7\right)$

If | D (A _j, B _j)-D|＞D, D are threshold value, think that then the coupling profile is to (A _j, B _j) be that a false coupling profile is right, thereby with its rejecting.

4, will mate to open profile right to being converted into closed profile

What mate opens profile to not possessing some right character of closed contour, such as the barycenter and the minimum rotation axis of inertia.In order to utilize out profile that the information that provides is carried out registration, it is right to being converted into closed contour at first will to open profile, thus method be exactly with open profile centering open profile two end points connect to form the profile of sealing with straight-line segment.

5, ask for the direction of the barycenter and the minimum rotation axis of inertia of all closed contour centering closed contours

At first will closed contour the zone be expressed as enclosed region after filling, the direction of asking for the barycenter of this enclosed region and the minimum rotation axis of inertia then is as the barycenter of this closed contour and the deflection of moment of inertia axle (being major axis).

6, ask for registration parameter

The coordinate of two image I subject to registration and I ' under rigid body translation (x, y) with between (x ', y ') satisfy following relation (the x axle is a vertical direction, and the y axle is a horizontal direction, and initial point is a picture centre):

$\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right]=\left[\begin{array}{cc}\cos \mathrm{\&theta;}& \sin \mathrm{\&theta;}\\ -\sin \mathrm{\&theta;}& \cos \mathrm{\&theta;}\end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]+\left[\begin{array}{c}\mathrm{\&Delta;x}\\ \mathrm{\&Delta;y}\end{array}\right],---\left(8\right)$

Wherein (θ, Δ x, Δ y) is rotation amount and translational movement between two images to be asked.

Note (θ _i, x _i, y _i) (i=1,2 ..., Num, Num is the right number of closed contour of all couplings) be the direction and the barycenter of the minimum rotation axis of inertia of i the closed contour centering closed contour that is arranged in image I, (θ ' _i, x ' _i, y ' _i) be i closed contour centering be arranged in image I ' the direction and the barycenter of the minimum rotation axis of inertia of closed contour, calculate:

$\left\{\begin{array}{c}{\mathrm{\&phi;}}_i=({\mathrm{\&theta;}}_i-{\mathrm{\&theta;}}_i^{\&prime;})\\ \mathrm{\&Delta;}{x}_i=x_i\cos \mathrm{\&theta;}+y_i\sin \mathrm{\&theta;}-x_i^{\&prime;},\\ \mathrm{\&Delta;}{y}_i=-x_i\sin \mathrm{\&theta;}+y_i\cos \mathrm{\&theta;}-y_i^{\&prime;}\end{array}\right.---\left(9\right)$

Then registration parameter is calculated as follows:

$\left\{\begin{array}{c}\mathrm{\&theta;}=\frac{1}{\mathrm{Num}-2[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]}\underset{j=[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]+1}{\overset{\mathrm{Nmu}-[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]}{\mathrm{\&Sigma;}}}{\mathrm{\&phi;}}_{\mathrm{Num}:j}\\ \mathrm{\&Delta;x}=\frac{1}{\mathrm{Num}-2[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]}\underset{j=[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]+1}{\overset{\mathrm{Nmu}-[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]}{\mathrm{\&Sigma;}}}{\mathrm{\&Delta;x}}_{\mathrm{Num}:j,}\\ \mathrm{\&Delta;y}=\frac{1}{\mathrm{Num}-2[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]}\underset{j=[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]+1}{\overset{\mathrm{Nmu}-[\mathrm{Num}\&CenterDot;\mathrm{\&alpha;}]}{\mathrm{\&Sigma;}}}{\mathrm{\&Delta;y}}_{\mathrm{Num}:j}\end{array}\right.---\left(10\right)$

 wherein _Num={  _i| i=1,2 ..., Num},  _Num:jBe set  _NumIn all elements by from small to large the ordering after j element, Δ x _Num:jWith Δ y _Num:jWith  N _Um:jConnotation is similar, and α (0＜α＜1) is a fractile.

Angle and the right centroid position relation of closed contour according to the right minimum rotation axis of inertia of the above-mentioned closed contour that obtains, can obtain two between the image subject to registration rotation amount and the translational movement between two images subject to registration, realize the multi-source image registration under the rigid body translation.

In an embodiment of the present invention, experimental image adopts infrared image and visible images (as Fig. 2).During registration with infrared image as the reference image, just only visible images is done coordinate transform.After the image pre-service, from two width of cloth input pictures, carry out profile and extract, extract result such as Fig. 3.According to top arthmetic statement the profile after extracting is mated the result after the coupling such as Fig. 4.Ask for the barycenter and the minimum rotation axis of inertia (as Fig. 5) of each profile of closed contour centering then, the barycenter and the minimum rotation axis of inertia according to each profile of closed contour centering are asked for registration parameter, according to registration parameter, treat registering images and carry out coordinate transform, the result behind the registration such as Fig. 6 then.Registration parameter (θ, Δ x, Δ y)=(8.1318 °, 20.0229,20.2923).

Claims (1)

1, under a kind of rigid body translation based on the multi-source image method for registering of profile, it is characterized in that comprising following concrete steps:

1) image pre-service and profile extract: adopt median filtering method that two multi-source images subject to registration of original input are removed the wide extraction of the laggard road wheel of the pre-service of making an uproar respectively, profile extracts and comprises rim detection and edge link, at first adopt the Canny operator that pretreated input picture is carried out rim detection, according to the spatial relation of edge pixel adjacent edge pixel is coupled together the profile of compositing area then, comprise closed contour and open profile, and only keep in the image length greater than the profile of a certain threshold value;

2) outline: it is right to find out the identical coupling profile with one-to-one relationship of shape between two width of cloth image outlines after extracting from above-mentioned profile, using normalized square invariant for closed contour mates, to be labeled as the closed contour of a coupling right if the distance between the normalized square invariant of two closed contours is about to this two closed contour less than a certain threshold value, uses the method that the profile chain code represents and mate for opening profile;

The profile of 3) rejecting mistake coupling is right: the right distance of definition closed contour is the distance between closed contour centering two closed contour barycenter, the definition right distance of opening profile is out profile average to the distance between last all corresponding point, ask for all closed contours right apart from summation and all open profile right apart from summation, both additions, and divided by profile to number, obtain profile to mean distance, if the right distance of certain profile and profile surpass some threshold values to the absolute value of the difference of mean distance, it is right to the profile of thinking the mistake coupling to be about to this profile, and with its rejecting;

4) will mate to open profile right to being converted into closed profile: thereby two end points will opening the profile of profile centering connect the profile that forms sealing with straight-line segment;

5) ask for the barycenter of all closed contour centering closed contours and the deflection of the minimum rotation axis of inertia;

6) ask for registration parameter according to the barycenter of all closed contour centering closed contours of asking for and the relation between the minimum rotation axis of inertia: establish that first width of cloth image is a reference picture in the image subject to registration of two original inputs, only second width of cloth image is carried out changes in coordinates in the registration process, at first with the closed contour of the coupling in the image subject to registration of two original inputs to the angle of minimum rotation axis of inertia series arrangement by size, remove several minimums with several maximums, ask for the mean value of all residue closed contours to the angle of the minimum rotation axis of inertia, this mean value is the rotation amount between the image subject to registration of two original inputs, according to this rotation amount second width of cloth image is rotated then, ask for the right center-of-mass coordinate difference of rotation back closed contour and sort by size, remove several minimums with maximum, ask for the mean value of all residue coordinate differences, this mean value is the translational movement between the image subject to registration of two original inputs, registration parameter is institute's rotation amount of asking and translational movement, second width of cloth image for the treatment of in the registering images according to registration parameter carries out coordinate transform, finishes image registration.

Images