CN103455992A - Method for splicing aviation multi-pixel parallel scanning images - Google Patents

Abstract

The invention provides a method for splicing aviation multi-pixel parallel scanning images. According to the method, 'dimension plus rotation' transformational models are adopted in geometric transformational models of two adjacent frames, perspective projection models of area-array cameras are not adopted, the method adapts to the geometric transformational models between the adjacent aviation multi-pixel parallel scanning images, and the overall transformation problem between the adjacent images is well solved.

Description

Splice the method that aviation is polynary and sweep image

Technical field

The present invention relates to technical field of remote sensing image processing, relate in particular to a kind of method that aviation is polynary and sweep image of splicing.

Background technology

Aviation is polynary and to sweep be the full shot camera that comes from the conventional film epoch, has very large side to field angle, characteristics that resolution is high, only needs linear array detector, just can obtain the wide cut image, usually is applied to the quick obtaining of infrared/night vision image.But, because focal length polynary and that sweep camera remains unchanged, in scanning, aircraft travels forward, and the factor such as non-linear that swings of scanning mirror, make the distortion of image comparatively complicated.In order to obtain certain regional image of ground, usually utilize polynary and sweep camera left and right sweeping by object lens in flight course and obtain image, and the adjacent image obtained in front and back designs certain overlapping according to flying speed, then by image, process and obtain area image, but because disturbing factor in aircraft flight is more, making overlapping between the area image obtained is constantly to change, and has increased complexity and difficulty that area image is processed.

At present both at home and abroad polynary to aviation and to sweep the achievement in research report that the automatic Mosaic of image processes few, and research and comparison many be the ccd image that obtains of unmanned plane and the splicing of other aviation faces system of battle formations picture.These splicing processing methods are all to extract the same place of overlapping region by matching algorithm, then calculate the spatial alternation relation of adjacent surface system of battle formations picture, complete the splicing of image on the basis of conversion.Such as the people such as Hu Qingwu (Hu Qingwu, Ai Mingyao, Yin Wanling, Yuan Hui, the full-automatic joining method research of large swing angle unmanned plane image, computer engineering, the 38th volume, the 15th phase, 2012) the full-automatic joining method of large swing angle unmanned plane image proposed, adopt the list of SIFT feature should retrain Image Matching Algorithm, calculate the optimal transform matrix of adjacent image, the multiresolution spline that has provided on this basis optimal transform matrix merges the image joint algorithm.Identical people (the Lu Heng in Shandong, Li Yongshu, He Jing, Chen Qiang, Ren Zhiming, a kind of unmanned plane image method for automatically split-jointing based on unique point, geography and Geographical Information Sciences, the 26th the 5th phase of volume, 2010) the unmanned plane image method for automatically split-jointing based on unique point, sane SIFT algorithm is introduced in unmanned plane image automatic Mosaic, and in conjunction with the characteristics of unmanned plane self, algorithm is improved, before carrying out feature point extraction by estimating that the degree of overlapping that adjacent image is asked dwindled hunting zone, application LM method is tried to achieve the accurate transformation matrix between adjacent image, complete the splicing of image and inlay.

The aerial image that current existing aerial remote sensing images splicing, especially unmanned plane obtain is all for the area array CCD camera Image Mosaics.When area array cameras obtains image, be all to obtain all pixels in field range,, when considering the transformation relation of adjacent image, be therefore all to take face battle array imaging geometry as basis simultaneously.Transformation model between adjacent image adopts the perspective projection model, needs to extract four of overlapping region or more same place reference mark and is calculated.For example above mentioned two kinds of unmanned plane image split-joint methods, be all to have adopted the perspective projection model.And a width aviation is polynary and sweep image and adopt line array CCD to obtain by scan mode, piece image is scanning imagery in chronological order, do not meet the perspective projection model of area array cameras, and in scanning process, image has local distortion.Therefore adjacent polynary and sweep between image and should consider whole transformation relation, also need to consider nonaffine deformation local in scanning process.

Summary of the invention

(1) technical matters that will solve

In view of above-mentioned technical matters, the invention provides a kind of method that aviation is polynary and sweep image of splicing, polynary and sweep image accurately to splice aviation.

(2) technical scheme

According to an aspect of the present invention, provide a kind of method that aviation is polynary and sweep image of splicing.The method comprises: steps A: in the overlapping region of current frame image and previous frame image, extract respectively two the best unique points of the same name at two two field picture two ends; Step B, calculate the scale factor m between two two field pictures, rotation parameter θ according to two the best unique points of the same name of current frame image and previous frame image; Step C, carry out coordinate transform according to the scale factor m calculated and rotation parameter θ to each pixel of current frame image; Step D extracts current frame image after the pixel coordinate conversion and the same place between the previous frame image every one section default pixel distance on direction of scanning; Step e, for each pixel of current frame image, its along slope coordinate all remains unchanged, and according to two same places of its left and right sides, adjusts each row of each pixel of present frame to coordinate; And step F, by the current frame image after adjusting together with the previous frame image combining, thereby it is polynary and sweep image to obtain spliced aviation.

(3) beneficial effect

From technique scheme, can find out, the present invention is spliced the polynary and method that sweep image of aviation and is had following beneficial effect:

(1) adopted the transformation model of " yardstick+rotation " in the geometric transformation model of adjacent two frames, and do not adopt the perspective projection model of area array cameras, adapt to the geometric transformation model between the polynary and adjacent image swept of aviation, solved preferably the integral transformation problem between adjacent image.

(2) at integral transformation model application in calculation the same place of scan image both sides calculate yardstick and the rotation parameter existed between the consecutive frame image, process by whole yardstick and rotational transform the geometry of having realized between adjacent image roughly and slightly splice.

(3) on the basis of integral transformation model, a large amount of local same place obtained according to the normalized crosscorrelation coupling carries out transformation of local coordinates and adjustment to the direction of scanning of image, has solved preferably the local geometric unevenness distortion between adjacent image.Thereby can guarantee the effect of topography's splicing

The accompanying drawing explanation

Fig. 1 is polynary according to embodiment of the present invention splicing aviation and sweeps the process flow diagram of image method;

Fig. 2 is that adjacent two frame aviations are polynary and sweep the schematic diagram that unique point in image distributes;

Fig. 3 A is for the front adjacent two frame aviations of splicing are polynary and sweep image;

Fig. 3 B utilizes method shown in Fig. 1 polynary and sweep image and carry out spliced design sketch to two frame aviations shown in Fig. 3 A.

Embodiment

For making the purpose, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and, with reference to accompanying drawing, the present invention is described in more detail.It should be noted that, in accompanying drawing or instructions description, similar or identical part is all used identical figure number.The implementation that does not illustrate in accompanying drawing or describe is form known to a person of ordinary skill in the art in affiliated technical field.In addition, although this paper can provide the demonstration of the parameter that comprises particular value, should be appreciated that, parameter is without definitely equaling corresponding value, but can in acceptable error margin or design constraint, be similar to corresponding value.The direction term of mentioning in embodiment, such as " on ", D score, 'fornt', 'back', " left side ", " right side " etc., be only the direction with reference to accompanying drawing.Therefore, the direction term of use is not to be used for limiting the scope of the invention for explanation.

The present invention is spliced the polynary and same place that sweep the method application scanning image both sides of image of aviation and is calculated yardstick and the rotation parameter existed between the consecutive frame image, by coordinate transform processing, has realized between adjacent image that geometry roughly slightly splices; On the basis of thick splicing, according to the same place of local normalized crosscorrelation coupling, the coordinate intense adjustment is carried out in the direction of scanning of image, thereby can guarantee the effect of partial splice.

In one exemplary embodiment of the present invention, provide a kind of method that aviation is polynary and sweep image of splicing.Fig. 1 is polynary according to embodiment of the present invention splicing aviation and sweeps the process flow diagram of image method.Please according to Fig. 1, the polynary and method that sweep image of the present embodiment splicing aviation comprises:

Steps A: in the overlapping region of current frame image and previous frame image, adopt yardstick invariant features conversion (SIFT) operator to extract respectively two the best unique point of the same name-first same place P at two two field picture two ends ₁with the second same place P ₂, as shown in Figure 2;

In this step, the method for utilizing yardstick invariant features transformation operator to extract the best unique point of the same name in adjacent two two field picture two ends is the general method in this area, and very roughly, it mainly comprises:

Sub-step A1, respectively for the point of interest in current frame image and previous frame imagery exploitation difference Gauss operator detected image metric space;

Sub-step A2 applies the gradient information compute gradient principal direction in this point of interest neighborhood window on the point of interest basis of detecting, and builds the proper vector of 128 dimensions that this point of interest is corresponding according to histogram of gradients;

Sub-step A3, calculate the Euclidean distance between the point of interest proper vector detected in the point of interest proper vector that detects in current frame image and previous frame image one by one, and wherein two nearest points of interest of Euclidean distance are exactly best unique point of the same name.

Step B, calculate the scale factor m between two two field pictures according to two the best unique points of the same name of current frame image and previous frame image, rotation parameter θ, wherein:

$m=\frac{\mathrm{\Δd}-\mathrm{\ΔD}}{\mathrm{\ΔD}}---\left(1\right)$

θ＝α ₁-α ₂ (2)

$\mathrm{\Δd}=\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2}---\left(3\right)$

$\mathrm{\ΔD}=\sqrt{{(X_1-X_2)}^2+{(Y_1-Y_2)}^2}---\left(4\right)$

${\mathrm{\α}}_1={\mathrm{tg}}^{-1}\left(\frac{x_1-x_2}{y_1-y_2}\right)---\left(5\right)$

${\mathrm{\α}}_2={\mathrm{tg}}^{-1}\left(\frac{X_1-X_2}{Y_1-Y_2}\right)---\left(6\right)$

Wherein: (x ₁, y ₁) and (X ₁, Y ₁) mean respectively the coordinate of the first same place P1 at previous frame image and current frame image; (x ₂, y ₂) and (X ₂, Y ₂) mean respectively the coordinate of the second same place P2 at previous frame image and current frame image; Tg ^-1mean the arc tangent trigonometric function.

Step C, carry out coordinate transform to each pixel of current frame image according to following formula according to the scale factor m calculated and rotation parameter θ:

X _new＝mcosθ(X _old-x ₁)-msinθ(Y _old-y ₁)+X ₁ (7)

Y _new＝msinθ(X _old-x ₁)+mcosθ(Y _old-y ₁)+Y ₁ (8)

Wherein: (X _old, Y _old) mean the original coordinate of processed pixel in current frame image, (X _new, Y _new) mean the coordinate after processed pixel conversion in current frame image.

In the preferred embodiment of the invention, if airborne platform flight is more steady, course is controlled well, while utilizing scale factor m and rotation parameter θ to carry out the conversion process of image coordinate of present frame.The conversion formula can simplify into:

X _new＝-mcosθ(X _old-x ₁)+X ₁ (9)

Y _new＝mcosθ(Y _old-y ₁)+Y ₁ (10)

Wherein, in formula 9 and formula 10, the implication of each parameter is identical with formula 8 with formula 7, no longer repeats herein.

Step D, adopt the normalized crosscorrelation matching process, on direction of scanning every a presetted pixel apart from extracting current frame image after the pixel coordinate conversion and the same place between the previous frame image;

The method that application normalized crosscorrelation matching process extracts current frame image and previous frame image same place is the general method in this area, and very roughly, it comprises:

Sub-step D1, extract the rectangular window image of current frame image as target image, and the center of this target image has represented an image characteristic point of present frame;

Sub-step D2, also get onesize rectangular window image at the previous frame image, and calculate the normalized crosscorrelation coefficient of these two groups of image data;

Sub-step D3, corresponding video in window position when the normalized crosscorrelation coefficient is maximum in the previous frame image by continuous calculating and the target image of relatively trying to achieve present frame, this location point has just represented in the previous frame image and the same place position with same characteristic features of current frame image unique point, has obtained same place.

In this step, default pixel distance is comprehensively determined by the stability of aircraft platform and the rate behavior of scanning, generally, the scope of this presetted pixel distance is between 50 to 200 pixels, the value of flying when more unstable correspondingly also will become less, and sweep speed is higher, value will become larger.In the present embodiment, this predeterminable range is got 100 pixels.

Step e, all pixels for current frame image keep along slope coordinate constant, and the row of adjusting each pixel are to coordinate (being direction of scanning),, the row of each pixel are to coordinate, to two nearest same places of the extraction left and right sides, the residing position of coordinate, according to following formula, calculate row after adjusting to coordinate according to this pixel column:

$C_{\mathrm{new}}=\left(\frac{Y_k-Y_{k-1}}{y_k-y_{k-1}}\right)(C_{\mathrm{old}}-y_{k-1})+Y_{k-1}---\left(9\right)$

Wherein, Y _kmean that the row in the previous frame image of k the unique point in current pixel left side are to coordinate figure, Y _k-1mean that the row in the previous frame image of k-1 the unique point in current pixel right side are to coordinate figure, y _kmean that the row in current frame image of k unique point are to coordinate figure, y _k-1mean that the row in current frame image of k-1 unique point are to coordinate figure, C _oldmean that original row of current frame image pixel are to coordinate, C _newmean that the row of image after the present frame adjustment are to coordinate, the sequence number of k representation feature point (since 1).

Step F, by the current frame image after adjusting together with the previous frame image combining, thereby it is polynary and sweep image to obtain spliced aviation.

It is polynary and sweep the method that image is realized splicing that the present embodiment has only provided two frame aviations.Polynary and sweep image for the aviation of multiframe, only each two field picture all need be carried out to top similar processing with the image spliced gets final product, while namely processing at every turn all using spliced image as the previous frame image, and the image that will need splicing is as current frame image, just obtained the aviation of multiframe splicing after processing through splicing repeatedly polynary and sweep image.

In order to verify the effect of the present embodiment, polynary and sweep imagery exploitation the present embodiment method and processed to two frame aviations shown in Fig. 3 A, Fig. 3 B is the design sketch after processing.From Fig. 3 A and Fig. 3 B, can find out: spliced image does not have gap at adjacent bound fraction, yet atural object has reached the accurate splicing of adjacent image excessively also very from accurate and smooth.

So far, by reference to the accompanying drawings the present embodiment be have been described in detail.According to above description, those skilled in the art should be spliced the polynary and method that sweep image of aviation to the present invention clearly understanding.

In addition, the above-mentioned definition to each element and method is not limited in various concrete structures, shape and the mode of mentioning in embodiment, and those of ordinary skill in the art can replace simply to it with knowing.

In sum, it is polynary and sweep in the method for image that aviation is spliced in the present invention, calculate yardstick and rotational transform parameter whole between adjacent image by the best same place that extracts the image both sides, overcome the deficiency of the perspective projection model of conventional area array cameras, solved preferably the integral transformation problem between adjacent image, and then the same place by local matching carries out partial transformation and the adjustment of coordinate to the direction of scanning of image again, solved the local geometric unevenness distortion between adjacent image.Thereby realized preferably the splicing that aviation is polynary and sweep image.

Above-described specific embodiment; purpose of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the foregoing is only specific embodiments of the invention; be not limited to the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.

Claims (10)

1. one kind splices the method that aviation is polynary and sweep image, it is characterized in that, comprising:

Steps A: in the overlapping region of current frame image and previous frame image, extract respectively two the best unique points of the same name at two two field picture two ends;

Step B, calculate the scale factor m between two two field pictures, rotation parameter θ according to two the best unique points of the same name of current frame image and previous frame image;

Step C, carry out coordinate transform according to the scale factor m calculated and rotation parameter θ to each pixel of current frame image;

Step D extracts current frame image after the pixel coordinate conversion and the same place between the previous frame image every one section default pixel distance on direction of scanning;

Step e, for each pixel of current frame image, its along slope coordinate all remains unchanged, and according to two same places of its left and right sides, adjusts each row of each pixel of present frame to coordinate; And

Step F, by the current frame image after adjusting together with the previous frame image combining, thereby it is polynary and sweep image to obtain spliced aviation.

2. method according to claim 1, is characterized in that, in described step B, according to following formula, calculates the scale factor m between two two field pictures, rotation parameter θ:

$m=\frac{\mathrm{\Δd}-\mathrm{\ΔD}}{\mathrm{\ΔD}},\mathrm{\θ}={\mathrm{\α}}_1-{\mathrm{\α}}_2$

${\mathrm{\α}}_1={\mathrm{tg}}^{-1}\left(\frac{x_1-x_2}{y_1-y_2}\right),{\mathrm{\α}}_2={\mathrm{tg}}^{-1}\left(\frac{X_1-X_2}{Y_1-Y_2}\right)$

$\mathrm{\Δd}=\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2},\mathrm{\ΔD}=\sqrt{{(X_1-X_2)}^2+{(Y_1-Y_2)}^2}$

Wherein: (x ₁, y ₁) and (X ₁, Y ₁) mean respectively the coordinate of the first same place P1 at previous frame image and current frame image; (x ₂, y ₂) and (X ₂, Y ₂) mean respectively the coordinate of the second same place P2 at previous frame image and current frame image; Tg ^-1mean the arc tangent trigonometric function.

3. method according to claim 2, is characterized in that, in described step C, according to following formula, the pixel of current frame image carried out to coordinate transform:

X _new＝mcosθ(X _old-x ₁)-msinθ(Y _old-y ₁)+X ₁

Y _new＝msinθ(X _old-x ₁)+mcosθ(Y _old-y ₁)+Y ₁

Wherein, (X _old, Y _old) mean original coordinate of processed pixel in current frame image, (X _new, Y _new) mean the coordinate after processed pixel conversion in current frame image.

4. method according to claim 2, is characterized in that, in described step C, according to following formula, the pixel of current frame image carried out to coordinate transform:

X _new＝-mcosθ(X _old-x ₁)+X ₁

Y _new＝mcosθ(Y _old-y ₁)+Y ₁

Wherein, (X _old, Y _old) mean original coordinate of processed pixel in current frame image, (X _new, Y _new) mean the coordinate after processed pixel conversion in current frame image.

5. method according to claim 1, is characterized in that, in described step e, for the row of each pixel of present frame, to coordinate, according to following formula, calculates row after adjusting to coordinate:

$C_{\mathrm{new}}=\frac{(Y_k-Y_{k-1})}{(y_k-y_{k-1})}(C_{\mathrm{old}}-y_{k-1})+Y_{k-1}$

Wherein, Y _kmean that the row in the previous frame image of k the unique point in current pixel left side are to coordinate figure, Y _k-1mean that the row in the previous frame image of k-1 the unique point in current pixel right side are to coordinate figure, y _kmean that the row in current frame image of k unique point are to coordinate figure, y _k-1mean that the row in current frame image of k unique point are to coordinate figure, C _oldmean that original row of current frame image pixel are to coordinate, C _newmean that the row of image after the present frame adjustment are to coordinate, the sequence number of k representation feature point.

6. according to the described method of any one in claim 1 to 5, it is characterized in that, in described steps A, adopt yardstick invariant features transformation operator to extract two the best unique points of the same name at two two field picture two ends.

7. method according to claim 6, is characterized in that, the step that described employing yardstick invariant features transformation operator extracts two the best unique points of the same name at two two field picture two ends comprises:

Sub-step A1, utilize respectively the point of interest in difference Gauss operator detected image metric space for current frame image and previous frame image;

Sub-step A2 applies the gradient information compute gradient principal direction in this point of interest neighborhood window on the point of interest basis of detecting, and builds the proper vector of 128 dimensions that this point of interest is corresponding according to histogram of gradients;

Sub-step A3, calculate the Euclidean distance between the point of interest proper vector detected in the point of interest proper vector that detects in current frame image and previous frame image one by one, and wherein two nearest points of interest of Euclidean distance are exactly best unique point of the same name.

8. according to the described method of any one in claim 1 to 5, it is characterized in that, in described step D, adopt the normalized crosscorrelation matching process, on direction of scanning every presetted pixel apart from extracting current frame image after the pixel coordinate conversion and the same place between the previous frame image.

9. method according to claim 8, is characterized in that, the scope of described presetted pixel distance is between 50 to 200 pixels.

10. method according to claim 8, it is characterized in that, described employing normalized crosscorrelation matching process comprises apart from extracting current frame image after the pixel coordinate conversion and the step of the same place between the previous frame image every presetted pixel on direction of scanning:

Sub-step D1, extract the rectangular window image of current frame image as target image, and the center of this target image has represented an image characteristic point of present frame;

Sub-step D2, also get onesize rectangular window image at the previous frame image, and calculate the normalized crosscorrelation coefficient of these two groups of image data;

Sub-step D3, corresponding video in window position when the normalized crosscorrelation coefficient is maximum in the previous frame image by continuous calculating and the target image of relatively trying to achieve present frame, this location point has just represented in the previous frame image and the same place position with same characteristic features of current frame image unique point, has obtained same place.

Images