CN105608670B - A kind of aerial video images without the quick joining method of ghost Cross slat belt - Google Patents
A kind of aerial video images without the quick joining method of ghost Cross slat belt Download PDFInfo
- Publication number
- CN105608670B CN105608670B CN201511025377.3A CN201511025377A CN105608670B CN 105608670 B CN105608670 B CN 105608670B CN 201511025377 A CN201511025377 A CN 201511025377A CN 105608670 B CN105608670 B CN 105608670B
- Authority
- CN
- China
- Prior art keywords
- image
- splicing
- rectification
- translation relation
- ghost
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
Abstract
A kind of aerial video images without links such as ghost Cross slat belt quick joining methods, including ortho-rectification, image registration, image mosaic.Ortho-rectification：The real-time attitude angular data ortho-rectification provided using Inertial Measurement Unit (IMU) is per frame image.Image registration：Translation relation between two frame orthograph pictures is calculated using phase correlation method.Image mosaic：On the basis of a frame orthograph picture, calculate per frame orthograph picture and the translation relation between it.Image mosaic：Estimation should participate in the part band of splicing per frame image, and these bands are merged, and form global image.The present invention does not need GPS location data and altitude data, does not need detection image characteristic point yet, and processing speed is fast, meets outfield rapid data quality examination and emergency response demand.In addition, the present invention only used it is corrected after every frame orthograph picture in very narrow band, cast out the redundancy section for causing ghost, avoided ghost phenomenon.
Description
Technical field
The present invention relates to photogrammetric field, remote sensing fields, computer vision fields, specifically provide a kind of for aviation
Video image without the quick joining method of ghost Cross slat belt.
Background technology
Quickly splicing always is unmanned vehicle remote sensing critical issue to be solved to image.Traditional photogrammetric skill
Art is excessively strictly not suitable for the image mosaic obtained based on unmanned vehicle.It can be by multiple image in spite of some business softwares
It is stitched together, but the limitation of these business softwares is excessive, the degree of automation is not high, and when image is excessive, can not splice.It is special
It is not that often there are ghost phenomena in transitional region for spliced image (present invention is referred to as ghost).The presence of ghost makes distant
Aliasing occurs for the spectral information for feeling image, and difficulty is caused to subsequent classification and inverting.
Invention content
The technical problem to be solved in the present invention：Overcome the deficiencies of the prior art and provide a kind of aerial video images without ghost
The quick joining method of shadow Cross slat belt.This method is divided into ortho-rectification, image registration, three step of image mosaic, in splicing only
Used it is corrected after orthograph picture in very narrow band, cast out the redundancy section for causing ghost, therefore splicing effect
Good, splicing speed is fast, no ghost, and unmanned vehicle is suitble to obtain image mosaic.
The technical solution adopted by the present invention：A kind of aerial video images without the quick joining method of ghost Cross slat belt,
It is characterized in that：
(1) in ortho-rectification link, the attitude angle data pair of IMU offers is directly used in the case of unknown ground elevation
Every frame image of data flow carries out ortho-rectification.
(2) it is approximately considered between the image crossed through ortho-rectification and only exists translation relation, therefore phase related side can be used
Method calculates the translation relation between image.
(3) using after correction orthograph picture and any adjacent two field pictures between translation relation estimation splicing after scheme
The size of picture；And the strip portion and its width of splicing should be participated in per frame image；And position of the band in global image
It sets.Finally these bands are merged to form global image.
Step (1) first uses the attitude angle data that IMU is provided to carry out ortho-rectification to image, after ortho-rectification image it
Between approximation only exist translation relation, meet the relevant use condition of phase.
When step (2) calculates translation relation using phase correlation method, the Part portions of image are used, Local map is used
As replacing general image, the speed of registration is further improved.
Using the translation relation between image in step (3), the band that splicing should be participated in per frame image, each frame image item are calculated
With misaligned, the generation of image ghost phenomenon is avoided.
The advantages of the present invention over the prior art are that：
(1) aerial video images are rapidly completed by three ortho-rectification, image registration and image mosaic links in the present invention
Splicing, a kind of solution rapidly and efficiently is provided for unmanned vehicle remote sensing images Bonding Problem.
(2) present invention utilizes IMU Data correction images, all only has parts of images to participate in calculating being registrated and splicing link,
While improving processing speed, ghost generation is avoided, avoids Spectra aliasing, spliced image can be used for subsequent point
Class or inverting.
Description of the drawings
Fig. 1 aerial video flow datas acquire schematic diagram；
Fig. 2 ortho-rectification schematic diagrames；
Fig. 3 cuts certain area on orthograph picture and uses phase correlation computations translation parameters；
Fig. 4 calculates the band that splicing should be participated in per frame orthograph picture and splicing；
Fig. 5 flow charts of the present invention.
Specific implementation mode
Step 1：Ortho-rectification
The present invention needs that aerial image is corrected to orthograph picture using IMU before image mosaic.Correction course and public affairs
Formula derives following article.
There are a point W in three dimensions, coordinate is W=[XW YW ZW], the form of homogeneous coordinates is [W 1]T.V=
[x y-f]TIndicate coordinate of this in image space coordinate system.It is represented by formula formula according to pinhole imaging system principle W and v relationship
1：
K, R and t are one 3 × 3 camera calibration matrix respectively, and 3 × 3 D world space coordinate system and image space are sat
Spin matrix and 3 × 1 translation vectors under mark system.When world coordinate system origin and camera coordinates system origin overlap, t is zero
Vector, formula 1 can be written as formula 2：
λ v=KRWT+ K0=KRWT (2)
It is assumed that there are a virtual camera, the camera coordinates system of the camera is overlapped with world coordinate system.So, the coke of camera
Plane is parallel to the ground, and primary optical axis is vertical perpendicular to ground.The image of camera acquisition is orthograph picture.Its image space is sat
Mark system coordinate can be expressed as formula 3：
λv0=K0IWT+K00=K0WT (3)
According to formula 2 and formula 3, we obtain formula 4：
v0=K0R-1K-1v (4)
If the intrinsic parameter that all images all pass through camera corrects, K0And K-1It is unit matrix, formula 4 can be reduced to formula
5：
v0=R-1v (5)
Roll angle φ, pitching angle theta and yaw angle ψ are provided by IMU systems.Yaw angle spin matrix Rψ, pitch angle spin moment
Battle array RθWith roll angle spin matrix RψMultiplication can be obtained spin matrix R, such as formula 6：
Each of original image point can calculate its position in orthograph picture using formula (7).
(x, y) and (x0, y0) position of some pixel of the pixel in orthograph picture and original image, f are indicated respectively
It is the focal length of camera, λ is scale modifying factor.Formula 6 can be used to calculate for parameter in R, and f can be obtained from camera operation instructions
It arrives, can also be demarcated using Matlab calibration tools case.This side can be used in the value of all pixels in virtual camera
Method calculates, and needs to use certain interpolation method if (x, y) is not integer.
Step 2：Step of registration
1975, Kuglin et al. was proposed according to the translation property of Fourier transformation and is calculated based on phase correlation theory
The method of translation parameters between image.The basic thought of this method is as follows：
Assuming that image f1(x, y), f2(x, y) is the two frames image to be registrated, therebetween existing translation, then, the two
Relationship can express an accepted way of doing sth 8：
f2(x, y)=f1(x-△x,y-△y) (8)
To f1(x,y)、f2(x, y) carries out Fourier transformation respectively, obtains F1(u,v)、F2(u, v), according to Fourier transformation
Translation property, formula 9：
F2(u, v)=F1(u,v)exp(-j2π(u△x+v△y)) (9)
Assuming that G (u, v) is f1(x,y)、f2The crosspower spectrum of (x, y) has formula 10：
F*(u, v) is the conjugation of F (u, v), there is formula 11：
G (u, v)=exp (- j2 π (u △ x+v △ y)) (11)
And an impulse function can be obtained by carrying out inverse Fourier transform to the middle section of formula 10, the letter at (△ x, △ y)
Numerical value is 1, remaining is 0.Since the middle section of formula 10 and rightmost part are equal, only need to carry out Fu to middle section
In leaf inverse transformation, can be obtained by a size and f in theory1(x, y) identical pulse image, and not due to two field pictures
It is completely superposed, and image is all regarded as continuous by above-mentioned formula, and practical epigraph is discrete, in specific operation process
It is middle with 2D-DFT instead of 2D-FT, the displacement between image is sometimes nor integer, obtained pulse image are actually approximate
Impulse function, value at peak value are less than 1, at the rest of value close to 0.Therefore the peak position of search pulse image is only needed,
Translation parameters between this peak position and two images is related.Translation parameters between two images is how many actually, it is also necessary to
Certain processing.Assuming that the size for participating in the image of operation is m, position where peak value is i, then determination method such as formula 13
With 14：
When m is even number：
When m is odd number：
In addition this method is also only applicable to two field pictures dislocation the case where interior, if two field pictures dislocation is not at this
Section, phase correlation method are just no longer applicable in.The determination method of △ y is similar to the determination method of △ x.
Phase correlation method be based on Fourier transformation, and the size of the speed of Fourier transformation and image have compared with
Big relationship, with the increase of picture size, calculating speed continuously decreases.To accelerate speed, in the same area of adjacent image
The regional area for determining a certain size calculates its translation relation to accelerate computational efficiency using regional area instead of entire image.
Step 3：Image mosaic
Global stitching image can be built by the orthograph picture after corrected.The orthograph picture for selecting a frame corrected is made
For reference picture, all images and the rotation relationship between it are calculated.Since every frame image has its overlay area alone, when
When spacing between two field pictures is excessive, two field pictures may not include public domain, therefore translation relation between the two without
Method directly calculates.Translation relation between the two is calculated the present invention provides a kind of round-about way.It is assumed that MI, jIndicate image Ii
With image IjBetween translation relation.MI, jIt is represented by formula 15, xI, jAnd yI, jImage I is indicated respectivelyiWith image IjBetween be expert at
Translation relation between direction and column direction.
If we need image IiProject to image Ij, can first use IiProject image onto Ik, then reuse
MK, jResult is projected into Ij.Matrix MI, jIt can be expressed as formula 16.All orthographs are as I in this way1, I2..., IN?
This method can be used to calculate its transition matrix between reference picture.If first frame orthograph picture is reference picture, PJ, 1
For image IjWith image I1Between projection matrix, PJ, 1It is represented by formula 17.
Mi,j=Mi,kMk,j (16)
It is assumed that the size of all orthograph pictures is m × n, the initial row line number that splicing is participated in per frame image is H.Then image
IjThe width of the middle band for needing to participate in splicing is xI, i-1.Assuming that spliced image is Z, the starting of real data in Z-image
Position is [r, c].It is only needed image I in splicingjIn H rows and H+xI, i-1Band between row directly writes to Z
In r+Pj-1,1(1,3) row and r+Pj,1(1,3) between row, c+Pj-1,1(2,3) row and c+Pj,1(2,3) region between arranging
The splicing of image can be completed.
It is recited above be only embody a kind of aerial video images of the present invention without the quick side of splicing of ghost Cross slat belt
The embodiment of method.The present invention is not limited to above-described embodiments.The specification of the present invention is not limit right for illustrating and want
The range asked.It will be apparent to those skilled in the art that can have many replacements, improvement and variation.All use equally is replaced
Change or equivalent transformation formed technical solution, be all fallen within the protection domain of application claims.
Claims (2)
1. a kind of aerial video images without the quick joining method of ghost Cross slat belt, it is characterised in that following steps：
(1) ortho-rectification step directly uses the attitude angle data that IMU is provided to every frame figure in the case of unknown ground elevation
As carrying out ortho-rectification；
(2) image registration step is approximately considered between the image that ortho-rectification is crossed only with translation relation, from the phase for participating in calculating
It is respectively taken in adjacent two field pictures and determines subregion using phase correlation method calculating translation relation to accelerate the efficiency of image registration；
(3) image mosaic step, using after correction orthograph picture and any adjacent two field pictures between translation relation estimate
Calculate the size of global image after splicing；And the strip portion and its width of splicing should be participated in per frame image；And the band exists
These bands finally are merged to form global image by the position in global image；
The step (1) first uses the attitude angle data that IMU is provided to carry out ortho-rectification to image, after ortho-rectification image it
Between approximation only exist translation relation, meet the relevant use condition of phase；
When calculating translation relation using phase correlation method, whole frame figure is replaced using the Part portions of image for the step (2)
Picture improves the speed of registration；
The step (3) calculates the band that splicing is participated in per frame image, each frame image band using the translation relation between image
It is misaligned, avoid the generation of image ghost phenomenon.
2. a kind of aerial video images according to claim 1 without the quick joining method of ghost Cross slat belt, it is special
Sign is：Using after correction orthograph picture and any adjacent two field pictures between translation relation estimate it is as follows：It is assumed that MI, j
Indicate image IiWith image IjBetween translation relation, MI, jIt is expressed as formula 1, xI, jAnd yI, jImage I is indicated respectivelyiWith image IjIt
Between translation relation between line direction and column direction；
If necessary by image IiProject to image Ij, can first use MI, kProject image onto Ik, then reuse Mk,jBy result
Project to Ij；Matrix MI, jIt is expressed as formula 2；All orthographs are as I1, I2..., INFormula 1 can be used to calculate itself and reference picture
Between transition matrix；If first frame orthograph picture is reference picture, PJ, 1For image IjWith image I1Between projection matrix,
PJ, 1It is represented by formula 3；
Mi,j=Mi,kMk,j (2)
It is assumed that the size of all orthograph pictures is m × n, the initial row line number that splicing is participated in per frame image is H；Then image IjIn
It is x to need the width for the band for participating in splicingI, i-1；Assuming that spliced image is Z, the initial position of real data in Z-image
For [r, c]；It is only needed image I in splicingjIn H rows and H+xI, i-1Band between row directly writes to r in Z
+Pj-1,1(1,3) row and r+Pj,1(1,3) between row, c+Pj-1,1(2,3) row and c+Pj,1(2,3) region between arranging can be complete
At the splicing of image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511025377.3A CN105608670B (en) | 2015-12-30 | 2015-12-30 | A kind of aerial video images without the quick joining method of ghost Cross slat belt |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201511025377.3A CN105608670B (en) | 2015-12-30 | 2015-12-30 | A kind of aerial video images without the quick joining method of ghost Cross slat belt |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105608670A CN105608670A (en) | 2016-05-25 |
| CN105608670B true CN105608670B (en) | 2018-11-02 |
Family
ID=55988586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201511025377.3A Active CN105608670B (en) | 2015-12-30 | 2015-12-30 | A kind of aerial video images without the quick joining method of ghost Cross slat belt |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105608670B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105894486B (en) * | 2016-06-29 | 2018-08-03 | 深圳市优象计算技术有限公司 | A kind of mobile phone night shooting method based on imu information |
| CN106526511B (en) * | 2016-10-21 | 2018-08-10 | 杭州电子科技大学 | SPEED MR imaging methods based on the positioning of k-space center ghost |
| CN108986025B (en) * | 2018-06-08 | 2023-02-10 | 西北工业大学 | High-precision different-time image splicing and correcting method based on incomplete attitude and orbit information |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810701A (en) * | 2014-01-15 | 2014-05-21 | 北京农业信息技术研究中心 | Method and system for geometric correction of UAV (unmanned aerial vehicle) loaded imaging hyperspectrum |
-
2015
- 2015-12-30 CN CN201511025377.3A patent/CN105608670B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810701A (en) * | 2014-01-15 | 2014-05-21 | 北京农业信息技术研究中心 | Method and system for geometric correction of UAV (unmanned aerial vehicle) loaded imaging hyperspectrum |
Non-Patent Citations (3)
| Title |
|---|
| Improving Vision-Based Planar Motion Estimation for Unmanned Aerial Vehicles through Online Mosaicing;Fernando Caballero et al.;《Proceedings 2006 IEEE International Conference on Robotics and Automation》;20060531;第2860-2865页 * |
| 一种利用Fourier-Mellin变换和曲线拟合的遥感图像亚像素配准方法;李含伦 等;《小型微型计算机系统》;20151215;第36卷(第12期);第2063-2068页 * |
| 航拍视频的多向条带拼接算法;廖超 等;《航空学报》;20121125;第33卷(第11期);第2065-2073页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105608670A (en) | 2016-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3565739B1 (en) | Rear-stitched view panorama for rear-view visualization | |
| US20180352165A1 (en) | Device having cameras with different focal lengths and a method of implementing cameras with different focal lenghts | |
| CN109314753B (en) | Method and computer-readable storage medium for generating intermediate views using optical flow | |
| US9729858B2 (en) | Stereo auto-calibration from structure-from-motion | |
| US10079975B2 (en) | Image distortion correction of a camera with a rolling shutter | |
| JP7164706B2 (en) | Image stitching method and device, in-vehicle image processing device, electronic device, storage medium | |
| US9686479B2 (en) | Method for combining multiple image fields | |
| US20220044442A1 (en) | Bi-level optimization-based infrared and visible light fusion method | |
| US8755624B2 (en) | Image registration device and method thereof | |
| US11307595B2 (en) | Apparatus for acquisition of distance for all directions of moving body and method thereof | |
| CN106447602B (en) | Image splicing method and device | |
| CN106886979A (en) | A kind of image splicing device and image split-joint method | |
| CN106023086A (en) | Aerial photography image and geographical data splicing method based on ORB feature matching | |
| WO2016188010A1 (en) | Motion image compensation method and device, display device | |
| CN105608670B (en) | A kind of aerial video images without the quick joining method of ghost Cross slat belt | |
| CN106127683A (en) | A kind of real-time joining method of unmanned aerial vehicle SAR image | |
| CN105427244A (en) | Remote sensing image splicing method and device | |
| CN102164269A (en) | Method and device for monitoring panoramic view | |
| US10991073B2 (en) | Apparatus and method of parallax-minimized stitching by using HLBP descriptor information | |
| CN106780297A (en) | Image high registration accuracy method under scene and Varying Illumination | |
| CN105657268A (en) | Multi-viewpoint video splicing and fusion algorithm based on multiple resolutions | |
| Huang et al. | Image registration among UAV image sequence and Google satellite image under quality mismatch | |
| CN110738696B (en) | Driving blind area perspective video generation method and driving blind area view perspective system | |
| CN104021535B (en) | The method of stepping framing ccd image splicing | |
| CN103455992B (en) | The method of splicing aviation multi-element scanning image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |