CN111010494A - Optical satellite video image stabilization method and system with geocoding function - Google Patents
Optical satellite video image stabilization method and system with geocoding function Download PDFInfo
- Publication number
- CN111010494A CN111010494A CN201911032211.2A CN201911032211A CN111010494A CN 111010494 A CN111010494 A CN 111010494A CN 201911032211 A CN201911032211 A CN 201911032211A CN 111010494 A CN111010494 A CN 111010494A
- Authority
- CN
- China
- Prior art keywords
- image
- points
- module
- auxiliary frame
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Abstract
The invention discloses an optical satellite video image stabilization method and system with geocoding, which comprises the steps of sequentially taking the previous frame of two adjacent video images as a main frame and the next frame as an auxiliary frame to obtain a plurality of pairs of main frame images and auxiliary frame images; matching video image frames to obtain matching points between each pair of main frame images and auxiliary frame images; constructing a geometric correction model between video image frames; solving the inter-frame geometric correction parameters of the auxiliary frame image; performing interframe geometric correction on the orientation parameters of the auxiliary frame image to obtain corrected orientation parameters; and carrying out geocoding on the video image to obtain a steady image with geocoding. The method ensures the image stabilization precision of the optical satellite video image, obtains the image stabilization image with the geocode, and is convenient for a user to directly obtain the geometric information of the interested target from the video data.
Description
Technical Field
The invention belongs to the technical field of geometric processing of optical satellite video images, and particularly relates to an optical satellite video image stabilizing method and system with geographic coding.
Background
A video camera carried on an optical video satellite can acquire a high-frequency time sequence image of a target area through a staring imaging mode. The high-frequency time sequence images can be used for realizing continuous observation of the motion process, and are particularly suitable for tracking analysis of instantaneous characteristics of moving targets such as airplanes and ships and the like and real-time monitoring of high-dynamic and short-time-slot phenomena such as forest fires, floods and the like.
In order to fully exert the performance of the optical video satellite, it is first required to solve the problem of image stabilization of video images. The video image stabilization precision usually reaches sub-pixels, and the requirement of high-precision real-time application of optical satellite video data can be met. At present, mathematical models such as image space translation, similarity transformation, affine transformation and the like are mostly adopted for video image stabilization to describe the geometric relationship between video image frames. Although simple and easy to use, these video stabilization models face two major problems for optical video satellites: 1) geometric deformation caused by lens distortion, topographic relief and the like is difficult to consider, and with the continuous improvement of the spatial resolution and the video width of a video camera, the geometric deformation among frames is more obvious, so that sub-pixel level video image stabilization is difficult to realize by using the video image stabilization models. Therefore, the influence of a plurality of error sources on the inter-frame geometric relationship of the video image is difficult to be completely described by adopting the traditional simple video image stabilization model, so that the geometric quality of the video image stabilization cannot be ensured. 2) For users of optical video satellites, it is generally necessary to obtain geometric information of an object of interest, such as a position and a speed of a moving object, from video data, and it is difficult to meet the application requirement of video data obtained by performing video stabilization processing using a conventional video stabilization model.
Disclosure of Invention
The invention provides an optical satellite video image stabilization method and system with geocoding, aiming at the defects of the prior art in the aspect of optical satellite video image stabilization.
The invention provides an optical satellite video image stabilizing method with geocoding, which is characterized by comprising the following steps:
step 1, according to the characteristics of high-frequency time sequence imaging of an optical video satellite, sequentially taking a previous frame of two adjacent video images as a main frame and a next frame as an auxiliary frame to obtain a plurality of pairs of main frame images and auxiliary frame images;
step 2, matching video image frames to obtain matching points between each pair of main frame and auxiliary frame images;
step 3, constructing a geometric correction model between video image frames, comprising the following substeps,
step 3.1, constructing an orientation model of the optical satellite video image;
step 3.2, constructing an interframe geometric correction model of the auxiliary frame image as follows,
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; p is a radical of1,p2,p3,p4(iii) as a cubic polynomial in the orientation model; (β)1,β2,β3,θ1,θ2,θ3) The inter-frame geometric correction parameters are auxiliary frame images;
step 4, solving the interframe geometric correction parameters of the auxiliary frame image;
step 5, performing interframe geometric correction on the orientation parameters of the auxiliary frame image to obtain corrected orientation parameters, comprising the following substeps,
step 5.1, dividing a regular grid on the auxiliary frame image, setting a plurality of elevation datum planes in the coverage range of the auxiliary frame image, and projecting each regular grid point on the auxiliary frame image onto the elevation datum planes according to the directional model of the auxiliary frame image to generate a virtual control point;
step 5.2, performing interframe geometric correction on the image space coordinates of the virtual control points to obtain corrected image space coordinates;
step 5.3, solving the orientation parameters of the auxiliary frame image by using the virtual control points after the interframe geometric correction;
step 6, geocoding the video image to obtain a steady image with geocoding, comprising the following substeps,
step 6.1, respectively projecting four corner points of the first frame image, the middle frame image and the last frame image onto the digital elevation model according to the orientation model of the video image to obtain the geographical ranges of the first frame image, the middle frame image and the last frame image, and then taking the intersection of the quadrangles in the geographical ranges of the three frame images as the geographical range of the image stabilizing image;
6.2, calculating the geographical coordinates of each ground point in the geographical range of the image-stabilized image according to the geographical coordinates, the image size and the spatial resolution of the upper left corner point of the image-stabilized image;
and 6.3, for each frame of video image, making a corresponding image stabilizing image.
Furthermore, the orientation model of the optical satellite video image obtained in step 3.1 is as follows:
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; cubic polynomial p1,p2,p3,p4The concrete form of (A) is as follows:
wherein (a)1,a2,…,a20,b1,b2,...,b20,c1,c2,...,c20,d1,d2,...,d20) Is the orientation parameter of the video image.
Moreover, in the step 4, the solution of the inter-frame geometric correction parameters of the auxiliary frame image is realized as follows,
step 4.1, according to the directional model of the main frame image, projecting the matching points on the main frame image onto the digital elevation model to obtain ground projection points;
step 4.2, projecting the ground projection point to the auxiliary frame image according to the directional model of the auxiliary frame image to obtain an image space projection point;
and 4.3, solving the interframe geometric correction parameters according to the interframe geometric correction model of the auxiliary frame image and the least square adjustment principle by using the image space projection points and the corresponding matching points on the auxiliary frame image.
In step 6.3, for each frame of video image, a corresponding image-stabilizing image is produced by adopting multi-core parallel computing based on OpenMP.
Furthermore, in step 6.3, for the single core computation process, the following steps are performed,
6.3.1, projecting ground points in the geographical range of the image stabilization image onto the video image according to the orientation model of the video image to obtain image space projection points;
6.3.2, performing gray level resampling on the video image according to the image space projection point coordinates, and assigning the obtained gray level value to the image stabilizing image;
and 6.3.3, repeating the steps 6.3.1 and 6.3.2 until all the ground points finish coordinate projection, gray level resampling and gray level assignment to obtain a stable image corresponding to the current video image.
The invention also provides an optical satellite video image stabilizing system with geocoding, which comprises the following modules:
the first module is used for sequentially taking the previous frame of two adjacent video images as a main frame and the next frame as an auxiliary frame according to the characteristic of high-frequency time sequence imaging of the optical video satellite to obtain a plurality of pairs of main frame images and auxiliary frame images;
the second module is used for matching video image frames to obtain matching points between each pair of main frame images and auxiliary frame images;
the third module is used for constructing a geometric correction model between video image frames;
the third module further comprises an orientation model construction module and an interframe geometric correction model construction module;
the orientation model building module is used for building an orientation model of the optical satellite video image;
the interframe geometric correction model used for constructing the auxiliary frame image is as follows,
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; p is a radical of1,p2,p3,p4(iii) as a cubic polynomial in the orientation model; (β)1,β2,β3,θ1,θ2,θ3) The inter-frame geometric correction parameters are auxiliary frame images;
the fourth module is used for solving the interframe geometric correction parameters of the auxiliary frame image;
the fifth module is used for carrying out interframe geometric correction on the orientation parameters of the auxiliary frame images to obtain corrected orientation parameters;
the fifth module further comprises a virtual control point generating module, a virtual control point correcting module and an orientation parameter solving module;
the virtual control point generating module is used for dividing a regular grid on the auxiliary frame image, setting a plurality of elevation datum planes in the coverage range of the auxiliary frame image, and projecting each regular grid point on the auxiliary frame image onto the elevation datum planes according to the directional model of the auxiliary frame image to generate virtual control points;
the virtual control point correction module is used for carrying out interframe geometric correction on the image space coordinates of the virtual control points to obtain corrected image space coordinates;
the orientation parameter solving module is used for solving the orientation parameters of the auxiliary frame images by using the virtual control points after the interframe geometric correction;
the sixth module is used for carrying out geocoding on the video image to obtain a steady image with geocoding;
the sixth module further comprises a geographic range calculation module, a geographic coordinate calculation module and an image stabilization image making module;
the geographical range calculation module is used for respectively projecting four corner points of the first frame image, the middle frame image and the last frame image onto the digital elevation model according to the orientation model of the video image to obtain the geographical ranges of the first frame image, the middle frame image and the last frame image, and then taking the intersection of the quadrangles in the geographical ranges of the three frame images as the geographical range of the image stabilizing image; (ii) a
The geographic coordinate calculation module is used for calculating the geographic coordinate of each ground point in the geographic range of the image-stabilizing image according to the geographic coordinate of the upper left corner point of the image-stabilizing image, the size of the image and the spatial resolution;
and the image stabilizing image making module is used for making a corresponding image stabilizing image for each frame of video image.
Furthermore, the orientation model of the optical satellite video image obtained in the orientation model building module is as follows:
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; cubic polynomial p1,p2,p3,p4The concrete form of (A) is as follows:
wherein (a)1,a2,...,a20,b1,b2,...,b20,c1,c2,...,c20,d1,d2,...,d20) Is the orientation parameter of the video image.
Moreover, the fourth module further comprises an image point projection module, a ground point projection module and an interframe geometric correction parameter solving module;
the image point projection module is used for projecting the matching points on the main frame image onto the digital elevation model according to the directional model of the main frame image to obtain ground projection points;
the ground point projection module is used for projecting the ground projection point to the auxiliary frame image according to the directional model of the auxiliary frame image to obtain an image space projection point;
and the interframe geometric correction parameter solving module is used for solving interframe geometric correction parameters according to the interframe geometric correction model of the auxiliary frame image and the least square adjustment principle by using the image space projection points and the corresponding matching points on the auxiliary frame image.
And in the image stabilization image making module, for each frame of video image, the corresponding image stabilization image is made by adopting multi-core parallel computation based on OpenMP.
Moreover, the image stabilization image production module comprises the following steps for the single-core calculation processing,
6.3.1, projecting ground points in the geographical range of the image stabilization image onto the video image according to the orientation model of the video image to obtain image space projection points;
6.3.2, performing gray level resampling on the video image according to the image space projection point coordinates, and assigning the obtained gray level value to the image stabilizing image;
and 6.3.3, repeating the steps 6.3.1 and 6.3.2 until all the ground points finish coordinate projection, gray level resampling and gray level assignment to obtain a stable image corresponding to the current video image.
The method ensures the image stabilization precision of the optical satellite video image, obtains the image stabilization image with the geocode, and is convenient for a user to directly obtain the geometric information of the interested target from the video data.
Drawings
FIG. 1 is a detailed flow chart of an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention and/or the technical solutions in the prior art, the following description will explain specific embodiments of the present invention with reference to the accompanying drawings. It is obvious that the drawings in the following description are only examples of the invention, and that for a person skilled in the art, other drawings and embodiments can be derived from them without inventive effort.
The invention will be further described with reference to the following figures and examples.
The embodiment of the invention constructs an interframe geometric correction model of the auxiliary frame image on the basis of the optical satellite video image orientation model, corrects the orientation parameters of the auxiliary frame image by using the interframe geometric correction parameters, and then carries out geocoding on the video image by using the corrected orientation parameters, so that the optical satellite image stabilization image with geocoding can be manufactured while the interframe geometric deformation of the video image caused by lens distortion, topographic relief and the like is eliminated. Referring to fig. 1, the optical satellite video image stabilization method with geocoding provided in the embodiment of the present invention includes the following specific steps:
step 1, according to the characteristics of high-frequency time sequence imaging of an optical video satellite, sequentially taking a previous frame of two adjacent video images as a main frame and a next frame as an auxiliary frame to obtain a plurality of pairs of main frame images and auxiliary frame images.
For example, the i-th frame image is used as a main frame, and the i + 1-th frame image is used as a sub frame.
And 2, matching video image frames to obtain matching points between each pair of main frame images and auxiliary frame images.
The method further comprises the following steps:
2.1, performing interframe matching by using an SIFT matching algorithm to obtain an initial matching point between the main frame image and the auxiliary frame image.
2.2 on the basis of the initial matching points, performing least square image matching to obtain accurate matching points between the main frame image and the auxiliary frame image.
And 2.3, eliminating error matching points between the main frame image and the auxiliary frame image by using a weight selection iteration method.
The SIFT matching algorithm, the least square image matching and the weight selection iteration method are specifically realized in the prior art, the invention is not repeated, and the invention obtains the matching points between the main frame image and the auxiliary frame image by using the methods.
And 3, constructing a geometric correction model between video image frames.
The method further comprises the following steps:
3.1 constructing an orientation model of the optical satellite video image, as shown in formula (1):
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; cubic polynomial p1,p2,p3,p4The concrete form of (A) is as follows:
wherein (a)1,a2,...,a20,b1,b2,...,b20,c1,c2,...,c20,d1,d2,...,d20) Is the orientation parameter of the video image.
3.2, constructing an interframe geometric correction model of the auxiliary frame image, wherein the interframe geometric correction model is shown as the formula (2):
wherein (β)1,β2,β3,θ1,θ2,θ3) And the inter-frame geometric correction parameters are the auxiliary frame images.
And 4, solving the interframe geometric correction parameters of the auxiliary frame image.
Through step 2, a plurality of pairs of matching points are obtained on the images of the main frame and the auxiliary frame, and p (x) is setp,yp),q(xq,yq)]Is one of the pair of matching points, the step further comprises:
4.1 matching points p (x) on the primary frame image according to the orientation model of the primary frame imagep,yp) Projecting the image to a digital elevation model to obtain ground projection points
And (4) realizing projection according to the matching points obtained in the step (2). Wherein phi isp,λp,hpLatitude, longitude and elevation coordinates, respectively.
4.2 projecting the ground point according to the oriented model of the auxiliary frame image
Projected on the auxiliary frame image to obtain an image side projection point q '(x'q,y′q) Wherein
4.3 Using the image side projection point q ' (x ' on the secondary frame image based on the inter-frame geometry correction model of the secondary frame image 'q,y′q) And its corresponding matching point q (x)q,yq) Establishing an error equation as shown in formula (3):
V＝AX-L (3)
wherein the content of the first and second substances,
for matching point coordinatesA residual vector;
to design the matrix, X ═ β1β2β3θ1θ2θ3]TIs an unknown number vector; l ═ xqyq]TIs a constant term.
According to the formula (3), the interframe geometric correction parameters are solved according to the least square adjustment principle, and the formula (4) is shown:
X＝(ATA)-1ATL (4)
and 5, performing interframe geometric correction on the orientation parameters of the auxiliary frame image to obtain corrected orientation parameters.
The method further comprises the following steps:
5.1 dividing a regular grid (such as 10 rows and 10 columns) on the auxiliary frame image, setting a plurality of (such as 5) elevation reference surfaces in the coverage area of the auxiliary frame image, then projecting each regular grid point on the auxiliary frame image onto the elevation reference surfaces according to the orientation model of the auxiliary frame image to obtain a group of virtual control points, wherein the image space coordinates and the object space coordinates of the virtual control points are respectively (x)t,yt) And
5.2 image side coordinates (x) of virtual control pointst,yt) Geometric correction is carried out between video image frames to obtain corrected image side coordinates (x't,y′t) As shown in formula (5):
5.3 Using virtual control points after interframe geometry correction
Solving the orientation parameters of the auxiliary frame image;
and 6, carrying out geocoding on the video image to obtain a steady image with geocoding.
The method further comprises the following steps:
6.1 respectively projecting four corner points of the first frame image, the intermediate frame image and the last frame image onto the digital elevation model according to the orientation model of the video image to obtain the geographical ranges of the first frame image, the intermediate frame image and the last frame image, and then taking the intersection of the quadrangles in the geographical ranges of the three frame images as the geographical range of the image stabilizing image;
6.2 according to the geographical coordinates of the upper left corner point of the steady image
Image size (W, H) and spatial resolution
Calculating the geographic coordinate (phi) of each ground point in the geographic range of the image-stabilizing imagej,λj) As shown in formula (6):
wherein, i and j are the ground point column number and the line number in the geographical range of the image stabilization image respectively.
6.3 for each frame of video image, adopting OpenMP-based multi-core parallel computation to make a corresponding image-stabilizing image, and for single-core computation processing, the sub-step further includes:
6.3.1 projecting the ground points in the geographical range of the image stabilization image onto the video image according to the orientation model of the video image to obtain image projection points;
6.3.2 according to the image space projection point coordinates, carrying out gray level resampling on the video image, and assigning the obtained gray value to the image stabilizing image;
6.3.3 repeating the steps 6.3.1 and 6.3.2 until all the ground points finish coordinate projection, gray level resampling and gray level assignment to obtain an image stabilizing image corresponding to the current video image.
Through the steps 6.2 and 6.3, each image point of the obtained image stabilization image has a geographic coordinate, namely, the geographic coding is realized.
The orientation model in the steps 4.2 and 5.1 is a video image orientation model constructed by using the original orientation parameters; the orientation model in steps 4.1, 6.1 and 6.3 is a video image orientation model constructed by using the orientation parameters after the interframe geometry correction.
In specific implementation, the automatic operation of the process can be realized by adopting a software mode. The apparatus for operating the process should also be within the scope of the present invention. The invention also provides an optical satellite video image stabilizing system with geocoding, which comprises the following modules:
the first module is used for sequentially taking the previous frame of two adjacent video images as a main frame and the next frame as an auxiliary frame according to the characteristic of high-frequency time sequence imaging of the optical video satellite to obtain a plurality of pairs of main frame images and auxiliary frame images;
the second module is used for matching video image frames to obtain matching points between each pair of main frame images and auxiliary frame images;
the third module is used for constructing a geometric correction model between video image frames;
the third module further comprises an orientation model construction module and an interframe geometric correction model construction module;
the orientation model building module is used for building an orientation model of the optical satellite video image;
the interframe geometric correction model used for constructing the auxiliary frame image is as follows,
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; p is a radical of1,p2,p3,p4(iii) as a cubic polynomial in the orientation model; (β)1,β2,β3,θ1,θ2,θ3) The inter-frame geometric correction parameters are auxiliary frame images;
the fourth module is used for solving the interframe geometric correction parameters of the auxiliary frame image;
the fifth module is used for carrying out interframe geometric correction on the orientation parameters of the auxiliary frame images to obtain corrected orientation parameters;
the fifth module further comprises a virtual control point generating module, a virtual control point correcting module and an orientation parameter solving module;
the virtual control point generating module is used for dividing a regular grid on the auxiliary frame image, setting a plurality of elevation datum planes in the coverage range of the auxiliary frame image, and projecting each regular grid point on the auxiliary frame image onto the elevation datum planes according to the directional model of the auxiliary frame image to generate virtual control points;
the virtual control point correction module is used for carrying out interframe geometric correction on the image space coordinates of the virtual control points to obtain corrected image space coordinates;
the orientation parameter solving module is used for solving the orientation parameters of the auxiliary frame images by using the virtual control points after the interframe geometric correction;
the sixth module is used for carrying out geocoding on the video image to obtain a steady image with geocoding;
the sixth module further comprises a geographic range calculation module, a geographic coordinate calculation module and an image stabilization image making module;
the geographical range calculation module is used for respectively projecting four corner points of the first frame image, the middle frame image and the last frame image onto the digital elevation model according to the orientation model of the video image to obtain the geographical ranges of the first frame image, the middle frame image and the last frame image, and then taking the intersection of the quadrangles in the geographical ranges of the three frame images as the geographical range of the image stabilizing image; (ii) a
The geographic coordinate calculation module is used for calculating the geographic coordinate of each ground point in the geographic range of the image-stabilizing image according to the geographic coordinate of the upper left corner point of the image-stabilizing image, the size of the image and the spatial resolution;
and the image stabilizing image making module is used for making a corresponding image stabilizing image for each frame of video image.
The specific implementation of each module can refer to the corresponding step, and the detailed description of the invention is omitted.
The above-described embodiments are intended to illustrate rather than to limit the invention, and any modifications and variations of the present invention are possible within the spirit and scope of the appended claims.
Claims (10)
1. An optical satellite video image stabilization method with geocoding is characterized by comprising the following steps:
step 1, according to the characteristics of high-frequency time sequence imaging of an optical video satellite, sequentially taking a previous frame of two adjacent video images as a main frame and a next frame as an auxiliary frame to obtain a plurality of pairs of main frame images and auxiliary frame images;
step 2, matching video image frames to obtain matching points between each pair of main frame and auxiliary frame images;
step 3, constructing a geometric correction model between video image frames, comprising the following substeps,
step 3.1, constructing an orientation model of the optical satellite video image;
step 3.2, constructing an interframe geometric correction model of the auxiliary frame image as follows,
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; p is a radical of1,p2,p3,p4(iii) as a cubic polynomial in the orientation model; (β)1,β2,β3,θ1,θ2,θ3) The inter-frame geometric correction parameters are auxiliary frame images;
step 4, solving the interframe geometric correction parameters of the auxiliary frame image;
step 5, performing interframe geometric correction on the orientation parameters of the auxiliary frame image to obtain corrected orientation parameters, comprising the following substeps,
step 5.1, dividing a regular grid on the auxiliary frame image, setting a plurality of elevation datum planes in the coverage range of the auxiliary frame image, and projecting each regular grid point on the auxiliary frame image onto the elevation datum planes according to the directional model of the auxiliary frame image to generate a virtual control point;
step 5.2, performing interframe geometric correction on the image space coordinates of the virtual control points to obtain corrected image space coordinates;
step 5.3, solving the orientation parameters of the auxiliary frame image by using the virtual control points after the interframe geometric correction;
step 6, geocoding the video image to obtain a steady image with geocoding, comprising the following substeps,
step 6.1, respectively projecting four corner points of the first frame image, the middle frame image and the last frame image onto the digital elevation model according to the orientation model of the video image to obtain the geographical ranges of the first frame image, the middle frame image and the last frame image, and then taking the intersection of the quadrangles in the geographical ranges of the three frame images as the geographical range of the image stabilizing image;
6.2, calculating the geographical coordinates of each ground point in the geographical range of the image-stabilized image according to the geographical coordinates, the image size and the spatial resolution of the upper left corner point of the image-stabilized image;
and 6.3, for each frame of video image, making a corresponding image stabilizing image.
2. The optical satellite video image stabilization method with geocoding of claim 1, wherein: the orientation model of the optical satellite video image obtained in step 3.1 is as follows:
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; cubic polynomial p1,p2,p3,p4The concrete form of (A) is as follows:
wherein (a)1,a2,...,a20,b1,b2,...,b20,c1,c2,...,c20,d1,d2,...,d20) Is the orientation parameter of the video image.
3. The optical satellite video image stabilization method with geocoding of claim 1, wherein: in step 4, the solution of the inter-frame geometric correction parameters of the auxiliary frame image is realized as follows,
step 4.1, according to the directional model of the main frame image, projecting the matching points on the main frame image onto the digital elevation model to obtain ground projection points;
step 4.2, projecting the ground projection point to the auxiliary frame image according to the directional model of the auxiliary frame image to obtain an image space projection point;
and 4.3, solving the interframe geometric correction parameters according to the interframe geometric correction model of the auxiliary frame image and the least square adjustment principle by using the image space projection points and the corresponding matching points on the auxiliary frame image.
4. The optical satellite video image stabilization method with geocoding of claim 1, wherein: in step 6.3, for each frame of video image, a corresponding image stabilizing image is produced by adopting multi-core parallel computing based on OpenMP.
5. The optical satellite video image stabilization method with geocoding of claim 4, wherein: in step 6.3, for the single core computation process, the following steps are performed,
6.3.1, projecting ground points in the geographical range of the image stabilization image onto the video image according to the orientation model of the video image to obtain image space projection points;
6.3.2, performing gray level resampling on the video image according to the image space projection point coordinates, and assigning the obtained gray level value to the image stabilizing image;
and 6.3.3, repeating the steps 6.3.1 and 6.3.2 until all the ground points finish coordinate projection, gray level resampling and gray level assignment to obtain a stable image corresponding to the current video image.
6. An optical satellite video image stabilization system with geocoding is characterized by comprising the following modules:
the first module is used for sequentially taking the previous frame of two adjacent video images as a main frame and the next frame as an auxiliary frame according to the characteristic of high-frequency time sequence imaging of the optical video satellite to obtain a plurality of pairs of main frame images and auxiliary frame images;
the second module is used for matching video image frames to obtain matching points between each pair of main frame images and auxiliary frame images;
the third module is used for constructing a geometric correction model between video image frames;
the third module further comprises an orientation model construction module and an interframe geometric correction model construction module;
the orientation model building module is used for building an orientation model of the optical satellite video image;
the interframe geometric correction model used for constructing the auxiliary frame image is as follows,
wherein (x, y) is likePoint coordinates;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; p is a radical of1,p2,p3,p4(iii) as a cubic polynomial in the orientation model; (β)1,β2,β3,θ1,θ2,θ3) The inter-frame geometric correction parameters are auxiliary frame images;
the fourth module is used for solving the interframe geometric correction parameters of the auxiliary frame image;
the fifth module is used for carrying out interframe geometric correction on the orientation parameters of the auxiliary frame images to obtain corrected orientation parameters;
the fifth module further comprises a virtual control point generating module, a virtual control point correcting module and an orientation parameter solving module;
the virtual control point generating module is used for dividing a regular grid on the auxiliary frame image, setting a plurality of elevation datum planes in the coverage range of the auxiliary frame image, and projecting each regular grid point on the auxiliary frame image onto the elevation datum planes according to the directional model of the auxiliary frame image to generate virtual control points;
the virtual control point correction module is used for carrying out interframe geometric correction on the image space coordinates of the virtual control points to obtain corrected image space coordinates;
the orientation parameter solving module is used for solving the orientation parameters of the auxiliary frame images by using the virtual control points after the interframe geometric correction;
the sixth module is used for carrying out geocoding on the video image to obtain a steady image with geocoding;
the sixth module further comprises a geographic range calculation module, a geographic coordinate calculation module and an image stabilization image making module;
the geographical range calculation module is used for respectively projecting four corner points of the first frame image, the middle frame image and the last frame image onto the digital elevation model according to the orientation model of the video image to obtain the geographical ranges of the first frame image, the middle frame image and the last frame image, and then taking the intersection of the quadrangles in the geographical ranges of the three frame images as the geographical range of the image stabilizing image; (ii) a
The geographic coordinate calculation module is used for calculating the geographic coordinate of each ground point in the geographic range of the image-stabilizing image according to the geographic coordinate of the upper left corner point of the image-stabilizing image, the size of the image and the spatial resolution;
and the image stabilizing image making module is used for making a corresponding image stabilizing image for each frame of video image.
7. The optical satellite video stabilization system with geocoding of claim 6, wherein: the orientation model of the optical satellite video image obtained in the orientation model building module is as follows:
wherein, (x, y) is the coordinates of the image point;
regularizing coordinates for ground points; (x)o,yo) Regularizing translation parameters for the coordinates of the image points; (x)s,ys) Regularizing scaling parameters for the coordinates of the image points; cubic polynomial p1,p2,p3,p4The concrete form of (A) is as follows:
wherein (a)1,a2,...,a20,b1,b2,...,b20,c1,c2,...,c20,d1,d2,...,d20) Is the orientation parameter of the video image.
8. The optical satellite video stabilization system with geocoding of claim 6, wherein: the fourth module further comprises an image point projection module, a ground point projection module and an interframe geometric correction parameter solving module;
the image point projection module is used for projecting the matching points on the main frame image onto the digital elevation model according to the directional model of the main frame image to obtain ground projection points;
the ground point projection module is used for projecting the ground projection point to the auxiliary frame image according to the directional model of the auxiliary frame image to obtain an image space projection point;
and the interframe geometric correction parameter solving module is used for solving interframe geometric correction parameters according to the interframe geometric correction model of the auxiliary frame image and the least square adjustment principle by using the image space projection points and the corresponding matching points on the auxiliary frame image.
9. The optical satellite video stabilization system with geocoding of claim 6, wherein: in the image stabilization image making module, for each frame of video image, multi-core parallel computation based on OpenMP is adopted to make a corresponding image stabilization image.
10. The optical satellite video stabilization system with geocoding of claim 9, wherein: in the image stabilization image production module, for the single-core calculation processing, the following steps are executed,
6.3.1, projecting ground points in the geographical range of the image stabilization image onto the video image according to the orientation model of the video image to obtain image space projection points;
6.3.2, performing gray level resampling on the video image according to the image space projection point coordinates, and assigning the obtained gray level value to the image stabilizing image;
and 6.3.3, repeating the steps 6.3.1 and 6.3.2 until all the ground points finish coordinate projection, gray level resampling and gray level assignment to obtain a stable image corresponding to the current video image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911032211.2A CN111010494B (en) | 2019-10-28 | 2019-10-28 | Optical satellite video image stabilization method and system with geocoding function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911032211.2A CN111010494B (en) | 2019-10-28 | 2019-10-28 | Optical satellite video image stabilization method and system with geocoding function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111010494A true CN111010494A (en) | 2020-04-14 |
| CN111010494B CN111010494B (en) | 2020-11-03 |
Family
ID=70111560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911032211.2A Expired - Fee Related CN111010494B (en) | 2019-10-28 | 2019-10-28 | Optical satellite video image stabilization method and system with geocoding function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111010494B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7840085B2 (en) * | 2006-04-06 | 2010-11-23 | Qualcomm Incorporated | Electronic video image stabilization |
| US8238694B2 (en) * | 2008-10-03 | 2012-08-07 | Microsoft Corporation | Alignment of sharp and blurred images based on blur kernel sparseness |
| CN106550172A (en) * | 2015-09-18 | 2017-03-29 | 河北工业大学 | Video stabilization in assemble of the satellite based on subregion Gray Projection |
| CN107040695A (en) * | 2017-04-28 | 2017-08-11 | 武汉大学 | Spaceborne video image stabilization method and system based on RPC location models |
| CN107705329A (en) * | 2017-10-24 | 2018-02-16 | 武汉大学 | High-resolution optical satellite based on geometrical constraint stares Image registration method |
| CN108076341A (en) * | 2017-12-19 | 2018-05-25 | 武汉大学 | A kind of video satellite is imaged in-orbit real-time digital image stabilization method and system |
| CN109712088A (en) * | 2018-12-14 | 2019-05-03 | 航天恒星科技有限公司 | A kind of remote sensing video satellite image processing method and system based on steady picture |
-
2019
- 2019-10-28 CN CN201911032211.2A patent/CN111010494B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7840085B2 (en) * | 2006-04-06 | 2010-11-23 | Qualcomm Incorporated | Electronic video image stabilization |
| US8238694B2 (en) * | 2008-10-03 | 2012-08-07 | Microsoft Corporation | Alignment of sharp and blurred images based on blur kernel sparseness |
| CN106550172A (en) * | 2015-09-18 | 2017-03-29 | 河北工业大学 | Video stabilization in assemble of the satellite based on subregion Gray Projection |
| CN107040695A (en) * | 2017-04-28 | 2017-08-11 | 武汉大学 | Spaceborne video image stabilization method and system based on RPC location models |
| CN107705329A (en) * | 2017-10-24 | 2018-02-16 | 武汉大学 | High-resolution optical satellite based on geometrical constraint stares Image registration method |
| CN108076341A (en) * | 2017-12-19 | 2018-05-25 | 武汉大学 | A kind of video satellite is imaged in-orbit real-time digital image stabilization method and system |
| CN109712088A (en) * | 2018-12-14 | 2019-05-03 | 航天恒星科技有限公司 | A kind of remote sensing video satellite image processing method and system based on steady picture |
Non-Patent Citations (1)
| Title |
|---|
| 张过: "《卫星视频处理与应用进展》", 《应用科学学报》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111010494B (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110211043B (en) | Registration method based on grid optimization for panoramic image stitching | |
| CN110648398B (en) | Real-time ortho image generation method and system based on unmanned aerial vehicle aerial data | |
| CN110675450B (en) | Method and system for generating orthoimage in real time based on SLAM technology | |
| CN102169573B (en) | Real-time distortion correction method and system of lens with high precision and wide field of view | |
| CN107040695B (en) | satellite-borne video image stabilization method and system based on RPC positioning model | |
| CN111174753B (en) | Optical image and laser height measurement data adjustment method based on rational function model | |
| CN110084785B (en) | Power transmission line vertical arc measuring method and system based on aerial images | |
| CN110276768B (en) | Image segmentation method, image segmentation device, image segmentation apparatus, and medium | |
| CN106709944B (en) | Satellite remote sensing image registration method | |
| CN106887016B (en) | Automatic relative registration method for GF-4 satellite sequence images | |
| CN113298947B (en) | Substation three-dimensional modeling method medium and system based on multi-source data fusion | |
| US20230025058A1 (en) | Image rectification method and device, and electronic system | |
| CN112215880A (en) | Image depth estimation method and device, electronic equipment and storage medium | |
| US8509522B2 (en) | Camera translation using rotation from device | |
| CN115131494A (en) | Optical remote sensing satellite imaging simulation method and device | |
| CN103778610A (en) | Geometric pretreatment method for vertical rail swing images of satellite-borne linear array sensor | |
| JP3490774B2 (en) | How to generate geospecific textures | |
| CN111010494B (en) | Optical satellite video image stabilization method and system with geocoding function | |
| CN110660099B (en) | Rational function model fitting method for remote sensing image processing based on neural network | |
| CN114913245B (en) | Multi-calibration-block multi-camera calibration method and system based on undirected weighted graph | |
| CN114998496A (en) | Orthoimage rapid generation method based on scene aerial photography image and sparse point cloud | |
| CN111861873B (en) | Method and device for generating simulation image | |
| CN113790711A (en) | Unmanned aerial vehicle low-altitude flight pose uncontrolled multi-view measurement method and storage medium | |
| CN108921908B (en) | Surface light field acquisition method and device and electronic equipment | |
| CN110608723A (en) | Real-time remote sensing image geographical reference method based on FPGA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20201103 Termination date: 20211028 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |