CN116071539B - Micro-nano satellite image on-orbit geometric precise correction method and device - Google Patents

Micro-nano satellite image on-orbit geometric precise correction method and device Download PDF

Info

Publication number
CN116071539B
CN116071539B CN202310307607.3A CN202310307607A CN116071539B CN 116071539 B CN116071539 B CN 116071539B CN 202310307607 A CN202310307607 A CN 202310307607A CN 116071539 B CN116071539 B CN 116071539B
Authority
CN
China
Prior art keywords
image
matched
points
region
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310307607.3A
Other languages
Chinese (zh)
Other versions
CN116071539A (en
Inventor
向俞明
胡玉新
王林徽
王峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aerospace Information Research Institute of CAS
Original Assignee
Aerospace Information Research Institute of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aerospace Information Research Institute of CAS filed Critical Aerospace Information Research Institute of CAS
Priority to CN202310307607.3A priority Critical patent/CN116071539B/en
Publication of CN116071539A publication Critical patent/CN116071539A/en
Application granted granted Critical
Publication of CN116071539B publication Critical patent/CN116071539B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/24Aligning, centring, orientation detection or correction of the image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/34Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462Salient features, e.g. scale invariant feature transforms [SIFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/13Satellite images
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a micro-nano satellite image in-orbit geometric precise correction method and a device, which relate to the field of micro-nano satellite image in-orbit geometric processing, and the method comprises the following steps: determining a mask of the region of interest based on an open source geographic vector file of a target region of interest of the micro-nano satellite; extracting a reference feature in a mask of a region of interest in a reference image of a target region; performing feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and corresponding geographic ranges; searching compressed reference features in a geographic range represented by the geographic information in a lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as reference features to be matched; performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points; and carrying out geometric fine correction on the image to be corrected according to the uniformly distributed matching points. The method and the device have high positioning precision and high positioning efficiency.

Description

Micro-nano satellite image on-orbit geometric precise correction method and device
Technical Field
The invention relates to the technical field of on-orbit geometric processing of micro-nano satellite images, in particular to a method and a device for precisely correcting the on-orbit geometric of micro-nano satellite images.
Background
With the gradual maturation of micro-nano satellite constellation technology, a multi-source satellite cluster shooting mode can realize short-period global revisit under the condition of taking high-resolution earth observation into consideration. The existing typical remote sensing satellite ground processing system at home and abroad adopts a process of downloading satellite original data to standard product production. Due to the limitation of bandwidth and other reasons, the downloading of a large amount of original data is time-consuming and labor-consuming, and the timeliness of a processing system is greatly limited. In order to relieve the pressure of satellite-to-ground transmission and improve the timeliness of information, on-orbit processing becomes a main processing mode of a micro-nano satellite constellation.
In order to improve timeliness of micro-nano satellite information processing, an on-orbit intelligent processing module carries out target detection and identification on the image and only downloads detected target slices. However, the micro-nano satellite has small load volume and light weight, and the positioning accuracy is poorer than that of the traditional large satellite, and the downloaded target slice has positioning errors of hundreds of meters or even kilometers, so that the accuracy of target information and multi-satellite combined application are greatly limited. Therefore, geometric fine correction of the micro-nano satellite image is a key step of subsequent application.
Conventional geometric fine correction relies on a control point slice library or a reference image as a basis to correct the positioning parameters of the image. However, due to the limited space of the on-board storage, it is difficult to directly acquire a control point slice library or a reference image required for fine correction. And the micro-nano satellite needs to consider the cost and weight of hardware equipment, the gap between equipment resources available for calculation and a ground processing system is too large, and meanwhile, due to the limitation of space heat dissipation conditions, the on-board embedded system cannot operate for a long time. These hardware limitations also present a significant challenge to the algorithm for on-orbit processing.
Disclosure of Invention
The invention provides a method and a device for accurately correcting the on-orbit geometry of a base micro-nano satellite image, which are used for at least partially solving the technical problems.
Based on this, the first aspect of the present invention provides a method for precisely correcting the on-orbit geometry of a micro-nano satellite image, comprising: determining a mask of the region of interest based on an open source geographic vector file of a target region of interest of the micro-nano satellite; extracting a reference feature in a mask of a region of interest in a reference image of a target region; performing feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features; searching compressed reference features in a geographic range represented by the geographic information in a lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as reference features to be matched; performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points; and carrying out geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
According to an embodiment of the present invention, determining a region of interest mask based on an open source geographic vector file of a target region of interest for a micro-nano satellite comprises: converting the open source geographic vector file into a geographic coding image; resampling the geocode image to obtain a geovector image; acquiring an interested region in a geographic vector image; and performing morphological expansion operation on the region of interest to obtain a mask of the region of interest.
According to an embodiment of the present invention, extracting a reference feature in a region of interest mask in a reference image of a target region includes: calculating the multi-directional image intensity change characteristics of the reference image through an anisotropic Gaussian differential operator, and determining the most obvious change characteristics in all directions as anisotropic differential characteristics; performing threshold processing on the anisotropic differential feature based on the intensity threshold to obtain a binarized anisotropic differential feature; and performing intersection operation on the binarized anisotropic differential feature and the region-of-interest mask to obtain a reference feature.
According to an embodiment of the invention, the anisotropic gaussian differential operator is:
Figure SMS_1
wherein x and y are respectively the horizontal and vertical coordinates of a pixel point in a reference image, sigma, rho and theta are respectively a scale parameter, an anisotropic parameter and a rotation angle, G sigma, rho, theta (x, y) are respectively anisotropic Gaussian kernels corresponding to the pixel point with the coordinates of (x, y), x theta is a coordinate after the coordinate x rotates by the angle theta, MAGD (x, y, theta) is an anisotropic Gaussian differential operator corresponding to the pixel point with the coordinates of (x, y) in the direction of the angle theta, I is an intensity value of the reference image, and I is convolution operation.
According to an embodiment of the present invention, performing feature compression on a reference feature to obtain a lightweight reference feature library includes: and performing travel coding on the reference features, and replacing the reference features with the same intensity value by using the string length to obtain a lightweight reference feature library.
According to the embodiment of the invention, according to the geographic information of the image to be corrected generated by on-board imaging, compressed reference features in a geographic range represented by the geographic information are searched in a lightweight reference feature library, and the compressed reference features as the reference features to be matched comprise: calculating longitude and latitude of four corners of the image to be corrected; and carrying out overlapping region inquiry on the region represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and searching out the compressed reference feature with the common region as the reference feature to be matched.
According to an embodiment of the present invention, performing block template matching on an image to be corrected and a reference feature to be matched, to obtain uniformly distributed matching points includes: decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features; uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks; extracting K points with the maximum Harris angular point response from each block as points to be matched, and obtaining M multiplied by N multiplied by K points to be matched; extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected; for M multiplied by N multiplied by K points to be matched, matching the decompressed reference characteristic with the anisotropic differential characteristic by using a graphic processor, wherein each matching combination is distributed with a thread on the graphic processor to perform correlation metric calculation; and determining the point to be matched corresponding to the maximum correlation metric as uniformly distributed matching points.
According to an embodiment of the invention, the correlation metric is calculated by:
Figure SMS_2
wherein, the method comprises the following steps ofx c ,y c ) As the coordinates of the point to be matched,C(x c ,y c ,dx',dy' is the point to be matched%x c ,y c ) Corresponding correlation measurementx',y') is the matching of each pixel point within the template window D in the graphics processorThe coordinates of the two points of the coordinate system,F AGD t- (x',y' is the anisotropic differential characteristic corresponding to the pixel point (x ', y '),F ref-roi (x',y' is the decompressed reference feature corresponding to the pixel point (x ', y '), dx ', dy ' is the offset of the coordinates (x ', y '), and mu 1 and mu 2 are respectivelyF AGD t- (x ', y')F ref-roi (x ', y') means in the matching template window D, σ1, σ2 are respectivelyF AGD t- (x',y') andF ref-roi (x',y') standard deviation in the matching template window D, l being the size of the matching template window D.
According to an embodiment of the present invention, performing geometric fine correction on an image to be corrected according to uniformly distributed matching points includes: and (3) using the uniformly distributed matching points to select the geographic coordinates of the corresponding points in the lightweight reference feature library as control points, and correcting the image positioning parameters of the micro-nano satellite to finish geometric fine correction.
In a second aspect of the present invention, an in-orbit geometric correction device for micro-nano satellite images includes: the determining module is used for determining a mask of the region of interest based on an open source geographic vector file of the target region of interest of the micro-nano satellite; the extraction module is used for extracting the reference characteristics in the mask of the region of interest in the reference image of the target region; the compression module is used for carrying out feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features; the searching module is used for searching compressed reference features in a geographic range represented by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as the reference features to be matched; the matching module is used for carrying out block template matching on the image to be corrected and the reference characteristic to be matched to obtain uniformly distributed matching points; and the correction module is used for performing geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
The on-orbit geometric precise correction method and device for the micro-nano satellite image provided by the embodiment of the invention at least comprise the following beneficial effects:
by extracting the reference features in the mask of the region of interest in the reference image of the target region as the matching features, the sufficiently obvious reference features are reserved, so that the positioning accuracy of the micro-nano satellite image is improved, and accurate target position information is provided for on-orbit target detection and identification. Further, by compressing the enough significant reference features to obtain a lightweight reference feature library, the control reference image for geometric fine correction can be effectively compressed to adapt to the limited storage space of on-board processing.
The anisotropic Gaussian differential operator is used for calculating the multi-directional image intensity change characteristic of the reference image to serve as a reference characteristic, so that the intensity change and the fine structure of the reference image can be accurately and comprehensively described, and the positioning accuracy of the micro-nano satellite image is further improved.
The decompressed reference characteristic and anisotropic differential characteristic are matched by using the graphic processor which is adapted to the on-board edge computing equipment, so that the problems of limited on-board computing resources and limited running time due to heat dissipation can be solved, the conditions of abnormality, noise and initial offset are relatively robust, the requirements on the storage space, hardware resources, downloading bandwidth and ground base station maintenance cost of the satellite are lower, the method is suitable for engineering projects, and the running efficiency is improved.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the present invention with reference to the accompanying drawings, in which:
fig. 1 schematically shows a flowchart of an in-orbit geometric fine correction method for micro-nano satellite images provided by an embodiment of the invention.
Fig. 2 schematically shows a block diagram of an in-orbit geometry correction device for micro-nano satellite images according to an embodiment of the invention.
Description of the embodiments
The present invention will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly and include, for example, either permanently connected, removably connected, or integrally formed therewith; may be mechanically connected, may be electrically connected or may communicate with each other; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the description of the present invention, it should be understood that the terms "longitudinal," "length," "circumferential," "front," "rear," "left," "right," "top," "bottom," "inner," "outer," and the like indicate an orientation or a positional relationship based on that shown in the drawings, merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the subsystem or element in question must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Like elements are denoted by like or similar reference numerals throughout the drawings. Conventional structures or constructions will be omitted when they may cause confusion in the understanding of the invention. And the shape, size and position relation of each component in the figure do not reflect the actual size, proportion and actual position relation. In addition, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.
Similarly, in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various disclosed aspects. The description of the terms "one embodiment," "some embodiments," "example," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Fig. 1 schematically shows a flowchart of an in-orbit geometric fine correction method for micro-nano satellite images provided by an embodiment of the invention.
As shown in FIG. 1, the on-orbit geometric fine correction method for the micro/nano satellite image comprises operations S110-S160.
In operation S110, a region of interest mask is determined based on an open source geographic vector file of a target region of interest of the micro-nano satellite.
In an embodiment of the present invention, the operation S110 may specifically include: and converting the open source geographic vector file into a geographic coding image. And resampling the geocode image to obtain a geovector image. And acquiring an interested region in the geographic vector image. And performing morphological expansion operation on the region of interest to obtain a mask of the region of interest.
For example, considering that the storage space on the satellite is limited, and the micro-nano satellite mainly focuses on the global important target area, an open source geographic vector file of the global important target area is adopted as an interested area mask; and converting the geographical vector of the key area in the vector diagram format into a geographical coding image, resampling the geographical vector according to the resolution of the image to be corrected to obtain a geographical vector image, and marking the geographical vector image as Mapt.
Cutting out a region of interest MapROI of Mapt in an open-source geographic vector image, performing morphological expansion operation on the region surface vector, for example, directly expanding the region surface vector of a heavy airport for a certain number of times, firstly solving an edge, namely a coastline, of the region vector of a heavy port by using a Canny operator, and then expanding the coastline for a certain number of times to serve as a mask ROI of the region of interest, wherein the mask ROI comprises the following steps:
Figure SMS_3
wherein d is the window size of morphological dilation operation, n is the number of morphological dilation operation, the finally obtained mask ROI is the mask of the region of interest, and enough obvious reference characteristics are reserved, A represents an airport, and H represents a port.
In operation S120, reference features within a region of interest mask in a reference image of a target region are extracted.
In the embodiment of the present invention, the operation S120 may specifically include: and calculating the multi-directional image intensity change characteristics of the reference image through an anisotropic Gaussian differential operator, and determining the most obvious change characteristics in all directions as anisotropic differential characteristics. And carrying out threshold processing on the anisotropic differential characteristic based on the intensity threshold value to obtain a binarized anisotropic differential characteristic. And performing intersection operation on the binarized anisotropic differential feature and the region-of-interest mask to obtain a reference feature.
For example, for a global key target area, a google optical image corresponding to the global key target area is acquired as a reference image, and is recorded as a Reft. Considering that the space occupation of the optical image is large, if the reference image of the global target area is uploaded to the satellite for processing, hundreds of GB of storage space is needed, and cannot be satisfied in actual engineering. Therefore, the multidirectional image intensity variation characteristic of the optical reference image is calculated through a multidirectional anisotropic Gaussian differential operator, and the binarized anisotropic differential characteristic is obtained.
The anisotropic gaussian differential operator can be defined as follows:
Figure SMS_4
wherein x and y are respectively the horizontal and vertical coordinates of the pixel point in the reference image, sigma, rho and theta are respectively the scale parameter, the anisotropic parameter and the rotation angle, gsigma, rho and theta (x, y) are respectively anisotropic Gaussian kernels corresponding to the pixel point with the coordinates of (x, y), xtheta is the coordinate after the coordinate x rotates by the angle theta, and ytheta is the coordinate after the coordinate y rotates by the angle theta.
Differentiating the anisotropic Gaussian kernel, and convolving the differentiation result with the image to obtain the anisotropic Gaussian differential operator, wherein the anisotropic Gaussian differential operator is as follows:
Figure SMS_5
wherein MAGD (x, y, θ) is an anisotropic Gaussian differential operator corresponding to a pixel point with coordinates (x, y) in the θ angle direction, I is an intensity value of the reference image, and x is convolution operation. It will be appreciated that the first term to the right of the equal sign represents the differentiation of gσ, ρ, θ (x, y) from x when xθ is greater than 0, and the second term represents the differentiation of gσ, ρ, θ (x, y) from x when xθ is less than 0.
The anisotropic Gaussian kernels in multiple directions are set up by the value of theta, and then the proposed multidirectional MAGD operator is constructed by the anisotropic Gaussian kernels, so that the intensity change and the fine structure of the image can be accurately and comprehensively described. By extracting the most significant MAGD operator value in each direction, the anisotropic differential feature FAGD can be obtained, which is in particular as follows:
Figure SMS_6
wherein, the value of theta is 0 to pi.
Further, the anisotropic differential feature FAGD is binarized by threshold segmentation, and the threshold is selected as the first 10% anisotropic differential feature response value, as follows:
Figure SMS_7
wherein FAGD-B is a binarized anisotropic differential feature, FAGD-AS is an anisotropic differential feature arranged in ascending order, W×H is the length and width of the image, and t is the first 10% threshold. Through the operation, the 16-bit reference image can be compressed into a binary image, so that the storage space is saved.
And further, intersection is taken between the binarized anisotropic differential feature and the mask roi of the region of interest obtained in the S110 operation, as follows:
Figure SMS_8
wherein, fref refers to the base features in the mask of the region of interest in the image, and n is the intersection operation. Therefore, only the reference features in the mask of the region of interest are stored in the feature library, the most favorable characteristics for image matching are reserved in some obvious and unique regions, the rest redundant and confusing parts are all 0, and related information is removed from the feature library.
In operation S130, feature compression is performed on the reference features to obtain a lightweight reference feature library.
In an embodiment of the invention, the lightweight reference feature library includes compressed reference features and geographic ranges of the compressed reference features. And carrying out travel coding on the reference features, and replacing the reference features with the same intensity value by using the string length to obtain a lightweight reference feature library.
Illustratively, the reference feature Fref within the binarized region of interest mask is further compressed in size using a run-length encoding algorithm, as follows:
Figure SMS_9
where Fref-lib is the final lightweight baseline signature library, count is the count operation, i.e., counting the number of identical pixels until the value of the next pixel has changed,
Figure SMS_10
representing the statistics starting from the upper left pixel until the lower right pixel of the reference image, m, n are each an increment of statistics for x, y, respectively. After the pixel arrangement sequence is defined, the continuous identical pixels are replaced by the numerical value and the length of the continuous pixel string, so that lossless compression is realized. The larger the image blocks having the same color in the image, the smaller the number of image blocks, and the higher the compression ratio. For the reference feature Fref of the region of interest after intersection, as the features reserved by the mask only occupy a small part of the image and the rest is 0, the algorithm can greatly save the space occupied by the information of mask rejection, ensure that the lightweight reference feature library Fref-lib contains the most valuable features, reject redundant information and obtain a higher compression ratio. The operations are all completed in the ground processing system, and after the lightweight reference feature library is constructed, the lightweight reference feature library is stored in the on-board edge computing equipment for subsequent on-board processing.
In operation S140, compressed reference features within a geographic range characterized by the geographic information are searched in a lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and are used as reference features to be matched.
In the embodiment of the present invention, the operation S140 may specifically include: and calculating the longitude and latitude of four corners of the image to be corrected. And carrying out overlapping region inquiry on the region represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and searching out the compressed reference feature with the common region as the reference feature to be matched.
Illustratively, the subsequent operations from operation S140 are all on-board processing operations. Firstly, calculating longitude and latitude of a four corner point of an image to be corrected generated by an on-board imaging algorithm, inquiring an overlapping region with a reference feature geographic range stored in a reference feature library, calculating the area of the overlapping region, and searching out reference features with common regions, wherein the method comprises the following steps:
Figure SMS_11
the Area is the Area of the overlapped Area, the longitude and latitude of four corner points of the image to be corrected are respectively marked as maxLon1, minLon1, maxLat1 and minLat1, and the longitude and latitude of four corner points of the reference feature are respectively marked as maxLon2, minLon2, maxLat2 and minLat2. The latitude and longitude resolutions of the reference features are respectively represented by resolutionLon and resolutionLat. When the overlapping area is greater than a given threshold, then the area reference feature Fref-lib-roi is selected from the reference feature library Fref-lib for subsequent processing.
In operation S150, the image to be corrected and the reference feature to be matched are subjected to block template matching, so as to obtain uniformly distributed matching points.
In an embodiment of the present invention, the operation S150 may specifically include: decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features. And uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks. K points with the largest Harris angular point response are extracted from each block to serve as points to be matched, and M multiplied by N multiplied by K points to be matched are obtained. And extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected. For M×N×K points to be matched, matching the decompressed reference features with anisotropic differential features by using a graphics processor (Graphic Processing Unit, GPU), wherein each matching combination is assigned a thread on the graphics processor for correlation metric computation. And determining the point to be matched corresponding to the maximum correlation metric as uniformly distributed matching points.
Illustratively, in the on-board processing stage, first, decompression processing is performed on the reference feature Fref-lib-roi of the selected region, and the binary feature is recovered, and the recovery process is as follows:
Figure SMS_12
wherein, (x ', y') is the decompressed pixel-by-pixel position, and d is the position of the reference feature library corresponding to the current pixel after compression. In the decompression process, two coding values in the reference feature library Fref-lib-roi are periodically read in, and the coding result is decoded into decompressed reference features Fref-roi.
And then uniformly partitioning the overlapping area of the image to be corrected and the reference feature into M multiplied by N blocks, extracting K points with the largest Harris angular point response from each block as points to be matched, and extracting anisotropic differential features FAGD-t of the image to be corrected in the common area by using a multidirectional anisotropic Gaussian differential operator. And carrying out GPU template matching on M multiplied by N multiplied by K points to be matched by utilizing anisotropic differential characteristics FAGD-t of the image to be corrected and Fref-roi decompressed by a reference characteristic library, wherein the matching correlation metric C is calculated as follows:
Figure SMS_13
wherein (xc, yc) is the image coordinate of the point to be matched, C (xc, yc, dx ', dy') is the correlation metric corresponding to the point to be matched (xc, yc), and (x ', y') is the coordinate of each pixel point in the matching template window D in the graphics processor, FAGD-t (x ', y') is the anisotropic differential feature corresponding to the pixel point (x ', y'), fref-roi (x ', y') is the decompressed reference feature corresponding to the pixel point (x ', y'), dx ', dy') is the offset of the coordinate (x ', y'), mu 1 and mu 2 are the average value of FAGD-t (x ', y') and Fref-roi (x ', y') in the matching template window D, and sigma 1 and sigma 2 are the standard deviation of FAGD-t (x ', y') and Fref-roi (x ', y') in the matching template window D, respectively.
Because the initial positioning error of the micro-nano satellite is larger, a larger value range is required to be set for the offset dx and dy, and a larger calculation load is generated. To reduce the repetition of calculations, the manner of calculation of the correlation metric may be optimized as follows:
Figure SMS_14
where l is the size of the matching template window D. And further, the correlation metric calculation of M multiplied by N multiplied by K points to be matched is solved by utilizing the parallel calculation capability of the GPU. Variables that can be calculated in parallel include: points to be matched (xc, yc); different offsets (dx ', dy'); each pixel (x ', y') within the template window D is matched. One matching combination is to fix a set of offsets (dx 0, dy 0) for each point to be matched (xc 0, yc 0) and calculate a correlation metric for a certain pixel (x 0, y 0) in the matching template window D. A Thread (Thread) is assigned to each matching combination on the GPU for correlation metric computation.
After the correlation metrics of all the matching combinations are calculated in parallel, the calculation results in the matching template window D need to be summed, and the parallel addition operation can be accelerated by using a Koggle-Stone algorithm. Summing the one-dimensional arrays, wherein the conventional cyclic sum time complexity is O (n), and the time complexity of the Koggle-Stone algorithm is O (log). For the two-dimensional summation problem of matching template windows, the temporal complexity of the cyclic summation is O (n 2). The pixel sums in the matching window can be obtained by summing each row in the horizontal direction and summing the row sum results in the vertical direction by using a Koggle-Stone parallel algorithm, and the time complexity is 2O (log n).
And finally, calculating the correlation metrics of all possible offsets of each point (xc, yc) to be matched, and selecting the largest correlation metric as a matching result to obtain the matching corresponding point coordinates (xm, ym) of the point (xc, yc) to be matched as follows:
Figure SMS_15
in operation S160, geometric fine correction is performed on the image to be corrected according to the uniformly distributed matching points.
In the embodiment of the present invention, the operation S160 may specifically include: and (3) using the uniformly distributed matching points to select the geographic coordinates of the corresponding points in the lightweight reference feature library as control points, and correcting the image positioning parameters of the micro-nano satellite to finish geometric fine correction.
Illustratively, the uniformly distributed matching point pairs (xc, yc; xm, ym) obtained by the partitioned template matching are utilized to reject the mismatching points in the matching point pairs by a random sample consensus algorithm (RANSAC). And selecting the geographic coordinates of the points of the reference feature library from the rest correct matching point pairs as control points, and correcting the positioning parameters of the micro-nano satellite images to finish the geometric fine correction processing.
In summary, the on-orbit geometric precise correction method for the micro-nano satellite image provided by the embodiment of the invention can rapidly and real-timely improve the positioning precision of the micro-nano satellite image and provide precise target position information for on-orbit target detection and identification. The lightweight standard feature library can effectively compress control reference images for geometric fine correction so as to adapt to a limited storage space for on-board processing, is adapted to a GPU block template matching algorithm of on-board edge computing equipment, can solve the problems of limited on-board computing resources and limitation of heat dissipation operation time, is relatively robust to abnormal, noise and initial offset, has low requirements on storage space, hardware resources, downloading bandwidth and ground base station maintenance cost of satellites, and is suitable for engineering projects. Therefore, the embodiment uses the lightweight feature library to perform template matching, simultaneously compresses the storage space occupied by the feature library, realizes the algorithm on the embedded GPU, improves the operation efficiency, and is suitable for the precise correction of the in-orbit satellite images.
Based on the same inventive concept, the embodiment of the invention also provides an on-orbit geometric fine correction device for the micro-nano satellite image.
Fig. 2 schematically shows a block diagram of an in-orbit geometry correction device for micro-nano satellite images according to an embodiment of the invention.
As shown in fig. 2, the on-orbit geometric correction apparatus 200 for micro-nano satellite image comprises: the system comprises a determining module 210, an extracting module 220, a compressing module 230, a retrieving module 240, a matching module 250 and a correcting module 260.
A determining module 210, configured to determine a region of interest mask based on an open source geographic vector file of a target region of interest of the micro-nano satellite.
The extracting module 220 is configured to extract a reference feature in the region of interest mask in the reference image of the target region.
The compression module 230 is configured to perform feature compression on the reference feature to obtain a lightweight reference feature library, where the lightweight reference feature library includes the compressed reference feature and a geographic range of the compressed reference feature.
The retrieving module 240 is configured to retrieve, from a lightweight reference feature library, compressed reference features within a geographic range represented by the geographic information according to geographic information of an image to be corrected generated by on-board imaging, as reference features to be matched.
And the matching module 250 is used for carrying out block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points.
The correction module 260 is configured to perform geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
It should be noted that the specific implementation details and the technical effects of the embodiment part of the apparatus do not correspond to those of the embodiment of the method, and are not repeated herein.
While the foregoing is directed to embodiments of the present invention, other and further details of the invention may be had by the present invention, it should be understood that the foregoing description is merely illustrative of the present invention and that no limitations are intended to the scope of the invention, except insofar as modifications, equivalents, improvements or modifications are within the spirit and principles of the invention.

Claims (7)

1. An on-orbit geometric precise correction method for micro-nano satellite images is characterized by comprising the following steps:
determining a region-of-interest mask based on an open source geographic vector file of the target region of interest of the micro-nano satellite;
extracting a reference feature in the region of interest mask in the reference image of the target region;
performing feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features;
searching compressed reference features in a geographic range represented by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, wherein the compressed reference features are used as the reference features to be matched, and the method comprises the following steps: calculating longitude and latitude of four corner points of the image to be corrected; overlapping area inquiry is carried out on the area represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and the compressed reference feature with the common area is searched out and used as the reference feature to be matched;
performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points, wherein the method comprises the following steps: decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features; uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks; extracting K points with the maximum Harris angular point response from each block as points to be matched, and obtaining M multiplied by N multiplied by K points to be matched; extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected; for M multiplied by N multiplied by K points to be matched, matching the decompressed reference features with anisotropic differential features by using a graphics processor, wherein each matching combination is distributed with a thread on the graphics processor to perform correlation metric calculation; determining the point to be matched corresponding to the maximum correlation measure as uniformly distributed matching points; the correlation metric is calculated by the following steps:
Figure QLYQS_1
wherein, the method comprises the following steps ofx c ,y c ) As the coordinates of the point to be matched,C(x c ,y c ,dx',dy') The points to be matched are%x c ,y c ) Corresponding correlation measurementx',y') Matching template windows for graphics processorDEach pixel point in the imageThe coordinates of the two points of the coordinate system,F AGD t- (x',y') Is pixel point #)x',y') A corresponding anisotropic differential characteristic is provided which,F ref-roi (x',y') Is pixel point #)x',y') Corresponding decompressed reference characteristics #dx',dy') Is coordinates [ ]x',y') Is set in the first stage of the process,μ 1μ 2 respectively isF AGD t- (x',y') AndF ref-roi (x',y') In a matching template windowDIs used as a mean value of the two,σ 1σ 2 respectively isF AGD t- (x',y') AndF ref-roi (x',y') In a matching template windowDIs used for the standard deviation of the above-mentioned components,lto match template windowDIs a dimension of (2);
and carrying out geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
2. The method for accurately correcting the on-orbit geometry of a micro-nano satellite image according to claim 1, wherein the determining the region of interest mask based on the open source geographical vector file of the target region of interest of the micro-nano satellite comprises:
converting the open source geographic vector file into a geographic coding image;
resampling the geocode image to obtain a geovector image;
acquiring an interested region in the geographic vector image;
and performing morphological expansion operation on the region of interest to obtain the mask of the region of interest.
3. The method of claim 1, wherein the extracting the fiducial features in the region of interest mask in the reference image of the target region comprises:
calculating the multi-directional image intensity change characteristics of the reference image through an anisotropic Gaussian differential operator, and determining the most obvious change characteristics in all directions as anisotropic differential characteristics;
performing threshold processing on the anisotropic differential feature based on an intensity threshold to obtain a binarized anisotropic differential feature;
and performing intersection operation on the binarized anisotropic differential feature and the region-of-interest mask to obtain the reference feature.
4. The method for accurately correcting the on-orbit geometry of a micro-nano satellite image according to claim 3, wherein the anisotropic gaussian differential operator is as follows:
Figure QLYQS_2
wherein,,xyrespectively the abscissa and the ordinate of the pixel points in the reference image,σρθrespectively the scale parameter, the anisotropy parameter and the rotation angle,G σ ρ θ,, (x,y) Is the coordinates of%x,y) An anisotropic gaussian kernel corresponding to the pixel points of (a),x θ is the coordinatesxRotatingθThe coordinates after the angle are used for the measurement,MAGD(x,y,θ) Is the coordinates of%x,y) Is at the pixel point of (2)θAn anisotropic gaussian derivative operator corresponding to the angular direction,Ithe intensity value of the reference image is convolution operation.
5. The method for accurately correcting the on-orbit geometry of the micro-nano satellite image according to claim 1, wherein the step of performing feature compression on the reference features to obtain a lightweight reference feature library comprises the steps of:
and performing travel coding on the reference features, and replacing the reference features with the same intensity value by using the string length to obtain the lightweight reference feature library.
6. The method for performing geometric fine correction on an in-orbit micro-nano satellite image according to claim 1, wherein the performing geometric fine correction on the image to be corrected according to the uniformly distributed matching points comprises:
and using the uniformly distributed matching points to select geographic coordinates of corresponding points in the lightweight reference feature library as control points, and correcting the image positioning parameters of the micro-nano satellite to finish geometric fine correction.
7. An on-orbit geometry correction device for micro-nano satellite images, which is characterized by comprising:
the determining module is used for determining a mask of the region of interest based on the open source geographic vector file of the target region of interest of the micro-nano satellite;
the extraction module is used for extracting the reference characteristics in the region of interest mask in the reference image of the target region;
the compression module is used for carrying out feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features;
the searching module is used for searching compressed reference features in a geographic range represented by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and the compressed reference features are used as the reference features to be matched, and comprise the following steps: calculating longitude and latitude of four corner points of the image to be corrected; overlapping area inquiry is carried out on the area represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and the compressed reference feature with the common area is searched out and used as the reference feature to be matched;
the matching module is used for carrying out block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points, and comprises the following steps: decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features; uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks; extracting K points with the maximum Harris angular point response from each block as points to be matched, and obtaining M multiplied by N multiplied by K points to be matched; extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected; for M multiplied by N multiplied by K points to be matched, matching the decompressed reference features with anisotropic differential features by using a graphics processor, wherein each matching combination is distributed with a thread on the graphics processor to perform correlation metric calculation; determining the point to be matched corresponding to the maximum correlation measure as uniformly distributed matching points; the correlation metric is calculated by the following steps:
Figure QLYQS_3
wherein, the method comprises the following steps ofx c ,y c ) As the coordinates of the point to be matched,C(x c ,y c ,dx',dy') The points to be matched are%x c ,y c ) Corresponding correlation measurementx',y') Matching template windows for graphics processorDThe coordinates of each pixel point within the image,F AGD t- (x',y') Is pixel point #)x',y') A corresponding anisotropic differential characteristic is provided which,F ref-roi (x',y') Is pixel point #)x',y') Corresponding decompressed reference characteristics #dx',dy') Is coordinates [ ]x',y') Is set in the first stage of the process,μ 1μ 2 respectively isF AGD t- (x',y') AndF ref-roi (x',y') In a matching template windowDIs used as a mean value of the two,σ 1σ 2 respectively isF AGD t- (x',y') AndF ref-roi (x',y') In a matching template windowDIs used for the standard deviation of the above-mentioned components,lto match template windowDIs a dimension of (2); and the correction module is used for performing geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
CN202310307607.3A 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device Active CN116071539B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310307607.3A CN116071539B (en) 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310307607.3A CN116071539B (en) 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device

Publications (2)

Publication Number Publication Date
CN116071539A CN116071539A (en) 2023-05-05
CN116071539B true CN116071539B (en) 2023-07-11

Family

ID=86175288

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310307607.3A Active CN116071539B (en) 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device

Country Status (1)

Country Link
CN (1) CN116071539B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116958833B (en) * 2023-09-20 2023-12-08 青岛浩海网络科技股份有限公司 GF4 data geometric fine correction method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110659369A (en) * 2019-03-22 2020-01-07 西安电子科技大学 On-orbit high-precision lightweight global image control point database construction method and system
CN115082359A (en) * 2022-07-27 2022-09-20 中国科学院空天信息创新研究院 Synchronous orbit optical satellite geometric fine correction method based on coastline data

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103383773B (en) * 2013-03-26 2016-09-28 中国科学院遥感与数字地球研究所 The remote sensing satellite image of a kind of dynamic extraction Image Control Point is the most just penetrating framework and the method for correction
US10838990B1 (en) * 2013-09-26 2020-11-17 EMC IP Holding Company LLC System and method for improving data compression of a storage system using coarse and fine grained similarity
CN114972555A (en) * 2022-05-25 2022-08-30 中国科学院空天信息创新研究院 Method, device, equipment and storage medium for compressing and correcting lightweight base map on satellite
CN115330619A (en) * 2022-08-11 2022-11-11 长光卫星技术股份有限公司 Local geometric fine correction method suitable for high-resolution remote sensing image

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110659369A (en) * 2019-03-22 2020-01-07 西安电子科技大学 On-orbit high-precision lightweight global image control point database construction method and system
CN115082359A (en) * 2022-07-27 2022-09-20 中国科学院空天信息创新研究院 Synchronous orbit optical satellite geometric fine correction method based on coastline data

Also Published As

Publication number Publication date
CN116071539A (en) 2023-05-05

Similar Documents

Publication Publication Date Title
CN111598823B (en) Multisource mobile measurement point cloud data space-ground integration method and storage medium
CN108230375B (en) Registration method of visible light image and SAR image based on structural similarity rapid robustness
CN116071539B (en) Micro-nano satellite image on-orbit geometric precise correction method and device
CN108428220B (en) Automatic geometric correction method for ocean island reef area of remote sensing image of geostationary orbit satellite sequence
CN109100719B (en) Terrain map joint mapping method based on satellite-borne SAR (synthetic aperture radar) image and optical image
CN106887016B (en) Automatic relative registration method for GF-4 satellite sequence images
EP3543910B1 (en) Cloud detection in aerial imagery
CN112419380B (en) Cloud mask-based high-precision registration method for stationary orbit satellite sequence images
CN110363758B (en) Optical remote sensing satellite imaging quality determination method and system
Jiang et al. Hierarchical motion consistency constraint for efficient geometrical verification in UAV stereo image matching
Gonçalves et al. CHAIR: Automatic image registration based on correlation and Hough transform
CN115797381B (en) Heterogeneous remote sensing image registration method based on geographic segmentation and hierarchical feature matching
CN107797110A (en) The system imaging method and aperture synthesis projection radiation meter of aperture synthesis projection radiation
CN112200845A (en) Image registration method and device
CN114581498B (en) Iterative model registration system combining vector data and raster image
CN115330619A (en) Local geometric fine correction method suitable for high-resolution remote sensing image
CN114004870A (en) Unmanned aerial vehicle SAR image geocoding method using optical orthographic image
CN114972453A (en) Improved SAR image region registration method based on LSD and template matching
CN113486728A (en) Method and device for detecting surface three-dimensional change based on feature fusion
Sadeq Using total probability in image template matching.
CN115049708B (en) SAR image registration method based on LSD line detection and template matching
Strunz et al. Automation of tie-pointing procedure for the geocoding of satellite images
Luo et al. Comparative Analysis of Advanced Feature Matching Algorithms in Challenging High Spatial Resolution Optical Satellite Stereo Scenarios
CN115019198B (en) Iterative updating and charting method for high-frequency remote sensing observation image
Ye et al. Improving Co-Registration for Sentinel-1 SAR and Sentinel-2 Optical Images. Remote Sens. 2021, 13, 928

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant