CN116071539A - Micro-nano satellite image on-orbit geometric precise correction method and device - Google Patents

Micro-nano satellite image on-orbit geometric precise correction method and device Download PDF

Info

Publication number
CN116071539A
CN116071539A CN202310307607.3A CN202310307607A CN116071539A CN 116071539 A CN116071539 A CN 116071539A CN 202310307607 A CN202310307607 A CN 202310307607A CN 116071539 A CN116071539 A CN 116071539A
Authority
CN
China
Prior art keywords
image
region
matched
micro
geographic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310307607.3A
Other languages
Chinese (zh)
Other versions
CN116071539B (en
Inventor
向俞明
胡玉新
王林徽
王峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aerospace Information Research Institute of CAS
Original Assignee
Aerospace Information Research Institute of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aerospace Information Research Institute of CAS filed Critical Aerospace Information Research Institute of CAS
Priority to CN202310307607.3A priority Critical patent/CN116071539B/en
Publication of CN116071539A publication Critical patent/CN116071539A/en
Application granted granted Critical
Publication of CN116071539B publication Critical patent/CN116071539B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/24Aligning, centring, orientation detection or correction of the image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/34Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462Salient features, e.g. scale invariant feature transforms [SIFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/13Satellite images
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a micro-nano satellite image in-orbit geometric precise correction method and a device, which relate to the field of micro-nano satellite image in-orbit geometric processing, and the method comprises the following steps: determining a mask of the region of interest based on an open source geographic vector file of a target region of interest of the micro-nano satellite; extracting a reference feature in a mask of a region of interest in a reference image of a target region; performing feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and corresponding geographic ranges; searching compressed reference features in a geographic range represented by the geographic information in a lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as reference features to be matched; performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points; and carrying out geometric fine correction on the image to be corrected according to the uniformly distributed matching points. The method and the device have high positioning precision and high positioning efficiency.

Description

Micro-nano satellite image on-orbit geometric precise correction method and device
Technical Field
The invention relates to the technical field of on-orbit geometric processing of micro-nano satellite images, in particular to a method and a device for precisely correcting the on-orbit geometric of micro-nano satellite images.
Background
With the gradual maturation of micro-nano satellite constellation technology, a multi-source satellite cluster shooting mode can realize short-period global revisit under the condition of taking high-resolution earth observation into consideration. The existing typical remote sensing satellite ground processing system at home and abroad adopts a process of downloading satellite original data to standard product production. Due to the limitation of bandwidth and other reasons, the downloading of a large amount of original data is time-consuming and labor-consuming, and the timeliness of a processing system is greatly limited. In order to relieve the pressure of satellite-to-ground transmission and improve the timeliness of information, on-orbit processing becomes a main processing mode of a micro-nano satellite constellation.
In order to improve timeliness of micro-nano satellite information processing, an on-orbit intelligent processing module carries out target detection and identification on the image and only downloads detected target slices. However, the micro-nano satellite has small load volume and light weight, and the positioning accuracy is poorer than that of the traditional large satellite, and the downloaded target slice has positioning errors of hundreds of meters or even kilometers, so that the accuracy of target information and multi-satellite combined application are greatly limited. Therefore, geometric fine correction of the micro-nano satellite image is a key step of subsequent application.
Conventional geometric fine correction relies on a control point slice library or a reference image as a basis to correct the positioning parameters of the image. However, due to the limited space of the on-board storage, it is difficult to directly acquire a control point slice library or a reference image required for fine correction. And the micro-nano satellite needs to consider the cost and weight of hardware equipment, the gap between equipment resources available for calculation and a ground processing system is too large, and meanwhile, due to the limitation of space heat dissipation conditions, the on-board embedded system cannot operate for a long time. These hardware limitations also present a significant challenge to the algorithm for on-orbit processing.
Disclosure of Invention
The invention provides a method and a device for accurately correcting the on-orbit geometry of a base micro-nano satellite image, which are used for at least partially solving the technical problems.
Based on this, the first aspect of the present invention provides a method for precisely correcting the on-orbit geometry of a micro-nano satellite image, comprising: determining a mask of the region of interest based on an open source geographic vector file of a target region of interest of the micro-nano satellite; extracting a reference feature in a mask of a region of interest in a reference image of a target region; performing feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features; searching compressed reference features in a geographic range represented by the geographic information in a lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as reference features to be matched; performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points; and carrying out geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
According to an embodiment of the present invention, determining a region of interest mask based on an open source geographic vector file of a target region of interest for a micro-nano satellite comprises: converting the open source geographic vector file into a geographic coding image; resampling the geocode image to obtain a geovector image; acquiring an interested region in a geographic vector image; and performing morphological expansion operation on the region of interest to obtain a mask of the region of interest.
According to an embodiment of the present invention, extracting a reference feature in a region of interest mask in a reference image of a target region includes: calculating the multi-directional image intensity change characteristics of the reference image through an anisotropic Gaussian differential operator, and determining the most obvious change characteristics in all directions as anisotropic differential characteristics; performing threshold processing on the anisotropic differential feature based on the intensity threshold to obtain a binarized anisotropic differential feature; and performing intersection operation on the binarized anisotropic differential feature and the region-of-interest mask to obtain a reference feature.
According to an embodiment of the invention, the anisotropic gaussian differential operator is:
Figure SMS_1
wherein ,xyrespectively the abscissa and the ordinate of the pixel points in the reference image,σρθrespectively the scale parameter, the anisotropy parameter and the rotation angle,G σ ρ θ,, (x,y) Is the coordinates of%x,y) An anisotropic gaussian kernel corresponding to the pixel points of (a),x θ is the coordinatesxRotatingθThe coordinates after the angle are used for the measurement,MAGD(x,y,θ) Is the coordinates of%x,y) Is at the pixel point of (2)θAn anisotropic gaussian derivative operator corresponding to the angular direction,Ithe intensity value of the reference image is calculated by convolution.
According to an embodiment of the present invention, performing feature compression on a reference feature to obtain a lightweight reference feature library includes: and performing travel coding on the reference features, and replacing the reference features with the same intensity value by using the string length to obtain a lightweight reference feature library.
According to the embodiment of the invention, according to the geographic information of the image to be corrected generated by on-board imaging, compressed reference features in a geographic range represented by the geographic information are searched in a lightweight reference feature library, and the compressed reference features as the reference features to be matched comprise: calculating longitude and latitude of four corners of the image to be corrected; and carrying out overlapping region inquiry on the region represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and searching out the compressed reference feature with the common region as the reference feature to be matched.
According to an embodiment of the present invention, performing block template matching on an image to be corrected and a reference feature to be matched, to obtain uniformly distributed matching points includes: decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features; uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks; extracting K points with the maximum Harris angular point response from each block as points to be matched, and obtaining M multiplied by N multiplied by K points to be matched; extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected; for M multiplied by N multiplied by K points to be matched, matching the decompressed reference characteristic with the anisotropic differential characteristic by using a graphic processor, wherein each matching combination is distributed with a thread on the graphic processor to perform correlation metric calculation; and determining the point to be matched corresponding to the maximum correlation metric as uniformly distributed matching points.
According to an embodiment of the invention, the correlation metric is calculated by:
Figure SMS_2
wherein, the method comprises the following steps ofx c ,y c ) As the coordinates of the point to be matched,C(x c ,y c ,dx,dy) The points to be matched are%x c ,y c ) Corresponding correlation measurementx,y) Matching template windows for graphics processorDThe coordinates of each pixel point within the image,F AGD t- (x,y) Is pixel point #)x,y) A corresponding anisotropic differential characteristic is provided which,F ref-roi (x,y) Is pixel point #)x,y) Corresponding decompressed reference characteristics #dx, dy) Is coordinates [ ]x,y) Is set in the first stage of the process,μ 1μ 2 respectively isF AGD t- (x,y) AndF ref-roi (x,y) In a matching template windowDIs used as a mean value of the two,σ 1σ 2 respectively isF AGD t- (x,y) AndF ref-roi (x,y) In a matching template windowDIs used for the standard deviation of the above-mentioned components,lto match template windowDIs a size of (c) a.
According to an embodiment of the present invention, performing geometric fine correction on an image to be corrected according to uniformly distributed matching points includes: and (3) using the uniformly distributed matching points to select the geographic coordinates of the corresponding points in the lightweight reference feature library as control points, and correcting the image positioning parameters of the micro-nano satellite to finish geometric fine correction.
In a second aspect of the present invention, an in-orbit geometric correction device for micro-nano satellite images includes: the determining module is used for determining a mask of the region of interest based on an open source geographic vector file of the target region of interest of the micro-nano satellite; the extraction module is used for extracting the reference characteristics in the mask of the region of interest in the reference image of the target region; the compression module is used for carrying out feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features; the searching module is used for searching compressed reference features in a geographic range represented by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as the reference features to be matched; the matching module is used for carrying out block template matching on the image to be corrected and the reference characteristic to be matched to obtain uniformly distributed matching points; and the correction module is used for performing geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
The on-orbit geometric precise correction method and device for the micro-nano satellite image provided by the embodiment of the invention at least comprise the following beneficial effects:
by extracting the reference features in the mask of the region of interest in the reference image of the target region as the matching features, the sufficiently obvious reference features are reserved, so that the positioning accuracy of the micro-nano satellite image is improved, and accurate target position information is provided for on-orbit target detection and identification. Further, by compressing the enough significant reference features to obtain a lightweight reference feature library, the control reference image for geometric fine correction can be effectively compressed to adapt to the limited storage space of on-board processing.
The anisotropic Gaussian differential operator is used for calculating the multi-directional image intensity change characteristic of the reference image to serve as a reference characteristic, so that the intensity change and the fine structure of the reference image can be accurately and comprehensively described, and the positioning accuracy of the micro-nano satellite image is further improved.
The decompressed reference characteristic and anisotropic differential characteristic are matched by using the graphic processor which is adapted to the on-board edge computing equipment, so that the problems of limited on-board computing resources and limited running time due to heat dissipation can be solved, the conditions of abnormality, noise and initial offset are relatively robust, the requirements on the storage space, hardware resources, downloading bandwidth and ground base station maintenance cost of the satellite are lower, the method is suitable for engineering projects, and the running efficiency is improved.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the present invention with reference to the accompanying drawings, in which:
fig. 1 schematically shows a flowchart of an in-orbit geometric fine correction method for micro-nano satellite images provided by an embodiment of the invention;
fig. 2 schematically shows a block diagram of an in-orbit geometry correction device for micro-nano satellite images according to an embodiment of the invention.
Detailed Description
The present invention will be further described in detail below with reference to specific embodiments and with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly and include, for example, either permanently connected, removably connected, or integrally formed therewith; may be mechanically connected, may be electrically connected or may communicate with each other; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the description of the present invention, it should be understood that the terms "longitudinal," "length," "circumferential," "front," "rear," "left," "right," "top," "bottom," "inner," "outer," and the like indicate an orientation or a positional relationship based on that shown in the drawings, merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the subsystem or element in question must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Like elements are denoted by like or similar reference numerals throughout the drawings. Conventional structures or constructions will be omitted when they may cause confusion in the understanding of the invention. And the shape, size and position relation of each component in the figure do not reflect the actual size, proportion and actual position relation. In addition, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.
Similarly, in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various disclosed aspects. The description of the terms "one embodiment," "some embodiments," "example," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Fig. 1 schematically shows a flowchart of an in-orbit geometric fine correction method for micro-nano satellite images provided by an embodiment of the invention.
As shown in FIG. 1, the on-orbit geometric fine correction method for the micro/nano satellite image comprises operations S110-S160.
In operation S110, a region of interest mask is determined based on an open source geographic vector file of a target region of interest of the micro-nano satellite.
In an embodiment of the present invention, the operation S110 may specifically include: and converting the open source geographic vector file into a geographic coding image. And resampling the geocode image to obtain a geovector image. And acquiring an interested region in the geographic vector image. And performing morphological expansion operation on the region of interest to obtain a mask of the region of interest.
For example, considering that the storage space on the satellite is limited, and the micro-nano satellite mainly focuses on the global important target area, an open source geographic vector file of the global important target area is adopted as an interested area mask; converting the geographical vector of the key area in the vector diagram format into a geocoded image, resampling the geocoded image according to the resolution of the image to be corrected to obtain a geographical vector image, and recording the geographical vector image asMap t
Cutting out geographic vector images of open sourcesMap t Is of interest in (a)Map ROI Performing morphological dilation operation on the regional face vector, e.g. directly dilating the regional face vector of the emphasis airport a certain number of times, for the emphasis harborThe edge of the port area vector is firstly calculated by using a Canny operator, namely a coastline, and then the coastline is expanded for a certain times to be used as a mask of the area of interestMask ROI The following is shown:
Figure SMS_3
wherein,dfor the window size of the morphological dilation operation,nis the number of morphological dilation operations, and is finally obtainedMask ROI I.e. the region of interest mask, and retains sufficiently significant fiducial features, a for airports and H for ports.
In operation S120, reference features within a region of interest mask in a reference image of a target region are extracted.
In the embodiment of the present invention, the operation S120 may specifically include: and calculating the multi-directional image intensity change characteristics of the reference image through an anisotropic Gaussian differential operator, and determining the most obvious change characteristics in all directions as anisotropic differential characteristics. And carrying out threshold processing on the anisotropic differential characteristic based on the intensity threshold value to obtain a binarized anisotropic differential characteristic. And performing intersection operation on the binarized anisotropic differential feature and the region-of-interest mask to obtain a reference feature.
For example, for a global key target area, a google optical image corresponding to the global key target area is obtained as a reference image, and is recorded asRef t . Considering that the space occupation of the optical image is large, if the reference image of the global target area is uploaded to the satellite for processing, hundreds of GB of storage space is needed, and cannot be satisfied in actual engineering. Therefore, the multidirectional image intensity variation characteristic of the optical reference image is calculated through a multidirectional anisotropic Gaussian differential operator, and the binarized anisotropic differential characteristic is obtained.
The anisotropic gaussian differential operator can be defined as follows:
Figure SMS_4
wherein,xyrespectively the abscissa and the ordinate of the pixel points in the reference image,σρθrespectively the scale parameter, the anisotropy parameter and the rotation angle,G σ ρ θ,, (x,y) Is the coordinates of%x,y) An anisotropic gaussian kernel corresponding to the pixel points of (a),x θ is the coordinatesxRotatingθThe coordinates after the angle are used for the measurement,y θ is the coordinatesyRotatingθCoordinates after the angle.
Differentiating the anisotropic Gaussian kernel, and convolving the differentiation result with the image to obtain the anisotropic Gaussian differential operator, wherein the anisotropic Gaussian differential operator is as follows:
Figure SMS_5
wherein,MAGD(x,y,θ) Is the coordinates of%x,y) Is at the pixel point of (2)θAn anisotropic gaussian derivative operator corresponding to the angular direction,Ithe intensity value of the reference image is convolution operation. It should be appreciated that the first item to the right of the equal sign representsx θ When the average particle diameter is greater than 0G σ ρ θ,, (x,y) For a pair ofxThe second term represents the differentiation ofx θ Less than 0G σ ρ θ,, (x,y) For a pair ofxIs a derivative of (a).
By setting upθIs used to construct the proposed multidirectionalMAGDThe operator can accurately and comprehensively describe the intensity change and the fine structure of the image. By extracting the most pronounced of the directionsMAGDThe value of the algorithm can obtain anisotropic differential characteristicsF AGD The method is characterized by comprising the following steps:
Figure SMS_6
wherein,θhas a value of 0 toπ
Further utilizing threshold segmentation to characterize anisotropyF AGD Binarization is carried out, and the threshold value is selected as the anisotropic differential characteristic response value of the first 10 percent, as follows:
Figure SMS_7
wherein,F AGD-B is an anisotropic differential feature of binarization,F AGD-AS for the anisotropic differential feature of the ascending order,W×Hfor the length and width of the image,tis the first 10% threshold. Through the operation, the 16-bit reference image can be compressed into a binary image, so that the storage space is saved.
Further, the binarized anisotropic differential feature is masked with the region of interest obtained in the S110 operationMask ROI Intersection is taken as follows:
Figure SMS_8
wherein,F ref reference is made to features within the region of interest mask in the image, and n is the intersection operation. Therefore, only the reference features in the mask of the region of interest are stored in the feature library, the most favorable characteristics for image matching are reserved in some obvious and unique regions, the rest redundant and confusing parts are all 0, and related information is removed from the feature library.
In operation S130, feature compression is performed on the reference features to obtain a lightweight reference feature library.
In an embodiment of the invention, the lightweight reference feature library includes compressed reference features and geographic ranges of the compressed reference features. And carrying out travel coding on the reference features, and replacing the reference features with the same intensity value by using the string length to obtain a lightweight reference feature library.
Illustratively, reference features within a binarized region of interest maskF ref Using run-length codingThe method further compresses its size as follows:
Figure SMS_9
wherein,F ref-lib for the final library of lightweight reference features,countto count the number of identical pixels, i.e. to count the number of identical pixels until the value of the next pixel has changed,
Figure SMS_10
representing statistics starting from the top left pixel, up to the bottom right corner of the reference image,mnrespectively isxyIncrement of each statistic. After the pixel arrangement sequence is defined, the continuous identical pixels are replaced by the numerical value and the length of the continuous pixel string, so that lossless compression is realized. The larger the image blocks having the same color in the image, the smaller the number of image blocks, and the higher the compression ratio. For the reference feature of the region of interest after intersectionF ref Because the features remained by the mask only occupy a small part of the image and the rest parts are all 0, the algorithm can greatly save the space occupied by the information of mask elimination and ensure a lightweight reference feature libraryF ref-lib The method comprises the steps of including the most valuable features, removing redundant information and obtaining a higher compression ratio. The operations are all completed in the ground processing system, and after the lightweight reference feature library is constructed, the lightweight reference feature library is stored in the on-board edge computing equipment for subsequent on-board processing.
In operation S140, compressed reference features within a geographic range characterized by the geographic information are searched in a lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and are used as reference features to be matched.
In the embodiment of the present invention, the operation S140 may specifically include: and calculating the longitude and latitude of four corners of the image to be corrected. And carrying out overlapping region inquiry on the region represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and searching out the compressed reference feature with the common region as the reference feature to be matched.
Illustratively, the subsequent operations from operation S140 are all on-board processing operations. Firstly, calculating longitude and latitude of a four corner point of an image to be corrected generated by an on-board imaging algorithm, inquiring an overlapping region with a reference feature geographic range stored in a reference feature library, calculating the area of the overlapping region, and searching out reference features with common regions, wherein the method comprises the following steps:
Figure SMS_11
wherein,Areathe longitude and latitude of the four corners of the image to be corrected are respectively marked as max for the area of the overlapped areaLon1,minLon1,maxLat1,minLat1, respectively marking longitude and latitude of four corner points of reference characteristic as maxLon2,minLon2,maxLat2,minLat2. The longitude and latitude resolutions of the reference features are respectively usedresolutionLonA kind of electronic device with high-pressure air-conditioning systemresolutionLatAnd (3) representing. When the area of the overlapped area is larger than a given threshold value, then the reference feature library is used forF ref-lib Selecting the region reference featureF ref-lib-roi For subsequent processing.
In operation S150, the image to be corrected and the reference feature to be matched are subjected to block template matching, so as to obtain uniformly distributed matching points.
In an embodiment of the present invention, the operation S150 may specifically include: decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features. And uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks. K points with the largest Harris angular point response are extracted from each block to serve as points to be matched, and M multiplied by N multiplied by K points to be matched are obtained. And extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected. For M×N×K points to be matched, matching the decompressed reference features with anisotropic differential features by using a graphics processor (Graphic Processing Unit, GPU), wherein each matching combination is assigned a thread on the graphics processor for correlation metric computation. And determining the point to be matched corresponding to the maximum correlation metric as uniformly distributed matching points.
Illustratively, during the on-board processing phase, first, reference features are generated for the selected regionF ref-lib-roi Decompression processing is carried out, the binary characteristic is restored, and the restoration process is as follows:
Figure SMS_12
wherein, the method comprises the following steps ofx,y) For the decompressed pixel-by-pixel position,dand the position of the corresponding reference feature library after the current pixel is compressed. During decompression, the reference feature library is periodically read inF ref-lib-roi Decoding the encoded result into decompressed reference featuresF ref-roi
Then, uniformly partitioning the overlapping area of the image to be corrected and the reference feature into M multiplied by N blocks, extracting K points with the largest Harris angular point response from each block as points to be matched, and extracting anisotropic differential features of the image to be corrected in the common area by using a multidirectional anisotropic Gaussian differential operatorF AGD-t . For M×N×K points to be matched, anisotropic differential characteristic of image to be corrected is utilizedF AGD-t And decompression of the reference feature libraryF ref-roi GPU template matching is performed, and a matching correlation metric C is calculated as follows:
Figure SMS_13
wherein, the method comprises the following steps ofx c ,y c ) For the image coordinates of the points to be matched,C(x c ,y c ,dx,dy) The points to be matched are%x c ,y c ) Corresponding correlation measurementx,y) Matching template windows for graphics processorDThe coordinates of each pixel point within the image,F AGD t- (x,y) Is pixel point #)x,y) A corresponding anisotropic differential characteristic is provided which,F ref-roi (x,y) Is pixel point #)x,y) Corresponding decompressed reference characteristics #dx, dy) Is coordinates [ ]x,y) Is set in the first stage of the process,μ 1μ 2 respectively isF AGD t- (x,y) AndF ref-roi (x,y) In a matching template windowDIs used as a mean value of the two,σ 1σ 2 respectively isF AGD t- (x,y) AndF ref-roi (x,y) In a matching template windowDStandard deviation of (3).
Because the initial positioning error of the micro-nano satellite is larger, the offset is neededdx,dyThe setting of a larger value range further creates a larger computational burden. To reduce the repetition of calculations, the manner of calculation of the correlation metric may be optimized as follows:
Figure SMS_14
wherein,lto match template windowDIs a size of (c) a. And further, the correlation metric calculation of M multiplied by N multiplied by K points to be matched is solved by utilizing the parallel calculation capability of the GPU. Variables that can be calculated in parallel include: the points to be matchedx c ,y c ) The method comprises the steps of carrying out a first treatment on the surface of the Different offset amountsdx,dy) The method comprises the steps of carrying out a first treatment on the surface of the Every pixel in the matching template window Dx,y). The matching combination is that every point to be matched isx c0 ,y c0 ) Fix a group of offsetdx 0 ,dy 0 ) For a certain pixel in the matched template window Dx 0 ,y 0 ) A correlation metric is calculated. A Thread (Thread) is assigned to each matching combination on the GPU for correlation metric computation.
All matching combinations are calculated in parallel after completionAfter the correlation metrics of (a), the calculation results in the matching template window D need to be summed, and the parallel addition operation can be accelerated by using the Koggle-Stone algorithm. Summing the one-dimensional arrays, wherein the conventional cyclic sum time complexity is O (n), and the time complexity of the Koggle-Stone algorithm is O (log). For the two-dimensional summation problem of matching template windows, the time complexity of the cyclic summation is O (n 2 ). The pixel sums in the matching window can be obtained by summing each row in the horizontal direction and summing the row sum results in the vertical direction by using a Koggle-Stone parallel algorithm, and the time complexity is 2O (log n).
Finally, for each point to be matchedx c ,y c ) The correlation measurement obtained by calculating all possible offset amounts of the two points to be matched can be obtained by selecting the largest correlation measurement as a matching resultx c ,y c ) Is matched with the corresponding point coordinatesx m ,y m ) The following are provided:
Figure SMS_15
in operation S160, geometric fine correction is performed on the image to be corrected according to the uniformly distributed matching points.
In the embodiment of the present invention, the operation S160 may specifically include: and (3) using the uniformly distributed matching points to select the geographic coordinates of the corresponding points in the lightweight reference feature library as control points, and correcting the image positioning parameters of the micro-nano satellite to finish geometric fine correction.
Illustratively, uniformly distributed matching point pairs obtained by partitioned template matchingx c ,y c ; x m ,y m ) And (5) eliminating the mismatching points in the matching point pairs through a random sampling consensus algorithm (RANSAC). And selecting the geographic coordinates of the points of the reference feature library from the rest correct matching point pairs as control points, and correcting the positioning parameters of the micro-nano satellite images to finish the geometric fine correction processing.
In summary, the on-orbit geometric precise correction method for the micro-nano satellite image provided by the embodiment of the invention can rapidly and real-timely improve the positioning precision of the micro-nano satellite image and provide precise target position information for on-orbit target detection and identification. The lightweight standard feature library can effectively compress control reference images for geometric fine correction so as to adapt to a limited storage space for on-board processing, is adapted to a GPU block template matching algorithm of on-board edge computing equipment, can solve the problems of limited on-board computing resources and limitation of heat dissipation operation time, is relatively robust to abnormal, noise and initial offset, has low requirements on storage space, hardware resources, downloading bandwidth and ground base station maintenance cost of satellites, and is suitable for engineering projects. Therefore, the embodiment uses the lightweight feature library to perform template matching, simultaneously compresses the storage space occupied by the feature library, realizes the algorithm on the embedded GPU, improves the operation efficiency, and is suitable for the precise correction of the in-orbit satellite images.
Based on the same inventive concept, the embodiment of the invention also provides an on-orbit geometric fine correction device for the micro-nano satellite image.
Fig. 2 schematically shows a block diagram of an in-orbit geometry correction device for micro-nano satellite images according to an embodiment of the invention.
As shown in fig. 2, the on-orbit geometric correction apparatus 200 for micro-nano satellite image comprises: the system comprises a determining module 210, an extracting module 220, a compressing module 230, a retrieving module 240, a matching module 250 and a correcting module 260.
A determining module 210, configured to determine a region of interest mask based on an open source geographic vector file of a target region of interest of the micro-nano satellite.
The extracting module 220 is configured to extract a reference feature in the region of interest mask in the reference image of the target region.
The compression module 230 is configured to perform feature compression on the reference feature to obtain a lightweight reference feature library, where the lightweight reference feature library includes the compressed reference feature and a geographic range of the compressed reference feature.
The retrieving module 240 is configured to retrieve, from a lightweight reference feature library, compressed reference features within a geographic range represented by the geographic information according to geographic information of an image to be corrected generated by on-board imaging, as reference features to be matched.
And the matching module 250 is used for carrying out block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points.
The correction module 260 is configured to perform geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
It should be noted that the specific implementation details and the technical effects of the embodiment part of the apparatus do not correspond to those of the embodiment of the method, and are not repeated herein.
While the foregoing is directed to embodiments of the present invention, other and further details of the invention may be had by the present invention, it should be understood that the foregoing description is merely illustrative of the present invention and that no limitations are intended to the scope of the invention, except insofar as modifications, equivalents, improvements or modifications are within the spirit and principles of the invention.

Claims (10)

1. An on-orbit geometric precise correction method for micro-nano satellite images is characterized by comprising the following steps:
determining a region-of-interest mask based on an open source geographic vector file of the target region of interest of the micro-nano satellite;
extracting a reference feature in the region of interest mask in the reference image of the target region;
performing feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features;
searching compressed reference features in a geographic range represented by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as reference features to be matched;
performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points;
and carrying out geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
2. The method for accurately correcting the on-orbit geometry of a micro-nano satellite image according to claim 1, wherein the determining the region of interest mask based on the open source geographical vector file of the target region of interest of the micro-nano satellite comprises:
converting the open source geographic vector file into a geographic coding image;
resampling the geocode image to obtain a geovector image;
acquiring an interested region in the geographic vector image;
and performing morphological expansion operation on the region of interest to obtain the mask of the region of interest.
3. The method of claim 1, wherein the extracting the fiducial features in the region of interest mask in the reference image of the target region comprises:
calculating the multi-directional image intensity change characteristics of the reference image through an anisotropic Gaussian differential operator, and determining the most obvious change characteristics in all directions as anisotropic differential characteristics;
performing threshold processing on the anisotropic differential feature based on an intensity threshold to obtain a binarized anisotropic differential feature;
and performing intersection operation on the binarized anisotropic differential feature and the region-of-interest mask to obtain the reference feature.
4. The method for accurately correcting the on-orbit geometry of a micro-nano satellite image according to claim 3, wherein the anisotropic gaussian differential operator is as follows:
Figure QLYQS_1
wherein,xyrespectively the abscissa and the ordinate of the pixel points in the reference image,σρθrespectively the scale parameter, the anisotropy parameter and the rotation angle,G σ ρ θ,, (x,y) Is the coordinates of%x,y) An anisotropic gaussian kernel corresponding to the pixel points of (a),x θ is the coordinatesxRotatingθThe coordinates after the angle are used for the measurement,MAGD(x,y,θ) Is the coordinates of%x,y) Is at the pixel point of (2)θAn anisotropic gaussian derivative operator corresponding to the angular direction,Ithe intensity value of the reference image is convolution operation.
5. The method for accurately correcting the on-orbit geometry of the micro-nano satellite image according to claim 1, wherein the step of performing feature compression on the reference features to obtain a lightweight reference feature library comprises the steps of:
and performing travel coding on the reference features, and replacing the reference features with the same intensity value by using the string length to obtain the lightweight reference feature library.
6. The method for accurately correcting the on-orbit geometry of the micro-nano satellite image according to claim 1, wherein the searching the compressed reference feature in the geographic range characterized by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, as the reference feature to be matched, comprises:
calculating longitude and latitude of four corner points of the image to be corrected;
and carrying out overlapping region inquiry on the region represented by the longitude and latitude of the four corners and the geographic range of the compressed reference feature stored in the lightweight reference feature library, and searching out the compressed reference feature with the common region as the reference feature to be matched.
7. The method for accurately correcting the on-orbit geometry of a micro-nano satellite image according to claim 1, wherein the step of performing block template matching on the image to be corrected and the reference feature to be matched to obtain uniformly distributed matching points comprises the steps of:
decompressing the reference features to be matched based on the stroke codes to obtain decompressed reference features;
uniformly partitioning the region overlapping with the geographic range of the reference feature to be matched in the image to be corrected into M multiplied by N blocks;
extracting K points with the maximum Harris angular point response from each block as points to be matched, and obtaining M multiplied by N multiplied by K points to be matched;
extracting anisotropic differential characteristics of a region overlapping with the geographic range of the reference characteristic to be matched in the image to be corrected;
for M multiplied by N multiplied by K points to be matched, matching the decompressed reference features with anisotropic differential features by using a graphics processor, wherein each matching combination is distributed with a thread on the graphics processor to perform correlation metric calculation;
and determining the point to be matched corresponding to the maximum correlation metric as uniformly distributed matching points.
8. The method for accurately correcting the on-orbit geometry of a micro-nano satellite image according to claim 7, wherein the correlation metric is calculated by:
Figure QLYQS_2
wherein, the method comprises the following steps ofx c ,y c ) As the coordinates of the point to be matched,C(x c ,y c ,dx,dy) The points to be matched are%x c ,y c ) Corresponding correlation measurementx,y) Matching template windows for graphics processorDThe coordinates of each pixel point within the image,F AGD t- (x,y) Is pixel point #)x,y) A corresponding anisotropic differential characteristic is provided which,F ref-roi (x,y) Is pixel point #)x,y) Corresponding decompressed reference characteristics #dx,dy) Is coordinates [ ]x,y) Is set in the first stage of the process,μ 1μ 2 respectively isF AGD t- (x,y) AndF ref-roi (x,y) In a matching template windowDIs used as a mean value of the two,σ 1σ 2 respectively isF AGD t- (x,y) AndF ref-roi (x,y) In a matching template windowDIs used for the standard deviation of the above-mentioned components,lto match template windowDIs a size of (c) a.
9. The method for performing geometric fine correction on an in-orbit micro-nano satellite image according to claim 1, wherein the performing geometric fine correction on the image to be corrected according to the uniformly distributed matching points comprises:
and using the uniformly distributed matching points to select geographic coordinates of corresponding points in the lightweight reference feature library as control points, and correcting the image positioning parameters of the micro-nano satellite to finish geometric fine correction.
10. An on-orbit geometry correction device for micro-nano satellite images, which is characterized by comprising:
the determining module is used for determining a mask of the region of interest based on the open source geographic vector file of the target region of interest of the micro-nano satellite;
the extraction module is used for extracting the reference characteristics in the region of interest mask in the reference image of the target region;
the compression module is used for carrying out feature compression on the reference features to obtain a lightweight reference feature library, wherein the lightweight reference feature library comprises compressed reference features and geographic ranges of the compressed reference features;
the retrieval module is used for retrieving compressed reference features in a geographic range represented by the geographic information in the lightweight reference feature library according to the geographic information of the image to be corrected generated by on-board imaging, and taking the compressed reference features as the reference features to be matched;
the matching module is used for carrying out block template matching on the image to be corrected and the reference characteristic to be matched to obtain uniformly distributed matching points;
and the correction module is used for performing geometric fine correction on the image to be corrected according to the uniformly distributed matching points.
CN202310307607.3A 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device Active CN116071539B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310307607.3A CN116071539B (en) 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310307607.3A CN116071539B (en) 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device

Publications (2)

Publication Number Publication Date
CN116071539A true CN116071539A (en) 2023-05-05
CN116071539B CN116071539B (en) 2023-07-11

Family

ID=86175288

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310307607.3A Active CN116071539B (en) 2023-03-28 2023-03-28 Micro-nano satellite image on-orbit geometric precise correction method and device

Country Status (1)

Country Link
CN (1) CN116071539B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116958833A (en) * 2023-09-20 2023-10-27 青岛浩海网络科技股份有限公司 GF4 data geometric fine correction method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103383773A (en) * 2013-03-26 2013-11-06 中国科学院遥感与数字地球研究所 Automatic ortho-rectification frame and method for dynamically extracting remote sensing satellite image of image control points
CN110659369A (en) * 2019-03-22 2020-01-07 西安电子科技大学 On-orbit high-precision lightweight global image control point database construction method and system
US10838990B1 (en) * 2013-09-26 2020-11-17 EMC IP Holding Company LLC System and method for improving data compression of a storage system using coarse and fine grained similarity
CN114972555A (en) * 2022-05-25 2022-08-30 中国科学院空天信息创新研究院 Method, device, equipment and storage medium for compressing and correcting lightweight base map on satellite
CN115082359A (en) * 2022-07-27 2022-09-20 中国科学院空天信息创新研究院 Synchronous orbit optical satellite geometric fine correction method based on coastline data
CN115330619A (en) * 2022-08-11 2022-11-11 长光卫星技术股份有限公司 Local geometric fine correction method suitable for high-resolution remote sensing image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103383773A (en) * 2013-03-26 2013-11-06 中国科学院遥感与数字地球研究所 Automatic ortho-rectification frame and method for dynamically extracting remote sensing satellite image of image control points
US10838990B1 (en) * 2013-09-26 2020-11-17 EMC IP Holding Company LLC System and method for improving data compression of a storage system using coarse and fine grained similarity
CN110659369A (en) * 2019-03-22 2020-01-07 西安电子科技大学 On-orbit high-precision lightweight global image control point database construction method and system
CN114972555A (en) * 2022-05-25 2022-08-30 中国科学院空天信息创新研究院 Method, device, equipment and storage medium for compressing and correcting lightweight base map on satellite
CN115082359A (en) * 2022-07-27 2022-09-20 中国科学院空天信息创新研究院 Synchronous orbit optical satellite geometric fine correction method based on coastline data
CN115330619A (en) * 2022-08-11 2022-11-11 长光卫星技术股份有限公司 Local geometric fine correction method suitable for high-resolution remote sensing image

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116958833A (en) * 2023-09-20 2023-10-27 青岛浩海网络科技股份有限公司 GF4 data geometric fine correction method
CN116958833B (en) * 2023-09-20 2023-12-08 青岛浩海网络科技股份有限公司 GF4 data geometric fine correction method

Also Published As

Publication number Publication date
CN116071539B (en) 2023-07-11

Similar Documents

Publication Publication Date Title
US7733961B2 (en) Remote sensing imagery accuracy analysis method and apparatus
CN116071539B (en) Micro-nano satellite image on-orbit geometric precise correction method and device
CN108230375B (en) Registration method of visible light image and SAR image based on structural similarity rapid robustness
CN109100719B (en) Terrain map joint mapping method based on satellite-borne SAR (synthetic aperture radar) image and optical image
EP3358530A1 (en) Compressed edge map representation for image aided navigation
EP3543910B1 (en) Cloud detection in aerial imagery
CN112419380B (en) Cloud mask-based high-precision registration method for stationary orbit satellite sequence images
CN106887016B (en) Automatic relative registration method for GF-4 satellite sequence images
Jiang et al. Hierarchical motion consistency constraint for efficient geometrical verification in UAV stereo image matching
CN116310883B (en) Agricultural disaster prediction method based on remote sensing image space-time fusion and related equipment
CN115797381B (en) Heterogeneous remote sensing image registration method based on geographic segmentation and hierarchical feature matching
CN107797110A (en) The system imaging method and aperture synthesis projection radiation meter of aperture synthesis projection radiation
CN114581498B (en) Iterative model registration system combining vector data and raster image
CN111581407B (en) Method, device and medium for constructing global geographic positioning reference image database
CN110826407A (en) Stereo matching method for high-resolution satellite generalized image pairs
CN115019198B (en) Iterative updating and charting method for high-frequency remote sensing observation image
Luo et al. Comparative Analysis of Advanced Feature Matching Algorithms in Challenging High Spatial Resolution Optical Satellite Stereo Scenarios
Strunz et al. Automation of tie-pointing procedure for the geocoding of satellite images
CN114863145B (en) Sparse light connection point matching method for luminous images
CN115035529B (en) VIN equal proportion reduction and character recognition method based on image
Ye et al. Improving Co-Registration for Sentinel-1 SAR and Sentinel-2 Optical Images. Remote Sens. 2021, 13, 928
Zhang et al. An improved approach for DSM generation from high‐resolution satellite imagery
Arif et al. Most favorable automatic georeferencing based on GCPs selection using least square method
WU et al. Improved panchromatic sharpening algorithm based on sparse representation
Wu et al. Registration of multisource satellite images by thin-plate splines with highly reliable conjugate points

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant