CN117422746A

CN117422746A - Partition nonlinear geographic registration method, device, equipment and storage medium

Info

Publication number: CN117422746A
Application number: CN202311385676.2A
Authority: CN
Inventors: 雷存款; 张红艳; 冷伟; 陈淑敏; 王艳杰; 聂磊; 徐轩; 李文强; 彭国樟; 符姗
Original assignee: Wuhan Jiahe Technology Co ltd
Current assignee: Wuhan Jiahe Technology Co ltd
Priority date: 2023-10-23
Filing date: 2023-10-23
Publication date: 2024-01-19
Anticipated expiration: 2043-10-23
Also published as: CN117422746B

Abstract

The invention relates to the field of remote sensing and discloses a partitioned nonlinear geographic registration method, a device, equipment and a storage medium, wherein the method can obtain high-dimensional features in an image by extracting partitioned features of a target reference image and a target image to be registered, reduce the subsequent operation amount of a model and improve the geographic registration efficiency; and the images to be registered are registered according to the homonymous point pairs between the reference image feature sets of the reference image feature sets by the nonlinear geometric transformation model, so that the problems of weak homonymous point matching capability, uneven spatial distribution, incapability of effectively fitting local heterogeneous offset, low operation efficiency and the like in the traditional geographic registration method are solved, and the precision and the efficiency of geographic registration are improved.

Description

Partition nonlinear geographic registration method, device, equipment and storage medium

Technical Field

The present invention relates to the field of remote sensing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for regional nonlinear geographic registration.

Background

The remote sensing image is widely applied to the fields of environment monitoring, natural resource management, city planning, agriculture, grain safety and the like as an efficient geospatial data carrier. The application fields generally relate to tasks such as multisource remote sensing image analysis, multi-temporal remote sensing image analysis, remote sensing image data fusion and mosaic, and the like, and the tasks have very strict requirements on the consistency of geographic space positions. However, factors such as the complex and lengthy atmosphere transmission environment, the orbit shooting angle of the heterologous satellite, the imaging time, the weather, the topography, etc. aggravate the current situation that the sites of the same name between the images are not consistent in the geographic space. The problem of inaccurate geographic position of domestic satellite images generally exists, and the problem of inaccurate geographic position of domestic satellite images brings great challenges to common remote sensing tasks such as multi-source image analysis and multi-time phase analysis. Therefore, researching a geographic registration method with high registration accuracy, high operation efficiency and strong generalization is the basis and key for solving the problems.

However, the conventional geographic registration method generally has the problems of few control points, uneven distribution, incapability of solving local heterogeneous offset, low operation calculation efficiency and the like, which brings great challenges to common remote sensing tasks such as multi-source image analysis, multi-time phase analysis and the like.

The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.

Disclosure of Invention

The invention mainly aims to provide a partition nonlinear geographic registration method, a device, equipment and a storage medium, and aims to solve the technical problems that the traditional geographic registration method in the prior art is usually few in control points, uneven in distribution, incapable of solving local heterogeneous offset and low in operation calculation efficiency.

To achieve the above object, the present invention provides a partitioned nonlinear geographical registration method, the method comprising the steps of:

acquiring a reference geographic image and a geographic image to be registered, and respectively preprocessing the reference geographic image and the geographic image to be registered to acquire a target reference image and a target image to be registered;

extracting image features of the target reference image through a preset LoFTR model partition to obtain a reference image feature set, and extracting image features of the target image to be registered through the preset LoFTR model partition to obtain an image feature set to be registered;

Matching according to the reference image feature set and the image features in the image feature set to be registered, and determining homonymy point pairs according to the successfully matched image features;

and constructing a nonlinear geometric transformation model based on the homonymous point pairs, and registering the target image to be registered through the nonlinear geometric transformation model.

Optionally, the homonymy point pair includes a reference homonymy point located in the target reference image and a homonymy point to be registered located in the target image to be registered corresponding to the reference homonymy point;

after the step of matching the image features in the reference image feature set and the image features in the image feature set to be registered and determining the homonymous point pair according to the successfully matched image features, the method further comprises the steps of:

filtering based on the space effectiveness of each homonymous point in the target reference image and the space effectiveness of each homonymous point in the target image to be registered to obtain an effective homonymous point pair containing the effective homonymous point to be registered;

constructing an offset field between the target reference image and the target image to be registered according to the effective homonymy point pairs;

constructing a KNN graph structure based on the position coordinates of the effective homonymous points to be registered in the target image to be registered;

Determining a filtering reference item of the effective homonymous points to be registered according to the offset field and the KNN graph structure;

filtering the filtering reference item of the effective homonymous points to be registered according to a preset constraint condition to obtain target homonymous points to be registered;

determining a target reference homonymy point and a target homonymy point pair corresponding to the target homonymy point to be registered;

correspondingly, the step of constructing a nonlinear geometric transformation model based on the homonymous point pairs and registering the target image to be registered through the nonlinear geometric transformation model comprises the following steps:

and constructing a nonlinear geometric transformation model based on the target homonymous point pairs, and registering the target images to be registered through the nonlinear geometric transformation model.

Optionally, the filtering reference item includes: offset distance, offset angle, offset distance range, offset angle range;

the step of determining the filtering reference item of the valid homonymous point to be registered according to the offset field and the KNN graph structure comprises the following steps:

determining the offset distance and the offset angle of the effective homonymous points to be registered based on the offset field;

determining the offset distance range of the effective homonymous point to be registered according to the offset distance of the effective homonymous point to be registered in the KNN graph structure, and determining the offset angle range of the effective homonymous point to be registered according to the offset angle of the effective homonymous point to be registered in the KNN graph structure.

Optionally, the preset LoFTR model includes: a feature extraction sub-module;

the step of extracting the image features of the target reference image through a preset LoFTR model partition to obtain a reference image feature set, and extracting the image features of the target image to be registered through the preset LoFTR model partition to obtain the image feature set to be registered includes:

partitioning the target reference image according to a preset size to obtain a plurality of reference blocks;

partitioning the target image to be registered according to a preset size to obtain a plurality of blocks to be registered;

and extracting the features in each reference image block and each image block to be registered through the feature extraction sub-module to obtain a reference image feature set and an image feature set to be registered.

Optionally, the preset LoFTR model further includes: the device comprises a feature matching sub-module, a local feature description sub-module and an iterative fine tuning sub-module;

the step of matching the image features in the reference image feature set and the image feature set to be registered and determining the homonymy point pair according to the successfully matched image features comprises the following steps:

matching the reference image feature set and the image feature set to be registered through the feature matching sub-module, and determining a rough homonymy point pair according to the successfully matched image features, wherein the rough homonymy point pair is used for representing the reference image block and the image block to be registered, wherein the image features of the reference image block and the image block to be registered are successfully matched;

Determining the reference image block and the image block to be registered related to the rough homonymy point;

determining a local reference feature descriptor of the reference image block and a local feature descriptor to be registered of the image block to be registered through the local feature description submodule;

and performing iterative optimization based on the local reference feature descriptors and the local features to be registered through the iterative fine adjustment, and determining homonymous point pairs according to an iterative optimization result.

Optionally, the step of preprocessing the reference geographic image and the geographic image to be registered to obtain a target reference image and a target image to be registered includes:

performing image type level processing on the reference image to obtain an intermediate reference image, and performing image type level processing on the geographic image to be registered to obtain an intermediate image to be registered, wherein the image type level processing comprises: radiometric calibration, atmospheric correction, geometric correction and data fusion;

performing spatial reference unification processing on the intermediate base image and the intermediate image to be registered to obtain a spatial reference base image and a spatial reference image to be registered of the same spatial reference system;

And performing overlap region clipping on the spatial reference image to be registered based on the spatial reference image to obtain a target reference image and a target image to be registered.

Optionally, the step of constructing a nonlinear geometric transformation model based on the homonymous point pair and registering the target image to be registered through the nonlinear geometric transformation model includes:

constructing a geographic control point structure array based on the target homonymy point pairs;

constructing a nonlinear geometric transformation model according to the geographic control point structure array;

and registering the target image to be registered according to the nonlinear geometric transformation model.

In addition, to achieve the above object, the present invention further proposes a zoned nonlinear geographical registration apparatus, including:

the image processing module is used for acquiring a reference geographic image and a geographic image to be registered, and respectively preprocessing the reference geographic image and the geographic image to be registered to acquire a target reference image and a target image to be registered;

the feature extraction module is used for extracting the image features of the target reference image through a preset LoFTR model partition to obtain a reference image feature set, and extracting the image features of the target image to be registered through the preset LoFTR model partition to obtain an image feature set to be registered;

The homonymy point matching module is used for matching according to the reference image feature set and the image features in the image feature set to be registered, and determining homonymy point pairs according to the successfully matched image features;

and the geographic registration module is used for constructing a nonlinear geometric transformation model based on the homonymous point pairs and registering the target image to be registered through the nonlinear geometric transformation model.

In addition, to achieve the above object, the present invention also proposes a zoned nonlinear geographical registration apparatus, the apparatus comprising: a memory, a processor, and a partitioned nonlinear geographical registration program stored on the memory and executable on the processor, the partitioned nonlinear geographical registration program being registered to implement the steps of the partitioned nonlinear geographical registration method as described above.

In addition, to achieve the above object, the present invention also proposes a storage medium having stored thereon a zoned nonlinear geographical registration program which, when executed by a processor, implements the steps of the zoned nonlinear geographical registration method as described above.

The method comprises the steps of obtaining a reference geographic image and a geographic image to be registered, and preprocessing the reference geographic image and the geographic image to be registered respectively to obtain a target reference image and a target image to be registered; extracting image features of a target reference image through a preset LoFTR model partition to obtain a reference image feature set, and extracting image features of a target image to be registered through the preset LoFTR model partition to obtain the image feature set to be registered; matching is carried out according to the image features in the reference image feature set and the image features in the image feature set to be registered, and the homonymy point pairs are determined according to the successfully matched image features; and constructing a nonlinear geometric transformation model based on the homonymous point pairs, and registering the target images to be registered through the nonlinear geometric transformation model. The high-dimensional features in the images can be obtained by carrying out regional feature extraction on the target reference image and the target image to be registered, so that the subsequent operation quantity of the model is reduced, and the geographic registration efficiency is improved; and the images to be registered are registered according to the homonymous point pairs between the reference image feature sets of the reference image feature sets by the nonlinear geometric transformation model, so that the problems of weak homonymous point matching capability, uneven spatial distribution, incapability of effectively fitting local heterogeneous offset, low operation efficiency and the like in the traditional geographic registration method are solved, and the precision and the efficiency of geographic registration are improved.

Drawings

FIG. 1 is a schematic diagram of a partitioned nonlinear geographic registration device of a hardware operating environment in accordance with an embodiment of the present invention;

FIG. 2 is a flow chart of a first embodiment of a zoned nonlinear geographic registration method of the present invention;

FIG. 3 is a flowchart illustrating the geographic image preprocessing according to the present invention;

FIG. 4 is a flow chart of a second embodiment of a zoned nonlinear geographic registration method of the present invention;

FIG. 5 is a schematic diagram of a filtering process of valid homonymous points to be registered according to the present invention;

FIG. 6 is a schematic diagram of a filtering scenario of valid homonymous points to be registered according to the present invention;

FIG. 7 is a flow chart of a third embodiment of a zoned nonlinear geographic registration method of the present invention;

FIG. 8 is a schematic view of a scenario in which the zoning nonlinear geographic registration method of the present invention performs zoning;

FIG. 9 is a schematic illustration of a registration scenario of the zoned nonlinear geographic registration method of the present invention;

fig. 10 is a block diagram of a first embodiment of a zoned nonlinear geographic registration apparatus of the present invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a partition nonlinear geographic registration device of a hardware running environment according to an embodiment of the present invention.

As shown in fig. 1, the zoned nonlinear geographic registration apparatus may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.

Those skilled in the art will appreciate that the structure shown in fig. 1 does not constitute a limitation of a zoned non-linear geographic registration device, and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.

As shown in fig. 1, an operating system, a network communication module, a user interface module, and a zoned nonlinear geographical registration program may be included in memory 1005, which is a storage medium.

In the zoned nonlinear geographic registration apparatus shown in fig. 1, the network interface 1004 is primarily used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the partition nonlinear geographic registration device of the present invention may be disposed in the partition nonlinear geographic registration device, where the partition nonlinear geographic registration device invokes a partition nonlinear geographic registration program stored in the memory 1005 through the processor 1001, and executes the partition nonlinear geographic registration method provided by the embodiment of the present invention.

An embodiment of the present invention provides a partition nonlinear geographical registration method, and referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the partition nonlinear geographical registration method of the present invention.

It should be noted that, at present, the geographic registration model is mainly divided into the following two types in the large direction: a registration mode based on region gray analysis and a registration mode based on characteristic points. The method can analyze the image in a finer granularity, but also depends on the constructed measurement strategy seriously, has low operation efficiency and is not suitable for remote sensing images with larger scale; the latter generally needs to go through steps of extracting feature points, matching homonymous points (Tie-points), constructing a space geometric transformation model and the like, and the method effectively reduces the complexity of calculation by using the homonymous points, improves the efficiency obviously, but has the phenomena of error accumulation and low matching precision.

The traditional characteristic point registration-based flow mainly comprises the following steps: and (3) matching homonymous points, filtering homonymous points and constructing a geometric transformation model based on the homonymous points. The most classical method in the homonymous point matching process is a method of over-Scale Invariant Feature Transform (SIFT), the method is characterized in that key points in an image are detected, local features are extracted in different scales and directions, the method has good scale invariance and rotation invariance, and high accuracy and robustness, can adapt to image matching tasks of different illumination conditions and visual angle changes, but the method is insufficient in calculation efficiency, in order to optimize the problem, an acceleration robust feature (SURF) method is used for quickly constructing a feature descriptor of a Hessian matrix, so that the operation efficiency of the algorithm is greatly improved, in addition, a scholars propose Oriented FAST and Rotated BRIEF (ORB), the method is an algorithm combining a FAST feature detector and a BRIEF feature descriptor, balance is achieved between speed and performance, the method has high operation speed and good description capacity, the matching method is mainly based on shallow features of images, the homonymous point detection capacity of the images is weak, and uniform control of the whole images cannot be achieved generally; in addition, after the homonym points are obtained, filtering operation is usually needed to prevent geometric distortion caused by mismatching, common filtering methods mainly include RANdom SAmple Consensus (RANSAC, PROSAC) and PROgressive SAmple Consensus (PROSAC), and although the methods can filter the homonym points of the outlier to a certain extent, the homonym points under the heterogeneous offset scene of the region can be removed, so that the methods fail to the scene; finally, a geometric transformation model is built based on the same name points of the filtering treatment, polynomial transformation is favored in the process due to the factors of high running speed, stable characteristics after geometric transformation and the like, but the model still cannot solve the problem of heterogeneous offset.

The present application is a typical feature point registration-based method. Unlike the conventional methods described above, the following are: firstly, a LoFTR deep learning model is newly introduced, and the model is combined with a transducer assembly, so that Gao Weishen-layer semantics of an image can be extracted more abstract, the matching capability of the model is obviously improved compared with SIFT and SURF, and the image can be basically uniformly controlled; secondly, aiming at the problem of local heterogeneous offset, the scheme abandons RANSAC, PROSAC and other filtering modes, and newly introduces a filtering method of an offset field and a graph model, so that outliers can be removed, and homonymous points of a heterogeneous offset region can be reserved; in addition, the scheme is different from a common polynomial model, and the selected nonlinear thin plate spline function (TPS) can utilize homonymous point information to a greater extent and can better fit the phenomenon of local heterogeneous offset; finally, in order to improve the processing efficiency of the model on the large-scale remote sensing image, a strategy of 'downscaling calculation-transformation reduction' is newly provided, and experimental results show that the method can not only remarkably improve the efficiency of geographic registration, but also almost has no loss of registration precision.

In this embodiment, the method for regional nonlinear geographic registration includes the following steps:

Step S10: and acquiring a reference geographic image and a geographic image to be registered, and respectively preprocessing the reference geographic image and the geographic image to be registered to obtain a target reference image and a target image to be registered.

It should be noted that, the execution body of the method of this embodiment may be a terminal device with functions of image partition, geographic registration and program running, for example, a computer, a server, etc., or may be an electronic device with the same or similar functions, for example, the above-mentioned partition nonlinear geographic registration device. This embodiment and the following embodiments will be described below by taking a zoned nonlinear geographical registration apparatus (hereinafter referred to as registration apparatus) as an example.

It should be noted that, at present, the geographical position of the satellite image generally has a problem of inaccurate deviation, which brings great challenges to common remote sensing tasks such as multi-source image analysis and multi-time phase analysis. Conventional geographic registration methods typically exist: the control points are few, the distribution is uneven, the problems of local heterogeneous offset, low operation calculation efficiency and the like cannot be solved. Therefore, the technical scheme aims to solve the problems of the traditional registration method and develop a geographic registration model which is strong in generalization, efficient in operation and capable of being realized in an engineering way.

According to the scheme, a LoFTR deep learning model is newly introduced, the model is combined with a transducer assembly, gao Weishen layers of semantics of the image can be extracted more abstract, the matching capability of the model is remarkably improved compared with that of the traditional matching, and the image can be basically and uniformly controlled; secondly, aiming at the problem of local heterogeneous offset, the scheme abandons RANSAC, PROSAC and other filtering modes, and newly introduces a filtering method of an offset field and a graph model, so that outliers can be removed, and homonymous points of a heterogeneous offset region can be reserved; in addition, the scheme is different from a common polynomial model, and the selected nonlinear thin plate spline function (TPS) can utilize homonymous point information to a greater extent and can better fit the phenomenon of local heterogeneous offset; finally, in order to improve the processing efficiency of the model on the large-scale remote sensing image, a strategy of 'downscaling calculation-transformation reduction' is newly provided, and experimental results show that the method can not only remarkably improve the efficiency of geographic registration, but also almost has no loss of registration precision.

It is understood that registration refers to matching geographic coordinates of different images obtained by different imaging means in the same region, and may include three processes of geometric correction, projective transformation and the same scale. Geo-registration may be the process of registering and overlaying two or more images acquired at different times, different imaging devices, or under different conditions.

It should be appreciated that geographic registration requires at least two geographic images, namely a reference geographic image and a geographic image to be registered. The reference geographic image is a geographic image serving as a geographic registration reference standard, and the geographic image to be registered is the geographic image needing registration. In general, the reference geographic image has the characteristics of high precision, abundant information quantity, intuitionistic and accuracy and the like, and the geographic registration of the geographic image to be registered is performed by using the reference geographic image, so that the obtained target image has higher accuracy.

It should be noted that, in this embodiment, the image types of the reference geographic image and the geographic image to be registered may be aerial images or satellite images, which is not limited in this embodiment.

It should be explained that the preprocessing may be a process of spatial reference unification and overlap region cropping for the geographic image. And carrying out spatial reference unification processing on the base geographic image and the geographic image to be registered, so that the spatial base image and the spatial image to be registered under the same spatial reference system can be obtained. The overlapping region clipping is based on the overlapping region of the spatial reference image and the spatial image to be registered, so that the spatial image to be registered and an effective image region in the spatial reference image, namely the target image to be registered and the target reference image, can be obtained.

It should be noted that, the preprocessing process of the data mainly considers the problem of inconsistent processing levels of remote sensing image data issued by different satellite sources, and the same data processing level is a precondition of comprehensive analysis. Therefore, firstly, the standard radiometric calibration, the atmospheric correction, the geometric correction, the data fusion and other processing procedures are respectively carried out on the input reference geographic image and the geographic image to be registered, so that the images are ensured to be in the same processing level. In addition, the images at the same processing level are subjected to uniform spatial reference unification and overlap region clipping operation (registration processing is only performed on the overlap region) so as to avoid invalid calculation, and in order to further improve the operation efficiency of the model, the images at the overlap region are subjected to optional downsampling processing so as to improve the processing efficiency of large-scale images, and finally the processed remote sensing images are subjected to graying. The specific implementation flow is shown in fig. 3, and fig. 3 is a schematic flow chart of geographic image preprocessing according to the present invention.

Preprocessing the geographic image to be registered to obtain a target reference image and a target image to be registered, wherein the method comprises the following steps of:

step S11: performing image type level processing on the reference image to obtain an intermediate reference image, and performing image type level processing on the geographic image to be registered to obtain an intermediate image to be registered, wherein the image type level processing comprises: radiometric calibration, atmospheric correction, geometric correction, and data fusion.

It should be noted that, the image type level of the geographic image may include a first level and a second level, and if the image type level of the geographic image is the first level, the image type level processing needs to be performed on the geographic image to obtain an intermediate image of the second level.

It will be appreciated that radiation calibration is a technique that may be used to accurately measure and analyze a radiation source. Through radiometric calibration, the brightness gray scale values of the reference geographic image and the geographic image to be registered can be converted into absolute radiance.

It should be appreciated that atmospheric calibration is a process for inverting the true surface reflectivity of a surface feature to eliminate radiation errors caused by atmospheric effects during geographic imaging.

It can be understood that geometric correction refers to that in the process of remote sensing imaging, the geometric position, shape, size, dimension, azimuth and other features of the ground feature on the original image are often inconsistent with the corresponding features of the ground feature due to the comprehensive influence of multiple factors, and the inconsistency is geometric deformation, which is also called geometric distortion.

It should be understood that the data fusion is to fuse the processed geographic image data. And obtaining the intermediate image of the second level by performing image type level processing on the geographic image of the first level.

If the input reference image or the image to be registered is the geographic image of the second level, the geographic image of the second level is directly used as the intermediate image without processing.

In one embodiment, the geographic image data used in this embodiment is Landsat data. Landsat data can be divided into two large categories, collection1 and Collection 2. Wherein collection2 is divided into two levels, i.e., a first level and a second level, of L1 and L2.

Step S12: and carrying out spatial reference unification processing on the intermediate base image and the intermediate image to be registered to obtain a spatial reference base image and a spatial reference image to be registered of the same spatial reference system.

Step S13: and performing overlap region clipping on the spatial reference image to be registered based on the spatial reference image to obtain a target reference image and a target image to be registered.

It should be understood that the spatial reference unification process is performed on two different geographic images, that is, the spatial reference parameters such as the coordinate system, resolution, and datum plane of the two geographic images are converted to be identical or nearly identical. By performing spatial reference unification processing on the base geographic image and the geographic image to be registered, the base geographic image and the geographic image to be registered can be located in the same spatial reference system, so that geographic calibration can be performed more conveniently.

It can be understood that the overlapping area clipping processing is performed on the space reference image and the space reference image to be registered, so that invalid image areas in the image to be registered and the reference image can be eliminated, the resource occupation of geographic registration is optimized, and the geographic registration efficiency is improved.

In this embodiment, the input mechanism geographic image and the geographic image to be registered are both described as the first level.

As shown in fig. 3, firstly, performing radiometric calibration carding on a reference geographic image ref0.GIF to obtain ref1.GIF; performing radiation calibration treatment on the geographic image img0.GIF to be registered to obtain img1.GIF;

respectively carrying out atmospheric correction treatment on ref1.GIF and img1.GIF to obtain ref2.GIF and img2.GIF;

and performing geometric calibration and data fusion processing on the ref2.GIF and the img2.GIF respectively to obtain ref3.GIF and img3.GIF, namely an intermediate reference image and an intermediate image to be registered.

Then, spatial reference unification processing is carried out on the intermediate reference image ref3.GIF and the intermediate image img3.GIF to be registered, so that a spatial reference image ref4.GIF and a spatial reference image img4.GIF of the same spatial reference system are obtained;

and then, overlapping region clipping is carried out on the spatial reference image ref4.GIF and the spatial reference image img4.GIF to be registered, so as to obtain ref5.GIF and img5.GIF.

It should be noted that, for a general geographic image, ref5.GIF gray processing can be used as a target reference image, and img5.GIF gray processing can be used as a target image to be registered for subsequent geographic registration. For partial geographic images with higher processing efficiency, the ref5.GIF and img5.GIF can be subjected to downsampling processing to obtain ref6.GIF and img6.GIF; and carrying out graying treatment on the ref6.GIF and the img6.GIF obtained by downsampling, so as to obtain a grayed target reference image ref7.GIF and a target image img7.GIF to be registered.

It can be understood that the processing efficiency of the large-scale image can be improved by performing downsampling processing on the image in the overlapping area, and finally, the processed geographic image is grayed.

It should be understood that, for large-scale geographic images, the embodiment adopts a downscaling calculation-transformation reduction strategy, so that the efficiency of geographic registration can be remarkably improved, and the registration accuracy is hardly lost.

In specific implementation, the registration device acquires a reference geographic image and a geographic image to be registered, and performs spatial reference unification processing and overlapping region clipping processing on the reference geographic image and the geographic image to be registered to obtain a target reference image and a target image to be registered.

Step S20: extracting the image features of the target reference image through a preset LoFTR model partition to obtain a reference image feature set, and extracting the image features of the target image to be registered through the preset LoFTR model partition to obtain the image feature set to be registered.

Note that the LoFTR (Local Feature based Transformer) model is a deep learning model for image feature matching and positioning. The method combines the traditional local feature descriptor and the transducer architecture, and has excellent feature representation and matching performance.

It should be explained that the above-mentioned partition extraction is to divide the target image into a plurality of image partitions, and extract the image features in each image partition to obtain the image feature set. Specifically, the image features of the target reference image are extracted through a preset LoFTR model partition to obtain a reference image feature set, and the image features of the target image to be registered are extracted through the preset LoFTR model partition to obtain the image feature set to be registered.

It should be noted that, by separately extracting the target reference image and the target image to be registered, high-dimensional features in the image can be obtained, the subsequent operation amount of the model is reduced, and the geographic registration efficiency is improved.

It can be understood that the reference image feature set is an image feature set obtained by extracting features of each reference image partition, and the image feature set to be registered is an image feature set obtained by extracting features of each image partition to be registered.

In specific implementation, the registration device extracts image features of a target reference image through a preset LoFTR model partition to obtain a reference image feature set; image features of the target image to be registered are extracted through a preset LoFTR model partition, an image feature set to be registered is obtained, subsequent model processing is conducted through the reference image feature set and the image feature set to be registered, and the operation amount of the model is reduced.

Step S30: and matching the image features in the reference image feature set and the image features in the image feature set to be registered, and determining the homonymy point pair according to the successfully matched image features.

It should be noted that, the image feature matching is to match the image features in the reference image set and the image set to be registered, and the image feature pair with the matching degree exceeding the preset matching degree is determined as the homonymy point pair.

It should be explained that the same name points are imaging points located at the same position in different images. The confidence coefficient between each image feature pair in the reference image set and the image set to be registered can be obtained by matching the feature images in the reference image set and the image set to be registered, and the image features with the confidence coefficient exceeding the preset confidence coefficient are judged to be the image features of the imaging points at the same position, so that the same-name points in the reference image set and the image set to be registered, namely the reference same-name points and the same-name points to be registered, are determined, and the same-name point pairs of the reference image set and the image set to be registered are determined.

It should be noted that, the confidence between the image feature pairs, that is, the matching degree between the reference image feature in the reference image set and the image feature to be registered in the image set to be registered, the higher the matching degree is, the higher the similarity between the two features is, and when the similarity is higher than the preset confidence, the imaging points corresponding to the two image features can be judged to be imaging points at the same position.

It will be appreciated that the above-mentioned preset confidence level, that is, the confidence level set by the user or automatically generated by the system, may be determined according to the requirement, for example, 90%, 85%, etc., which is not limited in this embodiment.

It should be understood that there may be multiple image features that are successfully matched in the image set to be registered in the reference image set, that is, there may be multiple reference homonymy points and homonymy points to be registered corresponding to the reference homonymy points, and multiple homonymy point pairs are formed.

In a specific implementation, the registration device may establish a correspondence between image features that are successfully matched, and determine a reference homonymy point, a homonymy point to be registered, and a homonymy point pair according to the correspondence.

Step S40: and constructing a nonlinear geometric transformation model based on the homonymous point pairs, and registering the target image to be registered through the nonlinear geometric transformation model.

It should be noted that when the same-name point pair is obtained, a nonlinear geometric transformation model can be constructed according to the corresponding relationship between the reference same-name point and the same-name point to be registered, so that the target image to be registered is registered according to the nonlinear set transformation model.

It should be explained that the nonlinear geometric transformation model is a model generated according to the reference homonymous points and used for describing nonlinear transformation relations in images or spatial data of the homonymous points to be registered. Common nonlinear geometric transformation models may include: polynomial transformation MODEL, B-Spline transformation MODEL, thin-Plate Spline transformation MODEL (TPS MODEL), nonlinear affine transformation MODEL, and the like. The geographic registration of the geographic images can be realized by transforming the target images to be registered through a proper nonlinear transformation model.

In one implementation, in order to make use of the homonymous point information to a greater extent and better fit the phenomenon of local heterogeneous offset, in this embodiment, a TSP transformation model is used as a construction type of a nonlinear set transformation model, so as to obtain a geographic registration result of more fitting the homonymous point information.

According to the method, a reference geographic image and a geographic image to be registered are obtained, and the reference geographic image and the geographic image to be registered are preprocessed respectively to obtain a target reference image and a target image to be registered; extracting image features of a target reference image through a preset LoFTR model partition to obtain a reference image feature set, and extracting image features of a target image to be registered through the preset LoFTR model partition to obtain the image feature set to be registered; matching is carried out according to the image features in the reference image feature set and the image features in the image feature set to be registered, and the homonymy point pairs are determined according to the successfully matched image features; and constructing a nonlinear geometric transformation model based on the homonymous point pairs, and registering the target images to be registered through the nonlinear geometric transformation model. The high-dimensional features in the images can be obtained by carrying out regional feature extraction on the target reference image and the target image to be registered, so that the subsequent operation quantity of the model is reduced, and the geographic registration efficiency is improved; and the images to be registered are registered according to the homonymous point pairs between the reference image feature sets of the reference image feature sets by the nonlinear geometric transformation model, so that the problems of weak homonymous point matching capability, uneven spatial distribution, incapability of effectively fitting local heterogeneous offset, low operation efficiency and the like in the traditional geographic registration method are solved, and the precision and the efficiency of geographic registration are improved.

With reference to the first embodiment of the present partitioned nonlinear geographical registration method as described above, a second embodiment of the present partitioned nonlinear geographical registration method is presented.

Referring to fig. 4, fig. 4 is a flow chart illustrating a second embodiment of the zoned nonlinear geographic registration method of the present invention.

As shown in fig. 4, in order to further improve the accuracy of the geographic registration, after the step of matching the reference image feature set with the image features in the image feature set to be registered and determining the homonym point pair according to the image features that are successfully matched, the method further includes:

step S31: and filtering based on the space effectiveness of each homonymous point in the target reference image and the space effectiveness of each homonymous point in the target image to be registered to obtain an effective homonymous point pair containing the effective homonymous point to be registered.

It should be noted that, since the homonymy point pair is an imaging point pair formed by a reference homonymy point and a homonymy point to be registered corresponding to the reference homonymy point, the homonymy point pair includes a reference homonymy point located in the target reference image and a homonymy point to be registered corresponding to the reference homonymy point and located in the target image to be registered.

It can be understood that the same-name points in the target reference image, namely the reference same-name points, and the same-name points in the target image to be registered, namely the same-name points to be registered. The space effectiveness of the imaging points in the geographic image is that is, the effective information duty ratio in the geographic image. If the effective information ratio in the geographic image is too low, for example, the effective information ratio in the geographic image is lower than a preset value by 5% or 10%, the space effectiveness of the partial geographic image can be indicated to be poor. In the application, the space effectiveness of each datum homonymous point in the target datum image and the space effectiveness of the target homonymous point to be registered in the target image to be registered can be judged, and homonymous points with poor space effectiveness are removed, so that the effective homonymous point pair containing the effective homonymous point to be registered and the effective datum homonymous point is obtained.

It should be understood that the homonymy point is a different imaging point at the same position in the target reference image and the target to-be-registered image, and when the homonymy point at the same position in the target reference image is removed, the homonymy point at the same position in the corresponding target to-be-registered image will also be removed, and vice versa, which is not described in detail in this embodiment.

The imaging point of the geographical image with the low effective information ratio may be represented by an imaging point with a missing information, a solid-color imaging point (i.e., an imaging point without information), or the like, which is not limited in this embodiment.

In specific implementation, the registration equipment base filters homonymous points with low space effectiveness in the target reference image and the target image to be registered respectively to obtain effective homonymous point pairs.

Step S32: and constructing an offset field between the target reference image and the target image to be registered according to the effective homonymy point pairs.

It can be understood that the registration device can construct an offset field between the target reference image and the target image to be registered by using the longitude and latitude coordinates of the reference homonymous point in each homonymous point pair and the longitude and latitude coordinates of the homonymous point to be registered.

It should be noted that the offset field is a data structure that may be used to describe latitude and longitude coordinate offset information between corresponding homonymous points in the target reference image and the target image to be registered. The method can be used for representing the offset vector between each homonymous point to be registered in the target image to be registered and each reference homonymous point corresponding to the homonymous point to be registered in the target reference image.

It can be understood that by constructing the offset field, the offset distance and the offset angle of the reference homonymous point and the homonymous point to be registered in each homonymous point pair can be determined, and the KNN graph structure is constructed according to the offset distance and the offset angle. Specifically, step S33: and constructing a KNN graph structure based on the position coordinates of the effective homonymous points to be registered in the target image to be registered.

S34: and determining a filtering reference item of the effective homonymous points to be registered according to the offset field and the KNN graph structure.

It should be noted that, the KNN (K-Nearest Neighbors) graph structure is a graph structure constructed based on a K nearest neighbor algorithm, the KNN graph structure is constructed through the offset distances and the offset angles of the reference homonymous points and the homonymous points to be registered in each homonymous point pair, and the target homonymous points to be registered and the target reference homonymous points and the target homonymous point pairs corresponding to the target homonymous points to be registered can be obtained by filtering according to the KNN graph structure and the offset field. In this embodiment, a KNN graph structure is constructed with a K value of 5, so that pairs of homonymous points are represented and filtered.

As shown in fig. 5 and fig. 6, fig. 5 is a schematic diagram of a filtering flow of valid homonymous points to be registered in the present invention; fig. 6 is a schematic diagram of a filtering scenario of valid homonymous points to be registered according to the present invention.

It should be noted that, the filtering reference items may include: the attributes such as local offset direction stability, spatial distribution uniformity, local offset distance stability, deformation degree adaptability, weighted offset degree and the like between the homonymous point pairs can be calculated through filtering the reference items, so that the homonymous points to be registered effectively are filtered according to preset constraint conditions.

It can be understood that, the common name points are paired points, and the common name points in the reference image are filtered, so that the corresponding common name points in the image to be registered are filtered, and the common name points in the image to be registered are filtered, so that the corresponding common name points in the reference image are filtered.

Specifically, the step of determining the filtering reference item of the valid homonymous point to be registered according to the offset field and the KNN graph structure includes:

It can be understood that the offset distance is a distance by which the longitude and latitude coordinates of the effective co-name point to be registered are offset relative to the longitude and latitude coordinates of the corresponding effective reference co-name point, and the offset angle is an angle by which the longitude and latitude coordinates of the effective co-name point to be registered are offset relative to the longitude and latitude coordinates of the corresponding effective reference co-name point; the offset distance range, namely the offset distance range, the offset angle range, namely the offset angle range, and the position of the effective homonymous point to be registered relative to the effective reference homonymous point can be determined through the offset distance, the offset distance range, the offset angle range and the offset angle range.

Step S35: filtering the filtering reference item of the effective homonymous points to be registered according to a preset constraint condition to obtain target homonymous points to be registered;

step S36: and determining a target reference homonymy point and a target homonymy point pair corresponding to the target homonymy point to be registered.

It can be understood that the registration device may perform offset distance range filtering, offset angle range filtering, spatial uniform distribution constraint, offset distance weight selection filtering, and the like on the valid homonymous points to be registered, so as to obtain the target homonymous points to be registered. Specifically, the registration device may determine attributes such as local offset direction stability, spatial distribution uniformity, local offset distance stability, deformation degree adaptability and the like between the same-name point pairs according to filtering reference items such as offset distance range, offset angle range, spatial uniformity distribution, offset distance weight and the like of the effective to-be-registered same-name points, so as to filter the effective to-be-registered same-name points which do not meet the conditions according to preset constraint conditions.

It will be appreciated that the preset constraint is a preset constraint, and the present embodiment does not limit the specific constraint.

Further, the step of constructing a nonlinear geometric transformation model based on the homonymous point pairs and registering the target image to be registered through the nonlinear geometric transformation model comprises the following steps:

Step S40': and constructing a nonlinear geometric transformation model based on the target homonymous point pairs, and registering the target images to be registered through the nonlinear geometric transformation model.

Specifically, constructing a geographic control point structure array based on the target homonymy point pairs; constructing a nonlinear geometric transformation model according to the geographic control point structure array; and registering the target image to be registered according to the nonlinear geometric transformation model.

It should be noted that, the target homonymy pair includes a target homonymy point to be registered and a target reference homonymy point. The geographical control point (Geographic Control Points, GCP) structure array is an array for storing and managing information of a plurality of geographical control points, through which data of the plurality of geographical control points can be more conveniently organized and managed, and related spatial analysis and processing can be performed.

It should be explained that the above-mentioned geographic control point is essentially the latitude and longitude coordinates (including elevation, which is usually 0 because geographic registration is a process of plane alignment) of the reference image at the common name unit position and the corresponding row and column index coordinates at the common name point of the image to be registered. For example, [ lon1, lat1,0, col1, row1], [ lon2, lat2,0, col2, row2], [ lon3, lat3,0, col3, row3] ]. The significance of constructing the geometric transformation model according to the geographic control points is that col, row of the images to be registered are endowed with correct lon, lat according to the reference images.

The embodiment filters based on the space effectiveness of each homonymous point in the target reference image and the space effectiveness of each homonymous point in the target to-be-registered image to obtain an effective homonymous point pair containing the effective to-be-registered homonymous point; constructing an offset field between the target reference image and the target image to be registered according to the effective homonymy point pairs; constructing a KNN graph structure based on position coordinates of effective homonymous points to be registered in the target image to be registered; determining a filtering reference item of the effective homonymous points to be registered according to the offset field and the KNN graph structure; filtering the filtering reference items of the valid homonymous points to be registered according to preset constraint conditions to obtain target homonymous points to be registered; and determining a target reference homonymy point and a target homonymy point pair corresponding to the target homonymy point to be registered. The method has the advantages that the target homonymous points are obtained by filtering the homonymous points through the space effectiveness of the homonymous point pairs and the construction of the offset field and the KNN graph structure, the homonymous points which are abnormally matched in the homonymous point pairs are removed, the purposes of uniformly distributing the homonymous point pairs in space and controlling the offset complex area in multiple points are restrained, and the stability of the follow-up construction of the nonlinear set transformation model is ensured.

Based on the embodiments of the zoned non-linear geographic registration method of the present invention as described above, a third embodiment of the present invention is presented. As shown in fig. 7, fig. 7 is a schematic flow chart of a third embodiment of the zoned nonlinear geographical registration method of the present invention.

In this embodiment, in order to better perform feature extraction on a geographic image, the steps of extracting, by a preset LoFTR model partition, image features of the target reference image to obtain a reference image feature set, and extracting, by the preset LoFTR model partition, image features of the target image to be registered to obtain a feature set of the image to be registered include:

step S21: partitioning the target reference image according to a preset size to obtain a plurality of reference blocks;

step S22: partitioning the target image to be registered according to a preset size to obtain a plurality of blocks to be registered;

step S23: and extracting the features in each reference image block and each image block to be registered through the feature extraction sub-module to obtain a reference image feature set and an image feature set to be registered.

It should be noted that the preset LoFTR model includes: the device comprises a feature extraction sub-module, a feature matching sub-module, a local feature description sub-module and an iterative fine tuning sub-module.

The feature extraction submodule is used for carrying out partition extraction on the image features of the geographic image to obtain an image block.

It should be appreciated that the image blocks may include a reference block partitioned from a target reference image and a block to be registered partitioned from a target image to be registered. The image features of each reference block can be obtained by extracting the features of each reference image block, and a reference image feature set is formed by the image features of the reference blocks; the image features of each block to be registered can be obtained by extracting the features of each block to be registered, and the image features of the blocks to be registered form an image feature set to be registered.

It should be noted that the preset size may be a ratio of the input target geographic image, and the target geographic image is divided into a fixed number of image blocks, for example, 1/6, 1/8, 1/5, etc., according to the ratio. It should be understood that the target geographic image is the target image to be registered or the target reference image.

It will be appreciated that the predetermined size may be a fixed cutting size, and the present embodiment is not limited thereto.

In one implementation, the image is cut by adopting a fixed cutting size, a partitioning scene of the partitioning nonlinear geographic registration method is shown in fig. 8, and fig. 8 is a schematic diagram of the partitioning scene of the partitioning nonlinear geographic registration method.

As shown in fig. 8, the target image img7.GIF and the target reference image ref. Tif to be registered are spatially partitioned, the preset partition size is 480 x 640, so that the target image img7.GIF and the target reference image ref. Tif to be registered are respectively divided into a plurality of blocks to be registered and reference blocks, the blocks to be registered and the reference blocks are extracted through a LOFTR model, the homonymous point pairs of the target reference image and the target image to be registered are determined, confidence filtering is performed according to the confidence of the homonymous point pairs, space effective area filtering is performed according to the space effectiveness of the target geographic image to obtain the target homonymous point pairs, a nonlinear geometric transformation model is constructed through the target homonymous point pairs, and the target image to be registered is registered based on the nonlinear geometric transformation model.

In this embodiment, the feature extraction of the reference block and the block to be registered is performed by the deep convolutional neural network, so that the high-order feature of the image can be extracted and the subsequent operation amount of the model is reduced.

As shown in fig. 9, fig. 9 is a schematic diagram of a registration scenario of the zoned nonlinear geographic registration method of the present invention.

In fig. 9, a Thin-Plate Spline transformation MODEL (TPS MODEL) is constructed according to the target homonymy point, and the target image to be registered is registered according to the Thin-Plate Spline transformation MODEL, so as to obtain a registration result.

Further, the step of matching the image features in the reference image feature set and the image feature set to be registered and determining the homonymy point pair according to the successfully matched image features includes:

and matching the reference image feature set with the image feature set to be registered through the feature matching sub-module, and determining a rough homonymy point pair according to the successfully matched image features, wherein the rough homonymy point pair is used for representing the reference image block and the image block to be registered, which are successfully matched in the image features.

It should be noted that, the feature matching sub-module may be configured to perform feature matching on the reference image feature set and the image feature set to be registered, so as to obtain a homonymy point pair.

It should be appreciated that when the first match is made, the obtained homonym point pair is a coarse homonym point pair.

In this embodiment, the feature matching submodule realizes image feature matching based on a transducer. Specifically, coarse homonymy points are obtained by matching input reference image features and image features to be registered by using Self-Attention units, cross-Attention units and other units.

It should be noted that, in order to enhance the robustness of matching and the discrimination capability of the model, the LoFTR model may calculate a local feature descriptor for each coarse homonymous point in the coarse registered geographic image. The local feature descriptors, namely the image features of the local area around the rough homonymous point, can more accurately distinguish the differences among different image blocks.

It can be understood that when the local feature descriptors are obtained, the feature matching sub-module can be used for matching the iterative image features formed by the image features with rough homonymy points and the local feature descriptors again.

It should be understood that the coarse homonymy point pairs can be optimized in an iterative manner by the method, and in each iterative step, a corresponding registration model is determined according to the current matching result, so that the positions and matching relations of the feature points are further optimized, more and more reliable homonymy points are found, and the robustness of the model is improved.

It can be understood that the LoFTR model can perform high-dimensional feature extraction of images, compress the image size to 1/8 of the original size, reduce the matching operation amount and simultaneously have a receptive field with a local area range, which is helpful for improving the matching success rate; determining coarse homonymy points based on coarse matching such as Self-Attention and Cross-Attention based on high-dimensional features (c×h=c×60×80, C is the feature quantity); roughly estimating the position of the rough homonymy point by combining the rough homonymy point with the Match Model (the position of the rough homonymy point is equivalent to a small area); according to the coarse homonymy point position, windows with the sizes (w, w) are respectively obtained on the reference image and the image to be registered, loFTR matching is carried out again to obtain fine matching homonymy points (at the moment, the homonymy points can still have the situation of mismatching, so that further filtering is needed to obtain geographical control points).

According to the embodiment, a target reference image is partitioned according to a preset size, and a plurality of reference blocks are obtained; partitioning the target image to be registered according to a preset size to obtain a plurality of blocks to be registered; and extracting the features in each reference image block and each image block to be registered through a feature extraction sub-module to obtain a reference image feature set and an image feature set to be registered. Because the reference block and the block to be registered are used for extracting the image features respectively, the high-dimensional image features in the target image to be registered and the target reference image are obtained, and the accuracy of feature extraction is improved.

In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium is stored with a partition nonlinear geographic registration program, and the partition nonlinear geographic registration program realizes the steps of the partition nonlinear geographic registration method when being executed by a processor.

Based on the first embodiment of the partition nonlinear geographical registration method of the present invention, a first embodiment of the partition nonlinear geographical registration device of the present invention is proposed, and referring to fig. 10, fig. 10 is a block diagram of the structure of the first embodiment of the partition nonlinear geographical registration device of the present invention.

As shown in fig. 10, a partitioned nonlinear geographical registration apparatus according to an embodiment of the present invention includes:

The image processing module 110 is configured to obtain a reference geographic image and a geographic image to be registered, and perform preprocessing on the reference geographic image and the geographic image to be registered respectively to obtain a target reference image and a target image to be registered;

the feature extraction module 120 is configured to extract image features of the target reference image through a preset LoFTR model partition, obtain a reference image feature set, and extract image features of the target image to be registered through the preset LoFTR model partition, obtain an image feature set to be registered;

the homonymy point matching module 130 is configured to match the reference image feature set with the image features in the image feature set to be registered, and determine homonymy point pairs according to the successfully matched image features;

and the geographic registration module 140 is used for constructing a nonlinear geometric transformation model based on the homonymous point pairs and registering the target image to be registered through the nonlinear geometric transformation model.

Further, the homonymy point pair comprises a reference homonymy point positioned in the target reference image and a homonymy point to be registered positioned in the target image to be registered, which corresponds to the reference homonymy point; the apparatus further comprises: the homonymy point filtering module; the homonymy point filtering module is used for filtering based on the space effectiveness of each homonymy point in the target reference image and the space effectiveness of each homonymy point in the target image to be registered to obtain an effective homonymy point pair containing the effective homonymy point to be registered; constructing an offset field between the target reference image and the target image to be registered according to the effective homonymy point pairs; constructing a KNN graph structure based on the position coordinates of the effective homonymous points to be registered in the target image to be registered; determining a filtering reference item of the effective homonymous points to be registered according to the offset field and the KNN graph structure; filtering the filtering reference item of the effective homonymous points to be registered according to a preset constraint condition to obtain target homonymous points to be registered; determining a target reference homonymy point and a target homonymy point pair corresponding to the target homonymy point to be registered; correspondingly, the step of constructing a nonlinear geometric transformation model based on the homonymous point pairs and registering the target image to be registered through the nonlinear geometric transformation model comprises the following steps: and constructing a nonlinear geometric transformation model based on the target homonymous point pairs, and registering the target images to be registered through the nonlinear geometric transformation model.

Further, the filtering reference term includes: offset distance, offset angle, offset distance range, offset angle range; the homonym filtering module is further used for determining offset distances and offset angles of the effective homonym points to be registered based on the offset field; determining the offset distance range of the effective homonymous point to be registered according to the offset distance of the effective homonymous point to be registered in the KNN graph structure, and determining the offset angle range of the effective homonymous point to be registered according to the offset angle of the effective homonymous point to be registered in the KNN graph structure.

Further, the preset LoFTR model includes: a feature extraction sub-module; the feature extraction module 120 is further configured to: partitioning the target reference image according to a preset size to obtain a plurality of reference blocks; partitioning the target image to be registered according to a preset size to obtain a plurality of blocks to be registered; and extracting the features in each reference image block and each image block to be registered through the feature extraction sub-module to obtain a reference image feature set and an image feature set to be registered.

Further, the preset LoFTR model further includes: the device comprises a feature matching sub-module, a local feature description sub-module and an iterative fine tuning sub-module; the homonymy point matching module 130 is further configured to match the reference image feature set and the image feature set to be registered through the feature matching sub-module, and determine a coarse homonymy point pair according to an image feature that is successfully matched, where the coarse homonymy point pair is used for characterizing the reference image block and the image block to be registered that are successfully matched with the image feature; determining the reference image block and the image block to be registered related to the rough homonymy point; determining a local reference feature descriptor of the reference image block and a local feature descriptor to be registered of the image block to be registered through the local feature description submodule; and performing iterative optimization based on the local reference feature descriptors and the local features to be registered through the iterative fine adjustment, and determining homonymous point pairs according to an iterative optimization result.

Further, the image processing module 110 is further configured to perform image type level processing on the reference image to obtain an intermediate reference image, and perform image type level processing on the geographic image to be registered to obtain an intermediate image to be registered, where the image type level processing includes: radiometric calibration, atmospheric correction, geometric correction and data fusion; performing spatial reference unification processing on the intermediate base image and the intermediate image to be registered to obtain a spatial reference base image and a spatial reference image to be registered of the same spatial reference system; and performing overlap region clipping on the spatial reference image to be registered based on the spatial reference image to obtain a target reference image and a target image to be registered.

Further, the geographic registration module 140 is further configured to construct a geographic control point structure array based on the target homonymy point pair; constructing a nonlinear geometric transformation model according to the geographic control point structure array; and registering the target image to be registered according to the nonlinear geometric transformation model.

Other embodiments or specific implementations of the zoned nonlinear geographic registration device of the present invention may refer to the above method embodiments, and are not described herein.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read-only memory/random-access memory, magnetic disk, optical disk), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims

1. A method of zoned nonlinear geographical registration, the method comprising:

2. The partitioned nonlinear geographical registration method of claim 1, wherein the pair of homonymous points includes a reference homonymous point located in the target reference image and a homonymous point to be registered located in the target image to be registered corresponding to the reference homonymous point;

3. The partitioned nonlinear geographical registration method of claim 2, wherein the filtering reference term comprises: offset distance, offset angle, offset distance range, offset angle range;

4. The method of zoned nonlinear geographical registration of claim 1,

the preset LoFTR model comprises: a feature extraction sub-module;

5. The zoned nonlinear geographical registration method of claim 4, wherein the preset LoFTR model further comprises: the device comprises a feature matching sub-module, a local feature description sub-module and an iterative fine tuning sub-module;

6. The method of zoned nonlinear geographic registration of claim 1, wherein the step of preprocessing the reference geographic image and the geographic image to be registered, respectively, to obtain a target reference image and a target image to be registered, comprises:

7. A method of partitioned nonlinear geographical registration as recited in claim 3, wherein the step of constructing a nonlinear geometric transformation model based on the homonymous point pairs and registering the target image to be registered via the nonlinear geometric transformation model comprises:

8. A zoned nonlinear geographical registration apparatus, the zoned nonlinear geographical registration apparatus comprising:

9. A zoned nonlinear geographical registration apparatus, the apparatus comprising: a memory, a processor, and a partitioned nonlinear geographical registration program stored on the memory and executable on the processor, the partitioned nonlinear geographical registration program configured to implement the steps of the partitioned nonlinear geographical registration method as recited in any one of claims 1-7.

10. A storage medium having stored thereon a zoned non-linear geographical registration program which when executed by a processor implements the steps of the zoned non-linear geographical registration method of any one of claims 1 to 7.