CN112560868B - Multi-process large area network image matching method and device based on feature point library - Google Patents

Multi-process large area network image matching method and device based on feature point library Download PDF

Info

Publication number
CN112560868B
CN112560868B CN202011355652.9A CN202011355652A CN112560868B CN 112560868 B CN112560868 B CN 112560868B CN 202011355652 A CN202011355652 A CN 202011355652A CN 112560868 B CN112560868 B CN 112560868B
Authority
CN
China
Prior art keywords
image
reference image
matching
characteristic point
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011355652.9A
Other languages
Chinese (zh)
Other versions
CN112560868A (en
Inventor
张桂滨
阴晓刚
王一
王重阳
赵莹芝
范晓敏
折晓宇
马亚斌
许凯波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xi'an Zhongke Xingtu Spatial Data Technology Co ltd
Original Assignee
Xi'an Zhongke Xingtu Spatial Data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xi'an Zhongke Xingtu Spatial Data Technology Co ltd filed Critical Xi'an Zhongke Xingtu Spatial Data Technology Co ltd
Priority to CN202011355652.9A priority Critical patent/CN112560868B/en
Publication of CN112560868A publication Critical patent/CN112560868A/en
Application granted granted Critical
Publication of CN112560868B publication Critical patent/CN112560868B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462Salient features, e.g. scale invariant feature transforms [SIFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29Geographical information databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/02Affine transformations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/80Geometric correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10032Satellite or aerial image; Remote sensing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20016Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Remote Sensing (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)

Abstract

A multi-process large area network image matching method and device based on a characteristic point library belongs to the field of image information processing, and is characterized by comprising the following steps: connecting the main process with a feature point library; determining a database table positioned in the reference image range in the characteristic point library according to the reference image range and the characteristic point library; packing the reference image, the feature point set and all images overlapped with the reference image as subtasks, and circulating until the image traversing of the measuring area is completed to form a plurality of subtask packages; the main process pushes all sub-task packages to each sub-process for processing; and the main process receives the return state of each sub-process, and when each sub-process is completed, the main process ends, returns to the total task state and completes image matching. The collection mode of the characteristic points effectively ensures the quantity and distribution of the characteristic points and meets the use requirement of large area network adjustment; meanwhile, a sub-table naming and storage mode of the feature point database is established, and the use efficiency of feature point searching is improved.

Description

Multi-process large area network image matching method and device based on feature point library
Technical Field
The invention belongs to the field of image information processing, and particularly relates to a characteristic point library-based multi-process large area network image matching method and device.
Background
With the continuous development of satellite technology and sensor technology, available satellite image data are more and more, and the accurate positioning technology of satellite images always depends on ground characteristic points, the quantity and distribution of the characteristic points directly influence the positioning accuracy of images on a ground target, however, due to various limiting conditions such as national boundaries, ground objects and the like, the manual acquisition of a sufficient quantity of ground control points is generally very difficult, high-precision image positioning is realized under the condition without the ground control points, high-precision reference images are required to be relied on, and in actual processing, a large number of characteristic points are extracted from the reference images and are penetrated into the images to be processed to carry out RPC refinement, so that the positioning accuracy of the images to be processed is improved.
In practical work, in order to meet the requirement of real-time processing, a large amount of reference images need to be stored, a large amount of storage space is needed, taking a google image with 5m resolution of 4-degree blocks as an example, the whole world is free of water areas, the single-band data volume of images containing land exceeds 10T, the size of a single-view image exceeds 8G, a large amount of computer storage resources are consumed in the process of reading and storing the reference images, and meanwhile, the migration difficulty of a system is increased due to the large amount of reference images in the process of system migration and deployment.
In the processing process of the large area network image, in order to ensure the border connecting precision and the uniform positioning precision of the large area network image, the large area network image is required to have a uniform reference standard, and secondly, the large area network image and the reference image are matched to belong to heterogeneous matching, so that the matching success rate is relatively low due to the differences of an imaging mechanism, acquisition time and the like. Therefore, under the condition of no ground control point, how to solve the storage and real-time updating of a large number of reference images, how to quickly and efficiently transfer the reference image characteristic points, and how to provide a high-precision reliable reference point for the regional network RPC refinement, is a key for improving the positioning and edge connection precision of the large regional network images.
Disclosure of Invention
The invention aims to solve the problems and provides a multi-process large area network image matching method and device based on a characteristic point library, which can rapidly provide a large number of high-reliability reference points.
In a first aspect, the present invention provides a method for matching images of a multi-process large area network based on a feature point library, including:
connecting the main process with a feature point library;
Selecting a reference image from the image traversal of the area, and determining a reference image range according to the RPC parameter of the reference image; determining a database table positioned in the reference image range in the characteristic point library according to the reference image range and the characteristic point library;
Characteristic points in the searching range in the determined database table are searched according to the reference image range, and a characteristic point set is formed; acquiring all images with overlapping areas with the reference image according to the RPC parameters;
Packing the reference image, the feature point set and all images overlapped with the reference image as subtasks to form a subtask package;
Traversing and selecting a reference image from the image of the detection area again, and repeating the process to form another subtask package; cycling until the image traversal of the detection area is completed, and forming a plurality of subtask packages;
The main process pushes all sub-task packages to each sub-process for processing;
And the main process receives the return state of each sub-process, and when each sub-process is completed, the main process ends, returns to the total task state and completes image matching.
Furthermore, the multi-process large area network image matching method based on the characteristic point library, disclosed by the invention, is characterized in that the construction of the characteristic point library is as follows:
Establishing a database according to the resolution of the reference image;
uniformly gridding and partitioning the global image according to standard degree intervals to obtain a plurality of gridding blocks;
Extracting uniformly grid characteristic points of the reference image to obtain longitude and latitude values of the characteristic points;
establishing N layers of pyramid layer data of M multiplied by M resolution reduction sampling by taking each characteristic point as a center as characteristic description of the characteristic point, wherein the range of N is 6-8, and the range of M is an odd number between 11-21;
Introducing 30m DEM and Egm96 elevation compensation data, performing bilinear elevation interpolation according to longitude and latitude values of the feature points, and obtaining an elevation to obtain feature points with feature description, longitude, latitude and elevation;
Establishing a database table name by the longitude and latitude integer value of the left lower corner of each grid block, and determining the storage characteristic point range of each database table, wherein the longitude range is from the longitude of the left lower corner to the longitude of the left lower corner plus the standard degree interval, and the latitude range is from the latitude of the left lower corner to the latitude of the left lower corner plus the standard degree interval;
And inserting the longitude and latitude values of each characteristic point into a corresponding database table to complete the construction of the characteristic point library.
Furthermore, the multi-process large area network image matching method based on the characteristic point library, which is disclosed by the invention, has the advantages that the main process pushes all sub-task packages to each sub-process for processing,
In each sub-process, firstly, carrying out dynamic pyramid image resolution consistency processing and distortion correction;
performing dynamic threshold correlation coefficient rough matching and least square fine matching;
the rough difference is removed to obtain correct matching point pairs as a reference image characteristic point set, and the characteristic point set is subjected to traversal matching with the overlapped images;
traversing the overlapped region images until the overlapped images are traversed;
And merging and storing the same-name point pair files of the task set, and feeding back the task state of the sub-process to the main process.
Further, according to the multi-process large area network image matching method based on the feature point library, the dynamic pyramid image resolution consistency processing and distortion correction are carried out, a virtual reference image is established according to the feature point set, and a whole image space transformation model is established according to the RPC parameters and the virtual reference image by the reference image; and calculating a reference image search window according to the transformation model by using the characteristic point target window, carrying out dynamic pyramid image resolution consistency processing on the target window and the search window, and carrying out distortion correction on the search window according to the transformation model.
Further, according to the multi-process large area network image matching method based on the feature point library, the dynamic threshold correlation coefficient rough matching and the least square fine matching are carried out on the target window and the search window, and the correlation coefficient threshold is from low to high according to pyramid resolution.
Further, according to the multi-process large area network image matching method based on the characteristic point library, the overlapping area images are traversed, a transformation model of the whole image space of the reference image and the overlapping image is built according to RPC parameters, and a characteristic point target window determines a reference image search window according to the transformation model;
The dynamic pyramid target window and the search window image resolution are processed in a consistent mode, and distortion correction is carried out on the search window according to the transformation model;
The target window and the search window are subjected to dynamic threshold correlation coefficient rough matching and least square fine matching, and the correlation coefficient threshold is from low to high according to pyramid resolution;
the rough difference is removed to obtain correct matching point pairs as a reference image characteristic point set, and the characteristic point set is subjected to traversal matching with the overlapped images;
until the overlapping image traversal is completed.
In a second aspect, the present invention provides a multi-process large area network image matching device based on a feature point library, including:
A connection unit: the method is used for connecting the main process with the characteristic point library;
a subtask unit: the method comprises the steps of traversing and selecting a reference image from a measured image, and determining a reference image range according to RPC parameters of the reference image; determining a database table positioned in the reference image range in the characteristic point library according to the reference image range and the characteristic point library; characteristic points in the searching range in the determined database table are searched according to the reference image range, and a characteristic point set is formed; acquiring all images with overlapping areas with the reference image according to the RPC parameters; packing the reference image, the feature point set and all images overlapped with the reference image as subtasks to form a subtask package; traversing and selecting a reference image from the image of the detection area again, and repeating the process to form another subtask package; cycling until the image traversal of the detection area is completed, and forming a plurality of subtask packages;
Multiprocessing processing unit: the main process is used for pushing all sub-task packages to each sub-process for processing; and the main process receives the return state of each sub-process, and when each sub-process is completed, the main process ends, returns to the total task state and completes image matching.
In a third aspect, the present invention provides a multi-process large area network image matching device based on a feature point library, including:
Feature point library unit: the method comprises the steps of establishing a database according to the resolution of a reference image; uniformly gridding and partitioning the global image according to standard degree intervals to obtain a plurality of gridding blocks; extracting uniformly grid characteristic points of the reference image to obtain longitude and latitude values of the characteristic points; establishing pyramid layer data with each feature point as a center to be used as feature description of the feature point; performing bilinear elevation interpolation according to the longitude and latitude values of the feature points to obtain elevations, and obtaining feature points with feature description, longitude, latitude and elevations; establishing a database table name according to the longitude and latitude integer value of the left lower corner of each grid block; inserting the longitude and latitude values of each characteristic point into corresponding database tables to complete the construction of a characteristic point library;
A connection unit: the method is used for connecting the main process with the characteristic point library;
A subtask unit: the method comprises the steps of traversing and selecting a reference image from a measured image, and determining a reference image range according to RPC parameters of the reference image;
Determining a database table positioned in the reference image range in the characteristic point library according to the reference image range and the characteristic point library; characteristic points in the searching range in the determined database table are searched according to the reference image range, and a characteristic point set is formed; acquiring all images with overlapping areas with the reference image according to the RPC parameters; packing the reference image, the feature point set and all images overlapped with the reference image as subtasks to form a subtask package; traversing and selecting a reference image from the image of the detection area again, and repeating the process to form another subtask package; cycling until the image traversal of the detection area is completed, and forming a plurality of subtask packages;
Multiprocessing processing unit: the main process is used for pushing all sub-task packages to each sub-process for processing; and the main process receives the return state of each sub-process, and when each sub-process is completed, the main process ends, returns to the total task state and completes image matching.
In a fourth aspect, the invention provides a multi-process large area network image matching device based on a feature point library, which comprises a memory and a processor; the memory is used for storing a computer program; the processor is configured to implement the multi-process large area network image matching method based on the feature point library according to the first aspect when executing the computer program.
In a fifth aspect, the present invention provides a computer readable storage medium, where a computer program is stored, where the computer program, when executed by a processor, implements the multi-process large area network image matching method based on the feature point library according to the first aspect.
The multi-process large area network image matching method and device based on the characteristic point library have the following technical effects: the collection mode of the characteristic points effectively ensures the quantity and distribution of the characteristic points and meets the use requirement of large area network adjustment; meanwhile, a sub-table naming and storage mode of the feature point database is established, and the use efficiency of feature point searching is improved.
By discretizing the reference image data into a characteristic point recording mode in a database table, the storage data volume is better than one percent of the reference image, the problem of storing a large number of global reference images is solved, and meanwhile, the migration deployment and the use of the system are facilitated. For the on-orbit real-time processing satellite, the reference image can not meet the requirement of real-time uploading, and the lightweight database can meet the requirement of real-time uploading, so that the real-time processing of the on-orbit satellite image is ensured.
In actual processing, a large amount of computer resources are consumed for reading a large amount of reference images, and the computer resource consumption is effectively reduced by a characteristic point library reading mode. And the dynamic pyramid creation and dynamic threshold matching strategies of the target window and the search window prevent pyramid images from falling off, reduce the resource consumption of a computer and improve the success rate of feature point matching. The large area network image matching technology based on the characteristic point library realizes the capability of improving the positioning precision and the edge connecting precision of the large area network image by means of the reference image under the condition of no ground control point.
The reasonable MPI process accelerates the design, and the matching task is processed by the separate processes, so that the efficiency of large area network image matching is accelerated.
Drawings
FIG. 1 is a schematic diagram of a characteristic point library construction flow according to the present invention;
FIG. 2 is a flow chart of a multi-process large area network image matching method based on a feature point library according to the present invention;
Fig. 3 is a diagram illustrating a global standard homogenization grid division according to an embodiment of the present invention.
Detailed Description
The method and the device for matching the multi-process large area network images based on the characteristic point library are described in detail below through drawings and embodiments.
Example 1
In the embodiment of the disclosure, a lightweight characteristic point database is constructed based on a Postgresql database, a corresponding relation is established between the database name and the resolution of the characteristic point, a characteristic point database with corresponding resolution is conveniently found in subsequent matching work, and the attributes of the database table comprise characteristic point feature description, characteristic point longitude, characteristic point latitude and characteristic point elevation. And establishing a sub-table storage mechanism and naming criteria, and improving the updating and searching efficiency of the database table.
In the embodiment of the disclosure, taking 4 degrees multiplied by 4 degrees as a standard interval as an example, performing uniform grid division on the world (-180, -90), wherein the grid quantity is (360×180)/(4*4) =4050 blocks, because the water area earth is usually an invalid image, according to 29% of land occupying the earth surface area, the land is calculated to be about 4050×0.29=1175 blocks, the data volume is calculated by taking 5 meter google images as reference images as examples, the single-band size of 4-degree segmented google images is above 8G, the single-band data volume of the global land reference images is 1175×8G/block=9400G calculated by 8G, according to practical use, each reference image is discretized into 40000 blocks, because part of the water area is also in the actual land block, there are no characteristic points, each table in the database is stored with 40000 records according to each block of 40000 points, the size of each record is 2.3KB, the characteristic point quantity of the global land is 1024×2.3×3×40000G, and the global reference image is better than the reference standard of the grid quantity of 1024×103×3.
The database name is established according to the resolution of the reference image, for example, the database can be named as G15 according to the resolution of 5 meters, a database table is correspondingly established for each grid block divided by the standard block, the database table name is named in a mode of combining the whole value of longitude and latitude at the lower left corner of the grid block with the north latitude (n) and the south latitude(s) of the east longitude (e) and the west longitude (w), for example, the "e100n22" is a grid block data table of 100 degrees north latitude and 22 degrees of east longitude, the longitude effective data range in the data table is a [100,104 ] half-closed half-open section, the latitude effective range is a [22,26 ] half-closed half-open section, and the database table has the attributes of id (primary key) and despt, lon, lat, alt.
The global (-180, -90) range is subjected to standardized grid partitioning according to a mode of multiplying 4 degrees by 4 degrees, each standardized grid block is used as a database table, the range of characteristic points in the table can be known according to the standard 4-degree grid partitioning, and the efficiency of database updating and searching is improved through sub-table storage.
Extracting characteristic points of a reference image grid-meshed uniform Forstner operator, establishing a 17 x 17 window by taking the characteristic points as the center, resampling by adopting a low-resolution mean pyramid, establishing 6 layers of pyramid data as despt attributes of a database table corresponding to characteristic description, extracting longitude and latitude object coordinates of the characteristic points, reserving longitude and latitude values to 9 bits after decimal points, carrying out bilinear interpolation on DEM30 m data and egm96 elevation compensation data according to the longitude and latitude values to obtain elevation values, reserving the elevation values to 2 bits after decimal points, respectively corresponding to lon, lat, alt attributes, and inserting the elevation values into the corresponding database table according to the longitude and latitude values of each characteristic point.
According to the construction mechanism of the characteristic point library, the database table required by the image can be rapidly determined by knowing the range of the image to be processed, the searching range is reduced, the searching speed is improved, and the consumption of computer resources can be effectively reduced by reading the record of the discretization data table relative to reading a large number of reference images. The method comprises the steps of establishing a dynamic pyramid layer matching strategy, eliminating scale change influence, not establishing a pyramid for the whole image, ensuring that pyramid data does not fall, only dynamically establishing pyramid layer data in a small range of a search window, reducing memory consumption for pyramid data reading, establishing a mechanism of traversing matching from a low resolution to a high resolution dynamic threshold, establishing a low resolution pyramid layer matching threshold in each layer matching process, and a high resolution pyramid layer matching threshold in each layer matching process, improving the matching success rate, establishing a virtual image by using a feature point set as a reference image, establishing two image whole image transformation models according to the relation between an object side of the image to be processed and the object side of the virtual reference image, determining the target window of the feature point according to the transformation models, transforming and resampling the search window, eliminating distortion influence of the target window and the search window, adopting an MPI-based multi-task distribution parallel processing mechanism for large area network image data, and respectively distributing a plurality of sets to a plurality of sets for parallel processing by discretizing the large area network image according to a plurality of overlapping area sets, thereby improving the matching efficiency. Through the various strategy mechanisms, the invention effectively improves the success rate and the efficiency of matching the large area network image with the feature points.
In the embodiment of the present disclosure, the step of constructing the feature point library as shown in fig. 1 includes:
S111, establishing a database according to the resolution of the reference image, and establishing a corresponding table of database names and resolution, so that different images to be corrected can be conveniently found out to the corresponding database;
S112, uniformly gridding and partitioning the whole world according to standard degree intervals;
S113, establishing a database table name according to the longitude and latitude integer value of the lower left corner of each grid block; e.g. "e100n22" is a grid block data table of 100 degrees east longitude and 22 degrees north latitude;
s114, extracting uniformly grid characteristic points of the reference image, and acquiring longitude and latitude values of the characteristic points;
s115, establishing pyramid layer data of 6 layers of reduced resolution samples of 17 times 17 by taking each corner point as a center as the corner point characteristic point description, wherein the embodiment of the disclosure adopts a mean pyramid;
S116, introducing 30 m DEM and Egm96 elevation compensation data, performing bilinear elevation interpolation according to the longitude and latitude values of the angular points, and obtaining the elevation to obtain feature points with feature description, longitude, latitude and elevation;
s117, inserting the longitude and latitude values of each characteristic point into a corresponding database table for storage.
In an embodiment of the present disclosure, as shown in fig. 2, the steps of a multi-process large area network image matching method based on a feature point library include:
s121, connecting a feature point library by a main process;
S122, selecting one image from the traversing area image as a reference image;
S123, calculating object coordinates of four corners of the reference image according to RPC parameters of the reference image, taking a maximum circumscribed rectangle as a reference image range, determining a database table name positioned in the reference image range according to a reference image range and a characteristic point database construction criterion, and searching for characteristic points in a range in the determined database table according to the reference image range to form a characteristic point set;
s124, acquiring all images with overlapping areas with the reference image according to rpc parameters;
s125, packing the reference image, the feature point set and all images overlapped with the reference image as a subtask;
S126, repeating the processes from S122 to S125 until the image traversal of the measurement area is completed;
s127, the main process pushes all task packages to each sub-process;
S128, in each sub-process, firstly establishing a virtual reference image according to a characteristic point set, establishing an integral image side transformation model by a reference image according to rpc parameters and the virtual reference image, calculating a reference image search window by a characteristic point target window according to the transformation model, carrying out dynamic pyramid image resolution consistency processing on the target window and the search window, and carrying out distortion correction on the search window according to the transformation model;
S129, performing dynamic threshold correlation coefficient rough matching and least square fine matching on the target window and the search window, and performing correlation coefficient threshold from low to high according to pyramid resolution;
s130, removing the rough differences to obtain correct matching point pairs as a reference image characteristic point set;
S131, traversing the overlapping region images, establishing a transformation model of the integral image space of the reference image and the overlapping image according to rpc parameters, and determining a reference image search window according to the transformation model by a characteristic point target window;
s132, carrying out consistency processing on the image resolution of the dynamic pyramid target window and the search window, and carrying out distortion correction on the search window according to the transformation model;
s133, performing dynamic threshold correlation coefficient rough matching and least square fine matching on the target window and the search window, and performing correlation coefficient threshold from low to high according to pyramid resolution;
S134, removing the rough differences to obtain correct matching point pairs as a reference image characteristic point set;
s135, repeating the processes from S131 to S134 until the overlapping image traversal is completed;
s136, merging and storing the same-name point pair files of the task set, and feeding back the task state of the sub-process to the main process;
S137, the main process receives the return state of each sub-task, and when each sub-task is completed, the same-name point pair files of each sub-process are combined to generate a datum point file;
S138, the main process is ended, the total task state is returned, and the processing flow is ended.
Example two
In another embodiment of the present disclosure, the multi-process large area network image matching process based on the feature point library is:
S211, searching a characteristic point library;
According to the RPC parameters attached to the reference image, four corner points (0, 0), (width-1, 0), (0, height-1) of the reference image are calculated according to a formula 1, longitude and latitude coordinates corresponding to the four corner points of the reference image are obtained, wherein width and height of the reference image are taken as width and height of the reference image, longitude and latitude coordinates (P i,Lj) corresponding to the four corner points of the image are obtained, and the largest circumscribed rectangle of the four corner point coordinates is taken as the reference image range. And then according to the reference image range and the characteristic point database construction criterion, determining the table name of the database positioned in the reference image range, and finally searching the characteristic points in the range in the corresponding database table according to the reference image range to form a characteristic point set.
Wherein F 1、F2、F3、F4 is a general polynomial, and the calculation mode is as follows:
Wherein b ijk (i, j, k=0, 1..20) is the inverse RPC parameter; (P, L, H) is regularized ground coordinates, (X, Y) is regularized image coordinates, and the calculation formulas are respectively as follows:
wherein LAT_OFF, LAT_SCALE, LONE_OFF, LONG_SCALE, HEIGHT_OFF and HEIGHT_SCALE are regularized parameters of the ground coordinates; smap_off, samp_scale, line_off, and line_scale are regularization parameters for image coordinates.
S212, judging the overlapping of the images;
Carrying out intersection operation on the two quadrangles S1 and S2 by utilizing the maximum circumscribed rectangular areas S1 and S2 of the reference image and other images in the area calculated by the formula 1 to obtain a polygon, and determining the circumscribed rectangle of the polygon as an image overlapping area of the reference image and the image to be matched; and then according to the RPC parameters attached to the reference image and longitude and latitude coordinates corresponding to four corner points of the image overlapping region, performing back calculation according to the RPC back calculation shown in the formula 4, thereby obtaining the homonymous region of the reference image and the image to be matched.
Wherein F 1、F2、F3、F4 is a general polynomial, and the calculation mode is as follows:
Wherein a ijk (i, j, k=0, 1..20) is the RPC coefficient; (P, L, H) is regularized ground coordinates, (X, Y) is regularized image coordinates, and the calculation formulas are respectively as follows:
Wherein LAT_OFF, LAT_SCALE, LONE_OFF, LONG_SCALE, HEIGHT_OFF and HEIGHT_SCALE are regularized parameters of the ground coordinates; SMAP_OFF, SAMP_SCALE, LINE_OFF and LINE_SCALE are regularized parameters of the image coordinates; if there is an intersection there is an overlap area, otherwise there is no.
S213, creating a virtual reference image of the feature point set;
the creation of the reference image is mainly to determine six parameters of the reference image and the width and height of the reference image.
A 0 and a 3 in six parameters of the reference image are respectively assigned with the minimum longitude and the maximum latitude of the maximum circumscribed rectangle of the reference image, the minimum longitude and the maximum latitude of the left upper corner of the reference image are represented, and a 1 and a 5 represent that the resolution of the reference image is obtained by the construction criteria of the characteristic point library, and a 2 and a 4 are defaults to 0.
The width of the reference image is obtained by dividing the resolution by the maximum longitude and the minimum longitude of the reference image, and the height of the reference image is obtained by dividing the resolution by the maximum latitude and the minimum latitude of the reference image.
S214, calculating an integral transformation model;
The integral transformation model respectively and reversely calculates the image space of the two images according to the formula 4 according to the object space coordinates of the overlapped quadrilateral areas of the images obtained in the step S212, and solves the six-parameter transformation coefficient model of the two images according to least square fitting.
S215, a dynamic pyramid dynamic threshold correlation coefficient and a least square matching strategy;
and dynamically establishing pyramid layers, and not establishing multi-layer pyramid for the whole image, wherein 6 layers of pyramid layer target windows taking the feature points as the centers are established in the feature points in the feature point library, only the pyramid layer data with the size of a search window corresponding to the target window is required to be established according to a transformation model, the search window is generally set to 61 x 61, pyramid layers with the same or similar resolution are selected for carrying out a layer-by-layer correlation coefficient and least square matching mechanism from low resolution to high resolution, a dynamic matching threshold mechanism is adopted for correlation coefficient matching in the pyramid layer matching process, the pyramid layer matching threshold with low resolution is lower, and the pyramid layer matching threshold with high resolution is higher.
The matching principle of the correlation coefficient is as follows:
By counting the correlation coefficient ρ (c, r) between the target window and the search window, the position with the largest correlation number is used as the homonymy point, and the calculation formula is as follows:
Wherein g and g' are pixel gray values corresponding to the target window and the search window, i and j are row and column coordinates of the window to be matched, and r and c are row and column coordinates of the corresponding search area; And Calculating formulas for the average value of gray scales in the target window and the search window; defining W1 as a target window in a reference image, wherein the size is (2N+1) multiplied by (2N+1) (N=1, 2,3.,) W2 as a search window of the image to be matched, and the size is (2M+1) multiplied by (2M+1) (M=1, 2,3.,) typically M > N, traversing the target window W1 in the W2 window according to a row-column sequence, and calculating a normalized cross-correlation coefficient rho (c, r), wherein the corresponding pixel point in the image to be matched is the same name point of the center point of the reference image when rho (c, r) is maximum.
The least squares matching principle is as follows:
The criterion for judging similarity by least square image matching is that the square sum of gray level differences is minimum, and the gray level differences are marked as v, as shown in formula 10:
Σvv=min equation 10
In general, there is geometric deformation between two-dimensional images, and only the geometric deformation of the images is fully considered, so that the best image matching result can be obtained, in actual calculation, because the size of the image matching window is generally smaller, the geometric deformation between the images is represented by using an affine transformation relationship, as shown in formula 11:
wherein a 0、a1、a2 and b 0、b1、b2 are parameters for correcting geometric deformation between left and right images, namely affine transformation parameters, (x, y) are pixel coordinates of a left image window, and (x 2,y2) are pixel coordinates of a right image window after affine transformation.
Meanwhile, considering the linear gray level distortion of the left and right images, equation 12 is obtained:
g(x,y)+n1(x,y)=h0+h1g2(a0+a1x+a2y,b0+b1x+b2y)+n2(x,y) Equation 12
Wherein g (x, y), g 2(x2,y2) are gray functions, and n 1(x,y)、n2 (x, y) are random noise; h 0、h1 is a linear distortion parameter.
After linearizing equation 12, a least squares match error equation is obtained as shown in equation 13:
v=c1dh0+c2dh1+c3da0+c4da1+c5da2+c6db0+c7db1+c8db2-Δg Equation 13
Wherein dh 0,dh1,da0,...,db2 is the correction value of the undetermined parameter, Δg is the gray level difference of the corresponding pixel, and their initial values are h0=0、h1=1、a0=0、a1=1、a2=0、b0=0、b1=0、b2=1.
C 1、C2、...、C8 is the error equation coefficient, as shown in equation 14, in whichG 2 is a gray value, and x and y are pixel coordinates;
the matrix form of the error equation is shown in equation 15:
v=cx-L equation 15
Wherein, C represents the coefficient matrix of the error equation, X represents the coefficient matrix of the undetermined parameter, L represents the gray level difference of the pixels in the window, as shown in formula 16, and T is the transpose of the matrix.
Since the gray scale in image matching is a regularly meshed discrete array, and the sampling interval is regarded as a unit length, the partial derivative in calculation can be replaced by difference, as shown in formula 17:
The least square image matching is an algorithm for matching through iteration, the initial value of each parameter at the beginning of the iteration is h 0=a0=a2=b0=b1=0,h1=a1=b2 =1, and the algorithm is set Is the parameter in the i-1 th deformation,For the parameter correction value at the ith iteration, the ith geometric distortion correction parameter value is shown in equation 18:
the ith radiation distortion correction parameter value is shown in equation 19:
Since the accuracy of least squares matching has a great relationship with the gray scale gradient of the image, the left image can accurately point by weighted average of the square weight of the gradient to the coordinates as shown in formula 20:
wherein, AndA gradient representing the gray scale of the image;
The accurate coordinates of the same-name points of the right image can be obtained according to the least square image matching affine transformation, as shown in a formula 21.
S216, distributing large area network image matching tasks based on MPI;
And the main process is responsible for decomposing the task into a plurality of image set matching tasks and distributing the image set matching tasks, and the sub-process waits for the task to be received. After the subprocesses receive the tasks, the plurality of subprocesses can simultaneously complete respective image matching tasks, and after all the subprocesses complete the matching tasks and feed back the execution state to the main process, the matching processing of all the images in the large area network is completed. The mechanism of parallel processing of a plurality of subprocesses ensures the completion of image matching in efficiency, and the advantages are more obvious compared with a single-process program especially when processing massive images.
S217, large area network image matching results based on the feature point library;
According to the invention, 600 high-resolution first-number image data are processed based on the characteristic point library, more than 140000 reference points are obtained in total, the 90-kernel MPI process is adopted for processing, only about 30 minutes is needed, the obtained reference points are sufficient in quantity and uniform in distribution, and the application of large-area network RPC refinement is satisfied.
Example III
The image matching device of the multi-process large area network based on the characteristic point library comprises a memory and a processor; the memory is used for storing a computer program; the processor is configured to, when executing the computer program, cause the computer to execute the multi-process large area network image matching method based on the feature point library in the first embodiment and the second embodiment, and specific matching steps are not described again.
Example IV
In another embodiment of the present disclosure, a computer readable medium stores a program code, where the program code when executed on a computer causes the computer to execute the multi-process large area network image matching method based on the feature point library in the first embodiment and the second embodiment, and specific matching steps are not repeated.
The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. The computer readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital Versatile Disk (DVD)), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
The software formed by the computer storage code can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media which are mature in the field.
The functional units in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (8)

1. A multi-process large area network image matching method based on a characteristic point library is characterized by comprising the following steps:
establishing a database according to the resolution of the reference image; uniformly gridding and partitioning the global image according to standard degree intervals to obtain a plurality of gridding blocks;
Extracting uniformly grid characteristic points of the reference image to obtain longitude and latitude values of the characteristic points;
establishing pyramid layer data with each feature point as a center to be used as feature description of the feature point;
Performing bilinear elevation interpolation according to the longitude and latitude values of the feature points to obtain elevations, and obtaining feature points with feature description, longitude, latitude and elevations;
Establishing a database table name according to the longitude and latitude integer value of the left lower corner of each grid block;
inserting the longitude and latitude values of each characteristic point into corresponding database tables to complete the construction of a characteristic point library;
connecting the main process with a feature point library;
selecting a reference image from the image traversal of the area, and determining a reference image range according to the RPC parameter of the reference image;
determining a database table positioned in the reference image range in the characteristic point library according to the reference image range and the characteristic point library;
characteristic points in the searching range in the determined database table are searched according to the reference image range, and a characteristic point set is formed;
Acquiring all images with overlapping areas with the reference image according to the RPC parameters;
Packing the reference image, the feature point set and all images overlapped with the reference image as subtasks to form a subtask package;
Traversing and selecting a reference image from the image of the detection area again, and repeating the process to form another subtask package; cycling until the image traversal of the detection area is completed, and forming a plurality of subtask packages;
The main process pushes all sub-task packages to each sub-process for processing;
And the main process receives the return state of each sub-process, and when each sub-process is completed, the main process ends, returns to the total task state and completes image matching.
2. The multi-process large area network image matching method based on the characteristic point library according to claim 1, wherein the construction of the characteristic point library is characterized in that:
Establishing a database according to the resolution of the reference image;
uniformly gridding and partitioning the global image according to standard degree intervals to obtain a plurality of gridding blocks;
Extracting uniformly grid characteristic points of the reference image to obtain longitude and latitude values of the characteristic points;
establishing N layers of pyramid layer data of M multiplied by M resolution reduction sampling by taking each characteristic point as a center as characteristic description of the characteristic point, wherein the range of N is 6-8, and the range of M is an odd number between 11-21;
Introducing 30m DEM and Egm96 elevation compensation data, performing bilinear elevation interpolation according to longitude and latitude values of the feature points, and obtaining an elevation to obtain feature points with feature description, longitude, latitude and elevation;
Establishing a database table name according to the longitude and latitude integer value of the left lower corner of each grid block;
And inserting the longitude and latitude values of each characteristic point into a corresponding database table to complete the construction of the characteristic point library.
3. The multi-process large area network image matching method based on the feature point library according to claim 1, wherein the main process pushes all sub-task packages to each sub-process for processing, and the method is characterized in that:
Each sub-process firstly carries out dynamic pyramid image resolution consistency processing and distortion correction; establishing a virtual reference image according to the feature point set, and establishing an integral image space transformation model by the reference image and the virtual reference image according to the RPC parameters; calculating a reference image search window according to the transformation model by using the characteristic point target window, carrying out dynamic pyramid image resolution consistency processing on the target window and the search window, and carrying out distortion correction on the search window according to the transformation model;
performing dynamic threshold correlation coefficient rough matching and least square fine matching;
the rough difference is removed to obtain correct matching point pairs as a reference image characteristic point set;
Traversing the overlapping region image;
And merging and storing the same-name point pair files of the task set, and feeding back the task state of the sub-process to the main process.
4. The method for matching multi-process large area network images based on feature point library according to claim 3, wherein the steps of performing dynamic threshold correlation coefficient rough matching and least square fine matching are characterized in that: and carrying out dynamic threshold correlation coefficient rough matching and least square fine matching on the target window and the search window, and carrying out the correlation coefficient threshold from low to high according to pyramid resolution.
5. The feature point library-based multi-process large area network image matching method according to claim 4, wherein the traversing overlapping area images is characterized in that: establishing a transformation model of the integral image space of the reference image and the overlapped image according to the RPC parameters, and determining a reference image search window by a characteristic point target window according to the transformation model;
The dynamic pyramid target window and the search window image resolution are processed in a consistent mode, and distortion correction is carried out on the search window according to the transformation model;
The target window and the search window are subjected to dynamic threshold correlation coefficient rough matching and least square fine matching, and the correlation coefficient threshold is from low to high according to pyramid resolution;
the rough difference is removed to obtain correct matching point pairs as a reference image characteristic point set, and the characteristic point set is subjected to traversal matching with the overlapped images;
until the overlapping image traversal is completed.
6. A multi-process large area network image matching device based on a characteristic point library is characterized by comprising:
Feature point library unit: the method comprises the steps of establishing a database according to the resolution of a reference image; uniformly gridding and partitioning the global image according to standard degree intervals to obtain a plurality of gridding blocks; extracting uniformly grid characteristic points of the reference image to obtain longitude and latitude values of the characteristic points; establishing pyramid layer data with each feature point as a center to be used as feature description of the feature point; performing bilinear elevation interpolation according to the longitude and latitude values of the feature points to obtain elevations, and obtaining feature points with feature description, longitude, latitude and elevations; establishing a database table name according to the longitude and latitude integer value of the left lower corner of each grid block; inserting the longitude and latitude values of each characteristic point into corresponding database tables to complete the construction of a characteristic point library;
A connection unit: the method is used for connecting the main process with the characteristic point library;
A subtask unit: the method comprises the steps of traversing and selecting a reference image from a measured image, and determining a reference image range according to RPC parameters of the reference image;
determining a database table positioned in the reference image range in the characteristic point library according to the reference image range and the characteristic point library; characteristic points in the searching range in the determined database table are searched according to the reference image range, and a characteristic point set is formed;
Acquiring all images with overlapping areas with the reference image according to the RPC parameters; packing the reference image, the feature point set and all images overlapped with the reference image as subtasks to form a subtask package; traversing and selecting a reference image from the image of the detection area again, and repeating the process to form another subtask package; cycling until the image traversal of the detection area is completed, and forming a plurality of subtask packages;
Multiprocessing processing unit: the main process is used for pushing all sub-task packages to each sub-process for processing; and the main process receives the return state of each sub-process, and when each sub-process is completed, the main process ends, returns to the total task state and completes image matching.
7. A multi-process large area network image matching device based on a characteristic point library is characterized in that: comprising a memory and a processor; the memory is used for storing a computer program; the processor is configured to implement the multi-process large area network image matching method based on the feature point library according to any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, characterized by: the storage medium stores a computer program which, when executed by a processor, implements the multi-process large area network image matching method based on the feature point library as set forth in any one of claims 1 to 5.
CN202011355652.9A 2020-11-27 2020-11-27 Multi-process large area network image matching method and device based on feature point library Active CN112560868B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011355652.9A CN112560868B (en) 2020-11-27 2020-11-27 Multi-process large area network image matching method and device based on feature point library

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011355652.9A CN112560868B (en) 2020-11-27 2020-11-27 Multi-process large area network image matching method and device based on feature point library

Publications (2)

Publication Number Publication Date
CN112560868A CN112560868A (en) 2021-03-26
CN112560868B true CN112560868B (en) 2024-07-09

Family

ID=75046298

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011355652.9A Active CN112560868B (en) 2020-11-27 2020-11-27 Multi-process large area network image matching method and device based on feature point library

Country Status (1)

Country Link
CN (1) CN112560868B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113255610B (en) * 2021-07-02 2022-02-18 浙江大华技术股份有限公司 Feature base building method, feature retrieval method and related device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663028A (en) * 2012-03-23 2012-09-12 北京师范大学 Method suitable for fast spatially-indexing global digital elevation model and remote sensing image data
CN103115614A (en) * 2013-01-21 2013-05-22 武汉大学 Associated parallel matching method for multi-source multi-track long-strip satellite remote sensing images

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6757445B1 (en) * 2000-10-04 2004-06-29 Pixxures, Inc. Method and apparatus for producing digital orthophotos using sparse stereo configurations and external models
CN103455624B (en) * 2013-09-16 2017-07-25 湖北文理学院 A kind of global multidimensional remote sensing image Map Service of Network implementation method of lightweight
KR101742373B1 (en) * 2015-05-07 2017-06-01 한국과학기술원 Landslide gis database and method for management
CN105387846B (en) * 2015-10-26 2017-11-10 中国农业大学 The ortho-rectification method and system of satellite image
CN105913435B (en) * 2016-04-13 2019-05-28 西安航天天绘数据技术有限公司 A kind of multiscale morphology image matching method and system suitable for big region
KR101965965B1 (en) * 2017-09-05 2019-08-13 순천대학교 산학협력단 A method of automatic geometric correction of digital elevation model made from satellite images and provided rpc
CN108304593B (en) * 2018-04-19 2020-12-29 北京星球时空科技有限公司 Method for interactive display of paper map and electronic map
CN109166143A (en) * 2018-07-06 2019-01-08 航天星图科技(北京)有限公司 A kind of big regional network stereo mapping satellite image matching process
CN110659369B (en) * 2019-03-22 2023-01-17 西安电子科技大学 On-orbit high-precision lightweight global image control point database construction method and system
CN110990516B (en) * 2019-12-13 2023-06-27 中科图新(苏州)科技有限公司 Map data processing method, device and server

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663028A (en) * 2012-03-23 2012-09-12 北京师范大学 Method suitable for fast spatially-indexing global digital elevation model and remote sensing image data
CN103115614A (en) * 2013-01-21 2013-05-22 武汉大学 Associated parallel matching method for multi-source multi-track long-strip satellite remote sensing images

Also Published As

Publication number Publication date
CN112560868A (en) 2021-03-26

Similar Documents

Publication Publication Date Title
Rupnik et al. MicMac–a free, open-source solution for photogrammetry
Rupnik et al. 3D reconstruction from multi-view VHR-satellite images in MicMac
CN106780712B (en) Three-dimensional point cloud generation method combining laser scanning and image matching
Agüera-Vega et al. Effects of point cloud density, interpolation method and grid size on derived Digital Terrain Model accuracy at micro topography level
CN112529946B (en) High-discrete body model optimization method and system, electronic device and readable storage medium
Zhao et al. Geometric-constrained multi-view image matching method based on semi-global optimization
Taylor et al. Monitoring landscape change in the National Parks of England and Wales using aerial photo interpretation and GIS
US7778808B2 (en) Geospatial modeling system providing data thinning of geospatial data points and related methods
CN113538501A (en) Low-altitude image DSM generation building edge refinement method
Debella-Gilo Bare-earth extraction and DTM generation from photogrammetric point clouds including the use of an existing lower-resolution DTM
CN112560868B (en) Multi-process large area network image matching method and device based on feature point library
CN114494371A (en) Optical image and SAR image registration method based on multi-scale phase consistency
Wang et al. Pursuing 3-D scene structures with optical satellite images from affine reconstruction to Euclidean reconstruction
Elkhrachy Feature extraction of laser scan data based on geometric properties
CN112509042A (en) Real-time positioning method and device based on-satellite control point library and storage medium
CN115713548A (en) Automatic registration method for multi-stage live-action three-dimensional model
Buckley et al. Integration, validation and point spacing optimisation of digital elevation models
Awange et al. Fundamentals of photogrammetry
Liu et al. Automatic building height estimation with shadow correction over heterogeneous compact cities using stereo Gaofen-7 data at sub-meter resolution
Wang et al. DEM construction method for slopes using three-dimensional point cloud data based on moving least square theory
CN112561805A (en) Correction method and device for geometric rough correction image
Soycan et al. Digital elevation model production from scanned topographic contour maps via thin plate spline interpolation
Cheng et al. Generation of pixel-level SAR image time series using a locally adaptive matching technique
Sholarin et al. Photogrammetry
Ye et al. Gaussian mixture model of ground filtering based on hierarchical curvature constraints for airborne lidar point clouds

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant