CN112233246B - Satellite image dense matching method and system based on SRTM constraint - Google Patents
Satellite image dense matching method and system based on SRTM constraint Download PDFInfo
- Publication number
- CN112233246B CN112233246B CN202011014959.2A CN202011014959A CN112233246B CN 112233246 B CN112233246 B CN 112233246B CN 202011014959 A CN202011014959 A CN 202011014959A CN 112233246 B CN112233246 B CN 112233246B
- Authority
- CN
- China
- Prior art keywords
- image
- determining
- epipolar line
- matching
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000003384 imaging method Methods 0.000 claims abstract description 13
- 230000006870 function Effects 0.000 claims description 35
- 230000008569 process Effects 0.000 claims description 11
- 241000668842 Lepidosaphes gloverii Species 0.000 claims description 8
- 230000008859 change Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 4
- 230000002238 attenuated effect Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012876 topography Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C7/00—Tracing profiles
- G01C7/02—Tracing profiles of land surfaces
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a satellite image dense matching method and system based on SRTM constraint, wherein the method comprises the following steps: correcting the satellite stereoscopic image into a epipolar stereoscopic pair according to imaging parameters of a satellite sensor; constructing pyramid images according to the epipolar line stereopair; projecting the terrain product of the SRTM into a left image of the pyramid top layer stereoscopic image to generate an initial parallax image; according to the initial parallax image, determining gray feature similarity between homonymous image points in the pyramid top-layer stereoscopic image; accumulating matching cost of the epipolar line stereoscopic image and determining a dense matching result of the pyramid top layer; and taking the dense matching result as a new initial parallax image, and transmitting the initial parallax image to the pyramid at the next stage until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain the three-dimensional point cloud. The invention can improve the reliability and precision of dense matching of the weak texture region, and can be widely applied to the technical field of remote sensing.
Description
Technical Field
The invention relates to the technical field of remote sensing, in particular to a satellite image dense matching method and system based on SRTM constraint.
Background
Dense matching of satellite stereo images is a process of finding the same name image point between two satellite images pixel by pixel. The corresponding three-dimensional point cloud can be calculated through parameters such as the position, the gesture and the like of the satellite sensor. Therefore, the dense matching of satellite stereoscopic images is one of core technologies for digitizing large-scale three-dimensional information, and can be widely applied to the fields of mapping and drawing, military national defense, virtual reality, game animation and the like.
The dense matching technology of satellite stereo image mainly searches for the same name image point according to the gray scale characteristics on the image. The more similar the gray scale features, the greater the likelihood of being homonymous points. However, in the weak texture region (such as snow mountain, snow field, lake, etc.), the gray scale characteristics on the satellite image are not obvious, and the mismatching points are serious, so that the accuracy and the display effect of the three-dimensional reconstruction of the weak texture region are seriously reduced.
SRTM (Shuttle Radar Topography Mission) is a product of global topographic mapping by space plane radar. SRTM topography products are disclosed for free worldwide with a resolution of 30 meters. Because the space plane radar adopts an active imaging technology, the precision of the SRTM terrain product is not affected by regional texture factors. A stable and reliable three-dimensional model product can be obtained even in a region of weak texture. However, the resolution of SRTM terrain products is low (30 meters), and the high-precision, high-resolution mapping requirements are far from being met.
Disclosure of Invention
In view of this, the embodiment of the invention provides a satellite image dense matching method and system based on SRTM constraint, so as to enhance the matching reliability of the weak texture region and improve the three-dimensional reconstruction accuracy of the weak texture region.
The first aspect of the invention provides a satellite image dense matching method based on SRTM constraint, comprising the following steps:
correcting the satellite stereoscopic image into a epipolar stereoscopic pair according to imaging parameters of a satellite sensor;
constructing an image of a pyramid according to the epipolar stereopair, wherein the top layer resolution of the pyramid is consistent with the SRTM resolution;
projecting the terrain product of the SRTM into a left image of the pyramid top layer stereoscopic image to generate an initial parallax image;
according to the constraint dense matching process of the initial parallax map, gray feature similarity between homonymous image points is determined in the pyramid top-layer stereoscopic image;
accumulating the matching cost of the epipolar line stereoscopic image and determining a dense matching result of the pyramid top layer;
and taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain the three-dimensional point cloud.
In some embodiments, the projecting the topographic product of the SRTM into the left image of the pyramid top layer stereoscopic image generates an initial disparity map comprising:
acquiring an RPC model of the pyramid top layer image;
projecting the SRTM topographic product to a left image in the pyramid top layer stereoscopic image to generate an initial parallax image;
wherein the form of the RPC model is expressed as:
P＝(lat-LAT_OFF)/LAT_SCALE；
L＝(long-LONG_OFF)/LONG_SCALE；
H＝(hei-HEI_OFF)/HEI_SCALE；
wherein row represents the row coordinates of the image points on the satellite image; col represents column coordinates of image points on the satellite image; line_scale represents a SCALE factor in the row direction; samp_scale represents a SCALE factor in the column direction; line_off represents an offset factor in the row direction; samp_off represents an offset factor in the column direction; LAT_SCALE represents a SCALE factor in latitude; long_scale represents a SCALE factor on longitude; HEI_SCALE represents an elevational SCALE factor; lat_off represents an inexpensive factor in latitude; long_off represents an inexpensive factor in longitude; HEI_OFF represents an inexpensive factor in elevation, respectively; p represents the object point coordinates of the normalized latitude; l represents the object point coordinates of the normalized longitude; h represents the object point coordinates of the normalized elevation; line_num_coef i 、LINE_DEN_COEF i 、SAMP_NUM_COEF i And samp_den_coef i Parameters respectively representing the RPC model; p is p i (P, L, H) represents a cubic polynomial for P, L and H.
In some embodiments, the determining gray feature similarity between the homonymous pixels in the pyramid top-level stereoscopic image according to the initial disparity map constraint dense matching process includes:
determining a left epipolar line image and a right epipolar line image;
determining a window operator;
determining pixels on the left epipolar line image;
determining an estimated disparity for the pixel;
determining pixels on the right epipolar line image according to the pixels on the left epipolar line image and the estimated parallax;
calculating the matching cost of the initial parallax map constraint according to the window operator, the pixels on the left epipolar line image, the estimated parallax and the pixels on the right epipolar line image;
and calculating the gray feature similarity between the homonymous image points in the pyramid top-layer stereoscopic image according to the matching cost of the initial parallax image constraint.
In some embodiments, the accumulating the matching cost of the epipolar stereo image, determining the dense matching result of the pyramid top layer includes:
determining a first order penalty term and a second order penalty term;
dynamically adjusting the first order penalty term and the second order penalty term through an initial disparity map, and determining a target energy function of a semi-global dense matching method;
and determining a dense matching result according to the target energy function.
In some embodiments, the expression of the target energy function is:
e represents a target energy function of dense matching of the epipolar line stereoscopic images; d represents a dense matching disparity map; cost (p, d) p ) Representing the corresponding disparity d of pixel p p Matching costs of (a); i L Representing left epipolar line images; n represents a set of adjacent pixels; q represents an adjacent pixel of the pixel p; p (P) 1 Representing a first order penalty term coefficient; p (P) 2 Representing a second order penalty term coefficient; d, d p Representing the parallax of pixel p; d, d q Representing the parallax of pixel q;representing the values of pixels p, q on the initial disparity map, respectively; t [. Cndot.]A boolean function is represented, whose value is 1 when the condition in brackets is true, and 0 otherwise.
In some embodiments, the dynamically adjusting the first order penalty term and the second order penalty term by the initial disparity map includes:
judging whether the parallax change of the adjacent pixels accords with the initial parallax map or not, if so, not adjusting the first-order penalty item; otherwise, correspondingly adjusting the first-order penalty term;
judging whether parallaxes of adjacent pixels in the initial parallax map are consistent, if yes, enhancing the second first order penalty term, and penalty parallax change of the adjacent pixels; conversely, the second order penalty term is attenuated, thereby encouraging parallax changes for adjacent pixels.
In some embodiments, the matching cost is calculated by the following formula:
wherein Cost (p, dp) represents the parallax d corresponding to pixel p p Matching costs of (a);representing left epipolar line image I L A matching window operator on the window; />Representing the right epipolar line image I R A matching window operator on the window; w represents the size of the initial disparity map constraint; />Indicating that pixel p is inInitial disparity map D 0 A value of (a); />Representing the mean value of the gray gradient in the matching window; alpha represents a normalization factor of the gray gradient; left epipolar line image is I L The right epipolar line image is I R M represents a window operator; p represents a pixel on the left epipolar line image; d, d p Representing an estimated disparity for pixel p; p-d p Representing pixels on the right epipolar line image.
A second aspect of the present invention provides a satellite image dense matching system based on SRTM constraints, comprising:
the imaging module is used for correcting the satellite stereoscopic image into a nuclear line stereoscopic pair according to imaging parameters of the satellite sensor;
the construction module is used for constructing images of a pyramid according to the epipolar line stereopair, and the top layer resolution of the pyramid is consistent with the SRTM resolution;
the initial parallax map generation module is used for projecting the terrain product of the SRTM into the left image of the pyramid top layer stereoscopic image to generate an initial parallax map;
the feature determining module is used for determining gray feature similarity between homonymous image points in the pyramid top-layer stereoscopic image according to the initial parallax image constraint dense matching process;
the semi-global dense matching module is used for accumulating the matching cost of the epipolar line stereoscopic image and determining the dense matching result of the pyramid top layer;
and taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain the three-dimensional point cloud.
In some embodiments, the semi-global dense matching module comprises:
the first determining unit is used for determining a first order penalty term and a second first order penalty term;
the dynamic adjustment unit is used for dynamically adjusting the first order penalty term and the second order penalty term through an initial disparity map and determining a target energy function of the semi-global dense matching method;
and the second determining unit is used for determining a dense matching result according to the target energy function.
A third aspect of an embodiment of the present invention provides a satellite image dense matching system based on SRTM constraints, including a processor and a memory;
the memory is used for storing programs;
the processor executes the program to implement the method according to the first aspect.
According to imaging parameters of a satellite sensor, correcting a satellite stereoscopic image into a nuclear line stereoscopic pair; constructing an image of a pyramid according to the epipolar stereopair, wherein the top layer resolution of the pyramid is consistent with the SRTM resolution; projecting the terrain product of the SRTM into a left image of the pyramid top layer stereoscopic image to generate an initial parallax image; according to the constraint dense matching process of the initial parallax map, gray feature similarity between homonymous image points is determined in the pyramid top-layer stereoscopic image; accumulating the matching cost of the epipolar line stereoscopic image and determining a dense matching result of the pyramid top layer; and taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain the three-dimensional point cloud. The embodiment of the invention is beneficial to solving the problem of dense matching of satellite stereoscopic images in the weak texture region, and improves the reliability and precision of dense matching of the weak texture region by introducing SRTM constraint.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of the overall steps provided by the present invention;
fig. 2 is a schematic diagram of a satellite epipolar stereopair pyramid provided by an embodiment of the present invention;
fig. 3 is a schematic diagram of a dense matching result of satellite stereoscopic images according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of an embodiment of the present invention.
Detailed Description
The invention is further explained and illustrated below with reference to the drawing and the specific embodiments of the present specification. The step numbers in the embodiments of the present invention are set for convenience of illustration, and the order of steps is not limited in any way, and the execution order of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.
Aiming at the problems existing in the prior art, the embodiment of the invention provides a satellite image dense matching method based on SRTM constraint, as shown in figure 1, which comprises the following steps:
s1, correcting a satellite stereoscopic image into a nuclear line stereoscopic pair according to imaging parameters of a satellite sensor;
s2, constructing a pyramid image according to the epipolar line stereopair, wherein the top layer resolution of the pyramid is consistent with the SRTM resolution;
s3, projecting the terrain product of the SRTM into a left image of the pyramid top layer stereoscopic image to generate an initial parallax image;
step S3 of the embodiment of the present invention includes S31 and S32:
s31, acquiring an RPC model of a pyramid top-layer image;
s32, projecting the SRTM topographic product to a left image in the pyramid top layer stereoscopic image to generate an initial parallax image;
wherein the form of the RPC model is expressed as:
P＝(lat-LAT_OFF)/LAT_SCALE；
L＝(long-LONG_OFF)/LONG_SCALE；
H＝(hei-HEI_OFF)/HEI_SCALE；
wherein row represents the row coordinates of the image points on the satellite image; col represents column coordinates of image points on the satellite image; line_scale represents a SCALE factor in the row direction; samp_scale represents a SCALE factor in the column direction; line_off represents an offset factor in the row direction; samp_off represents an offset factor in the column direction; LAT_SCALE represents a SCALE factor in latitude; long_scale represents a SCALE factor on longitude; HEI_SCALE represents an elevational SCALE factor; lat_off represents an inexpensive factor in latitude; long_off represents an inexpensive factor in longitude; HEI_OFF represents an inexpensive factor in elevation, respectively; p represents the object point coordinates of the normalized latitude; l represents the object point coordinates of the normalized longitude; h represents the object point coordinates of the normalized elevation; line_num_coef i 、LINE_DEN_COEF i 、SAMP_NUM_COEF i And samp_den_coef i Parameters respectively representing the RPC model; p is p i (P, L, H) represents a cubic polynomial for P, L and H.
S4, determining gray feature similarity between homonymous image points in the pyramid top-layer stereoscopic image according to the initial parallax image constraint dense matching process;
step S4 of the embodiment of the present invention includes S41-S47:
s41, determining a left epipolar line image and a right epipolar line image;
s42, determining a window operator;
s43, determining pixels on the left epipolar line image;
s44, determining estimated parallax of the pixel;
s45, determining the pixels on the right epipolar line image according to the pixels on the left epipolar line image and the estimated parallax;
s46, calculating the matching cost of the initial parallax map constraint according to the window operator, the pixels on the left epipolar line image, the estimated parallax and the pixels on the right epipolar line image;
and S47, calculating the gray feature similarity between the same-name image points in the pyramid top-layer stereoscopic image according to the matching cost of the initial parallax image constraint.
S5, accumulating the matching cost of the epipolar line stereoscopic image and determining a dense matching result of the pyramid top layer;
step S5 of the embodiment of the present invention includes S51-S53:
s51, determining a first order penalty term and a second order penalty term;
s52, dynamically adjusting the first order penalty term and the second order penalty term through an initial disparity map, and determining a target energy function of a semi-global dense matching method;
specifically, judging whether the parallax change of the adjacent pixels accords with the initial parallax map, if so, not adjusting the first-order penalty item; otherwise, correspondingly adjusting the first-order penalty term;
judging whether parallaxes of adjacent pixels in the initial parallax map are consistent, if yes, enhancing the second first order penalty term, and penalty parallax change of the adjacent pixels; conversely, the second order penalty term is attenuated, thereby encouraging parallax changes for adjacent pixels.
The expression of the target energy function is:
e represents a target energy function of dense matching of the epipolar line stereoscopic images; d represents a dense matching disparity map; cost (p, d) p ) Representing the corresponding disparity d of pixel p p Matching costs of (a); i L Representing left epipolar line images; n represents a set of adjacent pixels; q represents an adjacent pixel of the pixel p; p (P) 1 Representing a first order penalty term coefficient;P 2 representing a second order penalty term coefficient; d, d p Representing the parallax of pixel p; d, d q Representing the parallax of pixel q;representing the values of pixels p, q on the initial disparity map, respectively; t [. Cndot.]A boolean function is represented, whose value is 1 when the condition in brackets is true, and 0 otherwise.
S53, determining a dense matching result according to the target energy function.
And S6, taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain the three-dimensional point cloud.
The method of the present invention is described in detail below with reference to a pyramid schematic diagram of a satellite epipolar stereopair and a dense matching result schematic diagram of a satellite stereopair, and in this embodiment, as shown in fig. 4, the method includes the following steps:
step 1: the satellite stereopsis is corrected into a epipolar stereopair according to the imaging parameters RPC (Rational Polynomial Coefficients) of the satellite sensor. In the epipolar stereo image, the row coordinates of the same-name image points are equal, so that the dense matching of the satellite images can be converted from a two-dimensional problem to a one-dimensional problem, and the dense matching speed is greatly increased. The correction method of the satellite nuclear line stereoscopic image is a common method in the field, and the invention is not repeated. The correction method of the satellite epipolar stereo image can be seen in the article of a method for generating an approximate epipolar image of a linear array push-broom satellite stereo image based on a projection reference plane.
Step 2: image pyramids are constructed for the satellite epipolar stereopair, respectively, as shown in fig. 2. The pyramid top layer resolution remains consistent with the SRTM resolution (i.e., 30 meters). Then, according to the RPC model of the pyramid top layer image, the SRTM terrain product is projected to the left image in the pyramid top layer stereoscopic image to generate an initial parallax image D 0 . The RPC model is a mathematical model for describing the geometric relationship between the three-dimensional point of the object space and the two-dimensional point of the image space on the satellite image, and the concrete shape of the model is that of the modelThe formula is as follows.
P＝(lat-LAT_OFF)/LAT_SCALE
L＝(long-LONG_OFF)/LONG_SCALE
H＝(hei-HEI_OFF)/HEI_SCALE
In the above description, row and col respectively represent row coordinates and column coordinates of image points on the satellite image; LINE_SCALE and SAMP_SCALE respectively represent SCALE factors in the row direction and the column direction; line_off and samp_off represent offset factors in the row direction, respectively; LAT_SCALE, LONG_SCALE and HEI_SCALE represent SCALE factors in latitude, longitude and elevation, respectively; LAT_OFF, LONG_OFF and HEI_OFF represent inexpensive factors in latitude, longitude and elevation, respectively; p, L and H represent normalized object point coordinates (latitude, longitude, and elevation), respectively; line_num_coef i 、LINE_DEN_COEF i 、SAMP_NUM_COEF i And samp_den_coef i 80 parameters respectively representing the RPC model; p is p i (P, L, H) represents a cubic polynomial for P, L and H.
Step 3: in the pyramid top-layer stereoscopic image, a fixed matching window with the size of 9×9 pixels is adopted to calculate the gray feature similarity between the homonymous pixels, and the gray feature similarity is used as the matching cost between the homonymous pixels. However, in the weak texture region, there is no obvious gray scale feature within the fixed matching window, resulting in inaccurate matching cost calculation. Therefore, the invention introduces the initial disparity map D in the cost calculation process 0 As a constraint, and adaptively adjusts the constraint size according to gray scale characteristics within the matching window. The richer the texture within the matching window, the smaller the constraint; otherwise, the greater the constraint. Let the left epipolar line image be I L The right epipolar line image is I R M represents a window operator, and p represents a pixel on the left epipolar line image; d, d p Representing an estimate of pixel pParallax; p-d p Representing pixels on the right epipolar line image, then a matching cost calculation based on the initial disparity map constraint is performed as shown in the following equation.
In the above formula, cost (p, d p ) Representing the corresponding disparity d of pixel p p Matching costs of (a);representing left epipolar line image I L A matching window operator on the window; />Representing the right epipolar line image I R A matching window operator on the window; w represents the size of the initial disparity map constraint; />Representing pixel p in initial disparity map D 0 A value of (a); />Representing the mean value of the gray gradient in the matching window; alpha represents the normalization factor of the gray gradient.
And (3) adopting a traditional matching window operator as a measure, and adopting the above method to calculate the matching cost of each pixel on the left epipolar line image, so as to form a cost matrix. The traditional window operators have a plurality of types, including Census operators, sobel operators, ZNCC operators, HOG operators and the like, and the operators can obtain a good matching result. For specific formulas of these window operators, see article Evaluation of stereo matching costs on Images with radiometric differences; the present invention will not be described in detail.
Step 4: for satelliteThe matching cost of the epipolar stereo images is accumulated, so that a more accurate matching result is obtained. The invention adopts the traditional semi-global dense matching method to obtain the accurate dense matching parallax map. In the matching cost accumulation process, the semi-global dense matching method establishes two first order punishment items P 1 And P 2 . Wherein P is 1 For penalizing small (1 pixel) disparity variations between adjacent pixels; and P is 2 For penalizing large (greater than 1 pixel) disparity variations between adjacent pixels. But in the region of weak texture and bevel, the conventional penalty term P 1 And P 2 Which can lead to serious mismatch problems. Thus, the present invention is based on the initial disparity map D 0 The penalty term is adaptively adjusted. When the parallax change of the adjacent pixels is matched with the initial parallax map, no penalty exists; otherwise, a penalty is given adaptively, thereby modifying the energy function of the semi-global dense matching method to be shown in the following equation. The method can be more suitable for matching the inclined surface areas, so that the matching precision of the inclined surface areas is improved.
In the above formula, E represents a target energy function of dense matching of epipolar stereo images; d represents a dense matching disparity map; cost (p, d) p ) Representing the corresponding disparity d of pixel p p Matching costs of (a); IL represents left epipolar line image; n represents a set of adjacent pixels; q represents an adjacent pixel of the pixel p; p (P) 1 Representing a first order penalty term coefficient; p (P) 2 Representing a second order penalty term coefficient; d, d p Representing the parallax of pixel p; d, d q Representing the parallax of pixel q;representing the values of pixels p, q on the initial disparity map, respectively; t [. Cndot.]A boolean function is represented, whose value is 1 when the condition in brackets is true, and 0 otherwise.
The optimal solution of the target energy function is the dense matching result of the satellite epipolar line stereoscopic image. The global energy function can be converted into one-dimensional optimization problems in multiple directions by adopting a semi-global dense matching mode, and an approximate optimal solution is obtained by adopting a dynamic programming mode, wherein the approximate optimal solution is shown in the following formula:
in the above, L r Representing a one-dimensional optimization function on a scanning line in the r direction; cost (p, d) p ) Representing the corresponding disparity d of pixel p p Matching costs of (a); p-r represents the previous pixel of pixel p; d, d p-r Representing the disparity of the pixel p-r; p (P) 1 Representing a first order penalty term coefficient; p (P) 2 Representing a second order penalty term coefficient;the values of the pixels p, p-r on the initial disparity map are indicated, respectively.
Step 5: taking the dense matching result of the pyramid top layer as a new initial parallax map D 0 And transferring to the next level pyramid, and repeating the step 3 and the step 4 until the pyramid bottom layer is calculated. Finally, a high-resolution three-dimensional point cloud is obtained, as shown in fig. 3.
In addition, the invention also provides a satellite image dense matching system based on SRTM constraint, which comprises:
the imaging module is used for correcting the satellite stereoscopic image into a nuclear line stereoscopic pair according to imaging parameters of the satellite sensor;
the construction module is used for constructing images of a pyramid according to the epipolar line stereopair, and the top layer resolution of the pyramid is consistent with the SRTM resolution;
the initial parallax map generation module is used for projecting the terrain product of the SRTM into the left image of the pyramid top layer stereoscopic image to generate an initial parallax map;
the feature determining module is used for determining gray feature similarity between homonymous image points in the pyramid top-layer stereoscopic image according to the initial parallax image constraint dense matching process;
the semi-global dense matching module is used for accumulating the matching cost of the epipolar line stereoscopic image and determining the dense matching result of the pyramid top layer;
and taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain the three-dimensional point cloud.
In some embodiments, the semi-global dense matching module comprises:
the first determining unit is used for determining a first order penalty term and a second first order penalty term;
the dynamic adjustment unit is used for dynamically adjusting the first order penalty term and the second order penalty term through an initial disparity map and determining a target energy function of the semi-global dense matching method;
and the second determining unit is used for determining a dense matching result according to the target energy function.
The invention also provides a satellite image dense matching system based on SRTM constraint, which comprises a processor and a memory;
the memory is used for storing programs;
the processor executes the program to implement the method as shown in fig. 1.
The content in the method embodiment is applicable to the system embodiment, the functions specifically realized by the system embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.
Further, an embodiment of the present invention provides a storage medium storing a program that is executed by a processor to perform the method shown in fig. 1.
In some alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, and in which sub-operations described as part of a larger operation are performed independently.
Furthermore, while the invention is described in the context of functional modules, it should be appreciated that, unless otherwise indicated, one or more of the described functions and/or features may be integrated in a single physical device and/or software module or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be apparent to those skilled in the art from consideration of their attributes, functions and internal relationships. Accordingly, one of ordinary skill in the art can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and are not intended to be limiting upon the scope of the invention, which is to be defined in the appended claims and their full scope of equivalents.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, randomAccess Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the embodiments described above, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present invention, and these equivalent modifications and substitutions are intended to be included in the scope of the present invention as defined in the appended claims.
Claims (8)
1. The satellite image dense matching method based on SRTM constraint is characterized by comprising the following steps:
correcting the satellite stereoscopic image into a epipolar stereoscopic pair according to imaging parameters of a satellite sensor;
constructing an image of a pyramid according to the epipolar stereopair, wherein the top layer resolution of the pyramid is consistent with the SRTM resolution;
projecting the terrain product of the SRTM into a left image of the pyramid top layer stereoscopic image to generate an initial parallax image;
according to the constraint dense matching process of the initial parallax map, gray feature similarity between homonymous image points is determined in the pyramid top-layer stereoscopic image;
accumulating the matching cost of the epipolar stereopair and determining the dense matching result of the pyramid top layer;
taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain a three-dimensional point cloud;
the determining the gray feature similarity between the homonymous image points in the pyramid top-layer stereoscopic image according to the initial disparity map constraint dense matching process comprises the following steps:
determining a left epipolar line image and a right epipolar line image;
determining a window operator;
determining pixels on the left epipolar line image;
determining an estimated disparity for the pixel;
determining pixels on the right epipolar line image according to the pixels on the left epipolar line image and the estimated parallax;
calculating the matching cost of the initial parallax map constraint according to the window operator, the pixels on the left epipolar line image, the estimated parallax and the pixels on the right epipolar line image;
according to the matching cost of the initial parallax map constraint, calculating the gray feature similarity between the homonymous image points in the pyramid top-layer stereoscopic image;
the calculation formula of the matching cost is as follows:
wherein Cost (p, d p ) Representing the corresponding disparity d of pixel p p Matching costs of (a);representing left epipolar line image I L A matching window operator on the window; />Representing the right epipolar line image I R A matching window operator on the window; w represents the size of the initial disparity map constraint; />Representing pixel p in initial disparity map D 0 A value of (a); />Representing the mean value of the gray gradient in the matching window; alpha represents a normalization factor of the gray gradient; left epipolar line image is I L The right epipolar line image is I R M represents a window operator; p represents a pixel on the left epipolar line image; d, d p Representing an estimated disparity for pixel p; p-d p Representing pixels on the right epipolar line image.
2. The SRTM constraint-based satellite image dense matching method of claim 1, wherein the projecting the topographic product of SRTM into the left image of the pyramid top layer stereoscopic image generates an initial disparity map comprising:
acquiring an RPC model of the pyramid top layer image;
projecting the SRTM topographic product to a left image in the pyramid top layer stereoscopic image to generate an initial parallax image;
wherein the form of the RPC model is expressed as:
P＝(lat-LAT_OFF)/LAT_SCALE；
L＝(long-LONG_OFF)/LONG_SCALE；
H＝(hei-HEI_OFF)/HEI_SCALE；
wherein row represents the row coordinates of the image points on the satellite image; col represents column coordinates of image points on the satellite image; line_scale represents a SCALE factor in the row direction; samp_scale represents a SCALE factor in the column direction; line_off represents an offset factor in the row direction; samp_off represents an offset factor in the column direction; LAT_SCALE represents a SCALE factor in latitude; long_scale represents a SCALE factor on longitude; HEI_SCALE represents an elevational SCALE factor; lat_off represents an inexpensive factor in latitude; long_off represents an inexpensive factor in longitude; HEI_OFF represents an inexpensive factor in elevation, respectively; p represents the object point coordinates of the normalized latitude; l represents the object point coordinates of the normalized longitude; h represents the object point coordinates of the normalized elevation; line_num_coef i 、LINE_DEN_COEF i 、SAMP_NUM_COEF i And samp_den_coef i Parameters respectively representing the RPC model; p is p i (P, L, H) represents a cubic polynomial for P, L and H.
3. The SRTM constraint-based satellite image dense matching method of claim 1, wherein the accumulating the matching cost of the epipolar stereopair, determining the dense matching result of the pyramid top layer comprises:
determining a first order penalty term and a second order penalty term;
dynamically adjusting the first order penalty term and the second order penalty term through an initial disparity map, and determining a target energy function of a semi-global dense matching method;
and determining a dense matching result according to the target energy function.
4. The SRTM constraint based satellite image dense matching method of claim 3, wherein the expression of the target energy function is:
wherein E represents a target energy function of dense matching of epipolar stereopair; d represents a dense matching disparity map; cost (p, d) p ) Representing the corresponding disparity d of pixel p p Matching costs of (a); i L Representing left epipolar line images; n represents a set of adjacent pixels; q represents an adjacent pixel of the pixel p; p (P) 1 Representing a first order penalty term coefficient; p (P) 2 Representing a second order penalty term coefficient; d, d p Representing the parallax of pixel p; d, d q Representing the parallax of pixel q;representing the values of pixels p, q on the initial disparity map, respectively; t [. Cndot.]A boolean function is represented, whose value is 1 when the condition in brackets is true, and 0 otherwise.
5. The SRTM constraint based satellite video dense matching method of claim 3, wherein the dynamically adjusting the first order penalty term and the second order penalty term with the initial disparity map comprises:
judging whether the parallax change of the adjacent pixels accords with the initial parallax map or not, if so, not adjusting the first-order penalty item; otherwise, correspondingly adjusting the first-order penalty term;
judging whether parallaxes of adjacent pixels in the initial parallax map are consistent, if yes, enhancing the second first order penalty term, and penalty parallax change of the adjacent pixels; conversely, the second order penalty term is attenuated, thereby encouraging parallax changes for adjacent pixels.
6. Satellite image dense matching system based on SRTM constraint, which is characterized by comprising:
the imaging module is used for correcting the satellite stereoscopic image into a nuclear line stereoscopic pair according to imaging parameters of the satellite sensor;
the construction module is used for constructing images of a pyramid according to the epipolar line stereopair, and the top layer resolution of the pyramid is consistent with the SRTM resolution;
the initial parallax map generation module is used for projecting the terrain product of the SRTM into the left image of the pyramid top layer stereoscopic image to generate an initial parallax map;
the feature determining module is used for determining gray feature similarity between homonymous image points in the pyramid top-layer stereoscopic image according to the initial parallax image constraint dense matching process;
the semi-global dense matching module is used for accumulating the matching cost of the epipolar line stereopair and determining the dense matching result of the pyramid top layer;
taking the dense matching result as a new initial parallax image, transmitting the initial parallax image to a pyramid at the next stage, and repeating the steps until the dense matching result of the bottom layer of the pyramid is determined, so as to obtain a three-dimensional point cloud;
the feature determining module is specifically configured to:
determining a left epipolar line image and a right epipolar line image;
determining a window operator;
determining pixels on the left epipolar line image;
determining an estimated disparity for the pixel;
determining pixels on the right epipolar line image according to the pixels on the left epipolar line image and the estimated parallax;
calculating the matching cost of the initial parallax map constraint according to the window operator, the pixels on the left epipolar line image, the estimated parallax and the pixels on the right epipolar line image;
according to the matching cost of the initial parallax map constraint, calculating the gray feature similarity between the homonymous image points in the pyramid top-layer stereoscopic image;
the calculation formula of the matching cost is as follows:
wherein Cost (p, d p ) Representing the corresponding disparity d of pixel p p Matching costs of (a);representing left epipolar line image I L A matching window operator on the window; />Representing the right epipolar line image I R A matching window operator on the window; w represents the size of the initial disparity map constraint; />Representing pixel p in initial disparity map D 0 A value of (a); />Representing the mean value of the gray gradient in the matching window; alpha represents a normalization factor of the gray gradient; left epipolar line image is I L The right epipolar line image is I R M represents a window operator; p represents a pixel on the left epipolar line image; d, d p Representing an estimated disparity for pixel p; p-d p Representing pixels on the right epipolar line image.
7. The SRTM constraint based satellite image dense matching system of claim 6, wherein the semi-global dense matching module comprises:
the first determining unit is used for determining a first order penalty term and a second first order penalty term;
the dynamic adjustment unit is used for dynamically adjusting the first order penalty term and the second order penalty term through an initial disparity map and determining a target energy function of the semi-global dense matching method;
and the second determining unit is used for determining a dense matching result according to the target energy function.
8. The satellite image dense matching system based on SRTM constraint is characterized by comprising a processor and a memory;
the memory is used for storing programs;
the processor executing the program to implement the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011014959.2A CN112233246B (en) | 2020-09-24 | 2020-09-24 | Satellite image dense matching method and system based on SRTM constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011014959.2A CN112233246B (en) | 2020-09-24 | 2020-09-24 | Satellite image dense matching method and system based on SRTM constraint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112233246A CN112233246A (en) | 2021-01-15 |
| CN112233246B true CN112233246B (en) | 2024-04-16 |
Family
ID=74108147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011014959.2A Active CN112233246B (en) | 2020-09-24 | 2020-09-24 | Satellite image dense matching method and system based on SRTM constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112233246B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113965742B (en) * | 2021-02-28 | 2022-04-19 | 北京中科慧眼科技有限公司 | Dense disparity map extraction method and system based on multi-sensor fusion and intelligent terminal |
| CN115689965B (en) * | 2022-12-30 | 2023-03-21 | 武汉大学 | Whole-scene satellite image deep learning dense matching oriented multi-level parallax fusion method |
| CN116299596B (en) * | 2023-05-15 | 2023-08-01 | 中山大学 | Marine precise single-point positioning method considering station baseline length and troposphere constraint |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105160702A (en) * | 2015-08-20 | 2015-12-16 | 武汉大学 | Stereoscopic image dense matching method and system based on LiDAR point cloud assistance |
| CN105225233A (en) * | 2015-09-15 | 2016-01-06 | 武汉大学 | A kind of stereopsis dense Stereo Matching method and system expanded based on two classes |
| CN107170000A (en) * | 2017-04-18 | 2017-09-15 | 武汉市工程科学技术研究院 | The stereopsis dense Stereo Matching method optimized based on global block |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0125774D0 (en) * | 2001-10-26 | 2001-12-19 | Cableform Ltd | Method and apparatus for image matching |
-
2020
- 2020-09-24 CN CN202011014959.2A patent/CN112233246B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105160702A (en) * | 2015-08-20 | 2015-12-16 | 武汉大学 | Stereoscopic image dense matching method and system based on LiDAR point cloud assistance |
| CN105225233A (en) * | 2015-09-15 | 2016-01-06 | 武汉大学 | A kind of stereopsis dense Stereo Matching method and system expanded based on two classes |
| CN107170000A (en) * | 2017-04-18 | 2017-09-15 | 武汉市工程科学技术研究院 | The stereopsis dense Stereo Matching method optimized based on global block |
Non-Patent Citations (1)
| Title |
|---|
| 基于RFM的高分辨率卫星遥感影像自动匹配研究;季顺平;袁修孝;;测绘学报(06);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112233246A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112233246B (en) | Satellite image dense matching method and system based on SRTM constraint | |
| Zhao et al. | Geometric-constrained multi-view image matching method based on semi-global optimization | |
| US9208612B2 (en) | Systems and methods that generate height map models for efficient three dimensional reconstruction from depth information | |
| CN104820991B (en) | A kind of multiple soft-constraint solid matching method based on cost matrix | |
| KR101554241B1 (en) | A method for depth map quality enhancement of defective pixel depth data values in a three-dimensional image | |
| JP4685313B2 (en) | Method for processing passive volumetric image of any aspect | |
| WO2018127007A1 (en) | Depth image acquisition method and system | |
| CN106780712B (en) | Three-dimensional point cloud generation method combining laser scanning and image matching | |
| KR102249769B1 (en) | Estimation method of 3D coordinate value for each pixel of 2D image and autonomous driving information estimation method using the same | |
| KR20130138247A (en) | Rapid 3d modeling | |
| CN108876861B (en) | Stereo matching method for extraterrestrial celestial body patrolling device | |
| CN108399631B (en) | Scale invariance oblique image multi-view dense matching method | |
| CN110223222A (en) | Image split-joint method, image splicing device and computer readable storage medium | |
| EP2469501A2 (en) | Method for determining a positioning error in a georeferenced image and corresponding device | |
| CN113506372A (en) | Environment reconstruction method and device | |
| CN117745537B (en) | Tunnel equipment temperature detection method, device, computer equipment and storage medium | |
| CN113281779B (en) | 3D object rapid detection method, device, equipment and medium | |
| Esteban et al. | Fit3d toolbox: multiple view geometry and 3d reconstruction for matlab | |
| CN115719320B (en) | Tilt correction dense matching method based on remote sensing image | |
| GB2569609A (en) | Method and device for digital 3D reconstruction | |
| Zhu et al. | Triangulation of well-defined points as a constraint for reliable image matching | |
| CN114913500B (en) | Pose determination method and device, computer equipment and storage medium | |
| CN102236893A (en) | Space-position-forecast-based corresponding image point matching method for lunar surface image | |
| CN116246017A (en) | Sea wave three-dimensional reconstruction method based on binocular image data | |
| CN112258635B (en) | Three-dimensional reconstruction method and device based on improved binocular matching SAD algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |