CN115719407A - Distributed multi-view stereo reconstruction method for large-scale aerial images - Google Patents

Distributed multi-view stereo reconstruction method for large-scale aerial images Download PDF

Info

Publication number
CN115719407A
CN115719407A CN202310011438.9A CN202310011438A CN115719407A CN 115719407 A CN115719407 A CN 115719407A CN 202310011438 A CN202310011438 A CN 202310011438A CN 115719407 A CN115719407 A CN 115719407A
Authority
CN
China
Prior art keywords
images
point cloud
calculating
image
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310011438.9A
Other languages
Chinese (zh)
Other versions
CN115719407B (en
Inventor
曹明伟
王子洋
赵海峰
孙登第
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui University
Original Assignee
Anhui University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui University filed Critical Anhui University
Priority to CN202310011438.9A priority Critical patent/CN115719407B/en
Publication of CN115719407A publication Critical patent/CN115719407A/en
Application granted granted Critical
Publication of CN115719407B publication Critical patent/CN115719407B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Image Processing (AREA)

Abstract

The invention discloses a distributed multi-view stereo reconstruction method facing a large-scale aerial image. The method makes full use of the regionality among large-scale aerial images, converts the multi-view stereo reconstruction problem of a large-scale scene into a small-scale multi-view stereo reconstruction problem which can be solved on a low-performance computer, improves the time efficiency of three-dimensional reconstruction, and reduces the cost of three-dimensional reconstruction.

Description

Distributed multi-view stereo reconstruction method for large-scale aerial images
Technical Field
The invention relates to computer vision and image processing technology, in particular to a distributed multi-view stereo reconstruction method for large-scale aerial images.
Background
Multi-view Stereo (MVS) is a technique for computing a scene dense point cloud model from image data, usually with the output information of a Structure from Motion (SfM), i.e., a sparse point cloud model and camera parameters, as input information to the MVS. Currently, great research progress is made on the multi-view stereo reconstruction problem of small-scale image data (e.g., small-scale scene image data acquired by a handheld camera), however, for a large-scale outdoor scene, the existing multi-view stereo reconstruction method needs to be further improved. In addition, with the popularization of consumer-grade unmanned aerial vehicle equipment, large-scale data of outdoor scenes can be acquired easily. The existing multi-view stereo reconstruction method mainly has the following challenges when processing large-scale aerial image data: (a) The method is very time-consuming, the multi-view stereo reconstruction process is very time-consuming, and particularly when outdoor large-scale aerial image data are processed, the existing multi-view stereo reconstruction method cannot calculate a dense point cloud model within limited time and is difficult to meet the timeliness requirement of a high-level application system; (b) The storage overflows, the multi-view stereo reconstruction method has a large demand on a computer memory, and particularly when the data volume of an aerial image is large, the single-version multi-view stereo reconstruction method has the problem of memory overflow, so that the three-dimensional reconstruction process fails.
The above problems seriously hinder the development and application of the multi-view stereo reconstruction technology, and expose the shortcomings of the single-machine version multi-view stereo reconstruction method in processing large-scale aerial image data. Therefore, a distributed multi-view stereo reconstruction method oriented to large-scale aerial images is needed, so that dense point cloud models of scenes can be rapidly calculated from the large-scale aerial images.
At present, the classical papers on multi-view stereo research are mainly: [1] accurate, dense, and Robust Multi-View Steropsis, [2] Pixel View Selection for Unstructured Multi-View Stereo, [3] BlendedMVS: A large-scale dataset for generated Multi-View Stereo networks. A paper [1] published in 2007 on a CVPR conference is a multi-view stereo reconstruction method based on seed point diffusion; paper [2] published in the ECCV conference in 2016, is a multi-view stereo reconstruction method based on image block matching; paper [3] was published in CVPR in 2020, and is a multi-view stereo reconstruction method based on deep learning, and the depth map of each image is estimated mainly by using a deep learning technique. The important point of the multi-view stereo reconstruction methods is how to improve the precision of a multi-view stereo reconstruction model (three-dimensional dense point cloud), and the processed targets are image data of small-scale scenes.
Therefore, when the existing multi-view stereo reconstruction method is applied to large-scale aerial image data, the following challenges still face: (1) When the existing single-machine version multi-view stereo reconstruction method is used for processing large-scale aerial image data, a larger memory space is needed, for example, the content space needed for processing 1000 pieces of image data is 64 Gb, and even exceeds the maximum memory space supported by the existing hardware equipment, for example, when the image data reaches 1500 pieces, the memory space of 128Gb is needed, and the maximum memory space range supported by a single computer is far exceeded; (2) The existing multi-view stereo reconstruction method has too low operating efficiency, and is difficult to meet the time efficiency requirement of large-scale three-dimensional reconstruction based on aerial image data, for example, the time for processing 1000 aerial images takes 10 days.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to solve the defects in the prior art, provides a distributed multi-view stereo reconstruction method for large-scale aerial images,
the dense point cloud model of the scene is rapidly calculated from the large-scale aerial image data on the basis of the distributed operation environment, the progress of the multi-view three-dimensional reconstruction technology for the large-scale aerial image is promoted, and the purpose of rapidly calculating the high-quality dense point cloud of the large-scale scene is achieved.
The technical scheme is as follows: the invention discloses a distributed multi-view stereo reconstruction method for large-scale aerial images, which comprises the following steps of:
s1, for a given large-scale aerial image data set,
Figure 100002_DEST_PATH_IMAGE001
wherein, in the step (A),
Figure 537233DEST_PATH_IMAGE002
representing the number of aerial images, and calculating a sparse point cloud model and camera parameters of a corresponding scene:
the sparse point cloud model is S
Figure 100002_DEST_PATH_IMAGE003
Wherein
Figure 724632DEST_PATH_IMAGE004
The number of three-dimensional points in the sparse point cloud representing the entire scene,
Figure 100002_DEST_PATH_IMAGE005
denotes the firstiA three-dimensional point
Figure 398321DEST_PATH_IMAGE006
The position in the world coordinate system is,
Figure 100002_DEST_PATH_IMAGE007
a serial number representing a three-dimensional point;
camera parameters C
Figure 645763DEST_PATH_IMAGE008
Wherein, in the step (A),
Figure 100002_DEST_PATH_IMAGE009
indicating the number of aerial images to be taken,
Figure 44383DEST_PATH_IMAGE010
denotes the first
Figure 100002_DEST_PATH_IMAGE011
The internal parameter matrix of the individual cameras,
Figure 786205DEST_PATH_IMAGE012
denotes the first
Figure 157144DEST_PATH_IMAGE011
The rotation matrix of the individual cameras is,
Figure 100002_DEST_PATH_IMAGE013
is shown as
Figure 575487DEST_PATH_IMAGE011
The translation vector of each of the cameras is,
Figure 523720DEST_PATH_IMAGE011
a serial number representing a camera;
s2, dividing the large-scale sparse point cloud model S into different sub-regions to obtain S
Figure 911976DEST_PATH_IMAGE014
In which
Figure 100002_DEST_PATH_IMAGE015
The number of sub-areas is indicated,
Figure 543945DEST_PATH_IMAGE007
a serial number representing a three-dimensional point,
Figure 635921DEST_PATH_IMAGE016
is shown as
Figure 946817DEST_PATH_IMAGE016
A sub-region;
dividing camera parameters C into
Figure 748551DEST_PATH_IMAGE015
Obtaining the camera parameters C of each subarea corresponding to the sparse point cloud model S
Figure 100002_DEST_PATH_IMAGE017
；
S3, calculating a depth map of the image in each area, and recording the area
Figure 484294DEST_PATH_IMAGE018
Therein is provided with
Figure 100002_DEST_PATH_IMAGE019
Taking aerial images, areas
Figure 510019DEST_PATH_IMAGE018
The corresponding depth image is
Figure 308211DEST_PATH_IMAGE020
,
Figure 100002_DEST_PATH_IMAGE021
(ii) a Wherein the content of the first and second substances,
Figure 195527DEST_PATH_IMAGE022
is shown as
Figure 5351DEST_PATH_IMAGE022
A number of the amplitude depth map;
s4, for each region
Figure 795452DEST_PATH_IMAGE018
Of the depth image data
Figure 100002_DEST_PATH_IMAGE023
Selecting two optimal initial fusion views, and recording as
Figure 736732DEST_PATH_IMAGE024
In which
Figure 880269DEST_PATH_IMAGE019
Indicating area
Figure 934812DEST_PATH_IMAGE018
Number of aerial images inThe amount of the (B) component (A),
Figure 100002_DEST_PATH_IMAGE025
and
Figure 53072DEST_PATH_IMAGE026
is a subscript, used to distinguish different depth images;
Figure 100002_DEST_PATH_IMAGE027
and
Figure 966802DEST_PATH_IMAGE028
representing an optimal initial fusion view;
s5, fusion region
Figure 773084DEST_PATH_IMAGE018
All depth images in (1)
Figure 337926DEST_PATH_IMAGE023
I.e. obtaining the region
Figure 79617DEST_PATH_IMAGE018
Corresponding dense point cloud model, note
Figure 100002_DEST_PATH_IMAGE029
Wherein, in the process,
Figure 496954DEST_PATH_IMAGE030
representing the number of three-dimensional points in the dense cloud after the depth map in the sub-region is fused,
Figure 106927DEST_PATH_IMAGE005
indicating points
Figure 100002_DEST_PATH_IMAGE031
A position in a world coordinate system;
s6, modeling the dense point cloud of each area
Figure 542588DEST_PATH_IMAGE032
Are combined into a wholeThe dense point cloud model of the complete scene can be obtained and recorded as
Figure 100002_DEST_PATH_IMAGE033
Wherein, in the process,
Figure 235606DEST_PATH_IMAGE015
indicating the number of sub-regions.
Further, in step S1, a hybrid motion inference structure method is used to extract the aerial image data set from the aerial image data set
Figure 858348DEST_PATH_IMAGE034
Calculating sparse point cloud model and camera parameters of a scene, and specifically comprising the following steps:
step S1.1, aerial image matching
Firstly, detecting feature points and calculating feature descriptors by using a local feature detection method based on deep learning, then calculating a matching relation between the feature descriptors by using a local perception hash method, and finally eliminating wrong matching points according to geometric consistency between images to obtain correct feature matching points;
step S1.2, calculating camera parameters
Calculating the parameters of the camera by using an incremental motion inference structure method according to the feature matching points obtained in the S1.1; firstly, calculating relative attitude information of a camera by using a five-point algorithm, then calculating absolute attitude information of the camera by using a three-point method, and finally calculating focal length information of each image by using a camera self-calibration method;
s1.3, calculating a sparse point cloud model of the regional image
And calculating a sparse point cloud model of the area scene by using a global motion inference structure method according to the feature matching points obtained in the step S1.1 and the camera parameters obtained in the step S1.2, so that the time efficiency of three-dimensional reconstruction is improved. First, the region is divided
Figure 272012DEST_PATH_IMAGE018
The three-dimensional points corresponding to all the images are registered in a world coordinate system, and then Bundle adjustment (Bundle A) is useddjustment) method simultaneously optimizes the camera parameters and three-dimensional points in the world coordinate system until convergence, and then an accurate sparse point cloud model can be obtained; wherein, the first and the second end of the pipe are connected with each other,
Figure 890075DEST_PATH_IMAGE016
is shown as
Figure 249601DEST_PATH_IMAGE016
The serial number of each region.
Further, in step S2, a dominant set clustering method is used to divide the large-scale sparse point cloud into a plurality of sub-region scenes, and the specific method is as follows:
note the book
Figure 100002_DEST_PATH_IMAGE035
Represent an inclusion
Figure 359639DEST_PATH_IMAGE002
Aerial image and sparse point cloud model
Figure 311414DEST_PATH_IMAGE036
The set of (a) and (b),
Figure 100002_DEST_PATH_IMAGE037
indicates a device having
Figure 970935DEST_PATH_IMAGE002
Rows and columns
Figure 225330DEST_PATH_IMAGE002
A square matrix of columns for recording similarity between images;
Figure 681719DEST_PATH_IMAGE038
and
Figure 100002_DEST_PATH_IMAGE039
respectively representing images
Figure 594442DEST_PATH_IMAGE040
And image
Figure 100002_DEST_PATH_IMAGE041
Set of all three-dimensional points that can be observed, image
Figure 859202DEST_PATH_IMAGE040
And an image
Figure 799345DEST_PATH_IMAGE041
The similarity between them is defined as:
Figure 743030DEST_PATH_IMAGE042
（1）
wherein, the first and the second end of the pipe are connected with each other,
Figure 100002_DEST_PATH_IMAGE043
representing a vector
Figure 334810DEST_PATH_IMAGE044
Sum vector
Figure 100002_DEST_PATH_IMAGE045
The angle between the two (C) and the angle between the two (C) are determined,
Figure 454076DEST_PATH_IMAGE046
the calculation method of (2) is as follows:
Figure 100002_DEST_PATH_IMAGE047
（2）
and is
Figure 361858DEST_PATH_IMAGE048
The calculation method of (2) is as follows:
Figure 100002_DEST_PATH_IMAGE049
（3）
wherein the content of the first and second substances,
Figure 464943DEST_PATH_IMAGE050
and
Figure 100002_DEST_PATH_IMAGE051
are all intermediate calculation variables;
to this end, a similarity matrix between the images is calculated
Figure 719470DEST_PATH_IMAGE037
According to
Figure 21138DEST_PATH_IMAGE037
Construct a graph structure from the values of
Figure 319395DEST_PATH_IMAGE052
Wherein
Figure 100002_DEST_PATH_IMAGE053
The position of the vertex is represented and,
Figure 893465DEST_PATH_IMAGE054
representing an edge;
note the book
Figure 100002_DEST_PATH_IMAGE055
Is represented by containing
Figure 935370DEST_PATH_IMAGE056
A vector of elements, then arbitrary
Figure 100002_DEST_PATH_IMAGE057
Time of day, vector
Figure 39680DEST_PATH_IMAGE055
The values of each element in (a) are as follows:
Figure 243260DEST_PATH_IMAGE058
（4）
wherein the content of the first and second substances,
Figure 100002_DEST_PATH_IMAGE059
subscripts of (2)
Figure 39046DEST_PATH_IMAGE060
To represent
Figure 100002_DEST_PATH_IMAGE061
Time of day vector
Figure 150222DEST_PATH_IMAGE062
To (1) a
Figure 692062DEST_PATH_IMAGE060
The number of the components is such that,
Figure 100002_DEST_PATH_IMAGE063
to represent
Figure 348433DEST_PATH_IMAGE062
Transposing;
therefore, the temperature of the molten metal is controlled,
Figure 585511DEST_PATH_IMAGE057
time of day and
Figure 828273DEST_PATH_IMAGE061
time of day, vector
Figure 349253DEST_PATH_IMAGE055
The error between is:
Figure 19269DEST_PATH_IMAGE064
（5）
wherein the content of the first and second substances,
Figure 9222DEST_PATH_IMAGE063
represent
Figure 790096DEST_PATH_IMAGE062
Transposing;
if it is used
Figure 100002_DEST_PATH_IMAGE065
And terminating the iterative calculation process, namely dividing the large-scale sparse point cloud model into a plurality of sub-area sparse point cloud models.
Further, in the step S3, a stereo matching method based on image blocks is used to calculate a corresponding high-quality depth map for each image, so as to lay a foundation for reconstructing a high-quality dense point cloud model. The detailed calculation steps are as follows:
s3.1, performing parallax propagation by using an odd-even iteration test, wherein the strategy comprises space propagation, visual angle propagation and time propagation;
s3.2, when the space transmission is respectively iterated for odd and even times, the transmission directions respectively start from the upper left corner and the lower right corner, cost comparison is carried out on each pixel point and the parallax planes of the left pixel and the right pixel, and matching cost of the parallax of the upper left corner and the lower right corner and the current parallax value is calculated;
s3.3, finally, taking the parallax value with the minimum substitution value as the optimal parallax value;
and step S3.4, iterating and calculating the steps S3.1 to S3.4 until the optimal parallax values of all pixels are calculated, namely obtaining the optimal depth image.
Further, in the step S4, a multiple constraint method is used to select two optimal initial fusion images for each sub-region, so as to ensure the geometric consistency between the generated dense point cloud and the real scene; the detailed calculation steps are as follows:
s4.1, calculating the characteristic matching points meeting the homography matrix constraint between any two images in the input images by using a homography matrix constraint method, and recording the characteristic matching points as
Figure 463785DEST_PATH_IMAGE066
；
S4.2, calculating the characteristic matching points meeting the basic matrix constraint relation between any two images in the input images according to the basic matrix constraint relation, and recording the characteristic matching points as
Figure 100002_DEST_PATH_IMAGE067
；
S4.3, calculating characteristic matching points meeting the intrinsic matrix constraint relation between any two images in the input images according to the intrinsic matrix constraint relation, and recording the characteristic matching points as
Figure 445647DEST_PATH_IMAGE068
；
Step S4.4, matching point is taken
Figure 47530DEST_PATH_IMAGE066
Matching point
Figure 756729DEST_PATH_IMAGE067
And matching point
Figure 127667DEST_PATH_IMAGE068
The intersection between them to obtain the matching point
Figure 100002_DEST_PATH_IMAGE069
；
Step S4.5, matching points are satisfied
Figure 437688DEST_PATH_IMAGE069
And (5) taking the two images with the least error points as initial fusion images.
Further, in the step S5, the depth images in the region are fused together by fully utilizing the normal vector information of the depth map, so as to obtain a dense point cloud model corresponding to the image in the region, and the detailed steps are as follows:
s5.1, calculating the confidence coefficient of each vertex of the depth image to be fused;
s5.2, deleting some redundant overlapped points from the depth image to be fused according to the confidence coefficient of each vertex in the depth image to obtain the topological information of each area image in each depth image;
s5.3, carrying out weighting operation on the vertexes on the depth image according to the topological information to obtain the geometric information of the image;
and S5.4, stitching the area images according to the topological information and the geometric information so as to obtain a dense point cloud model of the corresponding area. Further, in step S6, a global iterative nearest neighbor method is used to merge a plurality of dense point cloud models with feature overlap into a dense point cloud model of a complete scene, and the detailed steps are as follows:
step S6.1, in the target point cloud
Figure 261288DEST_PATH_IMAGE070
In select point set
Figure 100002_DEST_PATH_IMAGE071
And is made of
Figure 56068DEST_PATH_IMAGE072
；
Step S6.2, finding out origin cloud
Figure 100002_DEST_PATH_IMAGE073
Corresponding point set in (1)
Figure 937306DEST_PATH_IMAGE074
And is and
Figure 100002_DEST_PATH_IMAGE075
so that
Figure 526550DEST_PATH_IMAGE074
And
Figure 723264DEST_PATH_IMAGE071
the distance between
Figure 649632DEST_PATH_IMAGE076
Minimum;
step S6.3, calculating point set
Figure 870529DEST_PATH_IMAGE071
Sum point set
Figure 489729DEST_PATH_IMAGE074
A rotation matrix of
Figure 100002_DEST_PATH_IMAGE077
And translation vector
Figure 943713DEST_PATH_IMAGE078
；
Step S6.4, use of rotation matrix
Figure 283558DEST_PATH_IMAGE077
And translation vector
Figure 483596DEST_PATH_IMAGE078
To pair
Figure 899796DEST_PATH_IMAGE071
The points are subjected to rotation transformation and translation transformation, and a new point set is calculated
Figure 100002_DEST_PATH_IMAGE079
；
Step S6.5, calculating point set
Figure 60650DEST_PATH_IMAGE080
And point set
Figure 859978DEST_PATH_IMAGE071
Average distance therebetween
Figure 100002_DEST_PATH_IMAGE081
，
Figure 304735DEST_PATH_IMAGE082
Representing the number of three-dimensional points in the point set;
step S6.6, if
Figure 100002_DEST_PATH_IMAGE083
If the number of iterations is less than the preset threshold or greater than the preset number of iterations, the calculation process is terminated, otherwise, the step S6.2 is returned until the calculation process converges.
Has the beneficial effects that: compared with the prior art, the invention has the following advantages:
(1) According to the method, the sparse point cloud model of the large-scale scene is divided into different subset areas, so that the problems of memory overflow and system crash of a single-version multi-view stereo reconstruction method are avoided, and the large-scale multi-view stereo reconstruction is possible.
(2) According to the method, on different nodes of a distributed system, sparse point cloud models and camera parameters (including camera focal length, rotation matrix and translation vector) of different sub-areas are independently processed, the dense point cloud models are rapidly calculated, and the time efficiency of large-scale multi-view three-dimensional reconstruction is improved.
(3) The method not only can solve the problem of memory space overflow of a single-version multi-view stereo reconstruction method, but also can improve the time efficiency of large-scale multi-view stereo reconstruction, and lays an important foundation for the application of unmanned aerial vehicle aerial images in the field of three-dimensional reconstruction and the development and application of three-dimensional reconstruction technology.
Drawings
FIG. 1 is a schematic overall flow diagram of the present invention;
FIG. 2 is an aerial image dataset in an embodiment;
FIG. 3 is a large-scale sparse point cloud model in an embodiment;
FIG. 4 is a sparse point cloud model and camera pose information for a subregion in an embodiment;
FIG. 5 is an original image (non-depth image) of a primary fused view of a sub-region in an embodiment;
FIG. 6 is a dense point cloud model of a sub-region in an embodiment;
FIG. 7 is a further prior art dense point cloud model;
FIG. 8 is a block diagram of a complete dense point cloud model according to an embodiment of the present invention.
Detailed Description
The technical solution of the present invention is described in detail below, but the scope of the present invention is not limited to the embodiments.
The invention discloses a distributed multi-view stereo reconstruction method for large-scale aerial images, which aims to reconstruct scene dense point cloud from the large-scale aerial images, and has the following core idea: firstly, dividing a sparse point cloud model corresponding to a large-scale aerial image into a plurality of sub-regions; secondly, distributing the aerial images, the sparse point cloud models and the camera parameters corresponding to the sub-regions on different nodes of a distributed environment for processing, and calculating a depth image of each image; then, fusing depth images in the sub-regions on different nodes of the distributed environment, so as to obtain dense point cloud models of the sub-regions; finally, merging the dense point clouds on the sub-nodes on a main control machine in the distributed environment, so as to obtain a dense point cloud model of a complete scene; finally, the method makes it possible to quickly calculate a high-quality dense point cloud model from large-scale aerial image data.
As shown in fig. 1, the distributed multi-view stereo reconstruction method for large-scale aerial images of the present invention includes the following steps:
s1, for a given large-scale aerial image data set,
Figure 672263DEST_PATH_IMAGE001
wherein, in the step (A),
Figure 71145DEST_PATH_IMAGE002
representing the number of aerial images, and calculating a sparse point cloud model and camera parameters of a corresponding scene:
the sparse point cloud model is S
Figure 143006DEST_PATH_IMAGE003
Wherein
Figure 927423DEST_PATH_IMAGE004
The number of three-dimensional points is represented,
Figure 793748DEST_PATH_IMAGE005
indicating points
Figure 912882DEST_PATH_IMAGE006
The position in the world coordinate system is,
Figure 788434DEST_PATH_IMAGE006
inSubscript
Figure 692936DEST_PATH_IMAGE007
A serial number representing a three-dimensional point;
camera parameters C
Figure 730163DEST_PATH_IMAGE008
Wherein, in the process,
Figure 103637DEST_PATH_IMAGE009
indicating the number of aerial images to be taken,
Figure 251722DEST_PATH_IMAGE010
denotes the first
Figure 135364DEST_PATH_IMAGE011
The internal parameter matrix of the individual cameras,
Figure 218858DEST_PATH_IMAGE012
is shown as
Figure 578164DEST_PATH_IMAGE011
The rotation matrix of the individual cameras is,
Figure 795519DEST_PATH_IMAGE013
is shown as
Figure 409034DEST_PATH_IMAGE011
The translation vector of the individual cameras is,
Figure 53642DEST_PATH_IMAGE011
a serial number indicating a camera;
s1.1, detecting feature points and calculating feature descriptors by using a SuperPoint method based on deep learning, then calculating a matching relation between the feature descriptors by using a local perception Hash method, and finally eliminating wrong matching points according to geometric consistency between images to obtain correct feature matching points;
s1.2, calculating camera parameters by using an incremental motion inference structure method according to the feature matching points obtained in the S1.1; firstly, calculating relative attitude information of a camera by using a five-point algorithm, then calculating absolute attitude information of the camera by using a three-point method, and finally calculating focal length information of each image by using a camera self-calibration method;
and S1.3, calculating a sparse point cloud model of the area scene by using a global motion inference structure method according to the feature matching points obtained in the S1.1 and the camera parameters obtained in the S1.2, and improving the time efficiency of three-dimensional reconstruction. First, the region is divided
Figure 153708DEST_PATH_IMAGE018
Registering three-dimensional points corresponding to all the images in a world coordinate system, and then optimizing camera parameters and the three-dimensional points in the world coordinate system by using a Bundle Adjustment (Bundle Adjustment) method until convergence to obtain an accurate sparse point cloud model; wherein the content of the first and second substances,
Figure 909174DEST_PATH_IMAGE016
is shown as
Figure 642775DEST_PATH_IMAGE016
The serial number of each region;
s2, dividing the large-scale sparse point cloud model S into different areas to obtain S
Figure 192705DEST_PATH_IMAGE014
Wherein, in the process,
Figure 261024DEST_PATH_IMAGE004
representing the number of three-dimensional points in the sparse point cloud throughout the scene,
Figure 820181DEST_PATH_IMAGE005
indicating points
Figure 408288DEST_PATH_IMAGE006
The position in the world coordinate system is such that,
Figure 394699DEST_PATH_IMAGE015
indicating the number of sub-regions，
Figure 451779DEST_PATH_IMAGE007
A number representing a three-dimensional point is indicated,
Figure 283469DEST_PATH_IMAGE016
is shown as
Figure 850716DEST_PATH_IMAGE016
A sub-region;
dividing camera parameters C into
Figure 148974DEST_PATH_IMAGE015
Different regions, obtaining camera parameters C of each region
Figure 926306DEST_PATH_IMAGE017
Wherein, in the step (A),
Figure 561686DEST_PATH_IMAGE002
indicating the number of aerial images to be taken,
Figure 249020DEST_PATH_IMAGE084
is shown as
Figure 452599DEST_PATH_IMAGE007
The internal parameter matrix of the individual cameras,
Figure DEST_PATH_IMAGE085
is shown as
Figure 749850DEST_PATH_IMAGE007
The rotation matrix of the individual cameras is,
Figure 188922DEST_PATH_IMAGE086
is shown as
Figure 606128DEST_PATH_IMAGE007
The translation vector of the individual cameras is,
Figure 105242DEST_PATH_IMAGE015
representing the number of sub-regions;
dividing a large-scale sparse point cloud into a plurality of sub-region scenes by using a dominance set clustering method, wherein the specific method comprises the following steps:
note the book
Figure 857167DEST_PATH_IMAGE035
Show a package of
Figure 834350DEST_PATH_IMAGE002
Aerial image and sparse point cloud model
Figure 371642DEST_PATH_IMAGE036
The set of (a) and (b),
Figure 41657DEST_PATH_IMAGE037
indicates a device having
Figure 516763DEST_PATH_IMAGE002
Rows and columns
Figure 563217DEST_PATH_IMAGE002
A square matrix of columns for recording similarity between images;
Figure 955015DEST_PATH_IMAGE038
and
Figure 795932DEST_PATH_IMAGE039
respectively representing images
Figure 256869DEST_PATH_IMAGE040
And image
Figure 841434DEST_PATH_IMAGE041
Set of all three-dimensional points that can be observed, image
Figure 87739DEST_PATH_IMAGE040
And image
Figure 99557DEST_PATH_IMAGE041
In betweenThe similarity is defined as:
Figure 808975DEST_PATH_IMAGE042
（1）
wherein the content of the first and second substances,
Figure 931652DEST_PATH_IMAGE043
representing a vector
Figure 298042DEST_PATH_IMAGE044
Sum vector
Figure 215183DEST_PATH_IMAGE045
The angle between the two (C) and the angle between the two (C) are determined,
Figure 916292DEST_PATH_IMAGE046
the calculation method of (2) is as follows:
Figure 577080DEST_PATH_IMAGE047
（2）
and is provided with
Figure 63556DEST_PATH_IMAGE048
The calculation method of (2) is as follows:
Figure 151598DEST_PATH_IMAGE049
（3）
wherein the content of the first and second substances,
Figure 949790DEST_PATH_IMAGE050
and
Figure 305947DEST_PATH_IMAGE051
are all intermediate calculation variables;
to this end, a similarity matrix between the images is calculated
Figure 505984DEST_PATH_IMAGE037
According to
Figure 905872DEST_PATH_IMAGE037
Value of (a) to construct a graph structure
Figure 191360DEST_PATH_IMAGE052
Wherein
Figure 849744DEST_PATH_IMAGE053
The position of the vertex is represented and,
Figure 904287DEST_PATH_IMAGE054
representing an edge;
note book
Figure 475077DEST_PATH_IMAGE055
Is represented by containing
Figure 247861DEST_PATH_IMAGE056
A vector of elements, then arbitrary
Figure 945821DEST_PATH_IMAGE057
Time of day, vector
Figure 854871DEST_PATH_IMAGE055
The values of each element in (a) are as follows:
Figure 862141DEST_PATH_IMAGE058
（4）
wherein the content of the first and second substances,
Figure 856642DEST_PATH_IMAGE059
subscripts of (2)
Figure 856828DEST_PATH_IMAGE060
To represent
Figure 620385DEST_PATH_IMAGE061
Time of day vector
Figure 798556DEST_PATH_IMAGE062
To (1)
Figure 280353DEST_PATH_IMAGE060
The number of the components is such that,
Figure 320116DEST_PATH_IMAGE063
to represent
Figure 938179DEST_PATH_IMAGE062
Transposing;
therefore, the temperature of the molten metal is controlled,
Figure 287252DEST_PATH_IMAGE057
time of day and
Figure 256345DEST_PATH_IMAGE061
time of day, vector
Figure 598333DEST_PATH_IMAGE055
The error between is:
Figure 336482DEST_PATH_IMAGE064
（5）
wherein, the first and the second end of the pipe are connected with each other,
Figure 715511DEST_PATH_IMAGE063
represent
Figure 47266DEST_PATH_IMAGE062
Transposing;
if it is used
Figure 802732DEST_PATH_IMAGE065
If the iterative computation process is ended, the large-scale sparse point cloud model can be divided into sparse point cloud models of a plurality of sub-areas;
s3, calculating a depth map of the image in each area, and recording the area
Figure 281206DEST_PATH_IMAGE018
Is provided with therein
Figure 831136DEST_PATH_IMAGE019
Taking aerial images, areas
Figure 384609DEST_PATH_IMAGE018
The corresponding depth image is
Figure DEST_PATH_IMAGE087
,
Figure 599558DEST_PATH_IMAGE021
(ii) a Calculating a corresponding high-quality depth map for each image by using a stereo matching method based on image blocks, and laying a foundation for reconstructing a high-quality dense point cloud model; the specific process is as follows:
s3.1, performing parallax propagation by using an odd-even iteration test, wherein the strategy comprises space propagation, visual angle propagation and time propagation;
s3.2, when the space transmission is respectively iterated for odd and even times, the transmission directions respectively start from the upper left corner and the lower right corner, cost comparison is carried out on each pixel point and the parallax planes of the left pixel and the right pixel, and the matching cost of the parallax of the upper left corner and the lower right corner and the current parallax value is calculated;
s3.3, finally, taking the parallax value with the minimum substitution value as the optimal parallax value;
step S3.4, iterative computation step S3.1 to step S3.4, until the optimal parallax value of all pixels is computed, the optimal depth image can be obtained
S4, for each region
Figure 312299DEST_PATH_IMAGE018
Depth image data of
Figure 439655DEST_PATH_IMAGE023
Selecting two optimal initial fusion views, and recording as
Figure 605057DEST_PATH_IMAGE024
Wherein
Figure 328425DEST_PATH_IMAGE019
Indicating area
Figure 895673DEST_PATH_IMAGE018
The number of aerial images in the interior,
Figure 928351DEST_PATH_IMAGE025
and
Figure 581049DEST_PATH_IMAGE026
is a subscript, used to distinguish different depth images; selecting two optimal initial fusion images for each sub-region by using a multiple constraint method, and ensuring the geometric consistency of the generated dense point cloud and a real scene; the specific process is as follows:
s4.1, calculating the characteristic matching points meeting the homography matrix constraint between any two images in the input images by using a homography matrix constraint method, and recording the characteristic matching points as
Figure 606642DEST_PATH_IMAGE066
；
S4.2, calculating the characteristic matching points meeting the basic matrix constraint relation between any two images in the input images according to the basic matrix constraint relation, and recording the characteristic matching points as
Figure 28397DEST_PATH_IMAGE067
；
S4.3, calculating characteristic matching points meeting the intrinsic matrix constraint relation between any two images in the input images according to the intrinsic matrix constraint relation, and recording the characteristic matching points as
Figure 356610DEST_PATH_IMAGE068
；
Step S4.4, matching point is taken
Figure 371970DEST_PATH_IMAGE066
Matching point
Figure 811042DEST_PATH_IMAGE067
And matching point
Figure 978980DEST_PATH_IMAGE068
The intersection between them to obtain the matching point
Figure 478095DEST_PATH_IMAGE069
；
Step S4.5, matching points are satisfied
Figure 715172DEST_PATH_IMAGE069
Two images which are constrained and have the least error points are used as initial fusion images;
s5, fusion region
Figure 957935DEST_PATH_IMAGE018
All depth images in (1)
Figure 478915DEST_PATH_IMAGE023
To obtain the region
Figure 148930DEST_PATH_IMAGE018
Corresponding dense point cloud model, note
Figure 404462DEST_PATH_IMAGE029
Wherein, in the step (A),
Figure 450916DEST_PATH_IMAGE030
representing the number of three-dimensional points in the dense cloud generated by the depth map fusion,
Figure 327867DEST_PATH_IMAGE005
indicating points
Figure 168784DEST_PATH_IMAGE031
A location in a world coordinate system; the method comprises the following steps of fully utilizing normal vector information of a depth map to fuse depth images in a region into a whole to obtain a dense point cloud model corresponding to the images in the region, wherein the specific process comprises the following steps:
s5.1, calculating the confidence coefficient of each vertex of the depth image to be fused;
s5.2, deleting some redundant overlapped points from the depth image to be fused according to the confidence of each vertex in the depth image to obtain the topological information of each region image in each depth image;
s5.3, carrying out weighting operation on the vertex on the depth image according to the topological information to obtain the geometric information of the image;
s5.4, stitching the area images according to the topological information and the geometric information so as to obtain dense point cloud models of corresponding areas;
s6, modeling the dense point cloud of each area
Figure 380454DEST_PATH_IMAGE032
Combining them into a whole body to obtain complete dense point cloud model, and marking it as
Figure 230598DEST_PATH_IMAGE033
Wherein, in the step (A),
Figure 726170DEST_PATH_IMAGE015
representing the number of sub-regions; combining a plurality of dense point cloud models with feature overlapping into a dense point cloud model of a complete scene by using a global iterative nearest neighbor method, wherein the specific process is as follows:
step S6.1, in the target point cloud
Figure 737989DEST_PATH_IMAGE070
In select point set
Figure 702534DEST_PATH_IMAGE071
And is and
Figure 825211DEST_PATH_IMAGE072
；
step S6.2, finding out origin cloud
Figure 670895DEST_PATH_IMAGE073
Corresponding point set in (1)
Figure 119194DEST_PATH_IMAGE074
And is and
Figure 305456DEST_PATH_IMAGE075
so that
Figure 231823DEST_PATH_IMAGE074
And
Figure 701988DEST_PATH_IMAGE071
the distance between
Figure 55609DEST_PATH_IMAGE076
Minimum;
step S6.3, calculating point set
Figure 729167DEST_PATH_IMAGE071
Sum point set
Figure 193646DEST_PATH_IMAGE074
A rotation matrix of
Figure 285361DEST_PATH_IMAGE077
And translation vector
Figure 544304DEST_PATH_IMAGE078
；
Step S6.4, using the rotation matrix
Figure 705158DEST_PATH_IMAGE077
And translation vector
Figure 973328DEST_PATH_IMAGE078
For is to
Figure 152506DEST_PATH_IMAGE071
The points are subjected to rotation transformation and translation transformation, and a new point set is calculated
Figure 847929DEST_PATH_IMAGE079
；
Step S6.5, calculating point set
Figure 620713DEST_PATH_IMAGE080
And point set
Figure 302361DEST_PATH_IMAGE071
Average distance between
Figure 476991DEST_PATH_IMAGE081
，
Figure 969414DEST_PATH_IMAGE082
Representing the number of three-dimensional points in the point set;
step S6.6, if
Figure 229494DEST_PATH_IMAGE088
If the value is less than the preset threshold value or greater than the preset iteration number, the calculation process is terminated, otherwise, the step S6.2 is returned until the calculation process converges.
Example 1:
the original aerial image of the embodiment is shown in fig. 2, the final reconstruction result of the embodiment is shown in fig. 8, and the dense point cloud model reconstructed from the large-scale aerial image has high geometric consistency with the real scene.
As can be seen from the above embodiments, the sparse point cloud model of the complete scene is calculated firstly (as shown in fig. 3), then the large-scale sparse point cloud model is divided into sparse point cloud models of a plurality of sub-regions (as shown in fig. 4), then the depth map of the image in each sub-region is calculated, and an initial fusion view is selected for each sub-region (as shown in fig. 5, the initial fusion view is an original image (non-depth image)) of the initial fusion view), and fourth, the depth images in the sub-regions are fused to obtain dense point cloud models corresponding to the sub-regions (as shown in fig. 6); and finally, combining the dense point clouds of the sub-regions into a whole to obtain a dense point cloud model of the complete scene (as shown in fig. 8). In addition, as can be seen from fig. 7, the dense point cloud model calculated by other prior art schemes has a large difference from the real scene in terms of geometric consistency.
In addition, the technical scheme of the embodiment only needs to use 1000 aerial images for 8 hours and only occupies 24Gb memory space, that is, the time efficiency of large-scale multi-view stereo reconstruction can be improved, and the problem of memory overflow can be avoided.
The invention can also be applied to other fields, such as the Yuan universe, digital Chinese construction, digital country construction, digital city construction, military simulation, unmanned driving, automatic navigation under the condition of no satellite, digital protection of cultural heritage, three-dimensional scene monitoring, shooting and manufacturing of large-scale movies and televisions, three-dimensional surveying of natural disaster sites, three-dimensional visual science popularization creation, virtual reality and augmented reality.

Claims (7)

1. A distributed multi-view stereo reconstruction method for large-scale aerial images is characterized by comprising the following steps: the method comprises the following steps:
s1, for a given large-scale aerial image data set,
Figure DEST_PATH_IMAGE001
wherein, in the step (A),
Figure 466551DEST_PATH_IMAGE002
representing the number of aerial images, and calculating a sparse point cloud model and camera parameters of a corresponding scene:
the sparse point cloud model is S
Figure DEST_PATH_IMAGE003
In which
Figure 590365DEST_PATH_IMAGE004
The number of three-dimensional points in the sparse point cloud representing the entire scene,
Figure DEST_PATH_IMAGE005
is shown asiA three-dimensional point
Figure 907339DEST_PATH_IMAGE006
The position in the world coordinate system is,
Figure DEST_PATH_IMAGE007
a serial number representing a three-dimensional point;
camera parameters C
Figure 975658DEST_PATH_IMAGE008
Wherein
Figure DEST_PATH_IMAGE009
Indicating the number of aerial images to be taken,
Figure 754389DEST_PATH_IMAGE010
is shown as
Figure DEST_PATH_IMAGE011
The internal parameter matrix of the individual cameras,
Figure 732709DEST_PATH_IMAGE012
is shown as
Figure 656803DEST_PATH_IMAGE011
The rotation matrix of the individual cameras is,
Figure DEST_PATH_IMAGE013
is shown as
Figure 854828DEST_PATH_IMAGE011
The translation vector of each of the cameras is,
Figure 14414DEST_PATH_IMAGE011
a serial number indicating a camera;
s2, dividing the large-scale sparse point cloud model S into different sub-regions to obtain S
Figure 784924DEST_PATH_IMAGE014
Wherein, in the process,
Figure DEST_PATH_IMAGE015
the number of sub-regions is indicated,
Figure 332449DEST_PATH_IMAGE007
representing three-dimensional points
Figure 922830DEST_PATH_IMAGE006
The serial number of (a) is included,
Figure 27053DEST_PATH_IMAGE016
denotes the first
Figure 295746DEST_PATH_IMAGE016
A sub-region;
dividing camera parameters C into
Figure 561642DEST_PATH_IMAGE015
Sub-regions corresponding to the sparse point cloud model S, and obtaining camera parameters C of each sub-region
Figure DEST_PATH_IMAGE017
；
S3, calculating a depth map of the image in each area, and recording the area
Figure 232795DEST_PATH_IMAGE018
Therein is provided with
Figure DEST_PATH_IMAGE019
Taking aerial images, i.e. areas
Figure 203025DEST_PATH_IMAGE018
The corresponding depth image is
Figure 807182DEST_PATH_IMAGE020
,
Figure DEST_PATH_IMAGE021
(ii) a Wherein, the first and the second end of the pipe are connected with each other,
Figure 73340DEST_PATH_IMAGE022
denotes the first
Figure 903893DEST_PATH_IMAGE022
A number of the depth map;
s4, for each region
Figure 943393DEST_PATH_IMAGE018
Depth image data of
Figure DEST_PATH_IMAGE023
Selecting two optimal initial fusion views, and recording as
Figure 870898DEST_PATH_IMAGE024
Wherein
Figure 603230DEST_PATH_IMAGE019
Indicating area
Figure 655500DEST_PATH_IMAGE018
The number of aerial images in the interior,
Figure DEST_PATH_IMAGE025
and
Figure 734577DEST_PATH_IMAGE026
is a subscript, used to distinguish different depth images;
Figure DEST_PATH_IMAGE027
and
Figure 516588DEST_PATH_IMAGE028
representing an optimal initial fused view;
s5, fusion region
Figure 419822DEST_PATH_IMAGE018
All depth images in (1)
Figure 959388DEST_PATH_IMAGE023
To obtain the region
Figure 606270DEST_PATH_IMAGE018
Corresponding dense point cloud model, note
Figure DEST_PATH_IMAGE029
Wherein, in the step (A),
Figure 744252DEST_PATH_IMAGE030
representing the number of three-dimensional points in the dense cloud after the depth map in the sub-region is fused,
Figure 552808DEST_PATH_IMAGE005
indicating points
Figure DEST_PATH_IMAGE031
A position in a world coordinate system;
s6, carrying out dense point cloud model of each area
Figure 173145DEST_PATH_IMAGE032
Merging into a whole to obtain a dense point cloud model of the complete scene, and recording the model as
Figure DEST_PATH_IMAGE033
。
2. The distributed multi-view stereo reconstruction method for large-scale aerial images according to claim 1, characterized in that: in step S1, a hybrid motion inference structure method is used to extract a set of aerial image data from a set of aerial image data
Figure 826981DEST_PATH_IMAGE034
Calculating sparse point cloud model and camera parameters of a scene, and specifically comprising the following steps:
step S1.1, aerial image matching
Firstly, detecting feature points and calculating feature descriptors by using a local feature method based on deep learning, then calculating a matching relation between the feature descriptors by using a local perception hash method, and finally eliminating wrong matching points according to geometric consistency between images to obtain correct feature matching points;
step S1.2, calculating camera parameters
Calculating camera parameters by using an incremental motion inference structure method according to the feature matching points obtained in the S1.1; firstly, calculating relative attitude information of a camera by using a five-point algorithm, then calculating absolute attitude information of the camera by using a three-point method, and finally calculating focal length information of each image by using a camera self-calibration method;
s1.3, calculating a sparse point cloud model of the regional image
And (3) according to the feature matching points obtained in the step (S1.1) and the camera parameters obtained in the step (S1.2), calculating a sparse point cloud model of the area scene by using a global motion inference structure method:
first, the region is divided
Figure 990109DEST_PATH_IMAGE018
Registering three-dimensional points corresponding to all the images in a world coordinate system, and then optimizing camera parameters and the three-dimensional points in the world coordinate system by using a bundling adjustment method until convergence, so as to obtain an accurate sparse point cloud model; wherein, the first and the second end of the pipe are connected with each other,
Figure 996330DEST_PATH_IMAGE016
is shown as
Figure 244909DEST_PATH_IMAGE016
The serial number of each region.
3. The distributed multi-view stereo reconstruction method for large-scale aerial images according to claim 1, characterized in that: in the step S2, the dominant set clustering method is used to divide the sub-regions, and the specific method is as follows:
note the book
Figure DEST_PATH_IMAGE035
Show a package of
Figure 702435DEST_PATH_IMAGE002
Aerial image and sparse point cloud model
Figure 110282DEST_PATH_IMAGE036
The set of (a) and (b),
Figure DEST_PATH_IMAGE037
indicates a device having
Figure 463903DEST_PATH_IMAGE002
Rows and columns
Figure 825877DEST_PATH_IMAGE002
A square matrix of columns for recording similarity between images;
Figure 493618DEST_PATH_IMAGE038
and
Figure DEST_PATH_IMAGE039
respectively representing images
Figure 959235DEST_PATH_IMAGE040
And image
Figure DEST_PATH_IMAGE041
Set of all three-dimensional points that can be observed, then image
Figure 873970DEST_PATH_IMAGE040
And an image
Figure 97141DEST_PATH_IMAGE041
The similarity between them is defined as:
Figure 663514DEST_PATH_IMAGE042
（1）
wherein, the first and the second end of the pipe are connected with each other,
Figure DEST_PATH_IMAGE043
representing a vector
Figure 249216DEST_PATH_IMAGE044
Sum vector
Figure DEST_PATH_IMAGE045
The angle between the two (C) and the angle between the two (C) are determined,
Figure 475798DEST_PATH_IMAGE046
the calculation method of (2) is as follows:
Figure DEST_PATH_IMAGE047
（2）
and is
Figure 514161DEST_PATH_IMAGE048
The calculation method of (2) is as follows:
Figure DEST_PATH_IMAGE049
（3）
wherein the content of the first and second substances,
Figure 212121DEST_PATH_IMAGE050
and
Figure DEST_PATH_IMAGE051
are all intermediate calculation variables;
to this end, a similarity matrix between the images is calculated
Figure 917909DEST_PATH_IMAGE037
According to
Figure 721917DEST_PATH_IMAGE037
Value of (a) to construct a graph structure
Figure 44314DEST_PATH_IMAGE052
Wherein
Figure DEST_PATH_IMAGE053
The position of the vertex is represented and,
Figure 415471DEST_PATH_IMAGE054
representing an edge;
note book
Figure DEST_PATH_IMAGE055
Is expressed as containing
Figure 975766DEST_PATH_IMAGE056
A vector of elements, then arbitrary
Figure DEST_PATH_IMAGE057
Time of day, vector
Figure 75309DEST_PATH_IMAGE055
The values of each element in (a) are as follows:
Figure 494789DEST_PATH_IMAGE058
（4）
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE059
subscript of
Figure 439611DEST_PATH_IMAGE060
Represent
Figure DEST_PATH_IMAGE061
Time of day vector
Figure 355877DEST_PATH_IMAGE062
To (1)
Figure 626321DEST_PATH_IMAGE060
The number of the components is such that,
Figure DEST_PATH_IMAGE063
to represent
Figure 126572DEST_PATH_IMAGE062
Transposing;
therefore, the temperature of the molten metal is controlled,
Figure 281610DEST_PATH_IMAGE057
time of day and
Figure 347655DEST_PATH_IMAGE061
time and vector
Figure 929946DEST_PATH_IMAGE055
The error between is:
Figure 950117DEST_PATH_IMAGE064
（5）
wherein the content of the first and second substances,
Figure 643267DEST_PATH_IMAGE063
represent
Figure 298239DEST_PATH_IMAGE062
Transposing;
if it is not
Figure DEST_PATH_IMAGE065
Then the iterative calculation is terminatedThe process can divide the large-scale sparse point cloud model into a plurality of subarea sparse point cloud models.
4. The distributed multi-view stereo reconstruction method for large-scale aerial images according to claim 1, characterized in that: in step S3, a stereo matching method based on image blocks is used to calculate a corresponding high-quality depth map for each image, and the detailed calculation steps are as follows:
s3.1, performing parallax propagation by using an odd-even iteration test, wherein the strategy comprises space propagation, visual angle propagation and time propagation;
s3.2, when the space transmission is respectively iterated for odd and even times, the transmission directions respectively start from the upper left corner and the lower right corner, cost comparison is carried out on each pixel point and the parallax planes of the left pixel and the right pixel, and the matching cost of the parallax of the upper left corner and the lower right corner and the current parallax value is calculated;
s3.3, finally, taking the parallax value with the minimum substitution value as the optimal parallax value;
and step S3.4, iterating and calculating the steps S3.1 to S3.4 until the optimal parallax values of all pixels are calculated, namely obtaining the optimal depth image.
5. The distributed multi-view stereo reconstruction method for large-scale aerial images according to claim 1, characterized in that: in the step S4, two optimal initial fusion images are selected for each sub-region by using a multiple constraint method, and the detailed calculation steps are as follows:
s4.1, calculating the characteristic matching points meeting the homography matrix constraint between any two images in the input images by using a homography matrix constraint method, and recording the characteristic matching points as
Figure 379327DEST_PATH_IMAGE066
；
S4.2, calculating the characteristic matching points meeting the basic matrix constraint relation between any two images in the input images according to the basic matrix constraint relation, and recording the characteristic matching points as
Figure DEST_PATH_IMAGE067
；
S4.3, calculating the feature matching points meeting the intrinsic matrix constraint relation between any two images in the input images according to the intrinsic matrix constraint relation, and recording the feature matching points as
Figure 854171DEST_PATH_IMAGE068
；
Step S4.4, matching point is taken
Figure 351012DEST_PATH_IMAGE066
Matching points
Figure 627534DEST_PATH_IMAGE067
And matching point
Figure 286049DEST_PATH_IMAGE068
The intersection between them to obtain the matching point
Figure DEST_PATH_IMAGE069
；
Step S4.5, matching points are satisfied
Figure 841664DEST_PATH_IMAGE069
And (5) taking the two images with the least error points as initial fusion images.
6. The distributed multi-view stereo reconstruction method for large-scale aerial images according to claim 1, characterized in that: in the step S5, the depth images in the region are fused into a whole by fully utilizing the normal vector information of the depth image, and a dense point cloud model corresponding to the images in the region is obtained; the detailed steps are as follows:
s5.1, calculating the confidence coefficient of each vertex of the depth image to be fused;
s5.2, deleting some redundant overlapped points from the depth image to be fused according to the confidence of each vertex in the depth image to obtain the topological information of each region image in each depth image;
s5.3, carrying out weighting operation on the vertexes on the depth image according to the topological information to obtain the geometric information of the image;
and S5.4, stitching the area images according to the topological information and the geometric information so as to obtain a dense point cloud model of the corresponding area.
7. The distributed multi-view stereo reconstruction method for large-scale aerial images according to claim 1, characterized in that: in the step S6, combining a plurality of dense point cloud models with feature overlapping into a dense point cloud model of a complete scene by using a global iterative nearest neighbor method; the detailed steps are as follows:
step S6.1, in the target point cloud
Figure 876616DEST_PATH_IMAGE070
In select point set
Figure DEST_PATH_IMAGE071
And is and
Figure 595261DEST_PATH_IMAGE072
；
step S6.2, finding out origin cloud
Figure DEST_PATH_IMAGE073
Corresponding point set in (1)
Figure 752573DEST_PATH_IMAGE074
And is and
Figure DEST_PATH_IMAGE075
so that
Figure 202009DEST_PATH_IMAGE074
And
Figure 899707DEST_PATH_IMAGE071
a distance therebetween
Figure 259144DEST_PATH_IMAGE076
Minimum;
step S6.3, calculating point set
Figure 885559DEST_PATH_IMAGE071
Sum point set
Figure 963237DEST_PATH_IMAGE074
A rotation matrix of
Figure DEST_PATH_IMAGE077
And translation vector
Figure 199046DEST_PATH_IMAGE078
；
Step S6.4, using the rotation matrix
Figure 537624DEST_PATH_IMAGE077
And translation vector
Figure 240000DEST_PATH_IMAGE078
To pair
Figure 929608DEST_PATH_IMAGE071
The points are subjected to rotation transformation and translation transformation, and a new point set is calculated
Figure DEST_PATH_IMAGE079
；
Step S6.5, calculating point set
Figure 939414DEST_PATH_IMAGE080
And point set
Figure 398077DEST_PATH_IMAGE071
Average distance between
Figure DEST_PATH_IMAGE081
，
Figure 599252DEST_PATH_IMAGE082
Representing the number of three-dimensional points in the point set;
step S6.6, if
Figure DEST_PATH_IMAGE083
If the value is less than the preset threshold value or greater than the preset iteration number, the calculation process is terminated, otherwise, the step S6.2 is returned until the calculation process converges.
CN202310011438.9A 2023-01-05 2023-01-05 Large-scale aerial image-oriented distributed multi-view three-dimensional reconstruction method Active CN115719407B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310011438.9A CN115719407B (en) 2023-01-05 2023-01-05 Large-scale aerial image-oriented distributed multi-view three-dimensional reconstruction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310011438.9A CN115719407B (en) 2023-01-05 2023-01-05 Large-scale aerial image-oriented distributed multi-view three-dimensional reconstruction method

Publications (2)

Publication Number Publication Date
CN115719407A true CN115719407A (en) 2023-02-28
CN115719407B CN115719407B (en) 2023-06-27

Family

ID=85257835

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310011438.9A Active CN115719407B (en) 2023-01-05 2023-01-05 Large-scale aerial image-oriented distributed multi-view three-dimensional reconstruction method

Country Status (1)

Country Link
CN (1) CN115719407B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116071504A (en) * 2023-03-06 2023-05-05 安徽大学 Multi-view three-dimensional reconstruction method for high-resolution image
CN116805355A (en) * 2023-08-25 2023-09-26 安徽大学 Multi-view three-dimensional reconstruction method for resisting scene shielding
CN116993925A (en) * 2023-09-25 2023-11-03 安徽大学 Distributed bundling adjustment method for large-scale three-dimensional reconstruction
CN117408999A (en) * 2023-12-13 2024-01-16 安格利(成都)仪器设备有限公司 Method for automatically detecting corrosion pits of containers and pipelines by utilizing point cloud complement
CN117437363A (en) * 2023-12-20 2024-01-23 安徽大学 Large-scale multi-view stereoscopic method based on depth perception iterator

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108416840A (en) * 2018-03-14 2018-08-17 大连理工大学 A kind of dense method for reconstructing of three-dimensional scenic based on monocular camera
CN111968218A (en) * 2020-07-21 2020-11-20 电子科技大学 Three-dimensional reconstruction algorithm parallelization method based on GPU cluster
CN112085845A (en) * 2020-09-11 2020-12-15 中国人民解放军军事科学院国防科技创新研究院 Outdoor scene rapid three-dimensional reconstruction device based on unmanned aerial vehicle image
US20210183080A1 (en) * 2019-12-13 2021-06-17 Reconstruct Inc. Interior photographic documentation of architectural and industrial environments using 360 panoramic videos
CN113284227A (en) * 2021-05-14 2021-08-20 安徽大学 Distributed motion inference structure method for large-scale aerial images
WO2021185322A1 (en) * 2020-03-18 2021-09-23 广州极飞科技有限公司 Image processing method and related device
US20210358206A1 (en) * 2020-05-14 2021-11-18 Star Institute Of Intelligent Systems Unmanned aerial vehicle navigation map construction system and method based on three-dimensional image reconstruction technology
CN115205489A (en) * 2022-06-06 2022-10-18 广州中思人工智能科技有限公司 Three-dimensional reconstruction method, system and device in large scene

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108416840A (en) * 2018-03-14 2018-08-17 大连理工大学 A kind of dense method for reconstructing of three-dimensional scenic based on monocular camera
US20210183080A1 (en) * 2019-12-13 2021-06-17 Reconstruct Inc. Interior photographic documentation of architectural and industrial environments using 360 panoramic videos
WO2021185322A1 (en) * 2020-03-18 2021-09-23 广州极飞科技有限公司 Image processing method and related device
US20210358206A1 (en) * 2020-05-14 2021-11-18 Star Institute Of Intelligent Systems Unmanned aerial vehicle navigation map construction system and method based on three-dimensional image reconstruction technology
CN111968218A (en) * 2020-07-21 2020-11-20 电子科技大学 Three-dimensional reconstruction algorithm parallelization method based on GPU cluster
CN112085845A (en) * 2020-09-11 2020-12-15 中国人民解放军军事科学院国防科技创新研究院 Outdoor scene rapid three-dimensional reconstruction device based on unmanned aerial vehicle image
CN113284227A (en) * 2021-05-14 2021-08-20 安徽大学 Distributed motion inference structure method for large-scale aerial images
CN115205489A (en) * 2022-06-06 2022-10-18 广州中思人工智能科技有限公司 Three-dimensional reconstruction method, system and device in large scene

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
BARBARA ROESSLE,ET AL.: "Dense Depth Priors for Neural Radiance Fields from Sparse Input Views", 《IEEE XPLORE》 *
MINGWEI CAO,ET AL.: "Parallel surface reconstruction for large-scale scenes in the wild", 《WILEY》 *
姜翰青;赵长飞;章国锋;王慧燕;鲍虎军;: "基于多视图深度采样的自然场景三维重建", 计算机辅助设计与图形学学报, no. 10 *
张琮毅 等: "尺度可变的快速全局点云配准方法", 《计算机学报》 *
颜深 等: "大规模室外图像3维重建技术研究进展", 《中国图象图形学报》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116071504A (en) * 2023-03-06 2023-05-05 安徽大学 Multi-view three-dimensional reconstruction method for high-resolution image
CN116805355A (en) * 2023-08-25 2023-09-26 安徽大学 Multi-view three-dimensional reconstruction method for resisting scene shielding
CN116805355B (en) * 2023-08-25 2023-11-21 安徽大学 Multi-view three-dimensional reconstruction method for resisting scene shielding
CN116993925A (en) * 2023-09-25 2023-11-03 安徽大学 Distributed bundling adjustment method for large-scale three-dimensional reconstruction
CN116993925B (en) * 2023-09-25 2023-12-01 安徽大学 Distributed bundling adjustment method for large-scale three-dimensional reconstruction
CN117408999A (en) * 2023-12-13 2024-01-16 安格利(成都)仪器设备有限公司 Method for automatically detecting corrosion pits of containers and pipelines by utilizing point cloud complement
CN117408999B (en) * 2023-12-13 2024-02-20 安格利(成都)仪器设备有限公司 Method for automatically detecting corrosion pits of containers and pipelines by utilizing point cloud complement
CN117437363A (en) * 2023-12-20 2024-01-23 安徽大学 Large-scale multi-view stereoscopic method based on depth perception iterator
CN117437363B (en) * 2023-12-20 2024-03-22 安徽大学 Large-scale multi-view stereoscopic method based on depth perception iterator

Also Published As

Publication number Publication date
CN115719407B (en) 2023-06-27

Similar Documents

Publication Publication Date Title
CN115719407A (en) Distributed multi-view stereo reconstruction method for large-scale aerial images
CN112765095B (en) Method and system for filing image data of stereo mapping satellite
CN113516769B (en) Virtual reality three-dimensional scene loading and rendering method and device and terminal equipment
CN113096250A (en) Three-dimensional building model library system construction method based on unmanned aerial vehicle aerial image sequence
CN113808261B (en) Panorama-based self-supervised learning scene point cloud completion data set generation method
CN108648264A (en) Underwater scene method for reconstructing based on exercise recovery and storage medium
CN113284227B (en) Distributed motion inference structure method for large-scale aerial images
CN106408531A (en) GPU acceleration-based hierarchical adaptive three-dimensional reconstruction method
CN113192200A (en) Method for constructing urban real scene three-dimensional model based on space-three parallel computing algorithm
CN114202632A (en) Grid linear structure recovery method and device, electronic equipment and storage medium
Wang et al. Fast 3D reconstruction method based on UAV photography
Lai et al. 2D3D-MVPNet: Learning cross-domain feature descriptors for 2D-3D matching based on multi-view projections of point clouds
CN113886510A (en) Terminal interaction method, device, equipment and storage medium
CN117456136A (en) Digital twin scene intelligent generation method based on multi-mode visual recognition
US20080111814A1 (en) Geometric tagging
CN111260712B (en) Depth estimation method and device based on refocusing polar line graph neighborhood distribution
CN116152442A (en) Three-dimensional point cloud model generation method and device
CN116612235A (en) Multi-view geometric unmanned aerial vehicle image three-dimensional reconstruction method and storage medium
CN115578260A (en) Attention method and system for direction decoupling for image super-resolution
CN113487713B (en) Point cloud feature extraction method and device and electronic equipment
CN113850293A (en) Positioning method based on multi-source data and direction prior joint optimization
Li et al. BDLoc: Global localization from 2.5 D building map
CN116805355B (en) Multi-view three-dimensional reconstruction method for resisting scene shielding
CN117078982B (en) Deep learning-based large-dip-angle stereoscopic image alignment dense feature matching method
CN117437363B (en) Large-scale multi-view stereoscopic method based on depth perception iterator

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant