CN108986218B - PMVS-based rapid building dense point cloud reconstruction method - Google Patents

PMVS-based rapid building dense point cloud reconstruction method Download PDF

Info

Publication number
CN108986218B
CN108986218B CN201810575339.2A CN201810575339A CN108986218B CN 108986218 B CN108986218 B CN 108986218B CN 201810575339 A CN201810575339 A CN 201810575339A CN 108986218 B CN108986218 B CN 108986218B
Authority
CN
China
Prior art keywords
diffusion
vertex
triangle
point cloud
mesh
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810575339.2A
Other languages
Chinese (zh)
Other versions
CN108986218A (en
Inventor
张小国
王果
张恒
王慧青
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN201810575339.2A priority Critical patent/CN108986218B/en
Publication of CN108986218A publication Critical patent/CN108986218A/en
Application granted granted Critical
Publication of CN108986218B publication Critical patent/CN108986218B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Generation (AREA)

Abstract

The invention discloses a PMVS-based rapid building dense point cloud reconstruction method, which uses quasi-dense point cloud to replace PMVS algorithm self-built sparse point cloud as a diffusion seed point under the application scene of building three-dimensional reconstruction, and the main flow of the method comprises the following steps: firstly, an initial space triangular mesh model of a building is built, operations such as spatial interpolation diffusion, optimization and filtering of a surface patch, triangular mesh information updating and the like are carried out on a triangular mesh cycle, accurate quasi-dense point cloud is obtained, and the quasi-dense point cloud is used for replacing a self-built sparse point cloud of a PMVS algorithm as a diffusion seed point, so that the PMVS algorithm reconstruction process is accelerated. The invention can effectively improve the operation efficiency of the PMVS algorithm. The invention has certain filtering function on point cloud noise brought by natural landscape and point cloud noise of other non-building main bodies.

Description

PMVS-based rapid building dense point cloud reconstruction method
Technical Field
The invention relates to the field of image processing, in particular to a PMVS-based rapid building dense point cloud reconstruction method.
Background
The PMVS (patch-based multi-view stereo) algorithm proposed by Furukawa et al is one of the mainstream dense point cloud reconstruction algorithms that currently perform well. The basic flow of the algorithm is as follows: detecting characteristic points of an input image set by using Harris and DoG operators, obtaining sparse seed point clouds by using characteristic matching and triangularization reconstruction, iteratively performing steps of surface patch diffusion, optimization, filtering and the like by using the seed points, and finally reconstructing a dense surface patch set which has normal information and is tightly covered on the surface of an object to be reconstructed. The algorithm has the advantages that: (1) No initial information such as convex hull, bounding box, etc. is required; (2) The algorithm has a wide application range, and can be applied from single object reconstruction to large-scale complex scene reconstruction; (3) The algorithm has high reconstruction precision and good reconstruction efficiency.
Although the PMVS algorithm has good performance, there are some disadvantages, one of which is that the operation efficiency of the algorithm is low, and when processing high resolution image data and large scale scene reconstruction, the operation efficiency of the PMVS algorithm is obviously reduced, so that the PMVS algorithm cannot meet the requirement of fast reconstruction of large scale urban scenes in smart cities.
One of the reasons that the PMVS algorithm operates less efficiently is that: the PMVS algorithm uses weak constraint conditions to carry out feature matching on the image to reconstruct sparse point cloud of an object to be reconstructed, the process has no global optimality due to the weak constraint conditions of the feature matching, and the reconstructed sparse point cloud is full of noise. The subsequent surface patch diffusion optimization operation performed by using the sparse point cloud rich in noise as a seed point can cause more meaningless diffusion, and the surface patch diffusion and optimization process is a main time-consuming process in the PMVS algorithm, which increases the PMVS algorithm running time to a certain extent.
Various improvements have been proposed by many scholars for the efficiency problem of the PMVS algorithm, but there is no precedent for improving the efficiency of the PMVS algorithm from the viewpoint of improving the density and precision of seed points by using the geometric characteristics of buildings at present. The existing PMVS algorithm is low in operation efficiency and cannot meet the requirement of quick reconstruction of large-scale city scenes in smart cities.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a PMVS-based rapid building dense point cloud reconstruction method, which can rapidly acquire dense point clouds of buildings with higher precision.
The invention adopts the following technical scheme for solving the technical problems:
the invention provides a PMVS-based rapid building dense point cloud reconstruction method, which comprises the following steps:
step a, using a point cloud plane cluster obtained by a hypothesis plane fitting algorithm as input data, constructing an initial space triangular mesh model of a building to be reconstructed by using a Delaunay triangulation algorithm, and performing information initialization on the triangular mesh;
b, performing diffusion encryption on the initial space triangular mesh model by using a triangular mesh space interpolation diffusion algorithm;
c, performing patch optimization and filtering on the point cloud formed by the diffused triangular mesh vertexes by using an improved patch optimization and filtering algorithm;
the improved patch optimization algorithm specifically comprises the following steps:
reducing three variables in the nonlinear optimization of PMVS into one variable; the nonlinear optimization variable is only one patch position c (p), and the normal vector of the patch keeps the initial value unchanged in the optimization process; limiting the position of the dough sheet on a normal vector of the dough sheet, wherein the dough sheet can only move along the direction of the normal vector of the dough sheet in the optimization process;
the improved patch filtering algorithm specifically comprises the following steps:
for the a-th initial spatial triangular mesh T in the hypothetical planar model a The point cloud P is obtained after interpolation diffusion and surface patch optimization a First, P is clustered by mean shift algorithm a Performing clustering, calculating a clustering center c of a cluster with the largest number of three-dimensional points in a clustering result, and calculating P a "all points in the hypothetical plane normal vector direction are the average distance to the cluster center c
Figure BDA0001686634160000021
Then traverse P a "each new diffusion point v in j Calculating v j Distance d from the plane of the parent triangle j If is greater or greater>
Figure BDA0001686634160000022
Then v will be j Judging to be a mixed point and filtering the mixed point;
d, reconstructing a triangular mesh for the point cloud after the improved surface patch optimization and filtering, and updating mesh vertex normal information and diffusion punishment values;
step e, circulating steps b, c and d, and step n 1 And thirdly, taking the obtained quasi dense point cloud as a seed point of the PMVS algorithm, and repeatedly performing the three-step flow n of surface patch diffusion, surface patch optimization and surface patch filtering of the PMVS algorithm 2 And obtaining a final dense point cloud.
As a further optimization scheme of the PMVS-based rapid building dense point cloud reconstruction method, the step a comprises the following steps:
a1, point cloud P contained in the a-th plane in the hypothetical plane model a A point cloud P a All three-dimensional point coordinates in the three-dimensional point coordinate system are converted into a two-dimensional local coordinate system of a hypothetical plane where the three-dimensional point coordinates are located from a three-dimensional global coordinate system to obtain a two-dimensional point set P a ′；
Definition ofO-xyz is a global coordinate system, O-x ' y ' z ' is a target local coordinate system, the two coordinate systems both use O as an origin, and unit vectors in three coordinate axis directions of O-xyz are respectively as follows:
Figure BDA0001686634160000023
P a the unit normal vector of the plane is ^ 4>
Figure BDA0001686634160000024
The unit vector in the direction of the three coordinate axes o-x ' y ' z ' is determined->
Figure BDA0001686634160000025
Figure BDA0001686634160000026
Figure BDA0001686634160000027
And &>
Figure BDA0001686634160000031
Is perpendicular to>
Figure BDA0001686634160000032
Any two mutually perpendicular unit vectors on the plane of (a); by v b Represents P a The (b) th three-dimensional point, based on the comparison of the measured values>
Figure BDA0001686634160000033
Respectively represent v b Global coordinates and local coordinates of (a); p is expressed according to the formula (1) a By>
Figure BDA0001686634160000034
Is converted into->
Figure BDA0001686634160000035
Taking or combining>
Figure BDA0001686634160000036
As P, the x 'coordinate and the y' coordinate of a The two-dimensional plane coordinates of (a) and (b),thereby obtaining a two-dimensional point set P a ′；
Figure BDA0001686634160000037
a2, collecting P with two-dimensional points a ' and P a The plane contains a line segment set E a As input data, P is constructed using a two-dimensional finite Delaunay triangulation algorithm a ' obtaining an initial triangular mesh
Figure BDA0001686634160000038
Figure BDA0001686634160000039
Wherein n is 3 Represents the number of triangles in the triangular grid, and>
Figure BDA00016866341600000310
is T a M in 1 A plurality of triangles; due to the point cloud in P a And P a The arrangement order in ` does not change, so T a The represented triangular mesh relationships apply directly to P a I.e. T a Represents P a An initial spatial triangular mesh of the assumed plane;
a3, to T a Initializing information; the triangular mesh information includes: grid vertex coordinates, normal vectors, visual information and diffusion penalty values; due to T a Is equal to P a Three-dimensional point of (1), so using v b Representing the b-th mesh vertex by
Figure BDA00016866341600000311
Coordinates representing a vertex of the b-th grid in &>
Figure BDA00016866341600000312
Unit normal vector representing the b-th mesh vertex, denoted by V b Visual information, V, representing the b-th mesh vertex b I.e. set of indices of visual pictures, using f b Denotes the b-thA diffusion penalty value for each mesh vertex;
the coordinates of all mesh vertices use their corresponding P a Initializing global coordinates of the medium three-dimensional points; the diffusion penalty values of all the grid vertexes are initialized to 0; initializing the visual information of all the grid vertexes by the visual information of the three-dimensional line segment in the reconstruction result of the three-dimensional line segment model reconstruction algorithm, wherein the visual information of each vertex is consistent with the visual information of the three-dimensional line segment where the vertex is located; the normal vector initialization process for the mesh vertices is as follows:
traverse T a Calculating a normal vector of a plane unit of each triangle; m th 1 A triangle
Figure BDA00016866341600000313
Has a plane unit normal vector of->
Figure BDA00016866341600000314
And ensures that it is taken up or taken up>
Figure BDA00016866341600000315
And/or>
Figure BDA00016866341600000316
The included angle is an acute angle; wherein->
Figure BDA00016866341600000317
Is triangular>
Figure BDA00016866341600000318
Is calculated as shown in equation (2), and is based on the equation (2)>
Figure BDA00016866341600000319
And &>
Figure BDA00016866341600000320
Respectively triangular>
Figure BDA00016866341600000321
Coordinate of three vertices of (1), normal () tableRepresenting the unitization of vectors;
Figure BDA00016866341600000322
traverse T a Calculating a unit normal vector of each grid vertex; unit normal vector of b-th grid vertex
Figure BDA00016866341600000323
That is, the normal vector of any mesh vertex is the average of the plane unit normal vectors of all triangles including the vertex; wherein->
Figure BDA00016866341600000324
Is expressed as v b Is the m-th of the vertex 2 Plane unit normal vector of individual triangle, n 4 Is expressed as v b The total number of triangles that are vertices;
and a4, traversing all planes in the assumed plane model of the building, and repeating the steps a1 to a3.
As a further optimization scheme of the PMVS-based rapid building dense point cloud reconstruction method, the step b comprises the following steps:
b1, for the initial spatial triangular mesh T a M in 1 A triangle
Figure BDA0001686634160000041
Calculating the area threshold value
Figure BDA0001686634160000042
Wherein->
Figure BDA0001686634160000043
Represents the average triangle area of the triangle mesh, loop represents the preset diffusion times, and fine represents ^ greater than or equal to ^ greater than>
Figure BDA0001686634160000044
Has a formula &>
Figure BDA0001686634160000045
Wherein the content of the first and second substances,
Figure BDA0001686634160000046
respectively represent->
Figure BDA0001686634160000047
A diffusion penalty value for three vertices; using two sets of UT and UT' to represent a triangle set to be diffused and a temporary triangle set generated in the diffusion intermediate process; calculate->
Figure BDA0001686634160000048
Area of (4)>
Figure BDA0001686634160000049
If/or>
Figure BDA00016866341600000410
Will->
Figure BDA00016866341600000411
Adding into UT; if/or>
Figure BDA00016866341600000412
Then to T a C, performing the operation of the step b1 on the next triangle;
b2, m-th in the set UT 2 A triangle
Figure BDA00016866341600000413
Calculate->
Figure BDA00016866341600000414
Central point v of c Its coordinate->
Figure BDA00016866341600000415
The calculation is shown in formula (2); centering the center point v c Added as a new diffusion vertex to the point cloud P a V according to the following method c Updating the normal vector and the visual information: v. of c Is equal to->
Figure BDA00016866341600000416
A plane normal vector of (a); v. of c Visual information V of c ＝V 0 ∪V 1 ∪V 2 In which V is 0 、V 1 、V 2 Respectively represent->
Figure BDA00016866341600000417
Visual information of three vertices;
b3, insertion point v c After that, point v c And triangle
Figure BDA00016866341600000418
Three new triangles are respectively formed by the three vertexes, and the three new triangles are added into the set UT';
b4, traversing each triangle in the set UT, repeating the steps b2 and b3, and exchanging the set UT and UT' after traversing;
b5, if the set UT is not empty, repeating the step b4; if the set UT is empty, repeating the steps b1 to b5 for the next triangle until T is traversed a Up to all triangles in (1).
As a further optimization scheme of the PMVS-based rapid building dense point cloud reconstruction method, the method for reconstructing the triangular mesh and updating the normal information of the mesh vertex in step d is as follows:
constructing a triangular mesh for the point cloud after the improved patch optimization and filtering operation by using the method shown in the steps a1 to a2, wherein the visual information of the mesh vertex is kept unchanged; and then, using the normal vector updating method of the grid vertex in the step a3 to update the normal information of the vertex of the grid.
As a further optimization scheme of the PMVS-based rapid building dense point cloud reconstruction method, the method for updating the diffusion penalty value in the step d specifically comprises the following steps:
using T a ' representing an initial spatial triangular mesh T a Passing through the gridA new triangular mesh reconstructed after the steps of interpolation diffusion, improved patch optimization and filtering;
d1, calculating an initial space triangular grid T a The diffusion success rate r of all triangles in the tree is determined;
using the diffusion success rate r as the basis for judging whether a triangle receives punishment or not, and regarding T a M in 1 A triangle
Figure BDA0001686634160000051
Its success rate of diffusion->
Figure BDA0001686634160000052
Wherein n is a Indicates that the triangle is greater or less>
Figure BDA0001686634160000053
Total number of diffusion vertexes n obtained after diffusion r Representing triangles @afterimproved patch optimization and filtering>
Figure BDA0001686634160000054
The survival number of the contained diffusion vertexes; success rate of diffusion
Figure BDA0001686634160000055
Reflect->
Figure BDA0001686634160000056
Possibility of being in a hollow area, based on the status of the device>
Figure BDA0001686634160000057
The smaller the>
Figure BDA0001686634160000058
The more likely it is in a hole region, the more>
Figure BDA0001686634160000059
The included vertexes are punished, so that the included vertexes are weakened in diffusion strength in the next round of diffusion;
d2, for the new triangular mesh T a ' the b-th mesh vertex v b ', if v b ' is the diffusion vertex, then v is found b ' mother triangle t p ，t p I.e. diffusion of v b ' and performing step d3; if v is b If the vertex is the initial vertex, the diffusion penalty value is kept unchanged;
d3 if v b ' mother triangle t p Diffusion success rate of
Figure BDA00016866341600000510
Then it is necessary to point v b ' punishing, updating v according to equation (3) b 'diffusion penalty value f' b (ii) a If t is p Has a diffusion success rate>
Figure BDA00016866341600000511
Then v is not aligned with the point b ' punishment is carried out, and the diffusion punishment value f ' is updated according to a formula (4) ' b (ii) a Based on the formula (3) and (4->
Figure BDA00016866341600000512
Respectively represent a parent triangle t p Diffusion penalty, f, of three vertices u Is a preset penalty value;
Figure BDA00016866341600000513
Figure BDA00016866341600000514
d4, traverse T a ' repeating steps d2 to d3 for each mesh vertex in T a ' the updating of the mesh vertex diffusion penalty value.
The invention relates to a further optimization scheme of a PMVS-based rapid building dense point cloud reconstruction method, wherein n 1 Is an integer of 1 or more, n 2 Is an integer of 1 or more.
As an originalThe invention relates to a PMVS-based building dense point cloud rapid reconstruction method further optimization scheme, n 1 Is 3,n 2 Is 2.
Compared with the prior art, the invention adopting the technical scheme has the following technical effects:
(1) The invention fully utilizes the multi-plane geometric characteristics of the building, and provides a PMVS (pulse-modulated visual system) improvement method based on spatial triangular mesh interpolation diffusion based on plane priori knowledge provided by a hypothetical plane model of the building, so that the operation efficiency of the PMVS method can be effectively improved, and experiments prove that the improvement of the PMVS method provided by the invention can achieve 15% -23% of improvement of the operation speed of the original PMVS method when the loss rate of the reconstructed point cloud is lower than 5%;
(2) The method has a certain filtering effect on point cloud noise caused by natural landscapes, and compared with the reconstruction result of the original PMVS (passive matrix visual simulation) method, the point cloud noise of a non-building main body contained in the dense point cloud of the building reconstructed by the improved method provided by the invention is obviously reduced;
(3) The space grid interpolation algorithm based on the building hypothesis plane model provided by the method can quickly acquire the quasi-dense point cloud of the building with higher precision.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
As shown in FIG. 1, the invention discloses a PMVS (pseudo-symmetric triangular space) improvement method based on spatial triangular mesh interpolation diffusion, which is characterized in that a triangular mesh is constructed by utilizing a building hypothesis plane model, accurate and effective quasi-dense point clouds are quickly obtained by carrying out spatial interpolation diffusion on the triangular mesh, and the quasi-dense point clouds are used as diffusion seed points instead of self-constructed sparse point clouds of a PMVS algorithm, so that the operation efficiency of the PMVS algorithm can be effectively improved, and the PMVS improvement method comprises the following steps:
step a, using a point cloud plane cluster obtained by a hypothesis plane fitting algorithm as input data, using a Delaunay triangulation algorithm to construct an initial space triangular mesh model of a building to be reconstructed, and initializing information of the triangular mesh, wherein the specific steps are as follows:
a1. for the point cloud P contained in the a-th plane in the hypothetical plane model a A point cloud P a All three-dimensional point coordinates in the three-dimensional point coordinate system are converted into a two-dimensional local coordinate system of a hypothetical plane where the three-dimensional point coordinates are located from a three-dimensional global coordinate system to obtain a two-dimensional point set P a ′；
Defining O-xyz as a global coordinate system, O-x ' y ' z ' as a target local coordinate system, taking O as an origin in both coordinate systems, and respectively setting unit vectors in three coordinate axis directions of O-xyz as:
Figure BDA0001686634160000061
P a the unit normal vector of the plane is ^ 4>
Figure BDA0001686634160000062
The unit vector in the direction of the three coordinate axes o-x ' y ' z ' is determined->
Figure BDA0001686634160000063
Figure BDA0001686634160000064
Figure BDA0001686634160000065
And &>
Figure BDA0001686634160000066
Is perpendicular to>
Figure BDA0001686634160000067
Any two mutually perpendicular unit vectors on the plane of (a); by v b Represents P a The (b) th three-dimensional point, based on the comparison of the measured values>
Figure BDA0001686634160000068
Respectively represent v b Global coordinates and local coordinates of; p is expressed according to the formula (1) a By>
Figure BDA0001686634160000069
Is converted into->
Figure BDA00016866341600000610
Fetch and hold>
Figure BDA00016866341600000611
As P, the x 'coordinate and the y' coordinate of a To obtain a two-dimensional point set P a ′；
Figure BDA00016866341600000612
a2. With two-dimensional point set P a ' and P a The plane contains a line segment set E a As input data, P is constructed using a two-dimensional finite Delaunay triangulation algorithm a ' obtaining an initial triangular mesh
Figure BDA00016866341600000613
Figure BDA0001686634160000071
Wherein n is 3 Represents the number of triangles in the triangular grid, and>
Figure BDA0001686634160000072
is T a M in 1 A triangle; due to point cloud in P a And P a The arrangement order in ` does not change, so T a The represented triangular mesh relationship can be directly applied to P a I.e. T a Represents P a An initial spatial triangular mesh of the assumed plane;
a3. for T a Initializing information; the triangular mesh information includes: grid vertex coordinates, normal vectors, visual information and diffusion penalty values; due to T a Is equal to P a Three-dimensional point of (1), so using v b Representing the b-th mesh vertex by
Figure BDA0001686634160000073
Denotes the coordinates of the vertex of the b-th grid in->
Figure BDA0001686634160000074
Unit normal vector representing the b-th mesh vertex, denoted by V b Visual information (i.e., set of indices of the visible slice) representing the b-th mesh vertex, using f b Representing a diffusion penalty value for the b-th mesh vertex;
coordinates of all mesh vertices use their corresponding P a Initializing global coordinates of the medium three-dimensional points; the diffusion penalty values of all the grid vertexes are initialized to 0; initializing the visual information of all the grid vertexes by the visual information of the three-dimensional line segment in the reconstruction result of the three-dimensional line segment model reconstruction algorithm, wherein the visual information of each vertex is consistent with the visual information of the three-dimensional line segment where the vertex is located; the normal vector initialization process for the mesh vertices is as follows:
traverse T a Calculating a normal vector of a plane unit of each triangle; m th 1 A triangle
Figure BDA0001686634160000075
Has a plane unit normal vector of>
Figure BDA0001686634160000076
And ensures that it is taken up or taken up>
Figure BDA0001686634160000077
And/or>
Figure BDA0001686634160000078
The included angle is an acute angle; wherein +>
Figure BDA0001686634160000079
Is triangular>
Figure BDA00016866341600000710
Is calculated as shown in equation (2), and is based on the equation (2)>
Figure BDA00016866341600000711
And &>
Figure BDA00016866341600000712
Are respectively triangular>
Figure BDA00016866341600000713
The normaize () represents the unitization of the vector;
traverse T a Calculating a unit normal vector of each grid vertex; unit normal vector of b-th grid vertex
Figure BDA00016866341600000714
That is, the normal vector of any mesh vertex is the average of the plane unit normal vectors of all triangles including the vertex; wherein->
Figure BDA00016866341600000715
Is expressed as v b Is the m-th of the vertex 2 Plane unit normal vector of individual triangle, n 4 Is expressed as v b The total number of triangles that are vertices;
Figure BDA00016866341600000716
a4. and traversing all planes in the building hypothesis plane model, and repeating the steps a1 to a3.
B, using a triangular mesh space interpolation diffusion algorithm to perform diffusion encryption on the initial space triangular mesh model, wherein the specific steps are as follows:
b1. for the initial spatial triangular mesh T a M in m 1 A triangle
Figure BDA00016866341600000717
Calculating its area threshold
Figure BDA00016866341600000718
Wherein->
Figure BDA00016866341600000719
Represents the average triangle area of the triangle mesh, loop represents a predetermined number of diffusion times, fine represents +>
Figure BDA00016866341600000720
Is calculated by ^ 4>
Figure BDA00016866341600000721
Wherein the content of the first and second substances,
Figure BDA00016866341600000722
respectively represent->
Figure BDA00016866341600000723
A diffusion penalty value for three vertices; using two sets of UT and UT' to represent a triangle set to be diffused and a temporary triangle set generated in the diffusion intermediate process; counting/or>
Figure BDA0001686634160000081
Area of (4)>
Figure BDA0001686634160000082
If it is
Figure BDA0001686634160000083
Will then >>
Figure BDA0001686634160000084
Adding into UT; if/or>
Figure BDA0001686634160000085
Then to T a The next triangle is processed by the operation of step b1,
b2. For m in the set UT 2 A triangle
Figure BDA0001686634160000086
Calculate->
Figure BDA0001686634160000087
Central point v of c Its coordinate->
Figure BDA0001686634160000088
The calculation is shown in formula (2); centering the center point v c Added as a new diffusion vertex to the point cloud P a V is prepared by the following method c Updating the normal vector and the visual information: v. of c Is equal to->
Figure BDA0001686634160000089
A plane normal vector of (a); v. of c Visual information V of c ＝V 0 ∪V 1 ∪V 2 In which V is 0 、V 1 、V 2 Respectively represent->
Figure BDA00016866341600000810
Visual information of three vertices;
b3. insertion point v c After that, point v c And triangle
Figure BDA00016866341600000811
Three new triangles are respectively formed by the three vertexes of the three triangles, and the three new triangles are added into the set UT';
b4. traversing each triangle in the set UT, repeating the steps b2 and b3, and exchanging the sets UT and UT' after traversing;
b5. if the set UT is not empty, repeating the step b4; if the set UT is empty, repeating steps b1 to b5 for the next triangle until T is traversed a Up to all triangles in (1).
C, performing patch optimization and filtering on the point cloud formed by the diffused triangular mesh vertexes by using an improved patch optimization and filtering algorithm, wherein the method specifically comprises the following steps of:
step c uses an improved patch optimization algorithm, which specifically comprises the following steps:
three variables in the nonlinear optimization of PMVS were reduced to one variable. The three optimization variables of the original PMVS algorithm are: the patch position c (p) and two Euler angles related to the normal vector n (p) of the patch, while the nonlinear optimization variable of the invention is only one patch position c (p), and the normal vector of the patch keeps the initial value unchanged in the optimization process. The position of a patch of the original PMVS algorithm is limited on a connecting line of the center of the patch and the optical center of a camera corresponding to the reference picture, and the position of the patch can only move along the line; the algorithm limits the position of the dough sheet on the normal vector of the dough sheet, and the dough sheet can only move along the direction of the normal vector of the dough sheet in the optimization process.
The patch filtering algorithm used in the step c is added with a mean shift clustering-based median filtering algorithm on the basis of the original PMVS patch filtering algorithm, and the specific flow is as follows:
for the a-th initial spatial triangular mesh T in the hypothetical planar model a The point cloud P is obtained after interpolation diffusion and surface patch optimization a First, P is clustered by mean shift algorithm a "clustering, calculating c of the cluster center of the cluster containing the largest number of three-dimensional points in the clustering result, and calculating P a All points in "average distance to cluster center c in the direction of the hypothetical plane normal vector
Figure BDA00016866341600000812
Then traverse P a "each new diffusion point v in j Calculating v j Distance d from the plane of the parent triangle j If is greater or greater>
Figure BDA00016866341600000813
Then v will be j And judging as the mixed point and filtering the mixed point.
D, reconstructing a triangular mesh for the point clouds after the improved surface patch optimization and filtering, and updating point cloud normal information and a diffusion penalty value, wherein the method specifically comprises the following steps:
the method for reconstructing the triangular mesh and updating the normal information of the mesh vertex in the step d comprises the following steps:
and (3) performing triangular mesh construction on the point cloud after the improved patch optimization and filtering operation by using the method shown in the steps a1 to a2, wherein the visual information of the mesh vertexes is kept unchanged. And then, updating normal information of the mesh by using the mesh vertex normal vector updating method in the step a3.
The grid vertex diffusion penalty value updating method used in the step d specifically comprises the following steps:
using T a ' representing an initial spatial triangular mesh T a Reconstructing a new triangular mesh after the steps of mesh interpolation diffusion, patch optimization and filtering;
d1. computing an initial spatial triangular mesh T a The diffusion success rate r of all triangles in the tree is determined;
using the diffusion success rate r as the basis for judging whether a triangle receives punishment or not, and regarding T a M in 1 A triangle
Figure BDA0001686634160000091
Its success rate of diffusion->
Figure BDA0001686634160000092
Wherein n is a Indicates that the triangle is greater or less>
Figure BDA0001686634160000093
Total number of diffusion vertexes, n, obtained after diffusion r Representing triangles pickafter patch optimization and filtering>
Figure BDA0001686634160000094
The survival number of the contained diffusion vertexes; success rate of diffusion>
Figure BDA0001686634160000095
Reflect the fact that
Figure BDA0001686634160000096
Possibility of being in a hole area>
Figure BDA0001686634160000097
The smaller the>
Figure BDA0001686634160000098
The more likely it is in a hole region, the more>
Figure BDA0001686634160000099
The included vertexes are punished, so that the included vertexes are weakened in diffusion strength in the next round of diffusion;
d2. for new triangular mesh T a ' the b-th mesh vertex v b ', if v b ' is the diffusion vertex, then v is found b ' mother triangle t p (i.e., diffusion of v) b ' triangle), and step d3; if v is b If the vertex is the initial vertex, the diffusion penalty value is kept unchanged;
d3. if v is b ' mother triangle t p Success rate of diffusion
Figure BDA00016866341600000910
Then it is necessary to point v b ' punishing, updating v according to equation (3) b 'diffusion penalty value f' b (ii) a If t is p Has a diffusion success rate>
Figure BDA00016866341600000911
Then v is not aligned with the point b ' punishment is carried out, and the diffusion punishment value f ' is updated according to a formula (4) ' b (ii) a Based on the formula (3) and (4->
Figure BDA00016866341600000912
Respectively represent a parent triangle t p Diffusion penalty, f, for three vertices u Is a preset penalty value; />
Figure BDA00016866341600000913
Figure BDA00016866341600000914
d4. Traverse T a ' Each mesh vertex inRepeating the steps d2 to d3 to finish T a ' the updating of the mesh vertex diffusion penalty value.
Step e, circulating the three steps of b, c and d, and n 1 And thirdly, taking the obtained quasi dense point cloud as a seed point of the PMVS algorithm, and repeatedly performing the three-step process n of surface patch diffusion, surface patch optimization and surface patch filtering of the original PMVS algorithm 2 Secondly, obtaining a final dense point cloud; in the experiment n 1 3,n can be taken 2 It is possible to take 2.

Claims (4)

1. A PMVS-based building dense point cloud rapid reconstruction method is characterized by comprising the following steps:
step a, using a point cloud plane cluster obtained by a hypothesis plane fitting algorithm as input data, constructing an initial space triangular mesh model of a building to be reconstructed by using a Delaunay triangulation algorithm, and performing information initialization on the triangular mesh;
b, performing diffusion encryption on the initial space triangular mesh model by using a triangular mesh space interpolation diffusion algorithm;
c, performing patch optimization and filtering on the point cloud formed by the diffused triangular mesh vertexes by using an improved patch optimization and filtering algorithm;
the improved patch optimization algorithm specifically comprises the following steps:
reducing three variables in the nonlinear optimization of PMVS into one variable; the nonlinear optimization variable is only one patch position c (p), and the normal vector of the patch keeps the initial value unchanged in the optimization process; limiting the position of the dough sheet on a normal vector of the dough sheet, wherein the dough sheet can only move along the direction of the normal vector of the dough sheet in the optimization process;
the improved patch filtering algorithm specifically comprises the following steps:
for the a-th initial spatial triangular mesh T in the hypothetical planar model a The point cloud P is obtained after interpolation diffusion and surface patch optimization a First, P is clustered by mean shift algorithm a "clustering", calculating the clustering center c of the cluster with the most three-dimensional points in the clustering result, and calculating P a All points in "are in the direction of the hypothetical planar normal vectorAverage distance to cluster center c
Figure FDA0003993272750000011
Then go through P a "each new diffusion point v in j Calculating v j Distance d from the plane of the parent triangle j If, if
Figure FDA0003993272750000012
Then v will be j Judging to be a mixed point and filtering the mixed point;
d, reconstructing a triangular mesh for the point cloud subjected to the improved patch optimization and filtering, and updating mesh vertex normal information and a diffusion penalty value;
step e, circulating step b, step c and step d 1 And thirdly, taking the obtained quasi dense point cloud as a seed point of the PMVS algorithm, and repeatedly performing the three-step flow n of surface patch diffusion, surface patch optimization and surface patch filtering of the PMVS algorithm 2 Secondly, obtaining a final dense point cloud;
the step a comprises the following steps:
a1, point cloud P contained in the a-th plane in the hypothetical plane model a The point cloud P a All three-dimensional point coordinates in the three-dimensional point coordinate system are converted into a two-dimensional local coordinate system of a hypothetical plane where the three-dimensional point coordinates are located from a three-dimensional global coordinate system to obtain a two-dimensional point set P a ′；
Defining o-xyz as a global coordinate system, o-x ' y ' z ' as a target local coordinate system, taking 0 as an origin in both coordinate systems, and respectively setting unit vectors in three coordinate axis directions of o-xyz as:
Figure FDA0003993272750000013
P a the unit normal vector of the plane is ^ 4>
Figure FDA0003993272750000014
The unit vector in the direction of the three coordinate axes o-x ' y ' z ' is determined->
Figure FDA0003993272750000015
Figure FDA0003993272750000016
And &>
Figure FDA0003993272750000021
Is perpendicular to>
Figure FDA0003993272750000022
Any two mutually perpendicular unit vectors on the plane of (a); by v b Represents P a The (b) th three-dimensional point, based on the comparison of the measured values>
Figure FDA0003993272750000023
Respectively represent v b Global coordinates and local coordinates of; p is expressed according to the formula (1) as By>
Figure FDA0003993272750000024
Convert to>
Figure FDA0003993272750000025
Taking or combining>
Figure FDA0003993272750000026
As P, the x 'coordinate and the y' coordinate of a To obtain a two-dimensional point set P a ′；
Figure FDA0003993272750000027
a2, collecting P with two-dimensional points a ' and P a The plane contains a line segment set E a As input data, P is constructed using a two-dimensional finite Delaunay triangulation algorithm a ' obtaining an initial triangular mesh
Figure FDA0003993272750000028
Figure FDA0003993272750000029
Wherein n is 3 Represents the number of triangles in the triangular grid, and>
Figure FDA00039932727500000210
is T a M in 1 A triangle; due to point cloud in P a And P a The arrangement order in ` does not change, so T a The represented triangular mesh relationships apply directly to P a I.e. T a Represents P a An initial spatial triangular mesh of the assumed plane;
a3, to T a Initializing information; the triangular mesh information includes: grid vertex coordinates, normal vectors, visual information and diffusion penalty values; due to T a Is equal to P a Three-dimensional point of (1), so using v b Representing the b-th mesh vertex by
Figure FDA00039932727500000211
Coordinates representing a vertex of the b-th grid in &>
Figure FDA00039932727500000212
Unit normal vector representing the b-th mesh vertex, denoted by V b Visual information, V, representing the b-th mesh vertex b I.e. set of indices of visual pictures, using f b Represents a diffusion penalty value for the b-th mesh vertex;
coordinates of all mesh vertices use their corresponding P a Initializing global coordinates of the medium three-dimensional points; the diffusion penalty values of all the grid vertexes are initialized to 0; initializing the visual information of all the grid vertexes by the visual information of the three-dimensional line segments in the reconstruction result of the three-dimensional line segment model reconstruction algorithm, wherein the visual information of each vertex is consistent with the visual information of the three-dimensional line segment where the vertex is located; the mesh vertices' normal vector initialization process is as follows:
traverse T a Calculating the normal vector of the plane unit of each triangle(ii) a M th 1 A triangle
Figure FDA00039932727500000213
Has a plane unit normal vector of>
Figure FDA00039932727500000214
And ensures that it is taken up or taken up>
Figure FDA00039932727500000215
And/or>
Figure FDA00039932727500000216
The included angle is an acute angle; wherein->
Figure FDA00039932727500000217
Is triangular>
Figure FDA00039932727500000218
Is calculated as shown in equation (2), and is based on the equation (2)>
Figure FDA00039932727500000219
And &>
Figure FDA00039932727500000220
Respectively triangular>
Figure FDA00039932727500000221
The normaize () represents the unitization of the vector;
Figure FDA00039932727500000222
traverse T a Calculating a unit normal vector of each grid vertex; unit normal vector of b-th grid vertex
Figure FDA00039932727500000223
That is, the normal vector of any mesh vertex is the average of the plane unit normal vectors of all triangles including the vertex; wherein->
Figure FDA00039932727500000224
Is expressed as v b Is the m-th of the vertex 2 Plane unit normal vector of individual triangle, n 4 Is expressed as v b The total number of triangles that are vertices;
a4, traversing all planes in the assumed plane model of the building, and repeating the steps a1 to a3;
the step b comprises the following steps:
b1. For the initial spatial triangular mesh T a M in 1 A triangle
Figure FDA0003993272750000031
Calculate its area threshold->
Figure FDA0003993272750000032
Wherein->
Figure FDA0003993272750000033
Represents the average triangle area of the triangle mesh, loop represents the preset diffusion times, and fine represents ^ greater than or equal to ^ greater than>
Figure FDA0003993272750000034
Is calculated by ^ 4>
Figure FDA0003993272750000035
Wherein +>
Figure FDA0003993272750000036
Respectively represent->
Figure FDA0003993272750000037
Diffusion penalty values for three vertices; tabulated with two sets of UT and UTShowing a triangle set to be diffused and a temporary triangle set generated in the middle process of diffusion; counting/or>
Figure FDA0003993272750000038
Area (d)>
Figure FDA0003993272750000039
If/or>
Figure FDA00039932727500000310
Will->
Figure FDA00039932727500000311
Adding into UT; if it is
Figure FDA00039932727500000312
Then to T a C, performing the operation of the step b1 on the next triangle;
b2, m-th in the set UT 2 A triangle
Figure FDA00039932727500000313
Calculate->
Figure FDA00039932727500000314
Central point v of c Its coordinate->
Figure FDA00039932727500000315
The calculation is shown in formula (2); centering the center point v c Added as a new diffusion vertex to the point cloud P a V is prepared by the following method c Updating the normal vector and the visual information: v. of c Is equal to->
Figure FDA00039932727500000316
The plane normal vector of (a); v. of c Visual information V of c ＝V 0 ∪V 1 ∪V 2 In which V is 0 、V 1 、V 2 Respectively represent/>
Figure FDA00039932727500000317
Visual information of three vertices;
b3, insertion point v c After that, point v c And triangle
Figure FDA00039932727500000318
Three new triangles are respectively formed by the three vertexes of the three triangles, and the three new triangles are added into the set UT';
b4, traversing each triangle in the set UT, repeating the steps b2 and b3, and exchanging the sets UT and UT' after traversing;
b5, if the set UT is not empty, repeating the step b4; if the set UT is empty, repeating steps b1 to b5 for the next triangle until T is traversed a Until all triangles in (1);
the diffusion penalty value updating method in the step d comprises the following specific steps:
using T a ' representing an initial spatial triangular mesh T a Reconstructing a new triangular mesh after the steps of mesh interpolation diffusion, improved patch optimization and filtering;
d1, calculating an initial space triangular mesh T a The diffusion success rate r of all triangles in the tree is determined;
using the diffusion success rate r as the basis for judging whether a triangle receives punishment or not, and regarding T a M in 1 A triangle
Figure FDA00039932727500000319
Its diffusion success rate>
Figure FDA00039932727500000320
Wherein n is a Indicates that the triangle is greater or less>
Figure FDA00039932727500000321
Total number of diffusion vertexes n obtained after diffusion r Representation in improved patch optimization and filteringAfter which triangle is>
Figure FDA00039932727500000322
The survival number of the contained diffusion vertexes; success rate of diffusion>
Figure FDA0003993272750000041
Reflect->
Figure FDA0003993272750000042
Possibility of being in a hollow area, based on the status of the device>
Figure FDA0003993272750000043
The smaller the>
Figure FDA0003993272750000044
The more likely it is in a hole region, the more>
Figure FDA0003993272750000045
The included vertexes are punished, so that the included vertexes are weakened in diffusion strength in the next round of diffusion;
d2, for the new triangular mesh T a ' the b-th mesh vertex v b ', if v b ' is the diffusion vertex, then v is found b ' mother triangle t p ，t p I.e. diffusion v b ' and performing step d3; if v is b ' if the vertex is initial, the diffusion penalty value is kept unchanged;
d3, if v b ' mother triangle t p Success rate of diffusion
Figure FDA0003993272750000046
Then point v needs to be pointed to b ' punishing, updating v according to equation (3) b ' diffusion penalty f b '; if t is p Success rate of diffusion of>
Figure FDA0003993272750000047
Then v is not aligned with the point b ' punishment is carried out, and the diffusion punishment value f is updated according to the formula (4) b '; +in equations (3) and (4)>
Figure FDA0003993272750000048
Respectively represent a parent triangle t p Diffusion penalty, f, of three vertices u Is a preset penalty value;
Figure FDA0003993272750000049
Figure FDA00039932727500000410
d4, traverse T a ' repeating steps d2 to d3 for each mesh vertex in T a ' the updating of the mesh vertex diffusion penalty value.
2. The PMVS-based rapid reconstruction method for dense point cloud of building according to claim 1, wherein the method for reconstructing triangular mesh and updating normal information of mesh vertex in step d is as follows:
constructing a triangular mesh for the point cloud after the improved patch optimization and filtering operation by using the method shown in the steps a1 to a2, wherein the visual information of the mesh vertex is kept unchanged; and then, using the normal vector updating method of the grid vertex in the step a3 to update the normal information of the vertex of the grid.
3. The PMVS-based rapid building dense point cloud reconstruction method as claimed in claim 1, wherein n is 1 Is an integer of 1 or more, n 2 Is an integer of 1 or more.
4. The PMVS-based building dense point cloud fast reconstruction method as claimed in claim 1Characterized in that n is 1 Is 3,n 2 Is 2.
CN201810575339.2A 2018-06-06 2018-06-06 PMVS-based rapid building dense point cloud reconstruction method Active CN108986218B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810575339.2A CN108986218B (en) 2018-06-06 2018-06-06 PMVS-based rapid building dense point cloud reconstruction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810575339.2A CN108986218B (en) 2018-06-06 2018-06-06 PMVS-based rapid building dense point cloud reconstruction method

Publications (2)

Publication Number Publication Date
CN108986218A CN108986218A (en) 2018-12-11
CN108986218B true CN108986218B (en) 2023-03-31

Family

ID=64540791

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810575339.2A Active CN108986218B (en) 2018-06-06 2018-06-06 PMVS-based rapid building dense point cloud reconstruction method

Country Status (1)

Country Link
CN (1) CN108986218B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111445472B (en) * 2020-03-26 2023-05-12 达闼机器人股份有限公司 Laser point cloud ground segmentation method, device, computing equipment and storage medium
CN112084288A (en) * 2020-09-16 2020-12-15 广东电网有限责任公司 Visual map management device
CN114429047B (en) * 2022-01-27 2023-08-22 成都理工大学 Triangular mesh-based interpolation method for travel time of quadratic circle equation
CN117523132B (en) * 2023-08-01 2024-05-03 中铁建大桥工程局集团第四工程有限公司 Steel truss arch bridge buckling tower verticality adjusting system and method
CN116681767B (en) * 2023-08-03 2023-12-29 长沙智能驾驶研究院有限公司 Point cloud searching method and device and terminal equipment

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103021017B (en) * 2012-12-04 2015-05-20 上海交通大学 Three-dimensional scene rebuilding method based on GPU acceleration
CN104200517A (en) * 2014-08-06 2014-12-10 北京工业大学 Three-dimensional reconstruction method
CN105825543B (en) * 2016-03-31 2018-06-19 武汉大学 Point off density cloud generation method and system are regarded based on low altitude remote sensing image more
CN106023312B (en) * 2016-05-13 2019-02-22 南京大学 Three-dimensional building object model automatic reconstruction method based on aviation LiDAR data
CN106097436B (en) * 2016-06-12 2019-06-25 广西大学 A kind of three-dimensional rebuilding method of large scene object
CN106570468B (en) * 2016-10-25 2019-11-22 中国人民解放军空军工程大学 A method of rebuilding LiDAR original point cloud contour of building line

Also Published As

Publication number Publication date
CN108986218A (en) 2018-12-11

Similar Documents

Publication Publication Date Title
CN108986218B (en) PMVS-based rapid building dense point cloud reconstruction method
Kato et al. Neural 3d mesh renderer
JP5111638B2 (en) Apparatus and method for dividing a parametric curve into smaller subpatches
US20080238919A1 (en) System and method for rendering of texel imagery
CN111445581A (en) Mesh reconstruction using data-driven priors
CN111640180B (en) Three-dimensional reconstruction method and device and terminal equipment
CN113781667A (en) Three-dimensional structure simplified reconstruction method and device, computer equipment and storage medium
CN104157000B (en) The computational methods of model surface normal
Lionar et al. Neuralblox: Real-time neural representation fusion for robust volumetric mapping
CN116721231A (en) Three-dimensional reconstruction method and system for extensible scene based on unmanned aerial vehicle-mounted positioning
CN113421336B (en) Method and device for volume rendering of dynamic water area, storage medium and electronic device
US20240096022A1 (en) Low-poly mesh generation for three-dimensional models
Vyatkin Method of binary search for image elements of functionally defined objects using graphics processing units
Ma et al. Rapid reconstruction of a three-dimensional mesh model based on oblique images in the Internet of Things
CN110738726B (en) Robot vision-guided three-dimensional object reconstruction method based on octree
CN116543117B (en) High-precision large-scene three-dimensional modeling method for unmanned aerial vehicle images
Lin et al. Multiview textured mesh recovery by differentiable rendering
CN116452748A (en) Implicit three-dimensional reconstruction method, system, storage medium and terminal based on differential volume rendering
CN115631317A (en) Tunnel lining ortho-image generation method and device, storage medium and terminal
Fu et al. Reconstruct 3D geometry of target using differentiable SAR renderer
Seinturier et al. Towards a 3D based platform for cultural heritage site survey and virtual exploration
Poux How to represent 3D Data?
CN116229005B (en) Geodesic determining method and device for three-dimensional roadway model
CN117876625A (en) Multi-view reconstruction method and device combining explicit and implicit surface expression
Wei et al. Real-time rendering of algebraic B-spline surfaces via Bézier point insertion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant