CN105844639A - Depth map fusion and point cloud filtering algorithm based on geometric constraint - Google Patents
Depth map fusion and point cloud filtering algorithm based on geometric constraint Download PDFInfo
- Publication number
- CN105844639A CN105844639A CN201610170969.2A CN201610170969A CN105844639A CN 105844639 A CN105844639 A CN 105844639A CN 201610170969 A CN201610170969 A CN 201610170969A CN 105844639 A CN105844639 A CN 105844639A
- Authority
- CN
- China
- Prior art keywords
- depth map
- point cloud
- cloud
- energy
- filtering algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 7
- 238000001914 filtration Methods 0.000 title abstract description 5
- 238000000034 method Methods 0.000 claims description 22
- 238000005457 optimization Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 2
- 238000007500 overflow downdraw method Methods 0.000 abstract description 3
- 239000013598 vector Substances 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 3
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
A point cloud filtering algorithm based on brightness consistency has relatively low accuracy. Therefore the invention provides a new depth map fusion and point cloud filtering algorithm based on geometric constraint. For measuring geometric consistency and similarity between a partial point cloud and a global point cloud, the invention provides a new surface geometric characteristic descriptor (Histogram of Truncated Icosahedron, HTI). For realizing high reconstruction integrity, the invention further provides a multi-dimensional depth map fusion method based on an optimal frame, wherein conflict and supporting conditions between geometric consistency and three-dimensional points are simultaneously considered. Through test, compared with other algorithms, the point cloud filtering algorithm has an advantage of obtaining a more complete and accurate three-dimensional model.
Description
Technical field
The present invention designs a kind of depth map and merges and some cloud filter method, and the method is a kind of degree of depth based on geometrical constraint
Figure fusion method.
Background technology
Although the three-dimensional rebuilding method merged based on depth map has been widely studied, but such method is still not up to other
The method for reconstructing accuracy of type and integrity；Such as, just with the contiguous image of local in reconstruction based on depth map
Calculating concordance, and in block-based method (PMVS), coupling concordance is based on the overall situation, reconstructed results will be more
Accurately；Although some existing depth map fusion methods are (such as Cross-view Visibility Consistency
Filtering, CVCF) can strengthen the global coherency of depth map, but still face in terms of processing the depth map that noise is bigger
Face bigger difficulty.
Summary of the invention
Propose a kind of new depth map fusion and process the depth map comprising noise, the method base with some cloud filter method
In hypothesis below: estimated that the body surface obtained local should be with the thing obtained by multi-amplitude deepness image fusion by individual depth map
Surface is similar, but the geometric properties of a noise spot cloud is unstable, and therefore, the similarity of surface character is deep for merging
Degree figure provides clue.For measuring similarity, the present invention is extracted the surface of a kind of brand-new design from local with global point cloud
Geometric feature description；Meanwhile, in order to increase the integrity of reconstruction, the present invention make use of again method based on figure to merge many chis
Degree depth map；The particular content of the present invention can be divided into five steps.
The calculating of depth map, the present invention uses method based on multiple dimensioned piece and calculates depth image, the method master
There are three features: depth map obtains after Multi-Scale Calculation；After selecting figure through the overall situation, each pixel can obtain
Oneself randomly choose partial view, these views can be propagated together with normal vector with the degree of depth；Coupling optimum, and not
It is meansigma methods, as Matching power flow function；This step is that ensuing fusion process provides input depth map.
Local and the generation of global point cloud, local surfaces partial points cloud in other words refers to the point produced by single width depth image
Cloud, and overall situation surface or some cloud refer to all local surfaces or some cloud are fused into single model, owing to depth map is to use
Multi-Scale Calculation, first with arest neighbors interpolation, they should be adjusted to best yardstick；For every width depth map, partial points
Cloud can obtain through back projection, for each some p, utilizes from its k nearest point, calculates covariance matrix, and method is as follows
State formula
Calculate the eigenvalue λ of covariance matrix subsequentlyjWith associated characteristic vector vj(j=0,1,2).Afterwards, it is possible to will
Minimal eigenvalue characteristic of correspondence vector is as the normal vector at p point.And surface curvature c can be estimated by following formula
Afterwards, depth map is filtered by each yardstick, by back projection's every width depth map can to a partial points cloud,
Subsequently, initial global point cloud just can obtain by being merged by all partial points clouds.
Surface geometrical property extracts, and geometrical constraint is based on the similarity between local surfaces and overall situation surface.But, existing
Feature Descriptor can not meet description local geometric features and the overall situation conforming requirement of geometric properties.Therefore, the present invention proposes
A kind of simple geometric feature description HTI(Histogram of Truncated Icosahedron rapidly), HTI structure
Being made up of 12 positive five faces, limit and 20 positive six sides, this description is based on query point and gives in radius surface method
The result of vector statistical, all normal vectors are divided into 32 groups according to the HTI face of its indication, when all of method in search area
By rectangular histogram normalization after Vector Processing, form final HTI feature.
Geometry is supported, the HTI Feature Descriptor that geometry support can utilize top to mention calculates, for an inquiry
Point p, implements arest neighbors and searches and calculate corresponding local HTI feature and overall situation HTI feature, define local feature hpAnd the overall situation
Feature hgBetween similarity g be
In addition to the similarity of geometric properties, local surface curvature c can also be comprised in geometry and support, in s, to be defined as follows
Herein, α represents that weight parameter, t are followed by the threshold value during optimization based on figure；Curvature is covered several
What mainly has two reasons in supporting: first, owing to curvature can be used to estimate the slickness of local surfaces, big geometry support
(little curvature) means to make surface more to level off to plane, meets infinitesimal facet and assumes；Second, due to Curvature Estimate it
Depending on the minimal eigenvalue of covariance matrix, easily by noise jamming, less curvature generally means that less noise.
Multiscale Fusion based on figure, in order to merge multiple dimensioned depth map, the present invention propose a kind of based on figure
Optimization Framework.Basic thought is, if point can provide extra information, then just retain this in rougher yardstick
Point, if not having extra information just to delete these points.More specifically, a cloud is set up the figure with two kinds of limit types, G=
(P, S, C), herein, P represents vertex set, and corresponding to the point in some cloud, S and C represents support limit and the conflict limit collection of orientation respectively
Close, in order to set up these limits, first to each summit one radius of influence of definition be
Herein, d represents the degree of depth at this point, and f represents the focal length of the corresponding video camera of this point, and n is the yardstick of depth map, x
Represent the inner product between standard shaft opposite direction and normal vector；Now, two distinct types of limit can be set up as follows: if
There is another p in the radius of influence of q on one point, and the radius of influence of p is less than q, p both can support q, it is also possible to q
Conflict.If q is also within the radius of influence of p, (p, q) is added in figure, because p is more accurate, and q is not on conflict limit
It is provided that more information；Otherwise support that (p, q) would be added in figure, because q can provide within the coverage of p on limit
More information；Afterwards, all points are sorted according to their radius of influence size, from the beginning of the point with least radius,
We only need to detect other points within its radius of influence, and therefore this Algorithms T-cbmplexity is O (nlog n).Along with figure
Setting up, can give its energy equation, target energy E is supported ENERGY E by geometryg, point supports ENERGY EsWith a conflict energy
EcConstituting, it is defined as follows:
β and γ is two weight parameter herein, and variable l is the vector of pixel selection label composition, for a p, label l (p)=1,
If p is selected, otherwise l (p)=0.Geometry supports ENERGY EgIt is defined as follows:
For the support energy of a single point p, the geometry of the point of all support p is used to support that the opposite number of sum defines, then
Use the radius of influence of p to supporting that energy is normalized, be defined as follows:
Finally conflict energy definition is as follows:
In order to minimize E (l), the energy of pointwise test point is contributed and changes label l, and based on above-mentioned formula, a single point can
The contribution of amount E can be measured by following formula
In actual applications, generally whole some cloud is carried out 10 scanning, can ensure that this optimization method is close to local best points.
Claims (2)
1. new depth map merges and is used for processing the depth map comprising noise with some cloud filter method, and the method is based on as follows
Assume: the local objects surface obtained by depth map should be similar to the body surface obtained by multi-amplitude deepness image fusion, for
Increasing the integrity rebuild, the present invention make use of again method based on figure to merge multiple dimensioned depth map.
2. concretely comprise the following steps:
1) depth map calculates, and uses method based on multiple dimensioned piece to calculate depth image
2) partial points cloud and the generation of global point cloud, will be able to be owned to a partial points cloud by back projection's every width depth map
Partial points cloud finally merges into a global point cloud
3) surface geometrical property extracts, it is proposed that a kind of simple geometric feature description HTI(Histogram of rapidly
Truncated Icosahedron) this description is similar to that football, has 20 hexagons, 12 pentagons
4) geometry support, calculates HTI Feature Descriptor, utilizes following formula to obtain geometry support
5) Multiscale Fusion based on figure, uses optimization framework based on figure, gives one energy equation of figure, is defined as follows
This energy equation comprises three parts: geometry supports ENERGY Eg, point support ENERGY EsWith a conflict ENERGY Ec, it is defined as follows
。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610170969.2A CN105844639B (en) | 2016-03-24 | 2016-03-24 | A kind of depth map fusion based on geometrical constraint and point cloud filter method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610170969.2A CN105844639B (en) | 2016-03-24 | 2016-03-24 | A kind of depth map fusion based on geometrical constraint and point cloud filter method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105844639A true CN105844639A (en) | 2016-08-10 |
| CN105844639B CN105844639B (en) | 2019-04-12 |
Family
ID=56583135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610170969.2A Active CN105844639B (en) | 2016-03-24 | 2016-03-24 | A kind of depth map fusion based on geometrical constraint and point cloud filter method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105844639B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600675A (en) * | 2016-12-07 | 2017-04-26 | 西安蒜泥电子科技有限责任公司 | Point cloud synthesis method based on constraint of depth map |
| CN106709904A (en) * | 2016-11-21 | 2017-05-24 | 天津大学 | High-value target subtle change detection method based on active vision |
| US10733718B1 (en) | 2018-03-27 | 2020-08-04 | Regents Of The University Of Minnesota | Corruption detection for digital three-dimensional environment reconstruction |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120243774A1 (en) * | 2010-07-28 | 2012-09-27 | Shenzhen Institutes Of Advanced Technology Chinese Academy Of Sciences | Method for reconstruction of urban scenes |
| CN103714576A (en) * | 2014-01-08 | 2014-04-09 | 北京科技大学 | Three-dimensional key point detection method based on local weighted non-similarity measurement |
| CN104616278A (en) * | 2013-11-05 | 2015-05-13 | 北京三星通信技术研究有限公司 | Interest point detection method and system of three-dimensional (3D) point cloud |
-
2016
- 2016-03-24 CN CN201610170969.2A patent/CN105844639B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120243774A1 (en) * | 2010-07-28 | 2012-09-27 | Shenzhen Institutes Of Advanced Technology Chinese Academy Of Sciences | Method for reconstruction of urban scenes |
| CN104616278A (en) * | 2013-11-05 | 2015-05-13 | 北京三星通信技术研究有限公司 | Interest point detection method and system of three-dimensional (3D) point cloud |
| CN103714576A (en) * | 2014-01-08 | 2014-04-09 | 北京科技大学 | Three-dimensional key point detection method based on local weighted non-similarity measurement |
Non-Patent Citations (2)
| Title |
|---|
| PAUL MERRELL等，: "real-time visibility-based fusion of depth maps", 《IEEE 11TH INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
| SIMON FUHRMANN等: "Fusion of Depth Maps with Multiple Scales", 《ACM TRANSACTIONS ON GRAPHICS》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106709904A (en) * | 2016-11-21 | 2017-05-24 | 天津大学 | High-value target subtle change detection method based on active vision |
| CN106600675A (en) * | 2016-12-07 | 2017-04-26 | 西安蒜泥电子科技有限责任公司 | Point cloud synthesis method based on constraint of depth map |
| US10733718B1 (en) | 2018-03-27 | 2020-08-04 | Regents Of The University Of Minnesota | Corruption detection for digital three-dimensional environment reconstruction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105844639B (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106780524A (en) | A kind of three-dimensional point cloud road boundary extraction method | |
| CN106709948A (en) | Quick binocular stereo matching method based on superpixel segmentation | |
| CN103310421B (en) | The quick stereo matching process right for high-definition image and disparity map acquisition methods | |
| WO2022099958A1 (en) | Head-face dimension classification method based on three-dimensional point cloud coordinates | |
| CN111160214A (en) | 3D target detection method based on data fusion | |
| CN111508073B (en) | Method for extracting roof contour line of three-dimensional building model | |
| CN103632146B (en) | A kind of based on head and shoulder away from human body detecting method | |
| CN104616349A (en) | Local curved surface change factor based scattered point cloud data compaction processing method | |
| WO2012073894A1 (en) | Object detecting method and object detecting device using same | |
| CN103308000B (en) | Based on the curve object measuring method of binocular vision | |
| CN103473763B (en) | Road edge detection method based on heuristic Probabilistic Hough Transform | |
| Pound et al. | A patch-based approach to 3D plant shoot phenotyping | |
| Song et al. | Volumetric stereo and silhouette fusion for image-based modeling | |
| CN110807781A (en) | Point cloud simplification method capable of retaining details and boundary features | |
| CN105844639A (en) | Depth map fusion and point cloud filtering algorithm based on geometric constraint | |
| CN111145129A (en) | Point cloud denoising method based on hyper-voxels | |
| KR101549155B1 (en) | Method of automatic extraction of building boundary from lidar data | |
| WO2018133119A1 (en) | Method and system for three-dimensional reconstruction of complete indoor scene based on depth camera | |
| CN102740096A (en) | Space-time combination based dynamic scene stereo video matching method | |
| CN103366158A (en) | Three dimensional structure and color model-based monocular visual road face detection method | |
| Wang et al. | Glass object localization by joint inference of boundary and depth | |
| Kustra et al. | Robust segmentation of multiple intersecting manifolds from unoriented noisy point clouds | |
| CN115082716A (en) | Multi-source point cloud rough matching algorithm for road fine reconstruction | |
| CN107330930A (en) | Depth of 3 D picture information extracting method | |
| CN112508766B (en) | Intelligent interpretation method for rock mass structural plane based on point cloud and GPU (graphics processing Unit) technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Liu Yiguang Inventor after: Wu Pengfei Inventor after: Zheng Yunan Inventor after: Liu Kai Inventor after: Shi Xuelei Inventor after: Feng Jingming Inventor after: Dong Pengfei Inventor after: Cao Liping Inventor before: Liu Yiguang Inventor before: Wu Pengfei Inventor before: Dong Pengfei Inventor before: Cao Liping |
|
| CB03 | Change of inventor or designer information |