CN112862959B - Real-time probability monocular dense reconstruction method and system based on semantic prior - Google Patents

Real-time probability monocular dense reconstruction method and system based on semantic prior Download PDF

Info

Publication number
CN112862959B
CN112862959B CN202110309748.XA CN202110309748A CN112862959B CN 112862959 B CN112862959 B CN 112862959B CN 202110309748 A CN202110309748 A CN 202110309748A CN 112862959 B CN112862959 B CN 112862959B
Authority
CN
China
Prior art keywords
probability
depth
semantic
pixel
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110309748.XA
Other languages
Chinese (zh)
Other versions
CN112862959A (en
Inventor
季向阳
娄志强
邸研
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202110309748.XA priority Critical patent/CN112862959B/en
Publication of CN112862959A publication Critical patent/CN112862959A/en
Application granted granted Critical
Publication of CN112862959B publication Critical patent/CN112862959B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a real-time probability monocular dense reconstruction method and a system based on semantic prior, wherein the method comprises the following steps: s1, defining a frame image ItFrom the key frame IkCorresponding probability weighted depth map DkAnd relative pose between two frames
Figure DDA0002989043110000011
Calculation of ItCorresponding depth map Dt(ii) a S2, obtaining ItCorresponding depth map DtThen, obtaining a key frame I by adopting an SGM methodkLocal depth observation parameters of; s3, carrying out local initialization processing on the local depth observation parameters; s4, updating I time sequence by using updated local initialization resultkThen updating the depth map D using spatial probability weightingk(ii) a S5, when ItWhen converting into key frame, according to key frame IkIs propagated to ItTo continuously utilize timing information. According to the real-time probability monocular dense reconstruction method based on semantic priors, the dense reconstruction quality can be better improved.

Description

Real-time probability monocular density reconstruction method and system based on semantic prior
Technical Field
The invention relates to the technical field of three-dimensional vision, in particular to a real-time probability monocular dense reconstruction method and system based on semantic prior.
Background
At present, intelligent transportation and automatic driving develop rapidly, vehicles need to have the ability of sensing the three-dimensional structure of the surrounding environment in the driving process, although radars can complete the task, the cost is high, semantic information of a scene is difficult to utilize, the cost of a camera is low, and rich information such as colors and structures can be captured, so how to utilize images to complete dense reconstruction in a dynamic scene becomes a current research hotspot. The vision dense reconstruction technology can output the depths of most pixels in the image and recover the three-dimensional scene information lost by the image, and can be widely applied to tasks such as robot navigation, path planning, three-dimensional object detection and the like.
However, in dense reconstruction, there are many troublesome problems such as limited computational resources, reasonable introduction of variable scene depth range and semantic information, and the like. In the prior art, a conventional SFM (Structure from Motion recovery) method, a filtering-based method, and a CNN (conditional Neural Network) based method are generally used.
The traditional SFM method reconstructs the dense depth based on optimization, has huge calculation cost, and is difficult to meet the requirement of real-time dense reconstruction; although the filtering-based methods can efficiently solve the depth, the methods are difficult to introduce various semantic information, and most of the filtering-based methods model the depth of a single pixel, do not consider the depth relation among the pixels during modeling, do not consider the spatial structure information, and further reduce the accuracy of dense reconstruction results; the CNN-based method requires a data set for training, a real depth map is required in the case of a supervision method, the condition is more rigorous, and the real scene is complex and diverse, so that the conventional methods have strong data dependence and sometimes even have difficulty in outputting reasonable results in completely different scenes. Therefore, there is room for improvement in the above-described technology.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, an object of the present invention is to provide a real-time probabilistic monocular dense reconstruction method based on semantic priors, which can better improve dense reconstruction quality.
The invention also provides a system adopting the real-time probability monocular dense reconstruction method based on semantic priors.
The real-time probability monocular dense reconstruction method based on semantic prior comprises the following steps:
s1, defining a frame image ItFrom the key frame IkCorresponding probability weighted depth map DkAnd relative position between two framesPosture correction device
Figure BDA0002989043090000021
Calculation of ItCorresponding depth map Dt；
S2, obtaining ItCorresponding depth map DtThen, obtaining a key frame I by adopting an SGM methodkLocal depth observation parameters of;
s3, carrying out local initialization processing on the local depth observation parameters;
s4, updating k time sequence by using updated local initialization resultkThen updating the depth map D using spatial probability weightingk；
S5, when ItWhen converting into key frame, according to key frame IkIs propagated to ItTo continuously utilize timing information.
According to the real-time probability monocular dense reconstruction method based on semantic priors, the dense reconstruction quality can be better improved.
According to the real-time probability monocular dense reconstruction method based on semantic prior, provided by the invention, a key frame IkCorresponding to a pixel ukNamely:
Figure BDA0002989043090000022
wherein K is the camera reference matrix utIs ItU corresponding tokPixel coordinate, d (u), represents the depth of pixel u.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the loss function L in the SGM methodcIntroducing a semantic sparse function, namely:
LM＝α(S(p1)，S(p2))Lc
Figure BDA0002989043090000023
wherein p is1And p2And (b) representing the semantics of the pixel p by the corresponding matching point on the two frames, wherein beta is a preset parameter and is more than 1.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the local initialization processing of the local depth observation parameters comprises the following steps: pixel depth probability model initialization processing and region plane probability model initialization processing.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the local depth of the pixel depth probability model
Figure BDA0002989043090000024
The probability density function of (a) is:
Figure BDA0002989043090000031
wherein, pi represents the probability that u is an inner point,
Figure BDA0002989043090000032
are respectively Gaussian distribution
Figure BDA0002989043090000033
Mean and variance of zl，zrAre respectively uniformly distributed
Figure BDA0002989043090000034
Left and right boundaries of (2).
According to the real-time probability monocular dense reconstruction method based on semantic prior, after a proper prior probability distribution is selected, pi is relatedu，zuThe posterior probability distribution of (a) is the product of the gaussian distribution and the beta distribution, i.e.:
Figure BDA0002989043090000035
wherein a isu，buIs shellfishTower distribution
Figure BDA0002989043090000036
Parameter of (d), muu，σuIs a Gaussian distribution
Figure BDA0002989043090000037
The parameter (c) of (c).
According to the real-time probability monocular dense reconstruction method based on semantic prior, the area plane probability model is modeled by adopting vMF distribution, namely:
Figure BDA0002989043090000038
wherein xnIs a random three-dimensional unit vector, μ is the mean unit vector of the distribution, knAre lumped parameters.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the pixel depth probability model and the region plane probability model are related in a soft association mode, namely the pixel p belongs to the region RkThe probability of (c) is:
Pk(p)＝Pcolor(I(p)，I(pk))Pdistance(p，pc)
Figure BDA0002989043090000039
wherein P iscolor，Pdistance，PsemanticAnd PdepthRespectively representing color, pixel distance, semantics and depth probability, pkRepresents a region RkS (p) and i (p) respectively represent the semantics and color of the pixel p, dpRepresenting an estimate of the posterior depth of pixel p,
Figure BDA00029890430900000310
indicates the utilization region RkThe depth of the pixel p is obtained by calculating the estimated value of the posterior plane parameter.
According to the real-time probability monocular dense reconstruction method based on semantic prior, which is disclosed by the embodiment of the invention, when ItFor conversion to key frames, based on z key frames and ItRelative pose between them, obtaining key frame I through semantic SGMtInitial local depth observation parameters; when I istNot converted into key frames according to ItAnd key frame kkRelative pose between them, output key frame I by semantic SGMkLocal depth observation parameters.
According to the real-time probability monocular dense reconstruction system based on the semantic prior, the real-time probability monocular dense reconstruction method based on the semantic prior is adopted. Compared with the prior art, the system has the same advantages as the real-time probability monocular dense reconstruction method based on semantic prior, and is not described again.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flowchart illustration of a real-time probabilistic monocular dense reconstruction method based on semantic priors, in accordance with an embodiment of the present invention;
FIG. 2 is a second flowchart of a real-time probabilistic monocular dense reconstruction method based on semantic priors according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it is to be understood that the terms "central," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like are used in the orientations and positional relationships indicated in the drawings for convenience in describing the invention and to simplify the description, and are not intended to indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and are not to be considered limiting of the invention.
A real-time probabilistic monocular dense reconstruction method based on semantic priors according to an embodiment of the present invention is described below with reference to fig. 1 and 2. As shown in fig. 1, a real-time probabilistic monocular dense reconstruction method based on semantic priors according to an embodiment of the present invention includes the following steps:
s1, defining a frame image ItAccording to key frames IkCorresponding probability weighted depth map DkAnd relative pose between two frames
Figure BDA0002989043090000041
Calculating ItCorresponding depth map Dt；
S2, obtaining ItCorresponding depth map DtThen, obtaining a key frame k by adopting an SGM (Semi Global Matching) methodkLocal depth observation parameters of;
s3, carrying out local initialization processing on the local depth observation parameters;
s4, updating I time sequence by using updated local initialization resultkThen updating the depth map D using spatial probability weightingkTherefore, the method is beneficial to continuously optimizing the depth of the key frame and improving the accuracy of the key frame;
s5, when ItWhen converting into key frame, according to key frame IkIs propagated to ItTo continuously utilize timing information.
According to the real-time probability monocular dense reconstruction method based on semantic priors, the dense reconstruction quality can be better improved.
According to the real-time probability monocular dense reconstruction method based on semantic prior, provided by the embodiment of the invention, the key frame kkCorresponding to a pixel ukNamely:
Figure BDA0002989043090000051
wherein K is the camera reference matrix utIs ItU corresponding tokPixel coordinate, d (u), represents the depth of pixel u.
In addition, I is obtainedtMost of the pixel depth, then using guided filtering as post-processing to fill the hole and smooth the depth map, and finally outputting ItDepth map D oft。
According to the real-time probability monocular dense reconstruction method based on semantic prior, the loss function L in the SGM methodcIntroducing a semantic sparse function, namely:
LM＝α(S(p1)，S(p2))Lc
Figure BDA0002989043090000052
wherein p is1And p2And (b) representing the semantics of the pixel p by the corresponding matching point on the two frames, wherein beta is a preset parameter and is more than 1.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the local initialization processing of the local depth observation parameters comprises the following steps: pixel depth probability model initialization processing and region plane probability model initialization processing.
Further, for the pixel depth probability model, the depth map D is directly usedkThe depth of each pixel is taken as a local observation when DkAnd when the depth of the last certain pixel does not exist or is not in a reasonable depth range, considering the observation as an external point, and fitting by using uniform distribution, otherwise, fitting by using Gaussian distribution and uniform distribution.
Further, for the area plane probability model, it cannot be selected from DkLocal observation of regional plane parameters can be directly obtained, and I can be measuredkA certain area on
Figure BDA0002989043090000053
Is centered at
Figure BDA0002989043090000054
Is composed of
Figure BDA0002989043090000055
A neighborhood of a certain radius range. Further, by calculating
Figure BDA0002989043090000056
Probability P of each pixel in the regioni(P) when Pi(p) is greater than a preset threshold T2And when the pixel p has local depth observation, the p is included in the region
Figure BDA0002989043090000057
Set of trusted points
Figure BDA0002989043090000058
Further, at random
Figure BDA0002989043090000059
Three different points are selected, and the calculation can be carried out according to the three points
Figure BDA00029890430900000510
Further, by repeating this process, a set of depth observations of the center point can be obtained
Figure BDA0002989043090000061
And correspondingA set of planar normal vector observations
Figure BDA0002989043090000062
Further, C is obtaineddAnd CvThen, if CdAnd CvIf the observation quantity is too small, the observation time exterior point of the region is determined, so that the uniform distribution modeling can be directly used. Further, when the number is sufficient, the probability model of the local observation can be initialized using the EM algorithm, for example, in one specific embodiment, the EM algorithm for calculating the parameters of the normal vector probability model is used for specific description, and in the expected calculation step, the probability of the interior point of each observation is calculated, that is, the probability of the interior point of each observation is calculated
Figure BDA0002989043090000063
Figure BDA0002989043090000064
Figure BDA0002989043090000065
Further, in the maximization step, the parameters may be updated, i.e. the parameters may be updated
Figure BDA0002989043090000066
Figure BDA0002989043090000067
Figure BDA0002989043090000068
μ←μ/||μ||2
Figure BDA0002989043090000069
The iteration is carried out for a plurality of times until the maximum iteration time or convergence, and if the probability pi of the last inner point is less than the threshold value TπThen the outliers are considered and the uniform distribution is still used for fitting.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the local depth of the pixel depth probability model
Figure BDA00029890430900000610
The probability density function of (a) is:
Figure BDA00029890430900000611
wherein, pi represents the probability that u is an inner point,
Figure BDA00029890430900000612
are respectively Gaussian distribution
Figure BDA00029890430900000613
Mean and variance of zl，zrAre respectively uniformly distributed
Figure BDA00029890430900000614
Left and right boundaries of (2).
According to the real-time probability monocular dense reconstruction method based on semantic prior, after the proper prior probability distribution is selected, pi is relatedu，zuThe posterior probability distribution of (a) is the product of the gaussian distribution and the beta distribution, i.e.:
Figure BDA00029890430900000615
wherein a isu，buIs a beta distribution
Figure BDA00029890430900000616
Parameter of (d), muu，σuIs a Gaussian distribution
Figure BDA00029890430900000617
The parameter (c) of (c).
Further, when an updated depth observation is obtained for μ, the updated posterior probability is in the form:
Figure BDA0002989043090000071
in particular, this form can be converted to q (π) using a moment matching methodu，du|a′u，b′u，μ′u，σ′u) Namely, the fusion of the updated observation information and the parameter update of the posterior probability are completed, and simultaneously, the forms of Gaussian distribution and beta distribution are still maintained.
According to the real-time probability monocular dense reconstruction method based on semantic prior, the area plane probability model is modeled by adopting vMF distribution, namely:
Figure BDA0002989043090000072
wherein xnIs a random three-dimensional unit vector, μ is the mean unit vector of the distribution, knAre lumped parameters.
Further, the mean of the vMF distribution is:
Figure BDA0002989043090000073
wherein Ij(. cndot.) denotes a modified Bezier curve of the first type, D denotes xnThe dimension.
According to the real-time probability monocular dense reconstruction method based on semantic prior, a pixel depth probability model and a region plane probability model are processedWith soft-associative association, i.e. pixel p belongs to region RkThe probability of (c) is:
Pk(p)＝Pcolor(I(p)，I(pk))Pdistance(p，pc)
Figure BDA0002989043090000074
wherein P iscolor，Pdistance，PsemanticAnd PdepthRespectively representing color, pixel distance, semantics and depth probability, pkRepresents a region RkS (p) and i (p) respectively represent the semantics and color of the pixel p, dpRepresenting an estimate of the posterior depth of pixel p,
Figure BDA0002989043090000075
indicates the utilization region RkThe depth of the pixel p is obtained by calculating the estimated value of the posterior plane parameter.
Further, PkEach probability component in (p) has a uniform form, i.e.
P*(a，b)＝k1 exp(-k2 L(a，b))
Wherein k is1And k2A weight parameter corresponding to each probability component.
Further, for the semantic probability component PsemanticThe component formula is:
Figure BDA0002989043090000081
further, for other probability components, the component formula is:
L(a，b)＝||a-b||2。
further, for a single pixel p, it may belong to multiple regions { R }iI ∈ {1, 2, …, m } }, and further, the probability P can be distributed and calculated for each regioni(p) so that one can be obtained from the posterior plane parameters of each regionDepth d of pixel pi(p), the final depth calculation formula for pixel p is:
Figure BDA0002989043090000082
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0002989043090000083
for an illustrative function, the formula is:
Figure BDA0002989043090000084
according to the real-time probability monocular dense reconstruction method based on semantic prior, which is disclosed by the embodiment of the invention, when ItFor conversion to key frames, based on z key frames and ItRelative pose between them, obtaining key frame I through semantic SGMtInitial local depth observation parameters; when I istNot converted into key frames according to ItAnd key frame IxRelative pose between, output key frame k by semantic SGMkLocal depth observation parameters.
In conclusion, according to the real-time probability monocular dense reconstruction method based on semantic prior, dense reconstruction quality can be improved better.
The invention also provides a real-time probability monocular dense reconstruction system based on semantic priors, which comprises the real-time probability monocular dense reconstruction method based on the semantic priors, so that the system has the advantages of higher dense reconstruction quality and the like.
In the description of the present specification, reference to the description of "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (6)

1. A real-time probability monocular dense reconstruction method based on semantic prior is characterized by comprising the following steps:
s1, defining a frame image ItFrom the key frame IkCorresponding probability weighted depth map DkAnd relative pose between two frames
Figure FDA0003666640390000011
Calculating ItCorresponding depth map Dt；
S2, obtaining ItCorresponding depth map DtThen, obtaining a key frame I by adopting an SGM methodkLocal depth observation parameters of;
s3, carrying out local initialization processing on the local depth observation parameters;
s4, updating I time sequence by using updated local initialization resultkThen updating the depth map D using spatial probability weightingk；
S5, when ItWhen converting into key frame, according to key frame IkIs propagated to ItTo continuously utilize timing information;
the local initialization processing of the local depth observation parameters comprises the following steps: initializing a pixel depth probability model and a region plane probability model;
local depth of pixel depth probability model
Figure FDA0003666640390000012
The probability density function of (a) is:
Figure FDA0003666640390000013
wherein, pi represents the probability that u is an inner point, zu，
Figure FDA0003666640390000014
Are respectively Gaussian distribution
Figure FDA0003666640390000015
Mean and variance of zl，zrAre respectively uniformly distributed
Figure FDA0003666640390000016
Left and right boundaries of (d);
the area plane probability model is modeled by using vMF distribution, namely:
Figure FDA0003666640390000017
wherein xnIs a random three-dimensional unit vector, μ is the mean unit vector of the distribution, knIs a centralized parameter;
the pixel depth probability model and the region plane probability model are associated in a soft association manner, namely that the pixel p belongs to the region RkThe probability of (c) is:
Pk(p)＝Pcolor(I(p)，I(pk))Pdistance(p，pc)
Figure FDA0003666640390000018
wherein P iscolor，Pdistance，PsemanticAnd PdepthRespectively representing color, pixel distance, semantics and depth probability, pkRepresents a region RkS (p) and i (p) respectively represent the semantics and color of the pixel p, dpRepresenting an estimate of the posterior depth of pixel p,
Figure FDA0003666640390000021
indicates the utilization region RkThe depth of the pixel p is obtained by calculating the estimated value of the posterior plane parameter.
2. The real-time probabilistic monocular dense reconstruction method based on semantic priors of claim 1, wherein keyframe IkCorresponding to a pixel ukNamely:
Figure FDA0003666640390000022
wherein K is the camera reference matrix utIs shown as ItU corresponding tokPixel coordinates, d (u), represent the depth of pixel u.
3. The real-time probabilistic monocular dense reconstruction method based on semantic priors of claim 2, wherein the loss function L in SGM methodcIntroducing a semantic sparse function, namely:
LM＝α(S(p1)，S(p2))Lc
Figure FDA0003666640390000023
wherein p is1And p2And (b) representing the semantics of the pixel p by the corresponding matching point on the two frames, wherein beta is a preset parameter and is more than 1.
4. The real-time probabilistic monocular dense reconstruction method based on semantic priors of claim 1, wherein after selecting an appropriate prior probability distribution, with respect to piu，zuHas a posterior probability distribution of Gaussian distributionThe product of the cloth and beta distributions, namely:
Figure FDA0003666640390000024
wherein a isu，buIs a beta distribution
Figure FDA0003666640390000025
Parameter of (d), muu，σuIs a Gaussian distribution
Figure FDA0003666640390000026
The parameter (c) of (c).
5. The real-time probabilistic monocular dense reconstruction method based on semantic priors as claimed in claim 1, wherein when I istFor converting to key frames, based on z key frames and ItRelative pose between them, obtaining key frame I through semantic SGMtInitial local depth observation parameters; when I istNot converted into key frames according to ItAnd key frame IkRelative pose between them, output key frame I by semantic SGMkLocal depth observation parameters.
6. A real-time probability monocular dense reconstruction system based on semantic priors is characterized in that the real-time probability monocular dense reconstruction method based on the semantic priors according to any one of claims 1-5 is adopted.
CN202110309748.XA 2021-03-23 2021-03-23 Real-time probability monocular dense reconstruction method and system based on semantic prior Active CN112862959B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110309748.XA CN112862959B (en) 2021-03-23 2021-03-23 Real-time probability monocular dense reconstruction method and system based on semantic prior

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110309748.XA CN112862959B (en) 2021-03-23 2021-03-23 Real-time probability monocular dense reconstruction method and system based on semantic prior

Publications (2)

Publication Number Publication Date
CN112862959A CN112862959A (en) 2021-05-28
CN112862959B true CN112862959B (en) 2022-07-12

Family

ID=75992737

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110309748.XA Active CN112862959B (en) 2021-03-23 2021-03-23 Real-time probability monocular dense reconstruction method and system based on semantic prior

Country Status (1)

Country Link
CN (1) CN112862959B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107945265A (en) * 2017-11-29 2018-04-20 华中科技大学 Real-time dense monocular SLAM method and systems based on on-line study depth prediction network
CN108416840A (en) * 2018-03-14 2018-08-17 大连理工大学 A kind of dense method for reconstructing of three-dimensional scenic based on monocular camera
CN110349249A (en) * 2019-06-26 2019-10-18 华中科技大学 Real-time dense method for reconstructing and system based on RGB-D data
CN111998862A (en) * 2020-07-02 2020-11-27 中山大学 Dense binocular SLAM method based on BNN

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107945265A (en) * 2017-11-29 2018-04-20 华中科技大学 Real-time dense monocular SLAM method and systems based on on-line study depth prediction network
CN108416840A (en) * 2018-03-14 2018-08-17 大连理工大学 A kind of dense method for reconstructing of three-dimensional scenic based on monocular camera
CN110349249A (en) * 2019-06-26 2019-10-18 华中科技大学 Real-time dense method for reconstructing and system based on RGB-D data
CN111998862A (en) * 2020-07-02 2020-11-27 中山大学 Dense binocular SLAM method based on BNN

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
A Unified Framework for Piecewise Semantic Reconstruction in Dynamic Scenes via Exploiting Superpixel Relations;Yan Di 等;《2020 IEEE International Conference on Robotics and Automation (ICRA)》;20200915;全文 *

Also Published As

Publication number Publication date
CN112862959A (en) 2021-05-28

Similar Documents

Publication Publication Date Title
CN111739078B (en) Monocular unsupervised depth estimation method based on context attention mechanism
CN112132893B (en) Visual SLAM method suitable for indoor dynamic environment
CN108242079B (en) VSLAM method based on multi-feature visual odometer and graph optimization model
DE112018000332T5 (en) SEAL VISUAL SLAM WITH PROBABILISTIC SURFEL MAP
CN111161364B (en) Real-time shape completion and attitude estimation method for single-view depth map
CN111563415A (en) Binocular vision-based three-dimensional target detection system and method
CN111062326B (en) Self-supervision human body 3D gesture estimation network training method based on geometric driving
Qu et al. Depth completion via deep basis fitting
CN113313732A (en) Forward-looking scene depth estimation method based on self-supervision learning
CN110610486A (en) Monocular image depth estimation method and device
CN112132862B (en) Adaptive scale estimation target tracking algorithm based on unmanned aerial vehicle
Rosu et al. Semi-supervised semantic mapping through label propagation with semantic texture meshes
CN112085849A (en) Real-time iterative three-dimensional modeling method and system based on aerial video stream and readable medium
Min et al. Voldor+ slam: For the times when feature-based or direct methods are not good enough
CN107909597B (en) Multi-scale MRF model image segmentation method with edge preservation function
Shi et al. Self-supervised learning of depth and ego-motion with differentiable bundle adjustment
CN113034681B (en) Three-dimensional reconstruction method and device for spatial plane relation constraint
CN112862959B (en) Real-time probability monocular dense reconstruction method and system based on semantic prior
CN113628246A (en) Twin network target tracking method based on 3D convolution template updating
CN117523100A (en) Three-dimensional scene reconstruction method and device based on neural network and multi-view consistency
CN117726747A (en) Three-dimensional reconstruction method, device, storage medium and equipment for complementing weak texture scene
CN111598927A (en) Positioning reconstruction method and device
Zhao et al. Linear MonoSLAM: A linear approach to large-scale monocular SLAM problems
CN112802202A (en) Image processing method, image processing device, electronic equipment and computer storage medium
CN115933652A (en) Lunar vehicle direct-drive teleoperation driving method based on sequence image splicing and fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20210528

Assignee: ZHONGQU (BEIJING) TECHNOLOGY CO.,LTD.

Assignor: TSINGHUA University

Contract record no.: X2022110000074

Denomination of invention: Real-time probabilistic monocular density reconstruction method and system based on semantic prior

Granted publication date: 20220712

License type: Common License

Record date: 20221103

EE01 Entry into force of recordation of patent licensing contract