CN106251399B - A kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam - Google Patents
A kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam Download PDFInfo
- Publication number
- CN106251399B CN106251399B CN201610755833.8A CN201610755833A CN106251399B CN 106251399 B CN106251399 B CN 106251399B CN 201610755833 A CN201610755833 A CN 201610755833A CN 106251399 B CN106251399 B CN 106251399B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- frame
- image
- key frame
- lsd
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam, method treat scene modeling and record a video the following steps are included: S1, the video data of acquisition covering scene modeling；S2 calculates key frame posture from key frame by lsd-slam modeling technique and is restored the image of three-dimensional information, and generates three-dimensional space point cloud；S3 selects the image that can piece together panorama according to spatial relation from the image for restored three-dimensional information, and generates panoramic picture；Panoramic picture and three-dimensional space point cloud are included in identical spatial frame of reference, and carry out singulation processing to three-dimensional data by S4.It obtains and analyzes video data and obtain key frame, analysis of key frame is to generate three-dimensional space point cloud and panoramic picture, three-dimensional reconstruction is realized by three-dimensional space point cloud and panoramic picture, equipment is not demarcated, hardware device is not depended on, panoramic view data and model data are acquired simultaneously, solves the problems, such as that current three-dimensional reconstruction exists.
Description
Technical field
The present invention relates to the three-dimensional rebuilding method of computer vision field more particularly to a kind of outdoor scenes based on lsd-slam
Three-dimensional rebuilding method and implementing device.
Background technique
Three-dimensional reconstruction refers to the mathematical model established to three-dimension object and be suitble to computer representation and processing, is in computer ring
The basis of its property is handled it, operated and analyzed under border, and establishes the virtual of expression objective world in a computer
The key technology of reality.Common three-dimensional rebuilding method is first to demarcate to video camera, that is, the image for calculating video camera is sat
Then mark system and the relationship of world coordinate system go out three-dimensional information using the information reconstruction in multiple two-dimensional images, surveyed based on photography
The three-dimensional rebuilding method of amount, due to being allocated to residual error, results in light during the bundle adjustment of photograph
Distortion, so that torsional deformation occurs in the texture of threedimensional model, intuitive bandwagon effect is not so good as panoramic picture.
To guarantee that the effect of three-dimensional reconstruction meets displaying demand, three-dimensional reconstruction is carried out usually using panoramic picture at present, but
It is that the three-dimensional reconstruction of panoramic picture is needed using depth information, and the acquisition of depth information relies on substantially panoramic camera at present
With RGB-D video camera, it is strong that this just directly results in current panoramic picture three-dimensional reconstruction hardware device dependence, and panorama is needed to take the photograph
Camera and RGB-D video camera (video camera with distance measuring sensor) are difficult to popularize.Secondly, the three-dimensional reconstruction of current panoramic picture
It still needs to demarcate panoramic camera and RGB-D video camera, has accumulated error while increasing workload.Also,
The panoramic view data of panoramic picture three-dimensional reconstruction and model data separately acquire at present, low efficiency, the operation speed based on sift algorithm
It spends slower.Finally, the three-dimensional reconstruction result of panoramic picture is not carried out singulation at present, objectification can not be carried out to 3D solid
Management, it is difficult to meet profound application demand.In this regard, at present panoramic picture three-dimensional reconstruction still there are it is aforementioned it is various urgently
It solves the problems, such as.
Summary of the invention
The present invention provides a kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam, it is desirable to provide a kind of
Equipment is not demarcated, do not depend on hardware device, while acquiring the three-dimensional rebuilding method of panoramic view data and model data, into one
Step provides a kind of three-dimensional rebuilding method that can show singulation to reconstruction fructufy, and the three-dimensional reconstruction for solving current panoramic picture exists
The problem of.
A kind of outdoor scene three-dimensional rebuilding method based on lsd-slam of the present invention, comprising the following steps: S1 treats modeling
Scene is recorded a video, the video data of acquisition covering scene modeling；
S2 calculates key frame posture from key frame by lsd-slam modeling technique and is restored the figure of three-dimensional information
Picture, and generate three-dimensional space point cloud；
S3 selects the figure that can piece together panorama from the image for restored three-dimensional information according to spatial relation
Picture, and generate panoramic picture；
Panoramic picture and three-dimensional space point cloud are included in identical spatial frame of reference, and carry out monomer to three-dimensional data by S4
Change processing.
A kind of outdoor scene three-dimensional rebuilding method based on lsd-slam of the present invention, can arbitrarily be obtained using single-lens reflex camera, mobile phone etc.
The equipment for taking video treats scene modeling and carries out continuous annular video recording acquisition video data, and analysis video data obtains key frame
Posture generates three-dimensional space point cloud using key frame posture and obtains three-dimensional modeling data, passes through the analysis to keyframe sequence
Camera position is extracted, the plane centered on projection key frame images to fitting photo centre, split different frame is to obtain panorama
Panoramic picture and three-dimensional space unit are included in identical spatial frame of reference and complete three-dimensional reconstruction to panoramic picture by image.
Camera position is extracted in the analysis for the key frame that this method passes through, and without demarcating to equipment, reduces workload
While also avoid accumulated error；Three-dimensional space point cloud is generated using key frame posture to be not necessarily to provide depth letter by hardware
Breath is different from the three-dimensional reconstruction that the existing depth information obtained based on RGB-D video camera completes panoramic picture, without using spy
Device dependence is not present in kind equipment, the equipment that can arbitrarily obtain video data；Three-dimensional space is generated by analysis of key frame
Between put cloud (i.e. model data) and panoramic picture, while acquiring panoramic view data and model data, it is more efficient, and the present invention
Use surf as the innovatory algorithm of sift, arithmetic speed has a distinct increment.In addition, of the present invention based on lsd-slam's
Outdoor scene three-dimensional rebuilding method also carries out singulation processing to three-dimensional data, by single target (such as tables and chairs, electric appliance, pipeline) from unordered
Threedimensional model in extract, and get up with respective attributes database association, for searching and managing use, solve complete at present
The three-dimensional reconstruction of scape image can not carry out objectification management to 3D solid, it is difficult to the problem of meeting profound application demand.
A kind of implementing device of outdoor scene three-dimensional rebuilding method based on lsd-slam of the present invention, comprising: data acquisition
Unit treats scene modeling and records a video, the video data of acquisition covering scene modeling, and collected video data is sent to
The analysis of lsd-slam modeling unit；
Lsd-slam modeling unit analyzes the video data received, by lsd-slam modeling technique from key frame solution
It calculates key frame posture to be restored the image of three-dimensional information, it is single that the image for restoring three-dimensional information is sent to panoramic view data modeling
Member, and generate three-dimensional space point cloud and be sent to three-dimensional scenic integrated unit；
Panoramic view data modeling unit, receive restore three-dimensional information image, from the image for restored three-dimensional information according to
Spatial relation selects the image that can piece together panorama, and generates panoramic picture, sends panoramic picture three-dimensional scenic and melts
Close unit；
Three-dimensional scenic integrated unit receives panoramic picture and three-dimensional space point cloud, by panoramic picture and three-dimensional space point cloud
It is included in identical spatial frame of reference, and singulation processing is carried out to three-dimensional data.
A kind of implementing device of outdoor scene three-dimensional rebuilding method based on lsd-slam of the present invention is acquired single using data
Member obtains the video counts to scene modeling, and lsd-slam modeling unit analysis video data obtains key frame and calculates key frame
Posture is to generate three-dimensional space point cloud (i.e. model data), and panoramic view data modeling unit analysis of key frame is to extract camera position
It sets, the plane centered on projection key frame images to fitting photo centre, split different frame is to obtain panoramic picture, three-dimensional scenic
The panoramic picture received and three-dimensional space point cloud (i.e. three-dimensional modeling data) are included in identical spatial frame of reference by integrated unit
Complete the three-dimensional reconstruction to panoramic picture.The present apparatus is using panoramic view data modeling unit analysis of key frame to extract camera position
It sets, without demarcating to equipment, also avoids accumulated error while reducing workload；Pass through lsd-slam modeling unit solution
Calculation obtains key frame posture and generates three-dimensional space point cloud (i.e. three-dimensional modeling data), without obtaining depth information by hardware,
Then without using special equipments such as the RGB-D video cameras that can obtain depth information, the equipment that can arbitrarily obtain video data,
There is no device dependences；Lsd-slam modeling unit analysis of key frame generates three-dimensional space point cloud (i.e. model data), panorama
Data modeling unit analysis of key frame generates panoramic picture, acquires panoramic view data and pattern number simultaneously using data acquisition unit
According to, it is more efficient, and use surf as the innovatory algorithm of sift in apparatus of the present invention, arithmetic speed has a distinct increment.This
Outside, three-dimensional scenic integrated unit of the present invention also to three-dimensional data carry out singulation processing, by single target (such as tables and chairs,
Electric appliance, pipeline) it is extracted from unordered threedimensional model, and get up with respective attributes database association, for searching and managing
It uses, the three-dimensional reconstruction for solving current panoramic picture can not carry out objectification management to 3D solid, it is difficult to meet profound
Application demand the problem of.
Detailed description of the invention
Fig. 1 is a kind of work flow diagram of outdoor scene three-dimensional rebuilding method based on lsd-slam；
Fig. 2 is a kind of system block diagram of the implementing device of outdoor scene three-dimensional rebuilding method based on lsd-slam.
Specific embodiment
As shown in Figure 1, a kind of outdoor scene three-dimensional rebuilding method based on lsd-slam, comprising the following steps: S1 treats modeling
Scene is recorded a video, the video data of acquisition covering scene modeling；S2 is calculated by lsd-slam modeling technique from key frame
Key frame posture is restored the image of three-dimensional information, and generates three-dimensional space point cloud；S3, from the image for having restored three-dimensional information
In the image that can piece together panorama is selected according to spatial relation, and generate panoramic picture；S4, by panoramic picture and three
Dimension space point cloud is included in identical spatial frame of reference, and carries out singulation processing to three-dimensional data.It obtains and analyzes video data
Key frame is obtained, analysis of key frame passes through three-dimensional space point cloud and panoramic picture to generate three-dimensional space point cloud and panoramic picture
It realizes three-dimensional reconstruction, equipment is not demarcated, do not depend on special hardware equipment, while acquiring panoramic view data and model data
Three-dimensional rebuilding method, it is further provided a kind of three-dimensional rebuilding method that can show singulation to reconstruction fructufy solves current panorama
The three-dimensional reconstruction of image there are the problem of.
Step S2 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam is the following steps are included: S21, video
First frame as initial key frame insetion sequence, initial frame feature is extracted using Surf algorithm；S22, to video subsequent frame by
Frame is detected, and the new feature that frame is added, and the last frame F that frame Fi and keyframe sequence F will newly be added are extractednIt is moved
Estimation obtains the rotation and translation vector between adjacent image；S23, with the rotation and translation between the adjacent image of acquisition to
Amount is as last frame F in keyframe sequence FnFrame F is added with newiRelative motion e, relative motion e is compared with threshold value
Compared with, retain threshold value in FiAs new key frame, keyframe sequence is added, relative motion e is denoted as new addition frame FiWith pass
Last frame F in key frame sequence FnBetween motion vector EI-1, i, restore the three-dimensional letter of image for obtaining key frame posture
Breath；S24: after obtaining the exact posture of key frame, the bundle adjustment between key frame is carried out, it is of the same name to calculate image space
The space coordinate in object space is put, to obtain the three-dimensional space point cloud with space coordinate.Video counts of the step to acquisition
It according to being analyzed, determines key frame and resolves key frame posture and be restored the image of three-dimensional information, and generate three-dimensional space point
Cloud, realization obtain model data from video data.
Since during moving accumulation, the accumulation of error is inevitable.In this method the posture of subsequent frame according to
Rely the posture of front, it is desirable to guarantee the accurate of modeling, it is necessary to guarantee that matching is all accurate every time, and this is to be difficult to realize
, it needs to carry out global optimization processing to the kinematic error of accumulation, so there is also tired between step S23 and step S24
The step of product Error processing.Optimized in the present embodiment using posture figure to handle accumulated error.
XiInclude three line elements and three angle elements for key frame posture, is described as follows that (wherein R is with attitude matrix
The orthogonal spin matrix of 3*3, XtFor translation vector):
Ti,jIt is 4*4 matrix for the motion vector generated by frame matching, indicates that the i-th frame is moved to jth frame.xi=Ti, jxjG={ V, E }: optimization figure is made of, one key frame of each node on behalf side and node.Theoretically there is xi=Ti,jxjDue to
The presence of error, actual conditions are as follows: Ei,j=(xi-Ti,jxj) by defining two norms to error, obtain the mesh of optimization problem
Scalar functions:
Solving to it can be obtained optimum results.
Step S21 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam is the following steps are included: S211, to first
The pixel of beginning key frame carries out gaussian filtering；S212 carries out Hessian matrix disposal to filtered pixel and obtains tentatively
Characteristic point；S213 rejects the preliminary characteristic point for being less than threshold value, and the characteristic point of sub-pixel is obtained using linear interpolation method；
S214 determines the principal direction of characteristic point；S215 constructs surf feature point description operator, obtains initial frame feature.
Step S21 is in order to initialize keyframe sequence, using the first frame of video as key frame insetion sequence of publishing books, so
Initial frame feature is extracted using Surf algorithm afterwards.Hessian matrix is the core of Surf algorithm.In order to guarantee that characteristic point has ruler
Independence is spent, so needing to carry out gaussian filtering to it before carrying out Hessian matrix construction.It is being carried out after filtering
The calculating of Hessian, formula are as follows:
Its characteristic point transform is as follows:
A is constant in above formula, and it is a common empirical value 0.9 that A is chosen in the present embodiment.
By the three-dimensional space neighborhood of each pixel and a diameter of 3 pixels crossed by hessian matrix disposal
Other 26 points carry out size comparison, if it is the extreme value in this 26 points, reservation is changed the time as preliminary characteristic point.So
The characteristic point of sub-pixel is obtained using 3 dimensional linear interpolation methods afterwards, while rejecting the preliminary characteristic point for being less than threshold value.In characteristic point
Two-dimensional neighbourhood (in scale s), one 60 degree of statistics it is fan-shaped in all the points horizontal haar wavelet character and vertical haar it is small
Wave characteristic summation.Then by this 60 degree it is fan-shaped rotate at certain intervals, finally using that fan-shaped direction of maximum value as
The principal direction of this feature point.Finally construct surf feature point description operator.The window of 8 × 8 pixels is taken centered on characteristic point.It asks
The gradient magnitude and gradient direction of each pixel out, is then weighted it with Gauss window, each pixel corresponding one
A vector, length are, for the Gauss weight of the pixel.Then the gradient in 8 directions is calculated on the fritter of every 4 × 4 pixel
Direction histogram draws the accumulated value of each gradient direction, can form a seed point.Then each characteristic point is total by 2 × 2
4 seed point compositions, each seed point have 8 direction vector information.This united thought of neighborhood directivity information enhances
The antimierophonic ability of algorithm also provides preferable fault-tolerance simultaneously for the characteristic matching containing position error.
Step S22 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam is extracted the following steps are included: S221
The new feature that frame is added, the feature extracted using surf is as sample, one RANSAC sample of random selection from sample set, i.e., and 4
A matching double points；S222 calculates transformation matrix M according to this 4 matching double points；S223 according to sample set, transformation matrix M, and is missed
Metric function calculates the consistent collection for meeting current transform matrix, and returns to consistent concentration element number；S224, according to when previous
It causes that element number is concentrated to judge whether that optimal (maximum) unanimously collects, if then updating current optimal consistent collection；S225 updates current
Error probability p repeats step S221 to step S224 if p is greater than the minimum error probability allowed and continues iteration, until current
Error probability p is less than minimum error probability, estimates the movement of image with this, obtain rotation and translation between adjacent image to
Amount.The step detects video subsequent frame frame by frame, extracts new addition frame feature, carries out with the last frame of keyframe sequence
Estimation.It is matched according to the feature that surf is extracted, and with " stochastical sampling consistency "
(RandomSampleConsensus) movement of method estimation image, obtains the rotation and translation vector between adjacent image.
Step S23 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam specifically, for be newly added one
Frame Fi, calculate last frame F in FnWith FiRelative motion e, and judged according to threshold value.
If e > emax, that is, move too big, it may be possible to which it fails to match or camera strenuous exercise, abandons the frame；
If e < emin, illustrate too close from previous key frame, abandon the frame；
If emin≤e≤emax, then illustrate that matching result is preferable, motion estimation is errorless, while having again from a upper key frame
Certain distance, then FiAs new key frame, keyframe sequence is added.
In keyframe sequence, any one frame image can be calculated relative to original by the movement accumulation between adjacent image
The position of point.For arbitrary frame FiTo frame Fi+1Adjacent image, relative motion vector Ei,i+1, then any one frame image Fi
Calculation formula relative to origin position ω is as follows:
Different angle, different coordinate acquired images are snapped in a coordinate system by the ω being calculated.
Step S3 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam is the following steps are included: S31, to key
Image exterior orientation line element (i.e. moment photo centre coordinate) in frame sequence carries out ISODATA clustering；S32 is right
The key frame that one group is fallen in cluster result detects its three-dimensional coverage area, meets full-view image splicing range, then with minimum
Square law is fitted this group of photo centre, and each key frame images are projected to using the photo centre after being fitted as the flat of photo centre
On face；S33 carries out Surf feature extraction to the image in same group of key frame, according to the result after matching by different frame split,
Generate panoramic picture.The step extracts camera position from image, is fitted camera center, each key frame images are projected to
Using the photo centre after being fitted as in the plane of photo centre, split different frame is to obtain panoramic picture.To guarantee panoramic picture
Transition naturally, can be used at edge fit emergence processing.
Step S31 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam is specified the following steps are included: S311
Initial clustering parameter；S312 calculates the range index function of all samples in all kinds of；S313 is once obtained by given requirement by preceding
The cluster set obtained carries out division and merging treatment, to obtain new cluster centre；S314 re-starts interative computation, calculates
Indices, judge whether cluster result meets the requirements, and after successive ignition, if result restrains, operation terminates.The step
Camera position is extracted, orientation judgement is carried out, is fitted camera center, carries out image rectification.
Step S4 described in outdoor scene three-dimensional rebuilding method of this kind based on lsd-slam is the following steps are included: S41, for every
One width panoramic picture normalizes photo centre using it and establishes ball as radius to shoot space radius as the centre of sphere in position in space
Body, and panoramic picture is mapped to sphere inner surface, so that panoramic view data is detached from two-dimensional surface, it is included into three-dimensional point cloud institute really
Fixed spatial frame of reference；S42 is modeled by model single targetization on this basis, by single target from unordered three-dimensional mould
It is extracted in type, carries out singulation processing.Panoramic view data and threedimensional model are included in identical spatial frame of reference reality by the step
The three-dimensional reconstruction of existing panoramic picture, and singulation processing is carried out to three-dimensional data and meets profound application demand to meet.
As shown in Fig. 2, a kind of implementing device of the outdoor scene three-dimensional rebuilding method based on lsd-slam, comprising: data acquisition
Unit treats scene modeling and records a video, the video data of acquisition covering scene modeling, and collected video data is sent to
The analysis of lsd-slam modeling unit；Lsd-slam modeling unit, analyzes the video data received, models skill by lsd-slam
Art calculates key frame posture from key frame and is restored the image of three-dimensional information, and the image for restoring three-dimensional information is sent to entirely
Scape data modeling unit, and generate three-dimensional space point cloud and be sent to three-dimensional scenic integrated unit；Panoramic view data modeling unit receives
The image for restoring three-dimensional information, is selected and can be helped with split according to spatial relation from the image for restored three-dimensional information
The image of scape, and panoramic picture is generated, send panoramic picture three-dimensional scenic integrated unit；Three-dimensional scenic integrated unit receives complete
Panoramic picture and three-dimensional space point cloud are included in identical spatial frame of reference by scape image and three-dimensional space point cloud, and to three dimensions
According to progress singulation processing.The video counts to scene modeling are obtained by data acquisition unit, utilize lsd-slam modeling unit
It generates three-dimensional space point cloud (i.e. three-dimensional modeling data), panoramic view data modeling unit generates panoramic picture, and three-dimensional scenic fusion is single
The panoramic picture received and three-dimensional space point cloud (i.e. model data) are included in identical spatial frame of reference and completed to complete by member
The three-dimensional reconstruction of scape image, does not demarcate equipment, does not depend on hardware device, while acquiring panoramic view data and model data
Three-dimensional rebuilding method, it is further provided a kind of three-dimensional rebuilding method that can show singulation to reconstruction fructufy solves current panorama
The three-dimensional reconstruction of image there are the problem of.
Lsd-slam modeling unit packet described in implementing device of this kind based on the outdoor scene three-dimensional rebuilding method of lsd-slam
It includes, initializes keyframe sequence module, using the first frame of video as initial key frame insetion sequence, extracted using Surf algorithm
Initial frame feature；Motion calculation module detects video subsequent frame frame by frame, extracts the new feature that frame is added, and will newly add
Enter frame FiWith the last frame F of keyframe sequence FnEstimation is carried out, the rotation and translation vector between acquisition adjacent image is simultaneously
It is sent to key frame alignment module；Key frame alignment module, receive adjacent image between rotation and translation vector as
Last frame F in keyframe sequence FnFrame F is added with newiRelative motion e, relative motion e is compared with threshold value, retain
As new key frame keyframe sequence is added, relative motion e is denoted as new addition frame Fi and key frame sequence in Fi in threshold value
Arrange last frame F in FnBetween motion vector Ei-1, i, restore the three-dimensional information of image for obtaining key frame posture, will
Key frame posture is sent to a cloud generation module；Point cloud generation module, receives key frame posture, and it is flat to carry out flux of light method to key frame
Difference calculates image space same place in the space coordinate of object space, obtains the three-dimensional space point cloud with space coordinate.The unit is to obtaining
The video data taken is analyzed, and is determined key frame and is resolved key frame posture and is restored the image of three-dimensional information, and is generated
Three-dimensional space point cloud, realization obtain model data from video data.
Panoramic view data modeling unit packet described in implementing device of this kind based on the outdoor scene three-dimensional rebuilding method of lsd-slam
It includes, full-view image detection module, to the image exterior orientation line element (i.e. moment photo centre coordinate) in keyframe sequence
ISODATA clustering is carried out, cluster result is sent to photo centre's fitting module；Photo centre's fitting module receives simultaneously
Cluster result is analyzed, for falling in one group of key frame in cluster result, its three-dimensional coverage area is detected, meets full-view image spelling
Range is connect, then this group of photo centre is fitted with least square method, and each key frame images are projected in the photography after fitting
The heart is in the plane of photo centre；Image mosaic module carries out Surf feature extraction to the image in same group of key frame, according to
Different frame split is generated panoramic picture by the result after matching.The unit extracts camera position from image, is fitted camera
Center projects to each key frame images using the photo centre after being fitted as in the plane of photo centre, split different frame is to obtain
To panoramic picture.To guarantee the transition of panoramic picture naturally, emergence processing can be used at edge fit.
Claims (6)
1. a kind of outdoor scene three-dimensional rebuilding method based on lsd-slam, comprising the following steps:
S1 treats scene modeling and records a video, the video data of acquisition covering scene modeling；
S2 calculates key frame posture from key frame by lsd-slam modeling technique and is restored the image of three-dimensional information, and
Generate three-dimensional space point cloud；
S3 selects the image that can piece together panorama from the image for restored three-dimensional information according to spatial relation, and
Generate panoramic picture；
Panoramic picture and three-dimensional space point cloud are included in identical spatial frame of reference, and carried out at singulation to three-dimensional data by S4
Reason, which is characterized in that the step S3 the following steps are included:
S31 carries out ISODATA clustering to the image exterior orientation line element in keyframe sequence；
S32 detects its three-dimensional coverage area, meets full-view image splicing model for falling in one group of key frame in cluster result
It encloses, then this group of photo centre is fitted with least square method, and each key frame images are projected to and are with the photo centre after being fitted
In the plane of photo centre；
S33 carries out Surf feature extraction to the image in same group of key frame, according to the result after matching by different frame split,
Generate panoramic picture.
2. a kind of outdoor scene three-dimensional rebuilding method based on lsd-slam according to claim 1, which is characterized in that the step
Rapid S31 the following steps are included:
S311 sets initial clustering parameter；
S312 calculates the range index function of all samples in all kinds of；
The preceding cluster set once obtained is carried out division and merging treatment, to obtain new cluster by given requirement by S313
Center；
S314 re-starts interative computation, calculates indices, judges whether cluster result meets the requirements, by successive ignition
Afterwards, if result restrains, operation terminates.
3. a kind of outdoor scene three-dimensional rebuilding method based on lsd-slam according to claim 1, which is characterized in that the step
Rapid S4 the following steps are included:
S41, for each width panoramic picture, normalizing photo centre using it, the centre of sphere is in position in space, to shoot space half
Diameter is that radius establishes sphere, and panoramic picture is mapped to sphere inner surface, so that panoramic view data is detached from two-dimensional surface, is included into
Into spatial frame of reference determined by three-dimensional point cloud；
S42 is modeled on this basis by model single targetization, single target is extracted from unordered threedimensional model
Come, carries out singulation processing.
4. for realizing the implementing device of any outdoor scene three-dimensional rebuilding method based on lsd-slam of claim 1-3,
It is characterised by comprising:
Data acquisition unit treats scene modeling and records a video, the video data of acquisition covering scene modeling, and will be collected
Video data is sent to the analysis of lsd-slam modeling unit；
Lsd-slam modeling unit analyzes the video data received, is calculated by lsd-slam modeling technique from key frame
Key frame posture is restored the image of three-dimensional information, and the image for restoring three-dimensional information is sent to panoramic view data modeling unit,
And it generates three-dimensional space point cloud and is sent to three-dimensional scenic integrated unit；
Panoramic view data modeling unit receives the image for restoring three-dimensional information, according to space from the image for restored three-dimensional information
Positional relationship selects the image that can piece together panorama, and generates panoramic picture, and it is single to send the fusion of panoramic picture three-dimensional scenic
Member；
Three-dimensional scenic integrated unit receives panoramic picture and three-dimensional space point cloud, panoramic picture and three-dimensional space point cloud is included in
Identical spatial frame of reference, and singulation processing is carried out to three-dimensional data.
5. a kind of implementing device of outdoor scene three-dimensional rebuilding method based on lsd-slam according to claim 4, feature
It is, the lsd-slam modeling unit includes keyframe sequence module being initialized, using the first frame of video as initial key
Frame insetion sequence extracts initial frame feature using Surf algorithm；
Motion calculation module detects video subsequent frame frame by frame, extracts the new feature that frame is added, and frame F will newly be addediWith
The last frame F of keyframe sequence FnEstimation is carried out, the rotation and translation vector between adjacent image is obtained and is sent to
Key frame alignment module；
Key frame alignment module receives the rotation and translation vector between adjacent image as last in keyframe sequence F
One frame FnFrame F is added with newiRelative motion e, relative motion e is compared with threshold value, retain threshold value in FiAs
New key frame, is added keyframe sequence, and relative motion e is denoted as new addition frame FiWith last frame F in keyframe sequence FnBetween
Motion vector Ei-1,i, restore the three-dimensional information of image for obtaining key frame posture, key frame posture be sent to a cloud
Generation module；
Point cloud generation module, receives key frame posture, carries out bundle adjustment to key frame, calculates image space same place in object space
Space coordinate, obtain the three-dimensional space point cloud with space coordinate.
6. a kind of implementing device of outdoor scene three-dimensional rebuilding method based on lsd-slam according to claim 4, feature
It is, the panoramic view data modeling unit includes:
Full-view image detection module carries out ISODATA clustering to the image exterior orientation line element in keyframe sequence, will gather
Class result is sent to photo centre's fitting module；
Photo centre's fitting module receives and analyzes cluster result, for falling in one group of key frame in cluster result, detects it
Three-dimensional coverage area meets full-view image splicing range, is then fitted this group of photo centre with least square method, and by each key frame
Image projection is to using the photo centre after being fitted as in the plane of photo centre；
Image mosaic module carries out Surf feature extraction to the image in same group of key frame, will not according to the result after matching
Split at same frame generates panoramic picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610755833.8A CN106251399B (en) | 2016-08-30 | 2016-08-30 | A kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610755833.8A CN106251399B (en) | 2016-08-30 | 2016-08-30 | A kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106251399A CN106251399A (en) | 2016-12-21 |
| CN106251399B true CN106251399B (en) | 2019-04-16 |
Family
ID=57597239
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610755833.8A Active CN106251399B (en) | 2016-08-30 | 2016-08-30 | A kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106251399B (en) |
Families Citing this family (40)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875437B (en) * | 2016-12-27 | 2020-03-17 | 北京航空航天大学 | RGBD three-dimensional reconstruction-oriented key frame extraction method |
| WO2018129715A1 (en) * | 2017-01-13 | 2018-07-19 | 浙江大学 | Simultaneous positioning and dense three-dimensional reconstruction method |
| CN106954012A (en) * | 2017-03-29 | 2017-07-14 | 武汉嫦娥医学抗衰机器人股份有限公司 | A kind of high definition polyphaser full-view stereo imaging system and method |
| CN107358624B (en) * | 2017-06-06 | 2020-01-07 | 武汉几古几古科技有限公司 | Monocular dense instant positioning and map reconstruction method |
| CN107451540B (en) * | 2017-07-14 | 2023-09-01 | 南京维睛视空信息科技有限公司 | Compressible 3D identification method |
| CN109389677B (en) * | 2017-08-02 | 2022-10-18 | 珊口（上海）智能科技有限公司 | Real-time building method, system, device and storage medium of house three-dimensional live-action map |
| CN107862720B (en) * | 2017-11-24 | 2020-05-22 | 北京华捷艾米科技有限公司 | Pose optimization method and pose optimization system based on multi-map fusion |
| CN109903660A (en) * | 2017-12-11 | 2019-06-18 | 重庆海扶医疗科技股份有限公司 | Image generating method and device, the Training Methodology of Medical Devices and training system |
| CN108052914A (en) * | 2017-12-21 | 2018-05-18 | 中国科学院遥感与数字地球研究所 | A kind of forest forest resource investigation method identified based on SLAM and image |
| CN110148205B (en) * | 2018-02-11 | 2023-04-25 | 北京四维图新科技股份有限公司 | Three-dimensional reconstruction method and device based on crowdsourcing image |
| CN108566545A (en) * | 2018-03-05 | 2018-09-21 | 中德（珠海）人工智能研究院有限公司 | The method that three-dimensional modeling is carried out to large scene by mobile terminal and ball curtain camera |
| CN108564617B (en) * | 2018-03-22 | 2021-01-29 | 影石创新科技股份有限公司 | Three-dimensional reconstruction method and device for multi-view camera, VR camera and panoramic camera |
| CN108830925B (en) * | 2018-05-08 | 2020-09-15 | 中德（珠海）人工智能研究院有限公司 | Three-dimensional digital modeling method based on spherical screen video stream |
| CN108776989B (en) * | 2018-06-08 | 2022-06-24 | 北京航空航天大学 | Low-texture planar scene reconstruction method based on sparse SLAM framework |
| CN109073398B (en) * | 2018-07-20 | 2022-04-08 | 达闼机器人有限公司 | Map establishing method, positioning method, device, terminal and storage medium |
| CN109461180B (en) * | 2018-09-25 | 2022-08-30 | 北京理工大学 | Three-dimensional scene reconstruction method based on deep learning |
| CN110969574A (en) * | 2018-09-29 | 2020-04-07 | 广州汽车集团股份有限公司 | Vehicle-mounted panoramic map creation method and device |
| CN109785278B (en) * | 2018-12-21 | 2024-05-31 | 北京大学深圳研究生院 | Three-dimensional foot-type image processing method and device, electronic equipment and storage medium |
| CN110189399B (en) * | 2019-04-26 | 2021-04-27 | 浙江大学 | Indoor three-dimensional layout reconstruction method and system |
| CN110349253B (en) * | 2019-07-01 | 2023-12-01 | 达闼机器人股份有限公司 | Three-dimensional reconstruction method of scene, terminal and readable storage medium |
| CN110648398B (en) * | 2019-08-07 | 2020-09-11 | 武汉九州位讯科技有限公司 | Real-time ortho image generation method and system based on unmanned aerial vehicle aerial data |
| CN110706357B (en) * | 2019-10-10 | 2023-02-24 | 青岛大学附属医院 | Navigation system |
| CN111310853B (en) * | 2020-03-06 | 2022-08-02 | 埃洛克航空科技（北京）有限公司 | Single recognition algorithm based on neural network and elevation fusion |
| CN113379822B (en) * | 2020-03-16 | 2024-03-22 | 天目爱视（北京）科技有限公司 | Method for acquiring 3D information of target object based on pose information of acquisition equipment |
| CN111462326A (en) * | 2020-03-31 | 2020-07-28 | 武汉大学 | Low-cost 360-degree panoramic video camera urban pipeline three-dimensional reconstruction method and system |
| CN111724481A (en) * | 2020-06-24 | 2020-09-29 | 嘉应学院 | Method, device, equipment and storage medium for three-dimensional reconstruction of two-dimensional image |
| CN111739158B (en) * | 2020-06-29 | 2023-04-25 | 成都信息工程大学 | Three-dimensional scene image recovery method |
| CN111986086B (en) * | 2020-08-27 | 2021-11-09 | 贝壳找房(北京)科技有限公司 | Three-dimensional image optimization generation method and system |
| CN112200854B (en) * | 2020-09-25 | 2023-10-17 | 华南农业大学 | Leaf vegetable three-dimensional phenotype measuring method based on video image |
| CN112258391B (en) * | 2020-10-12 | 2022-05-17 | 武汉中海庭数据技术有限公司 | Fragmented map splicing method based on road traffic marking |
| CN112598780B (en) * | 2020-12-04 | 2024-04-05 | Oppo广东移动通信有限公司 | Instance object model construction method and device, readable medium and electronic equipment |
| CN112488022B (en) * | 2020-12-11 | 2024-05-10 | 武汉理工大学 | Method, device and system for monitoring panoramic view |
| CN112365406B (en) * | 2021-01-13 | 2021-06-25 | 芯视界（北京）科技有限公司 | Image processing method, device and readable storage medium |
| CN113124883B (en) * | 2021-03-01 | 2023-03-28 | 浙江国自机器人技术股份有限公司 | Off-line punctuation method based on 3D panoramic camera |
| CN113345084B (en) * | 2021-06-29 | 2022-10-21 | 吉林省交通规划设计院 | Three-dimensional modeling system and three-dimensional modeling method |
| CN116206050A (en) * | 2021-11-30 | 2023-06-02 | 中兴通讯股份有限公司 | Three-dimensional reconstruction method, electronic device, and computer-readable storage medium |
| CN114399597B (en) * | 2022-01-12 | 2022-10-28 | 贝壳找房(北京)科技有限公司 | Method and device for constructing scene space model and storage medium |
| CN114449247B (en) * | 2022-04-11 | 2022-07-22 | 深圳市其域创新科技有限公司 | Multi-channel video 3D (three-dimensional) superposition method and system |
| CN115103174A (en) * | 2022-05-27 | 2022-09-23 | 南昌威爱信息科技有限公司 | Method and apparatus for delivering volumetric video content |
| CN115631314B (en) * | 2022-12-19 | 2023-06-09 | 中汽研（天津）汽车工程研究院有限公司 | Point cloud map construction method based on multi-feature and self-adaptive key frames |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9495761B2 (en) * | 2013-11-04 | 2016-11-15 | The Regents Of The University Of California | Environment mapping with automatic motion model selection |
| US11051000B2 (en) * | 2014-07-14 | 2021-06-29 | Mitsubishi Electric Research Laboratories, Inc. | Method for calibrating cameras with non-overlapping views |
| CN105374011B (en) * | 2015-12-09 | 2019-03-15 | 中电科信息产业有限公司 | Point-clouds Registration method and device based on panoramic picture |
| CN105809681A (en) * | 2016-03-04 | 2016-07-27 | 清华大学 | Single camera based human body RGB-D data restoration and 3D reconstruction method |
| CN105783913A (en) * | 2016-03-08 | 2016-07-20 | 中山大学 | SLAM device integrating multiple vehicle-mounted sensors and control method of device |
-
2016
- 2016-08-30 CN CN201610755833.8A patent/CN106251399B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN106251399A (en) | 2016-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106251399B (en) | A kind of outdoor scene three-dimensional rebuilding method and implementing device based on lsd-slam | |
| CN107204010B (en) | A kind of monocular image depth estimation method and system | |
| CN109151439B (en) | Automatic tracking shooting system and method based on vision | |
| CN106327571B (en) | A kind of three-dimensional face modeling method and device | |
| CN105096269B (en) | The method and system of image radial distortion correction based on distortion linear structure detection | |
| WO2020240284A3 (en) | Vehicle environment modeling with cameras | |
| CN106127788B (en) | A kind of vision barrier-avoiding method and device | |
| CN103426200B (en) | Tree three-dimensional reconstruction method based on unmanned aerial vehicle aerial photo sequence image | |
| CN110598590A (en) | Close interaction human body posture estimation method and device based on multi-view camera | |
| CN109448105B (en) | Three-dimensional human body skeleton generation method and system based on multi-depth image sensor | |
| CN109118544A (en) | Synthetic aperture imaging method based on perspective transform | |
| CN114119739A (en) | Binocular vision-based hand key point space coordinate acquisition method | |
| CN110243390B (en) | Pose determination method and device and odometer | |
| CN107194985A (en) | A kind of three-dimensional visualization method and device towards large scene | |
| CN112200854B (en) | Leaf vegetable three-dimensional phenotype measuring method based on video image | |
| CN108519102A (en) | A kind of binocular vision speedometer calculation method based on reprojection | |
| CN111127556B (en) | Target object identification and pose estimation method and device based on 3D vision | |
| CN106530407A (en) | Three-dimensional panoramic splicing method, device and system for virtual reality | |
| CN110009675A (en) | Generate method, apparatus, medium and the equipment of disparity map | |
| CN111582036A (en) | Cross-view-angle person identification method based on shape and posture under wearable device | |
| CN105488780A (en) | Monocular vision ranging tracking device used for industrial production line, and tracking method thereof | |
| CN114882106A (en) | Pose determination method and device, equipment and medium | |
| CN117834839A (en) | Multi-view 3D intelligent imaging measurement system based on mobile terminal | |
| CN112365589A (en) | Virtual three-dimensional scene display method, device and system | |
| CN112085777A (en) | Six-degree-of-freedom VR glasses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 510800 No.929, Yueyang First Street, Guangzhou High-tech Industrial Development Zone, Guangdong Province Applicant after: Guangzhou City shadow Mdt InfoTech Ltd Address before: Room 3A16, 5 Hao 3 storeys, Baohua Road, Xinhua Street, Huadu District, Guangzhou City, Guangdong Province Applicant before: Guangzhou City shadow Mdt InfoTech Ltd |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |