CN110120093A - Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization - Google Patents
Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization Download PDFInfo
- Publication number
- CN110120093A CN110120093A CN201910227761.3A CN201910227761A CN110120093A CN 110120093 A CN110120093 A CN 110120093A CN 201910227761 A CN201910227761 A CN 201910227761A CN 110120093 A CN110120093 A CN 110120093A
- Authority
- CN
- China
- Prior art keywords
- camera
- dimensional
- depth
- rgb
- error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses plotting method three-dimensional in a kind of room RGB-D of diverse characteristics hybrid optimization and systems, the described method comprises the following steps: by the camera depth error in measurement of original grating difference, being corrected to depth data；Camera posture is updated by two 3D visions and three-dimensional geometry feature；According to the random error in camera tracing process, optimize camera track；By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension dot cloud.On the basis of original RGB-D SLAM, due to having fully considered that sensing system error and depth error in measurement influence, 2 three-dimensional feature points of fusion combine mapping, obtain higher three-dimensional point cloud precision.
Description
Technical field
The present invention relates to three-dimensional surveying & mapping field more particularly to a kind of rooms RGB-D of diverse characteristics hybrid optimization
Interior three-dimensional plotting method and system.
Background technique
High-precision three-dimensional spatial information is to be abstracted to closing with hemi-closure space complex environment, and realize diversified intelligence
Applicable data basis.Common closing/half envelope space three-dimensional mapping means mainly have laser scanning and view-based access control model image
The three-dimensional reconstruction of sequence.In the prior art, the three-dimensional reconstruction of view-based access control model image sequence can be clapped by digital camera
The two-dimensional image sequence taken the photograph carrys out three-dimensional scenic in recovery room, and visual information abundant can enhance closed loop detection well.However
It is long and low in dim environment or texture starvation areas job insecurity, precision that this method models the time.
Therefore, the existing technology needs to be improved and developed.
Summary of the invention
The technical problem to be solved in the present invention is that in view of the above drawbacks of the prior art, it is mixed to provide a kind of diverse characteristics
Close three-dimensional plotting method and system in the room RGB-D of optimization, it is intended to solve that three-dimensional reconstruction precision is low in the prior art asks
Topic.
The technical proposal for solving the technical problem of the invention is as follows:
Three-dimensional plotting method in a kind of room RGB-D of diverse characteristics hybrid optimization, wherein the following steps are included:
By the camera depth error in measurement of original grating difference, depth data is corrected；
Camera posture is updated by two 3D visions and three-dimensional geometry feature；
According to the random error in camera tracing process, optimize camera track；
By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension
Point cloud.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein described poor by original grating
Camera depth error in measurement, depth data is corrected before step further include:
Based on pinhole camera model, the inside and outside parameter of vision camera and depth camera is corrected.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein described poor by original grating
Camera depth error in measurement, step is corrected to depth data and is specifically included:
Depth data calibration model is constructed for the camera depth error in measurement of original grating difference, passes through different measurement distances
Lower camera distortion and systematic error optimize depth data calibration model, and correction depth data；
The depth data calibration model are as follows:
Wherein, W1And W2For camera tangential distortion, W3And W4For camera radial distortion, xc tAnd yc tFor after camera calibration
Image coordinate, xp tAnd yp tFor the image coordinate after the correction of infrared projection device.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein described to pass through two 3D visions
Camera attitude step is updated with three-dimensional geometry feature to specifically include:
It detects and matches two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature；
Critical data frame is screened using image blur, the constraint of Feature Points Matching rate and baseline constraint；
Two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature based on acquisition, by minimizing vision
The three-dimensional match point re-projection error of image two and depth image geometric match point range error obtain camera posture renewal.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein the camera posture renewal
Are as follows:
Wherein, F () is camera posture renewal function, and argmin is the collection for all independents variable for F () obtaining minimum value
It closes, KLFor adjacent key frame set, PL 2The two dimensional character obtained for the matching of all visual pattern key frames matches point set, PL 3For view
Feel the three-dimensional feature matching point set obtained in image key frame, DiThe three-dimensional match point obtained for i-th of depth key frame matching
Collect depth value, Obj () is Tukey biweight objective function, eji 2And eji 3It is characterized match point re-projection error, δji l2
And δji l3It is characterized match point and measures noise, σL2And σL3For re-projection error standard deviation, eji DFor geometric match point depth error,
δji dAnd σDIt corresponds respectively to geometric match point depth measurement noise and error to standard deviation, i is positive integer, n is characterized match point
Number, j be characterized matching point set number, ∑ be summation symbol.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein described to be tracked according to camera
Random error in journey, optimization camera Trace step specifically include:
Closed loop detection is carried out to depth image, and determines the boundary weight matrix of depth image；
Camera track is optimized with global optimization cost function according to boundary weight matrix.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein the determining depth image
Boundary weight step specifically includes:
Characteristic matching point in two data frames of side constraint link is obtained into three-dimensional point set to being mapped in depth image,
And side is calculated by three-dimensional point set and constrains error covariance, side constrains error covariance are as follows:
Wherein, cov () is error covariance, and x, y, z is respectively X, Y, the line element in Z-direction, [Pc,i x,Pc,i y,
Pc,i z]TFor three-dimensional point set PcIn i-th point of three-dimensional coordinate, [Pt,i x,Pt,i y,Pt,i z]TFor three-dimensional point set PtIn i-th point three
Tie up coordinate, [ei x,ei y,ei z]TError covariance is constrained for i-th point of side, [R] and [t] is that side constrains transformation matrix；
Pass through the weight matrix when constraining error covariance and angle constraint error covariance obtains, weight matrix are as follows:
Wherein,θ, Ψ are angle element.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein the global optimization cost letter
Number are as follows:
P={ p1,p2,p3,···,pi,···,pj,···,pn}
Wherein, p is camera posture, and V is camera gesture set, trijFor the side of depth image, σijFor information matrix, information
Matrix is the inverse matrix of weight matrix.
Three-dimensional plotting method in the room RGB-D of the diverse characteristics hybrid optimization, wherein the depth by after correction
Degree accordingly and optimization after camera track, RGB-D data set is exported and is specifically included for color three dimension dot cloud step:
According to the depth data after correction, the depth image in RGB-D data set is converted into single frames three-dimensional point cloud；
According to the camera track after the single frames three-dimensional point cloud of acquisition and optimization, RGB-D data set is exported as color three dimension
Point cloud.
Three-dimensional mapping system in a kind of room RGB-D of diverse characteristics hybrid optimization, wherein include: processor, and with institute
The memory of processor connection is stated,
The memory is stored in the room RGB-D of diverse characteristics hybrid optimization three-dimensional mapping program, the diverse characteristics
It is performed the steps of when three-dimensional mapping program is executed by the processor in the room RGB-D of hybrid optimization
By the camera depth error in measurement of original grating difference, depth data is corrected；
Camera posture is updated by two 3D visions and three-dimensional geometry feature；
According to the random error in camera tracing process, optimize camera track；
By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension
Point cloud.
The utility model has the advantages that on the basis of original RGB-D SLAM, due to having fully considered sensing system error and depth
Spending error in measurement influences, and 2 three-dimensional feature points of fusion combine mapping, obtains higher three-dimensional point cloud precision.
Detailed description of the invention
Fig. 1 is the flow chart of three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization in the present invention.
Fig. 2 is the comparison diagram of the depth data correction front and back error distribution of depth camera of the present invention.
Fig. 3 is the effect picture before the depth data correction of depth camera of the present invention.
Fig. 4 is the effect picture after the depth data correction of depth camera of the present invention.
Fig. 5 is that the present invention is based on three dimensional point clouds in the room of sensor track to generate coordinate system figure.
Fig. 6 is the functional schematic block diagram of three-dimensional mapping system in the room RGB-D of diverse characteristics hybrid optimization of the present invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer and more explicit, right as follows in conjunction with drawings and embodiments
The present invention is further described.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and do not have to
It is of the invention in limiting.
Please refer to Fig. 1-Fig. 5, the present invention provides mappings three-dimensional in a kind of room RGB-D of diverse characteristics hybrid optimization
Some embodiments of method.
In recent years, RGB-D sensor (i.e. depth camera), such as Kinect V1 and Structure Sensor etc., in room
Interior instant positioning is above widely used with map structuring application.Such sensor is different from conventional laser mapping principle, based on knot
Structure light realizes distance measuring.Its hardware is by laser projecting apparatus, diffractive-optical element and the big core devices of infrared camera three composition.
The laser of laser projecting apparatus transmitting, which first passes through, is diffused as random hot spot, is then copied into more parts and projects on subject, depth
Information can be resolved by the infrared light spot intensity difference for emitting with reflecting, and export 640*480 with the rate synchronization of 30 frame per second
Depth and visual pattern, effective distance is in 0.5m-5m or so, by the measurement distance after Data correction up to 8m.Such
The advantages that there is equipment range information to obtain in real time, and portability is strong, cheap is very suitable to quick three-dimensional under indoor environment and surveys
Figure.SLAM (simultaneous localization and mapping, instant positioning and map structuring).
As shown in Figure 1, three-dimensional plotting method in a kind of room RGB-D of diverse characteristics hybrid optimization of the invention, including with
Lower step:
Step S10, it is based on pinhole camera model, corrects the inside and outside parameter of vision camera and depth camera.
Specifically, it first on the basis of analysing in depth depth camera working mechanism, is regarded based on pinhole camera model construction
Camera and the inside and outside ginseng joint calibration model of depth camera are felt, by least square adjustment mode to positive vision camera and depth camera
Inside and outside parameter be corrected.
Step S100, by the camera depth error in measurement of original grating difference, depth data is corrected.
Specifically, step S100 is specifically included:
Step S110, depth data calibration model is constructed for the camera depth error in measurement of original grating difference, by not
Same amount ranging optimizes depth data calibration model, and correction depth data from lower camera distortion and systematic error.
According to the depth data acquisition principle of depth camera, depth information is obtained by grating difference information inverse, therefore
It is poor for grating mistake, construct error model.Wherein TdiPoor, the d for true gratingiFor grating residual quantity measured value, edTo there is depth phase
The error amount that machine lens distortion and systematic error generate, the error derive from depth transducer lens distortion and systematic error,
It can be indicated by following formula:
Tdi=di+ed
WhereinRespectively depth camera, infrared projection device tangential distortion,Respectively depth phase
Machine, infrared projection device radial distortion, can be indicated by following formula:
δtang=P1[(xt 2+yt 2)+2xt]+P2xtyt
δrad=xt[K1(xt 2+yt 2)+K2(xt 2+yt 2)2]
Wherein, P1And P2For depth camera tangential distortion parameter, K1And K2For depth camera radial distortion parameter, xtAnd ytFor
The image coordinate after camera calibration.Grating difference error model can be obtained in fusion formula, as shown by the following formula:
ed=(P1[(xt 2+yt 2)+2xt]+P2xtyt)c-(P1[(xt 2+yt 2)+2xt]+P2xtyt)p+(xt[K1(xt 2+yt 2)+K2
(xt 2+yt 2)2])c-(xt[K1(xt 2+yt 2)+K2(xt 2+yt 2)2])p
In view of depth camera and infrared projection device relative attitude are fixed and by factory correction, the camera distortion in the direction y
Negligible, the present invention describes radial distortion using distortion model, as shown by the equation.
Wherein, f () is Seidal distortion function,To be congruent to.
To be further simplified depth error model.Depth data error model (i.e. depth data calibration model) can pass through public affairs
Formula is expressed as follows:
Wherein, W1And W2For camera tangential distortion, W3And W4For camera radial distortion, xc tAnd yc tFor after camera calibration
Image coordinate, xp tAnd yp tFor the image coordinate after the correction of infrared projection device.
Depth data error model based on derivation, has tentatively drafted depth data correcting process.Correction data acquisition side
Face acquires target data within the scope of 0.5-6m using 0.5m distance as initial position with the interval of 0.5m or so, and pass through whole station
Instrument gets the accurate three-dimensional coordinate of each angle point, by depth error model iteration optimization, obtains optimal depth error model
Parameter, and then realize the high-precision correction of 0.5-6m range depth data.Fig. 2 show before RGB-D sensor calibration with correction
Error map afterwards, it can be seen that depth error is significantly reduced after correction, can maintain depth error within the scope of 4m
Within the scope of 2cm, depth error is less than 4cm within the scope of 6m.Fig. 3 and Fig. 4 is the example of depth correction in real scene, before correction
There are a large amount of noise, rough structures, after corrected, it can be seen that Triangulation Network Model is in ceiling plane for planar structure
On more smooth and overall model it is more regular.
It is worth noting that the depth data for different measurements under, building are related to camera distortion and systematic error
Depth data calibration model, not only improved depth data accuracy in measurement, but also expand depth camera measurement range.
Step S200, camera posture is updated by two 3D visions and three-dimensional geometry feature.
Specifically, step S200 is specifically included:
Step S210, detection and two 3D vision features of matching, three-dimensional geometry point feature and three-dimensional line feature.
Step S220, critical data frame is sieved using image blur, the constraint of Feature Points Matching rate and baseline constraint
Choosing.
Step S230, two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature based on acquisition, by most
The three-dimensional match point re-projection error of smallization visual pattern two and depth image geometric match point range error obtain camera posture more
Newly.
Specifically, the camera posture renewal are as follows:
Wherein, F () is camera posture renewal function, and argmin is the collection for all independents variable for F () obtaining minimum value
It closes, KLFor adjacent key frame set, PL 2The two dimensional character obtained for the matching of all visual pattern key frames matches point set, PL 3For view
Feel the three-dimensional feature matching point set obtained in image key frame, DiThe three-dimensional match point obtained for i-th of depth key frame matching
Collect depth value, Obj () is Tukey biweight objective function, eji 2And eji 3It is characterized match point re-projection error, δji l2
And δji l3It is characterized match point and measures noise, σL2And σL3For re-projection error standard deviation, eji DFor geometric match point depth error,
δji dAnd σDIt corresponds respectively to geometric match point depth measurement noise and error to standard deviation, i is positive integer, n is characterized match point
Number, j be characterized matching point set number, ∑ be summation symbol.
Specifically, the present invention is based on two 3D vision observations of acquisition (i.e. two 3D vision features) and three-dimensional geometry to see
Measured value (i.e. three-dimensional geometry point feature and three-dimensional line feature), can be by minimizing the three-dimensional match point re-projection error of visual pattern two
Sensor attitude is obtained with depth image geometric match point range error to update.It is mixed using two 3D visions and three-dimensional geometry feature
Optimization posture renewal model method is closed, the complementation with enhancing of diverse characteristics in complex environment sensor tracing process is realized, improves
Sensor tracks robustness and builds figure precision.
Step S300, according to the random error in camera tracing process, optimize camera track.
The error generated in sensor tracing process is distributed in figure in each side, causes accumulated error, is reduced accumulation and is missed
Poor problem can be exchanged into nonlinear least square problem.
Specifically, step S300 is specifically included:
Step S310, closed loop detection is carried out to depth image, and determines the boundary weight matrix of depth image.
Specifically, the boundary weight of depth image is determined using following steps:
Step S311, the characteristic matching point in two data frames of side constraint link is obtained to being mapped in depth image
Three-dimensional point set, and side is calculated by three-dimensional point set and constrains error covariance.
To quantify the uncertainty in each freedom degree, it may be assumed that the existing characteristics in two data frames of side constraint link
With point to pcAnd pt, characteristic matching point is mapped in depth image, point of the depth value within the scope of two meters is chosen, depth is missed
Difference is less than 15mm, rejects erroneous matching by RANSAC, residue character match point, which is mapped in depth image, can get correspondence
Three-dimensional point set PcAnd Pt, therefore the three-dimensional point set may be expressed as: in the tri- directions top X, Y, Z constraint residual error
Wherein, x, y, z is respectively X, Y, the line element in Z-direction, [Pc,i x,Pc,i y,Pc,i z]TFor three-dimensional point set PcIn i-th
The three-dimensional coordinate of a point, [Pt,i x,Pt,i y,Pt,i z]TFor three-dimensional point set PtIn i-th point of three-dimensional coordinate, [ei x,ei y,ei z]TFor
I-th point of side constrains error covariance, and [R] and [t] is that side constrains transformation matrix.
So, side constrains error covariance are as follows:
Wherein, cov () is error covariance.
Step S312, pass through the weight matrix when constraining error covariance and angle constraint error covariance obtains.
In view of the uncertainty of the side element precision in graph structure, and the error on the six-freedom degree of side is inconsistent, needs
Opposite side element assigns power.The weight on side can indicate that is, weight matrix gram is indicated by the covariance matrix of six-freedom degree are as follows:
Cn*n=(ci,j,ci,j=cov (Dimi,Dimj)) (0 < i <, 6,0 < j < 6), the higher freedom degree direction weight setting of error is more
It is small.
Specifically, weight matrix are as follows:
Wherein,θ, Ψ are angle element.
Step S320, camera track is optimized with global optimization cost function according to boundary weight matrix.
Specifically, the global optimization cost function are as follows:
P={ p1,p2,p3,···,pi,···,pj,···,pn}
Wherein, p is camera posture, and V is camera gesture set, trijFor the side of depth image, σijFor information matrix, information
Matrix is the inverse matrix of weight matrix, that is to say, that information matrix are as follows:
Step S400, by the depth data after correction and the camera track after optimization, it is by the output of RGB-D data set
Color three dimension dot cloud.
Specifically, step S400 is specifically included:
Step S410, according to the depth data after correction, the depth image in RGB-D data set is converted into single frames three-dimensional
Point cloud；
Specifically, according to the camera internal reference of acquisition, camera distortion parameter can depth image conversion in RGB-D data set
It is independent three-dimensional point cloud, XYZ coordinate is identified by following formula:
Zc=D
Wherein fxD, fyDFor depth camera focal length, cxD, cyDFor depth image principal point coordinate, u', v' are by camera distortion
The image coordinate of correction, these parameters are obtained by camera calibration.
Step S420, according to the camera track after the single frames three-dimensional point cloud of acquisition and optimization, it is by the output of RGB-D data set
Color three dimension dot cloud.
According to the camera track after the single frames three-dimensional point cloud of acquisition and optimization, camera exterior orientation parameter passes through spin matrix RD
With translation matrix tDIt is indicated, point cloud data fusion is carried out by formula, obtain global high-precision color three dimension dot cloud, Fig. 5
For the relational graph between depth camera coordinate and world coordinates.The coordinate of global high-precision color three dimension dot cloud is specific as follows:
The present invention also provides the preferable implementations of mapping system three-dimensional in a kind of room RGB-D of diverse characteristics hybrid optimization
Example:
As shown in fig. 6, three-dimensional mapping system in a kind of room RGB-D of diverse characteristics hybrid optimization of the embodiment of the present invention,
It include: processor 10, and the memory 20 being connect with the processor 10,
The memory 20 is stored in the room RGB-D of diverse characteristics hybrid optimization three-dimensional mapping program, the polynary spy
It levies and is performed the steps of when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of hybrid optimization
By the camera depth error in measurement of original grating difference, depth data is corrected；
Camera posture is updated by two 3D visions and three-dimensional geometry feature；
According to the random error in camera tracing process, optimize camera track；
By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension
Point cloud.
It is also real when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of the diverse characteristics hybrid optimization
Existing following steps:
Based on pinhole camera model, the inside and outside parameter of vision camera and depth camera is corrected, as detailed above.
It is also real when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of the diverse characteristics hybrid optimization
Existing following steps: depth data calibration model is constructed for the camera depth error in measurement of original grating difference, passes through different measurements
Optimize depth data calibration model, and correction depth data apart from lower camera distortion and systematic error；
The depth data calibration model are as follows:
Wherein, W1And W2For camera tangential distortion, W3And W4For camera radial distortion, xc tAnd yc tFor after camera calibration
Image coordinate, xp tAnd yp tFor by infrared projection device correction after image coordinate, as detailed above.
It is also real when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of the diverse characteristics hybrid optimization
Existing following steps:
It detects and matches two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature；
Critical data frame is screened using image blur, the constraint of Feature Points Matching rate and baseline constraint；
Two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature based on acquisition, by minimizing vision
The three-dimensional match point re-projection error of image two and depth image geometric match point range error obtain camera posture renewal, specifically such as
It is upper described.
The camera posture renewal are as follows:
Wherein, F () is camera posture renewal function, and argmin is the collection for all independents variable for F () obtaining minimum value
It closes, KLFor adjacent key frame set, PL 2The two dimensional character obtained for the matching of all visual pattern key frames matches point set, PL 3For view
Feel the three-dimensional feature matching point set obtained in image key frame, DiThe three-dimensional match point obtained for i-th of depth key frame matching
Collect depth value, Obj () is Tukey biweight objective function, eji 2And eji 3It is characterized match point re-projection error, δji l2
And δji l3It is characterized match point and measures noise, σL2And σL3For re-projection error standard deviation, eji DFor geometric match point depth error,
δji dAnd σDIt corresponds respectively to geometric match point depth measurement noise and error to standard deviation, i is positive integer, n is characterized match point
Number, j be characterized matching point set number, ∑ be summation symbol, as detailed above.
It is also real when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of the diverse characteristics hybrid optimization
Existing following steps:
Closed loop detection is carried out to depth image, and determines the boundary weight matrix of depth image；
Camera track is optimized with global optimization cost function according to boundary weight matrix, as detailed above.
It is also real when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of the diverse characteristics hybrid optimization
Existing following steps:
Characteristic matching point in two data frames of side constraint link is obtained into three-dimensional point set to being mapped in depth image,
And side is calculated by three-dimensional point set and constrains error covariance, side constrains error covariance are as follows:
Wherein, cov () is error covariance, and x, y, z is respectively X, Y, the line element in Z-direction, [Pc,i x,Pc,i y,
Pc,i z]TFor three-dimensional point set PcIn i-th point of three-dimensional coordinate, [Pt,i x,Pt,i y,Pt,i z]TFor three-dimensional point set PtIn i-th point three
Tie up coordinate, [ei x,ei y,ei z]TError covariance is constrained for i-th point of side, [R] and [t] is that side constrains transformation matrix；
Pass through the weight matrix when constraining error covariance and angle constraint error covariance obtains, weight matrix are as follows:
Wherein,θ, Ψ are angle element, as detailed above.
The global optimization cost function are as follows:
P={ p1,p2,p3,···,pi,···,pj,···,pn}
Wherein, p is camera posture, and V is camera gesture set, trijFor the side of depth image, σijFor information matrix, information
Matrix is the inverse matrix of weight matrix, as detailed above.
It is also real when three-dimensional mapping program is executed by the processor 10 in the room RGB-D of the diverse characteristics hybrid optimization
Existing following steps:
According to the depth data after correction, the depth image in RGB-D data set is converted into single frames three-dimensional point cloud；
According to the camera track after the single frames three-dimensional point cloud of acquisition and optimization, RGB-D data set is exported as color three dimension
Point cloud, as detailed above.
In conclusion in a kind of room RGB-D of diverse characteristics hybrid optimization provided by the present invention three-dimensional plotting method and
System the described method comprises the following steps: by the camera depth error in measurement of original grating difference, carry out school to depth data
Just；Camera posture is updated by two 3D visions and three-dimensional geometry feature；According to the random error in camera tracing process, optimization
Camera track；By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension
Point cloud.On the basis of original RGB-D SLAM, due to having fully considered sensing system error and depth error in measurement shadow
It rings, 2 three-dimensional feature points of fusion combine mapping, obtain higher three-dimensional point cloud precision.
It should be understood that the application of the present invention is not limited to the above for those of ordinary skills can
With improvement or transformation based on the above description, all these modifications and variations all should belong to the guarantor of appended claims of the present invention
Protect range.
Claims (10)
1. three-dimensional plotting method in a kind of room RGB-D of diverse characteristics hybrid optimization, which comprises the following steps:
By the camera depth error in measurement of original grating difference, depth data is corrected；
Camera posture is updated by two 3D visions and three-dimensional geometry feature；
According to the random error in camera tracing process, optimize camera track；
By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension dot cloud.
2. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 1, which is characterized in that
The camera depth error in measurement by original grating difference, is corrected before step depth data further include:
Based on pinhole camera model, the inside and outside parameter of vision camera and depth camera is corrected.
3. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 2, which is characterized in that
The camera depth error in measurement by original grating difference, is corrected step to depth data and specifically includes:
Depth data calibration model is constructed for the camera depth error in measurement of original grating difference, by different measurements apart from lower phase
Machine distortion and systematic error optimize depth data calibration model, and correction depth data；
The depth data calibration model are as follows:
ed=3W1(xc t 2-xp t 2)+yc tW2(xc t-yc t)+W3xc t(xc t 2-xp t 2)+W4[xc t 4-xp t 4+2yc t(xc t 2-xp t 2)]
Wherein, W1And W2For camera tangential distortion, W3And W4For camera radial distortion, xc tAnd yc tFor the image after camera calibration
Coordinate, xp tAnd yp tFor the image coordinate after the correction of infrared projection device.
4. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 1, which is characterized in that
It is described to be specifically included by two 3D visions and three-dimensional geometry feature update camera attitude step:
It detects and matches two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature；
Critical data frame is screened using image blur, the constraint of Feature Points Matching rate and baseline constraint；
Two 3D vision features, three-dimensional geometry point feature and three-dimensional line feature based on acquisition, by minimizing visual pattern
Two three-dimensional match point re-projection errors and depth image geometric match point range error obtain camera posture renewal.
5. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 4, which is characterized in that
The camera posture renewal are as follows:
Wherein, F () is camera posture renewal function, and argmin is the set for all independents variable for F () obtaining minimum value, KL
For adjacent key frame set, PL 2The two dimensional character obtained for the matching of all visual pattern key frames matches point set, PL 3For vision figure
As the three-dimensional feature obtained in key frame matches point set, DiThe three-dimensional match point depth of field obtained for i-th of depth key frame matching
Angle value, Obj () are Tukey biweight objective function, eji 2And eji 3It is characterized match point re-projection error, δji l2And δji l3
It is characterized match point and measures noise, σL2And σL3For re-projection error standard deviation, eji DFor geometric match point depth error, δji dAnd σD
It corresponding respectively to geometric match point depth measurement noise and error to standard deviation, i is positive integer, n is characterized the number of match point,
J is characterized matching point set number, and ∑ is summation symbol.
6. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 2, which is characterized in that
The random error according in camera tracing process, optimization camera Trace step specifically include:
Closed loop detection is carried out to depth image, and determines the boundary weight matrix of depth image；
Camera track is optimized with global optimization cost function according to boundary weight matrix.
7. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 6, which is characterized in that
The boundary weight step of the determining depth image specifically includes:
Characteristic matching point in two data frames of side constraint link is obtained into three-dimensional point set to being mapped in depth image, and is led to
It crosses three-dimensional point set and calculates side constraint error covariance, side constrains error covariance are as follows:
Wherein, cov () is error covariance, and x, y, z is respectively X, Y, the line element in Z-direction, [Pc,i x,Pc,i y,Pc,i z]TFor
Three-dimensional point set PcIn i-th point of three-dimensional coordinate, [Pt,i x,Pt,i y,Pt,i z]TFor three-dimensional point set PtIn i-th point of three-dimensional coordinate,
[ei x,ei y,eiz]TError covariance is constrained for i-th point of side, [R] and [t] is that side constrains transformation matrix；
Pass through the weight matrix when constraining error covariance and angle constraint error covariance obtains, weight matrix are as follows:
Wherein,θ, Ψ are angle element.
8. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 7, which is characterized in that
The global optimization cost function are as follows:
P={ p1,p2,p3,···,pi,···,pj,···,pn}
Wherein, p is camera posture, and V is camera gesture set, trijFor the side of depth image, σijFor information matrix, information matrix
For the inverse matrix of weight matrix.
9. three-dimensional plotting method in the room RGB-D of diverse characteristics hybrid optimization according to claim 8, which is characterized in that
It is described by correction after depth data and the camera track after optimization, RGB-D data set is exported as color three dimension dot cloud
Step specifically includes:
According to the depth data after correction, the depth image in RGB-D data set is converted into single frames three-dimensional point cloud；
According to the camera track after the single frames three-dimensional point cloud of acquisition and optimization, RGB-D data set is exported as color three dimension dot cloud.
10. three-dimensional mapping system in a kind of room RGB-D of diverse characteristics hybrid optimization characterized by comprising processor, with
And the memory being connected to the processor,
The memory is stored in the room RGB-D of diverse characteristics hybrid optimization three-dimensional mapping program, the diverse characteristics mixing
It is performed the steps of when three-dimensional mapping program is executed by the processor in the room RGB-D of optimization
By the camera depth error in measurement of original grating difference, depth data is corrected；
Camera posture is updated by two 3D visions and three-dimensional geometry feature；
According to the random error in camera tracing process, optimize camera track；
By the depth data after correction and the camera track after optimization, RGB-D data set is exported as color three dimension dot cloud.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910227761.3A CN110120093A (en) | 2019-03-25 | 2019-03-25 | Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910227761.3A CN110120093A (en) | 2019-03-25 | 2019-03-25 | Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110120093A true CN110120093A (en) | 2019-08-13 |
Family
ID=67520591
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910227761.3A Pending CN110120093A (en) | 2019-03-25 | 2019-03-25 | Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110120093A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807799A (en) * | 2019-09-29 | 2020-02-18 | 哈尔滨工程大学 | Line feature visual odometer method combining depth map inference |
| CN111429344A (en) * | 2020-02-19 | 2020-07-17 | 上海交通大学 | Laser S L AM closed loop detection method and system based on perceptual hashing |
| CN113077512A (en) * | 2021-03-24 | 2021-07-06 | 浙江中体文化集团有限公司 | RGB-D pose recognition model training method and system |
| CN113592989A (en) * | 2020-04-14 | 2021-11-02 | 广东博智林机器人有限公司 | Three-dimensional scene reconstruction system, method, equipment and storage medium |
| CN113635299A (en) * | 2021-07-23 | 2021-11-12 | 深圳市朗驰欣创科技股份有限公司 | Mechanical arm correction method, terminal device and storage medium |
| CN113920254A (en) * | 2021-12-15 | 2022-01-11 | 深圳市其域创新科技有限公司 | Monocular RGB (Red Green blue) -based indoor three-dimensional reconstruction method and system thereof |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354875A (en) * | 2015-09-25 | 2016-02-24 | 厦门大学 | Construction method and system for two-dimensional and three-dimensional joint model of indoor environment |
| CN105702151A (en) * | 2016-03-31 | 2016-06-22 | 百度在线网络技术（北京）有限公司 | Indoor map constructing method and device |
| US20170064287A1 (en) * | 2015-08-24 | 2017-03-02 | Itseez3D, Inc. | Fast algorithm for online calibration of rgb-d camera |
| CN106887037A (en) * | 2017-01-23 | 2017-06-23 | 杭州蓝芯科技有限公司 | A kind of indoor three-dimensional rebuilding method based on GPU and depth camera |
| CN107590827A (en) * | 2017-09-15 | 2018-01-16 | 重庆邮电大学 | A kind of indoor mobile robot vision SLAM methods based on Kinect |
| CN108648240A (en) * | 2018-05-11 | 2018-10-12 | 东南大学 | Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration |
| CN108765548A (en) * | 2018-04-25 | 2018-11-06 | 安徽大学 | Three-dimensional scenic real-time reconstruction method based on depth camera |
| CN109087394A (en) * | 2018-08-02 | 2018-12-25 | 福州大学 | A kind of real-time indoor three-dimensional rebuilding method based on inexpensive RGB-D sensor |
-
2019
- 2019-03-25 CN CN201910227761.3A patent/CN110120093A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170064287A1 (en) * | 2015-08-24 | 2017-03-02 | Itseez3D, Inc. | Fast algorithm for online calibration of rgb-d camera |
| CN105354875A (en) * | 2015-09-25 | 2016-02-24 | 厦门大学 | Construction method and system for two-dimensional and three-dimensional joint model of indoor environment |
| CN105702151A (en) * | 2016-03-31 | 2016-06-22 | 百度在线网络技术（北京）有限公司 | Indoor map constructing method and device |
| CN106887037A (en) * | 2017-01-23 | 2017-06-23 | 杭州蓝芯科技有限公司 | A kind of indoor three-dimensional rebuilding method based on GPU and depth camera |
| CN107590827A (en) * | 2017-09-15 | 2018-01-16 | 重庆邮电大学 | A kind of indoor mobile robot vision SLAM methods based on Kinect |
| CN108765548A (en) * | 2018-04-25 | 2018-11-06 | 安徽大学 | Three-dimensional scenic real-time reconstruction method based on depth camera |
| CN108648240A (en) * | 2018-05-11 | 2018-10-12 | 东南大学 | Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration |
| CN109087394A (en) * | 2018-08-02 | 2018-12-25 | 福州大学 | A kind of real-time indoor three-dimensional rebuilding method based on inexpensive RGB-D sensor |
Non-Patent Citations (6)
| Title |
|---|
| SHENGJUN T , QING Z , WU C , ET AL.: "Enhanced RGB-D Mapping Method for Detailed 3D Indoor and Outdoor Modeling", 《SENSORS》 * |
| TANG S , LI Y , YUAN Z , ET AL.: "A Vertex-to-Edge Weighted Closed-Form Method for Dense RGB-D Indoor SLAM", 《IEEE ACCESS》 * |
| YANG S, YI X, WANG Z, ET AL.: "Visual SLAM using multiple RGB-D cameras", 《2015 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO). IEEE》 * |
| 朱凯，刘华峰，夏青元: "基于单目视觉的同时定位与建图算法研究综述", 《计算机应用研究》 * |
| 汤圣君: "多视图像增强的RGB-D室内高精度三维测图方法", 《中国博士学位论文全文数据库信息科技辑》 * |
| 黄金鑫，赵勇: "一种改进的未知环境无人机三维地图实时创建方法", 《机械与电子》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807799A (en) * | 2019-09-29 | 2020-02-18 | 哈尔滨工程大学 | Line feature visual odometer method combining depth map inference |
| CN110807799B (en) * | 2019-09-29 | 2023-05-30 | 哈尔滨工程大学 | Line feature visual odometer method combined with depth map inference |
| CN111429344A (en) * | 2020-02-19 | 2020-07-17 | 上海交通大学 | Laser S L AM closed loop detection method and system based on perceptual hashing |
| CN111429344B (en) * | 2020-02-19 | 2022-04-26 | 上海交通大学 | Laser SLAM closed loop detection method and system based on perceptual hashing |
| CN113592989A (en) * | 2020-04-14 | 2021-11-02 | 广东博智林机器人有限公司 | Three-dimensional scene reconstruction system, method, equipment and storage medium |
| CN113592989B (en) * | 2020-04-14 | 2024-02-20 | 广东博智林机器人有限公司 | Three-dimensional scene reconstruction system, method, equipment and storage medium |
| CN113077512A (en) * | 2021-03-24 | 2021-07-06 | 浙江中体文化集团有限公司 | RGB-D pose recognition model training method and system |
| CN113077512B (en) * | 2021-03-24 | 2022-06-28 | 浙江中体文化集团有限公司 | RGB-D pose recognition model training method and system |
| CN113635299A (en) * | 2021-07-23 | 2021-11-12 | 深圳市朗驰欣创科技股份有限公司 | Mechanical arm correction method, terminal device and storage medium |
| CN113635299B (en) * | 2021-07-23 | 2022-11-25 | 深圳市朗驰欣创科技股份有限公司 | Mechanical arm correction method, terminal device and storage medium |
| CN113920254A (en) * | 2021-12-15 | 2022-01-11 | 深圳市其域创新科技有限公司 | Monocular RGB (Red Green blue) -based indoor three-dimensional reconstruction method and system thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112894832B (en) | Three-dimensional modeling method, three-dimensional modeling device, electronic equipment and storage medium | |
| CN110120093A (en) | Three-dimensional plotting method and system in a kind of room RGB-D of diverse characteristics hybrid optimization | |
| CN103119611B (en) | The method and apparatus of the location based on image | |
| CN105758426B (en) | The combined calibrating method of the multisensor of mobile robot | |
| CN109059895A (en) | A kind of multi-modal indoor ranging and localization method based on mobile phone camera and sensor | |
| CN108921947A (en) | Generate method, apparatus, equipment, storage medium and the acquisition entity of electronic map | |
| CN107179086A (en) | A kind of drafting method based on laser radar, apparatus and system | |
| CN102622747B (en) | Camera parameter optimization method for vision measurement | |
| CN111060924A (en) | SLAM and target tracking method | |
| CN112799096B (en) | Map construction method based on low-cost vehicle-mounted two-dimensional laser radar | |
| CN107729893A (en) | A kind of vision positioning method of clapper die spotting press, system and storage medium | |
| CN108332752A (en) | The method and device of robot indoor positioning | |
| Gong et al. | Extrinsic calibration of a 3D LIDAR and a camera using a trihedron | |
| Mi et al. | A vision-based displacement measurement system for foundation pit | |
| CN103176606B (en) | Based on plane interaction system and the method for binocular vision identification | |
| CN114140539A (en) | Method and device for acquiring position of indoor object | |
| CN111429515A (en) | Learning method of robot obstacle avoidance behavior based on deep learning | |
| CN116625354A (en) | High-precision topographic map generation method and system based on multi-source mapping data | |
| Reitinger et al. | Augmented reality scouting for interactive 3d reconstruction | |
| Chaochuan et al. | An extrinsic calibration method for multiple RGB-D cameras in a limited field of view | |
| Huang et al. | VWR-SLAM: Tightly coupled SLAM system based on visible light positioning landmark, wheel odometer, and RGB-D camera | |
| CN114137564A (en) | Automatic indoor object identification and positioning method and device | |
| Chen et al. | Low cost and efficient 3D indoor mapping using multiple consumer RGB-D cameras | |
| Wan et al. | Enhance accuracy: Sensitivity and uncertainty theory in LiDAR odometry and mapping | |
| Zhang et al. | An overlap-free calibration method for LiDAR-camera platforms based on environmental perception |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190813 |