CN103106688A - Indoor three-dimensional scene rebuilding method based on double-layer rectification method - Google Patents

Indoor three-dimensional scene rebuilding method based on double-layer rectification method Download PDF

Info

Publication number
CN103106688A
CN103106688A CN2013100538293A CN201310053829A CN103106688A CN 103106688 A CN103106688 A CN 103106688A CN 2013100538293 A CN2013100538293 A CN 2013100538293A CN 201310053829 A CN201310053829 A CN 201310053829A CN 103106688 A CN103106688 A CN 103106688A
Authority
CN
China
Prior art keywords
point
amp
kinect
image
matrix
Prior art date
Application number
CN2013100538293A
Other languages
Chinese (zh)
Other versions
CN103106688B (en
Inventor
贾松敏
郭兵
王可
李秀智
Original Assignee
北京工业大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京工业大学 filed Critical 北京工业大学
Priority to CN201310053829.3A priority Critical patent/CN103106688B/en
Publication of CN103106688A publication Critical patent/CN103106688A/en
Application granted granted Critical
Publication of CN103106688B publication Critical patent/CN103106688B/en

Links

Abstract

The invention belongs to the crossing field of computer vision and intelligent robots, relates to an indoor three-dimensional scene rebuilding method based on a double-layer rectification method and solves the problems that an existing indoor scene rebuilding method is expensive in required equipment, high in computation complexity and poor in real-time performance. The indoor three-dimensional scene rebuilding method based on the double-layer rectification method comprises Kinect calibration, SURF feature point extraction and matching, mapping from a feature point pair to a three-dimensional space point pair, three dimensional space point double-layer rectification based on random sample consensus (RANSAC) and inductively coupled plasma (ICP) methods and scene updating. According to the indoor three-dimensional scene rebuilding method based on the double-layer rectification method, the Kinect is adopted to obtain environmental data, and the double-layer rectification method is provided based on the RANSAC and the ICP. Indoor three-dimensional scene rebuilding which is economical and rapid is achieved, and the real-time performance of rebuilding algorithm and the rebuilding precision are effectively improved. The indoor three-dimensional scene rebuilding method based on the double-layer rectification method is applicable to the service robot field and other computer vision fields which are relative to the three-dimensional scene rebuilding.

Description

Indoor method for reconstructing three-dimensional scene based on double-deck method for registering

Technical field

The invention belongs to the crossing domain of computer vision and intelligent robot, relate to a kind of indoor environment three-dimensional reconstruction, relate in particular to a kind of method for reconstructing of the indoor scene on a large scale based on double-deck method for registering.

Background technology

In recent years, along with the development of infotech, the demand of 3 D scene rebuilding technology is constantly increased, economy indoor method for reconstructing three-dimensional scene fast becomes the guardian technique problem that numerous areas needs to be resolved hurrily.In family's service robot field, the Intelligent home service robot market demand that aging population causes is day by day strong.At present, on market, most of service robot is because can't providing single simple service by the perception three-dimensional environment under specific scene, and this problem is seriously restricting the development of home services Robot industry.

3 D scene rebuilding is one of the study hotspot problem in the fields such as computer vision, intelligent robot, virtual reality.Traditional three-dimensional rebuilding method can be divided into two classes according to the mode difference of obtaining three-dimensional data: based on the three-dimensional rebuilding method of laser scanner technique with based on the three-dimensional rebuilding method of vision.Still deposit larger limitation for the existing method of indoor large-scale 3 D scene rebuilding problem.

Based on the three-dimensional rebuilding method of laser, obtain depth data or the range image of scene by laser scanner, utilize the registration of depth data to realize aliging of frame data and global data.So just obtained the geological information of three-dimensional scenic, need to obtain the texture information of scene and be mapped to and reconstruct on geometric model by increasing a video camera, this will solve one by the mapping problems of photo to how much.Although can obtain the 3-D geometric model of degree of precision based on the three-dimensional rebuilding method of laser scanning, but the difficulty of texture is larger, thereby it is more difficult to generate realistic three-dimensional model, the laser equipment price is high simultaneously, generally be applied in the fields such as digital archaeology, topographic(al) reconnaissance, digital museum, be difficult at large-scale civil area universal.

Three-dimensional rebuilding method based on vision, namely adopt computer vision methods to carry out the object dimensional Model Reconstruction, refer to utilize digital camera as imageing sensor, the technology such as integrated use image processing vision calculating are carried out three-dimensional non-contact measurement, obtain the three-dimensional information of object with computer program.It is advantageous that not limited by body form, rebuild speed, can realize full-automatic or semi-automatic modeling etc., is an important development direction of three-dimensional reconstruction.Difference according to using the video camera number can be divided into monocular vision method, binocular vision method, trinocular vision method or used for multi-vision visual method.The monocular vision method uses a video camera to carry out three-dimensional reconstruction, derives depth information by two dimension characteristic of image, and these two dimensional characters comprise light and shade, texture, focus, profile etc.Its advantage is that device structure is simple, uses single width or several images just can reconstruct the object dimensional model.But more satisfactoryization of condition that usually requires, practical situations is not very desirable, the reconstruction effect is general.The binocular vision method also claims stereo vision method, and binocular parallax information is converted to depth information.Its advantage is that method is ripe, can stably obtain reconstructed results preferably; Not enough is that operand is still bigger than normal, and in the situation that the larger reconstruction of parallax range successful reduction.The basic thought of used for multi-vision visual method is to provide extra constraint by increasing video camera, avoids problem in binocular vision with this.Its advantage is to rebuild effect to be better than the binocular vision method, but device structure is more complicated, and cost is higher, and is also more difficult in control.

In recent years, along with RGBD(colour and the degree of depth) development of sensor technology, the Kinect that releases such as Microsoft is for 3 D scene rebuilding provides new scheme.At present about the research of Kinect three-dimensional rebuilding method having obtained some achievements aspect the three-dimensional reconstruction of single object, still be in the starting stage in the research aspect the indoor scene reconstruction.The people such as RichardA.Newcombe adopt Kinect to obtain environmental information, utilize the ICP method, realize the three-dimensional reconstruction of environmental information.Because the method realizes on GPU hardware, the GPU hardware configuration to be had relatively high expectations, and be subjected to the restriction of GPU internal memory, the scope that can only rebuild 3m * 3m * 3m can't satisfy the demand of indoor three-dimensional scenic establishment on a large scale.

Summary of the invention

In order to overcome the problem that exists in above-mentioned three-dimensional rebuilding method, the invention provides a kind of economy fast, based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering.

The technical solution used in the present invention is as follows:

Utilize Kinect to obtain RGB and the depth image information of environment, by extracting the SURF unique point of RGB image, with Feature Points Matching information as associated data, in conjunction with random sampling consistance (Random sample Consensus, RANSAC) point of proximity (the Iterative closest point of method and iteration, ICP) method proposes the double-deck method for registering of a kind of three-dimensional data.The method mainly comprises following content: the first, and utilize the RANSAC method to obtain the rotation translation transformation matrix of adjacent two frames (Frame-To-Frame) three-dimensional data, accumulate the relative position variation that this result is obtained Kinect.By setting threshold, increase by one frame data when the Kinect change in location surpasses a certain size are key frame (KeyFrame) with this data setting and complete first registration; The second, utilize the ICP method to obtain the accurate transformation matrix of adjacent key frame (KeyFrame-To-KeyFrame), complete accuracy registration.Utilize KeyFrame data that double-deck method for registering obtains and the transformation matrix between adjacent KeyFrame, complete the reconstruction of three-dimensional environment.

Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering, it comprises the following steps:

Step 1 is carried out Kinect and is demarcated.

In image measurement process and machine vision applications, for determining three-dimensional geometry position and its mutual relationship between corresponding point in image of surperficial certain point of space object, must set up the geometric model of camera imaging, these geometric model parameters consist of camera parameters.These parameters must just can obtain with calculating by experiment under most of conditions, and this process of finding the solution parameter just is referred to as camera calibration (or camera calibration).In image measurement and machine vision applications, the demarcation of camera parameters is unusual the key link, and the precision of its calibration result and stability directly affect the accuracy of net result.

Kinect is the XBOX360 body sense periphery peripheral hardware of a kind of Microsoft issue, and the degree of depth and colour (RGB) image information is provided simultaneously.Depth information utilizes thermal camera to adopt active mode to obtain, each frame is comprised of the 640*480 pixel, and the investigation depth scope is 0.5 ~ 4.0 meter, and the regulation of longitudinal angle scope is 43 °, the lateral angles scope is 57 °, can obtain the depth information of object in 6 square metres of scopes.The RGB camera of a 640*480 pixel is housed on Kinect simultaneously.Provide simultaneously this characteristic of RGB information and depth information most important for three-dimensional reconstruction, facilitate depth information to align with RGB information.

The calibrating parameters of Kinect sensor comprises thermal camera (depth transducer) internal reference, three parts of the outer ginseng between RGB video camera internal reference and thermal camera and RGB video camera.The present invention adopts the plane reference method of Zhang Zhengyou that the RGB video camera is demarcated.The data that outer ginseng between thermal camera intrinsic parameter and thermal camera and RGB video camera uses official of Microsoft to provide.

Step 2, the extraction of unique point and coupling.

Feature extraction: by analysis image information, determine whether each point in image belongs to a characteristics of image.The result of feature extraction is that the point on image is divided into different subsets, and these subsets often belong to isolated point, continuous curve or continuous zone.SURF(Speeded-Up RobustFeatures) unique point is the most popular method of present computed image feature, and the feature that the method is extracted has that yardstick is constant, the performance of invariable rotary, simultaneously illumination variation and affine, perspective transform is had unchangeability.SURF all surmounts or approaches the same class methods that in the past proposed, and have obvious advantage on computing velocity aspect 3 of multiplicities, uniqueness, robustness.

The present invention extracts the SURF unique point of RGB image, comprises that feature point detection and unique point describe two parts.Employing is carried out Feature Points Matching based on the nearest neighbor algorithm of Euclidean distance, utilizes the data structure of K-D tree to search for, and is more right than determining whether to accept this coupling according to the distance of nearest two unique points.

Step 3, images match are put the three-dimensional coordinate mapping.

Set up transformational relation between the plane of delineation and space three-dimensional point coordinate according to the calibration model of Kinect, determine that three dimensions puts the projection model of the plane of delineation, with following function representation:

u=π(p)

Wherein, p is the three dimensions point, and u is plane of delineation coordinate, and π (p) representation space three-dimensional point is to the mapping function of the plane of delineation.Coupling by image characteristic point obtains in the corresponding point of the plane of delineation pair, and the projection model that utilizes three dimensions to put the plane of delineation obtains three dimensions point coordinate corresponding to image characteristic point, further obtains three-dimensional point corresponding to two frame data pair.

Step 4 is based on the double-deck registration of the three dimensions point of RANSAC and ICP.

Registration refers to the interior coupling with the different images geographic coordinate that different imaging means was obtained of the same area.Comprise geometric correction, projective transformation and three kinds of processing of unified engineer's scale.Registration results is expressed as matrix:

T cw=[R cw,t cw]

Wherein, subscript " cw " expression is tied to current Kinect coordinate system, R from world coordinates cwThe expression world coordinates is tied to the rotation matrix of current coordinate system, t cwThe expression world coordinates is tied to the translation of current coordinate system.T cwDescribed Kinect and rotated translation relation under world coordinate system.Put p under the Kinect coordinate system cTo world coordinates p wTransformation relation be:

p c=T cwp w

The problem high for three dimensional point cloud method for registering complexity, that calculated amount is large the present invention is based on RANSAC and ICP method, proposes a kind of double-deck method for registering, is comprised of first registration and accuracy registration two parts.First registration adopts RANSAC, to obtain KeyFrame and Relative Transformation matrix; Adopt ICP to realize accuracy registration, realize the alignment of three-dimensional data points and provide three-dimension varying information accurately for upgrading three-dimensional scenic on the basis of first registration.

Step 5, scene update.

Each frame three-dimensional data of obtaining by Kinect approximately comprises 250,000 points.There is very large information redundancy in adjacent two frame data, in order to improve the sharpness of reconstructed results, and to the description that the three-dimensional map that generates provides an essence to want, the burden of minimizing system aspect internal memory, the present invention adopts KeyFrame Data Update three-dimensional scenic.

The invention has the beneficial effects as follows: adopt Kinect to obtain environmental data, for the characteristics of Kinect sensor, propose a kind of double-deck method for registering based on RANSAC and ICP, realize quick indoor 3 D scene rebuilding on a large scale.Effectively solve three-dimensional rebuilding method cost and real time problems, improved reconstruction precision.

Description of drawings

Fig. 1 is the indoor method for reconstructing three-dimensional scene block diagram based on Kinect;

Fig. 2 is Kinect coordinate system schematic diagram;

Fig. 3 is the double-deck method for registering process flow diagram based on RANSAC and ICP;

Fig. 4 creates the actual environment schematic diagram of three-dimensional scenic for using the present invention: in figure, (a) for the experiment real scene, (b) be the two-dimensional geometry schematic diagram of experimental situation;

Fig. 5 creates the result schematic diagram of three-dimensional scenic for using the present invention.

Embodiment

The present invention is described in further detail by reference to the accompanying drawings.As shown in Figure 1, the present invention includes following step:

Step 1 is carried out Kinect and is demarcated, and concrete grammar is as follows:

(1) print a chessboard template.The present invention adopts an A4 paper, chessboard be spaced apart 0.25cm.

(2) from a plurality of angle shot chessboards.During shooting, should allow chessboard take screen, and 8 template picture be taken altogether in each angle that guarantees chessboard in screen as far as possible.

(3) detect unique point in image, i.e. each black point of crossing of chessboard.

(4) obtain the parameter that Kinect demarcates.

The internal reference matrix K of thermal camera ir:

K ir = f uIR 0 u IR 0 f vIR v IR 0 0 1

Wherein, (f uIR, f vIR) be the focal length of thermal camera, value (5,5), (u IR, v IR) be thermal camera as the planar central coordinate, value (320,240).

The Intrinsic Matrix K of RGB video camera c:

K c = f u 0 u 0 0 f v v 0 0 0 1

Wherein, (f u, f v) be the focal length of RGB video camera, (u 0, v 0) be that the RGB video camera is as the planar central coordinate.

External parameter between thermal camera and RGB video camera is:

T=[R IRc,t IRc]

Wherein, R IRcBe rotation matrix, t bcTranslation vector, the parameter of directly using official of Microsoft to provide:

R IRc = 1 0 0 0 1 0 0 0 1

t IRc=[0.075?0] T

In the present invention, the Kinect coordinate system as shown in Figure 2, for y axle positive dirction, is upwards forward z axle positive dirction, is to the right the x positive dirction.The initial point position of Kinect is set as the world coordinate system initial point, and the X of world coordinate system, Y, Z direction are identical with x, y, the z direction of Kinect initial point position.

Step 2, the extraction of unique point and coupling, method is as follows:

(1) obtain integral image.Integral image refer to calculate given all pixels of gray level image accumulation and, for the integration I (X) of certain some X=(x, y) in image be:

I Σ ( X ) = Σ i = 0 i ≤ x Σ j = 0 j ≤ y I ( i , j )

Calculate the gray-scale value sum of a rectangular area with 3 plus and minus calculations in integral image, irrelevant with the area of rectangle.Can see in the step of back, the convolution mask that uses in the SURF feature point extraction is frame shape template, has greatly improved operation efficiency.

(2) ask for approximate Hessian matrix H ApproxFor certain some X=(x, y) in image I, the Hessian matrix H (X, s) on the s yardstick that X is ordered is defined as:

H ( X , s ) = L xx ( X , s ) L xy ( X , s ) L xy ( X , s ) L yy ( X , s )

Wherein, L xx(X, s), L xy(X, s), L yy(X, s) expression Gauss second-order partial differential coefficient is in the convolution of X place and image I.Use the second order Gauss filtering in the approximate Hessian of replacement of square frame filtering matrix.The value of frame shape Filtering Template after with image convolution is respectively D xx, D yy, D xy, further replace L with them xx, L yy, L xyObtain approximate Hessian matrix H Approx, its determinant is:

det(H approx)=D xxD yy-(wD xy) 2

Wherein, w is weight coefficient, and value is 0.9 in enforcement of the present invention.

(3) location feature point.The feature point detection of SURF is based on the Hessian matrix, according to the local maximum location feature point position of Hessian matrix determinant.

With the frame shape wave filter of different size, original image is processed and obtained the yardstick image pyramid, according to H ApproxObtain the extreme value that the scalogram picture is located at (X, s).

Use frame shape wave filter to build metric space, in every single order, select the scalogram picture of 4 layers, the structure parameter on 4 rank sees Table 1.

Size (the unit: s) of 16 templates in quadravalence before table 1 metric space

Use H ApproxMatrix is obtained extreme value, in 3 dimension (X, s) metric spaces, each regional area of 3 * 3 * 3 is carried out non-maximum value suppress (keep maximum value, other values are set to 0).Elect response as unique point greater than the point of 26 neighborhood values.Utilize the quadratic fit function that unique point is accurately located, fitting function D (X) is:

D ( X ) = D + ∂ D T ∂ X X + 1 2 X T ∂ D ∂ X 2 X

So far, position, the yardstick information (X, s) of unique point have been obtained.

(4) determine the direction character of unique point.With the Haar wavelet filter, circular neighborhood is processed, obtained the response of the corresponding x of each point, y direction in this neighborhood.Choose the Gaussian function (σ gets 2s, and s is yardstick corresponding to this unique point) centered by unique point, these responses are weighted, the vector of search length maximum, its direction is the corresponding direction of this unique point.

(5) construction feature description vectors.Determine a foursquare neighborhood centered by unique point, the length of side is got 20s, is the unique point direction setting y direction of principal axis of this neighborhood.Square area is divided into 4 * 4 sub regions, processes with the Haar wavelet filter in each subregion that (Haar small echo template size is 2s * 2s).Use d xThe little wave response of Haar of expression horizontal direction is used d yThe little wave response of Haar of expression vertical direction.For all d x, d yIn order to the Gaussian function weighting centered by unique point, the σ of this Gaussian function is 3.3s.In every sub regions respectively to d x, d y, d x|, d y| summation obtains 4 dimensional vector V (∑ d x, ∑ d y, ∑ d y|, ∑ d y|).The vector of 4 * 4 sub regions is coupled together just obtained one 64 vector of tieing up, this vector has rotation, yardstick unchangeability, then after carrying out normalization, has illumination invariant.So far, obtained the proper vector of Expressive Features point.

(6) characteristic matching.Employing is based on the nearest neighbor method of Euclidean distance, utilize the K-D tree to search in image to be matched, find with benchmark image in the nearest the first two unique point of unique point Euclidean distance, if minimum distance less than the proportion threshold value (0.7) of setting, is accepted this a pair of match point except the value that closely obtains in proper order.

Step 3, images match are put the three-dimensional coordinate mapping.

According to calibrating parameters, Kinect depth image and RGB image mid point mapping calculation method are as follows:

1 p=(x in depth image d, y d) coordinate P under the Kinect coordinate system 3D=(x, y, z) is:

P 3 D . x = ( x d - u IR ) × P 3 D . z / f uIR P 3 D . y = ( y d - v IR ) × P 3 D . z / f vIR P 3 D . z = depth ( x d , y d )

Wherein, depth (x d, y d) depth value of expression depth image mid point p.

So, derive the corresponding 3D coordinate of pixel of RGB image, and then obtain the coordinate (x in the RGB image rgb, y rgb).Computing formula is as follows:

x rgb = ( P 3 D ′ . x * f u / P 3 D ′ . z ) + u 0 y rgb = ( P 3 D ′ . y * f v / P 3 D ′ . z ) + v 0

Wherein, P ' 3D TR IRc* P 3D T+ t IRc

According to above-mentioned conversion relation, the matching double points that obtains in step 2 is converted to three dimensions point right.

Step 4, based on the double-deck registration of the three dimensions point of RANSAC and ICP, method comprises the following steps as shown in Figure 3:

(1) first registration.The corresponding point that Feature Points Matching obtains are to existing larger mistake coupling.Using RANSAC in the first registration stage, to remove the three dimensions point of mistake coupling right, finds the imperial palace point set that satisfies transformation model and estimate transformation matrix T by iteration.Accumulation KeyFrame obtains the transformation matrix of the relative KeyFrame of current Kinect to each Relative Transformation matrix of current data.Go out translational movement and the anglec of rotation mould value of Kinect according to this matrix computations, compare with the threshold value of setting, judging whether to choose this frame is KeyFrame.In embodiment, the threshold value setting of translational movement is 0.4, and angle threshold is 40 degree.

The detailed process of RANSAC is as follows:

1) from from the initial N of reference point collection A and point set B subject to registration to choosing at random 7 pairs of data three-dimensional matching double points;

2) utilize basis matrix to find the solution 7 methods of minimal configuration, be it is calculated that the transformation matrix T of benchmark point set and point set data subject to registration by 7 logarithms of choosing AB

3) utilize transformation matrix T ABWith Characteristic of Image point set subject to registration A remaining N-7 three-dimensional point in (expression comprises the point set B subject to registration of N point) Transform under reference point cloud coordinate system;

4) the point set P ' after computational transformation N-7With the benchmark point set Between error of coordinate;

5) from N to finding out the unique point of error of coordinate in certain threshold value matching double points to number, be designated as i;

6) repeat 1) ~ 5) step n(n sets by the user, in the present embodiment, iterations is set as 50) inferior, make the i value obtain maximum set for imperial palace point set, be interior point, all the other N-i are Mismatching point, are exterior point.Utilize imperial palace point set to estimate the least square solution of transformation model, as the transformation matrix T of current adjacent two frame data.

(2) accuracy registration.Obtain KeyFrame and Relative Transformation matrix thereof through first registration, the present invention adopts the accurate transformation matrix of ICP calculating K eyFrame, and first registration results as the priori conversion, is tried to achieve the accurate transformation matrix of KeyFrame-To-KeyFrame.

In the depth map of Kinect, pixel value is that 0 zone is invalid metrical information.Obtain effective information and invalid information in depth map for convenience of describing, be defined as follows function for above-mentioned phenomenon:

Wherein, X is picture planimetric coordinates point.

In order to obtain k Kinect pose constantly According to the transformational relation of Kinect coordinate system and world coordinate system, set up following energy function:

E = min Σ p k ∈ Ω | T cw k p w - p k |

Wherein, p wBe the point under world coordinate system, p kBe the point under current coordinate system, Ω has the set of the pixel of significant depth value in the k moment plane of delineation, that is:

Ω={ p k| u=π (p k) and M (u)=1}

The above-mentioned energy function of setting up is the mathematical description of three-dimensional ICP method.In the ICP algorithm, by the minimization energy function, obtain the pose under world coordinate system at k moment Kinect.Usually the ICP algorithm is under the prerequisite of supposition relative pose, constantly sets up the corresponding relation between the some cloud, and by optimizing corresponding point error iterative.Therefore in the solution procedure of ICP algorithm, initial relative pose is set and is played vital effect, and inappropriate initial pose will make the ICP algorithm be absorbed in local optimum, can't obtain correct result.Based on the ICP algorithm of unordered some cloud, along with an increase of cloud quantity, the space complexity of algorithm and time complexity will significantly improve, and greatly reduce the execution efficient of this algorithm.Therefore initial relative pose is the prerequisite of setting up corresponding relation between the some cloud, plays vital effect in the iterative process of ICP method.The Relative Transformation that this method obtains first registration is as initial relative pose, to obtain the optimal estimation of current KeyFrame.

Suppose in the side-play amount of the k-1 moment and k moment Kinect to be

T cw k = T inc k T cw k - 1

Kinect is at x, y, and the rotation amount on the z direction of principal axis (α, beta, gamma) and the translational movement on three directions are (t x, t y, t z).When enough hour of above-mentioned two vectors, launch according to the single order Taylor's formula, make x=(α, beta, gamma, t x, t y, t z):

T inc k = exp ( x )

= 1 γ - β - γ 1 α β - α 1 t x t y t z

= [ R inc | t inc ]

World coordinates for the k moment spatial point of obtaining is Under the coordinate system of k-1 Kinect during the moment, energy function is transformed to so with this spot projection:

E = min Σ p k ∈ Ω | | T cw k p w - p k | |

= min Σ p k ∈ Ω | | T inc k T cw k - 1 p w - p k | |

= min Σ p k ∈ Ω | | T inc k p w k - 1 p k | |

Ω={ p k| u=π (p k) and M (u)=1}

Wherein, p wAnd p kBe corresponding point, Be p wCoordinate under k-1 moment camera coordinates is.

By

T inc k p w k - 1 = R inc p w k - 1 + t inc = G ( p w k - 1 ) x + p w k - 1

Obtain the final expression of energy function:

min x ∈ se ( 3 ) Σ Ω | | G ( p w k - 1 ) x + p w k - 1 p k | |

Ω={ p k| u=π (p k) and M (u)=1}

Wherein, ( p w k - 1 ) [ [ p w k - 1 ] × | I 3 × 3 ] , [ p w k - 1 ] × Serve as reasons The antisymmetric matrix that consists of.The threshold value of setting energy function is 0.05, utilizes Cholesky to decompose and obtains hexa-atomic group of solution x=(α, beta, gamma, t x, t y, t z), be mapped to special European group (rigid motion group) SE (3) space in Lie group, and can obtain current Kinect pose in conjunction with the pose of k-1 moment Kinect.

Step 5, scene update.

The renewal of scene is divided into two kinds of situations, and a kind of is to carry out for the first time scene update, and this moment, the set positions with Kinect was the initial point of world coordinate system, and added the current contextual data of obtaining; Another kind is newly-increased frame KeyFrame data, according to formula Current newly-increased KeyFrame data are transformed in world coordinate system, complete the renewal of contextual data.

The below provides and uses the method for the invention is carried out the three-dimensional environment establishment under indoor true environment a experiment embodiment.

The depth camera that experiment is adopted is Kinect-XBOX360, and the RGB image resolution ratio is 640 * 480, and the highest frame frequency is 30fps.Indoor environment as shown in Figure 4, Fig. 4 (a) for the experiment real scene, Fig. 4 (b) is the two-dimensional geometry schematic diagram of experimental situation.During experiment, hand-held Kinect, begin from starting point, stops by behind the fixed route walking place of reaching home, and synchronously increases progressively in the process of walking the generation global map, the scope of 9m * 9m in the map covering chamber of establishment.Fig. 5 is the reconstructed results schematic diagram.

Experimental result shows, the method for the invention can be used on a large scale that indoor three-dimensional scenic creates, and has higher precision and good real-time.

The above is only preferred embodiment of the present invention, is not for limiting protection scope of the present invention, all any modifications of doing within the spirit and principles in the present invention, is equal to and replaces and improvement etc., within all should being included in protection scope of the present invention.

Claims (2)

1. indoor method for reconstructing three-dimensional scene based on double-deck method for registering is characterized in that comprising the following steps:
Step 1 is carried out Kinect and is demarcated, and method is as follows:
(1) print a chessboard template;
(2) from a plurality of angle shot chessboards;
(3) detect unique point in image, i.e. each black point of crossing of chessboard;
(4) obtain the parameter that Kinect demarcates:
The internal reference matrix K of thermal camera ir:
K ir = f uIR 0 u IR 0 f vIR v IR 0 0 1
Wherein, (f uIR, f vIR) be the focal length of thermal camera, value (5,5), (u IR, v IR) be thermal camera as the planar central coordinate, value (320,240);
The Intrinsic Matrix K of RGB video camera c:
K c = f u 0 u 0 0 f v v 0 0 0 1
Wherein, (f u, f v) be the focal length of RGB video camera, (u 0, v 0) be that the RGB video camera is as the planar central coordinate;
External parameter between thermal camera and RGB video camera is:
T=[R IRc,t IRc]
Wherein, R IRcBe rotation matrix, t IRcTranslation vector, the parameter of directly using official of Microsoft to provide:
R IRc = 1 0 0 0 1 0 0 0 1
t IRc=[0075??0??0] T
The Kinect coordinate system for y axle positive dirction, is upwards forward z axle positive dirction, is to the right the x positive dirction; The initial point position of Kinect is set as the world coordinate system initial point, and the X of world coordinate system, Y, Z direction are identical with x, y, the z direction of Kinect initial point position;
Step 2, the extraction of unique point and coupling, method is as follows:
(1) obtain integral image: the integration I (X) for certain some X=(x, y) in image is:
I Σ ( X ) = Σ i = 0 i ≤ x Σ j = 0 j ≤ y I ( i , j )
Calculate the gray-scale value sum of a rectangular area with 3 plus and minus calculations in integral image, irrelevant with the area of rectangle;
(2) ask for approximate Hessian matrix H Approx: for certain some X=(x, y) in image I, the Hessian matrix H (X, s) on the s yardstick that X is ordered is defined as:
H ( X , s ) = L xx ( X , s ) L xy ( X , s ) L xy ( X , s ) L yy ( X , s )
Wherein, L xx(X, s), L xy(X, s), L yy(X, s) expression Gauss second-order partial differential coefficient is in the convolution of X place and image I; Use the second order Gauss filtering in the approximate Hessian of replacement of square frame filtering matrix, the value of frame shape Filtering Template after with image convolution is respectively D xx, D yy, D xy, further replace L with them xx, L yy, L xyObtain approximate Hessian matrix H Approx, its determinant is:
det(H approx)=D xxD yy-(wD xy) 2
Wherein, w is weight coefficient;
(3) location feature point: the feature point detection of SURF is based on the Hessian matrix, according to the local maximum location feature point position of Hessian matrix determinant;
With the frame shape wave filter of different size, original image is processed and obtained the yardstick image pyramid, according to H ApproxObtain the extreme value that the scalogram picture is located at (X, s);
Use frame shape wave filter to build metric space, in every single order, select the scalogram picture of 4 layers, use H ApproxMatrix is obtained extreme value, in 3 dimension (X, s) metric spaces, each regional area of 3 * 3 * 3 is carried out non-maximum value suppress; Elect response as unique point greater than the point of 26 neighborhood values; Utilize the quadratic fit function that unique point is accurately located, fitting function D (X) is:
D ( X ) = D + ∂ D T ∂ X X + 1 2 X T ∂ D ∂ X 2 X
Thereby obtain position, the yardstick information (X, s) of unique point;
(4) determine the direction character of unique point: with the Haar wavelet filter, circular neighborhood is processed, obtained the response of the corresponding x of each point, y direction in this neighborhood; Choose the Gaussian function centered by unique point, σ gets 2s, and s is yardstick corresponding to this unique point, these responses are weighted, and the vector of search length maximum, its direction is the corresponding direction of this unique point;
(5) construction feature description vectors: determine a foursquare neighborhood centered by unique point, the length of side is got 20s, is the unique point direction setting y direction of principal axis of this neighborhood; Square area is divided into 4 * 4 sub regions, processes with the Haar wavelet filter in each subregion, Haar small echo template size is 2s * 2s; Use d xThe little wave response of Haar of expression horizontal direction is used d yThe little wave response of Haar of expression vertical direction is for all d x, d yIn order to the Gaussian function weighting centered by unique point, the σ of this Gaussian function is 3.3s; In every sub regions respectively to d x, d y, d x|, d y| summation obtains 4 dimensional vector V (∑ d x, ∑ d y, ∑ d y|, ∑ d y|), the vector of 4 * 4 sub regions being coupled together just obtained one 64 vector of tieing up, this vector has rotation, yardstick unchangeability, after normalization, has illumination invariant; This vector is the proper vector of Expressive Features point;
(6) characteristic matching: adopt the nearest neighbor method based on Euclidean distance, utilize the K-D tree to search in image to be matched, find with benchmark image in the nearest the first two unique point of unique point Euclidean distance, if minimum distance less than the proportion threshold value of setting, is accepted this a pair of match point except the value that closely obtains in proper order;
Step 3, images match are put the three-dimensional coordinate mapping, and method is as follows:
Ask 1 p=(x in depth image d, y d) coordinate P under the Kinect coordinate system 3D=(x, y, z):
P 3 D . x = ( x d - u IR ) × P 3 D . z / f uIR P 3 D . y = ( y d - v IR ) × P 3 D . z / f vIR P 3 D . z = depth ( x d , y d )
Wherein, depth (x d, y d) depth value of expression depth image mid point p;
Obtain coordinate (x in the RGB image by the corresponding 3D coordinate of RGB image pixel rgb, y rgb):
x rgb = ( P 3 D ′ . x * f u / P 3 D ′ . z ) + u 0 y rgb = ( P 3 D ′ . y * f v / P 3 D ′ . z ) + v 0
Wherein, P ' 3D T=R IRc* P 3D T+ t IRc
According to above-mentioned conversion relation, the matching double points that obtains in step 2 is converted to three dimensions point right;
Step 4, based on the double-deck registration of three dimensions point of RANSAC and ICP method, method is as follows:
(1) first registration: using RANSAC in the first registration stage, to remove the three dimensions point of mistake coupling right, finds the imperial palace point set that satisfies transformation model and estimate transformation matrix T by iteration; Accumulation KeyFrame obtains the transformation matrix of the relative KeyFrame of current Kinect to each Relative Transformation matrix of current data; Go out translational movement and the anglec of rotation mould value of Kinect according to this matrix computations, compare with the threshold value of setting, judging whether to choose this frame is KeyFrame;
(2) accuracy registration: for obtaining k Kinect pose constantly According to the transformational relation of Kinect coordinate system and world coordinate system, set up following energy function:
E = min Σ p k ∈ Ω | T cw k p w - p k |
Wherein, p wBe the point under world coordinate system, p kBe the point under current coordinate system, Ω has the set of the pixel of significant depth value in the k moment plane of delineation, that is:
Ω={ p k| u=π (p k) and M (u)=1}
Wherein, M(X) obtain the function of effective information and invalid information in depth map for describing:
Wherein, X is picture planimetric coordinates point;
Suppose in the side-play amount of the k-1 moment and k moment Kinect to be :
T cw k = T inc k T cw k - 1
As Kinect at x, y, the rotation amount on the z direction of principal axis (α, beta, gamma) and the translational movement on three directions are (t x, t y, t z) enough hour, launch according to the single order Taylor's formula, make x=(α, beta, gamma, t x, t y, t z):
T inc k = exp ( x )
= 1 γ - β - γ 1 α β - α 1 t x t y t z
= [ R inc | t inc ]
World coordinates for the k moment spatial point of obtaining is Under the coordinate system of k-1 Kinect during the moment, energy function is transformed to this spot projection:
E = min Σ p k ∈ Ω | | T cw k p w - p k | |
Σ p l ∈ Ω | | T inc k T cw k - 1 p w - p k | |
= min Σ p k ∈ Ω | | T inc k p w k - 1 p k | |
Ω={ p k| u=π (p k) and M (u)=1}
Wherein, p wAnd p kBe corresponding point, Be p wCoordinate under k-1 moment camera coordinates is;
By
T inc k p w k - 1 = R inc p w k - 1 + t inc = G ( p w k - 1 ) x + p w k - 1
Obtain the final expression of energy function:
min x ∈ se ( 3 ) Σ Ω | | G ( p w k - 1 ) x + p w k - 1 p k | |
Ω={ p k| u=π (p k) and M (u)=1}
Wherein, G ( p w k - 1 ) = [ [ p w k - 1 ] × | I 3 × 3 ] , [ p w k - 1 ] × Serve as reasons The antisymmetric matrix that consists of;
Set the threshold value of energy function, utilize Cholesky to decompose and obtain hexa-atomic group of solution x=(α, beta, gamma, t x, t y, t z), be mapped to special European group SE (3) space in Lie group, and can obtain current Kinect pose in conjunction with the pose of k-1 moment Kinect;
Step 5, scene update, method is as follows:
The renewal of scene is divided into two kinds of situations, and a kind of is to carry out for the first time scene update, and this moment, the set positions with Kinect was the initial point of world coordinate system, and added the current contextual data of obtaining; Another kind is newly-increased frame KeyFrame data, according to formula Current newly-increased KeyFrame data are transformed in world coordinate system, complete the renewal of contextual data.
2. the indoor method for reconstructing three-dimensional scene based on double-deck method for registering according to claim 1, is characterized in that, the method that matrix T is changed in the described application of step 4 RANSAC changes persuing is as follows:
(1) from from the initial N of reference point collection A and point set B subject to registration to choosing at random 7 pairs of data three-dimensional matching double points;
(2) utilize basis matrix to find the solution 7 methods of minimal configuration, be it is calculated that the transformation matrix T of benchmark point set and point set data subject to registration by 7 logarithms of choosing AB
(3) utilize transformation matrix T ABWith Characteristic of Image point set subject to registration In a remaining N-7 three-dimensional point Transform under reference point cloud coordinate system;
(4) the point set P ' after computational transformation N-7With the benchmark point set Between error of coordinate;
(5) from N to finding out the unique point of error of coordinate in certain threshold value matching double points to number, be designated as i;
(6) repeat (1) ~ (5) n time, make set that the i value obtains maximum for imperial palace point set, be interior point, all the other N-i are Mismatching point, are exterior point; Utilize imperial palace point set to estimate the least square solution of transformation model, as the transformation matrix T of current adjacent two frame data.
CN201310053829.3A 2013-02-20 2013-02-20 Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering CN103106688B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310053829.3A CN103106688B (en) 2013-02-20 2013-02-20 Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310053829.3A CN103106688B (en) 2013-02-20 2013-02-20 Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering

Publications (2)

Publication Number Publication Date
CN103106688A true CN103106688A (en) 2013-05-15
CN103106688B CN103106688B (en) 2016-04-27

Family

ID=48314513

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310053829.3A CN103106688B (en) 2013-02-20 2013-02-20 Based on the indoor method for reconstructing three-dimensional scene of double-deck method for registering

Country Status (1)

Country Link
CN (1) CN103106688B (en)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103260015A (en) * 2013-06-03 2013-08-21 程志全 Three-dimensional visual monitoring system based on RGB-Depth camera
CN103268729A (en) * 2013-05-22 2013-08-28 北京工业大学 Mobile robot cascading type map creating method based on mixed characteristics
CN103325142A (en) * 2013-05-29 2013-09-25 南京大学 Computer three-dimensional model establishing method based on Kinect
CN103413352A (en) * 2013-07-29 2013-11-27 西北工业大学 Scene three-dimensional reconstruction method based on RGBD multi-sensor fusion
CN103456038A (en) * 2013-08-19 2013-12-18 华中科技大学 Method for rebuilding three-dimensional scene of downhole environment
CN104126989A (en) * 2014-07-30 2014-11-05 福州大学 Foot surface three-dimensional information obtaining method based on multiple RGB-D cameras
CN104517287A (en) * 2014-12-10 2015-04-15 广州赛意信息科技有限公司 Image matching method and device
CN104517289A (en) * 2014-12-12 2015-04-15 浙江大学 Indoor scene positioning method based on hybrid camera
CN105222789A (en) * 2015-10-23 2016-01-06 哈尔滨工业大学 A kind of building indoor plane figure method for building up based on laser range sensor
CN105319991A (en) * 2015-11-25 2016-02-10 哈尔滨工业大学 Kinect visual information-based robot environment identification and operation control method
CN105335399A (en) * 2014-07-18 2016-02-17 联想(北京)有限公司 Information processing method and electronic device
CN105513128A (en) * 2016-01-13 2016-04-20 中国空气动力研究与发展中心低速空气动力研究所 Kinect-based three-dimensional data fusion processing method
CN105509748A (en) * 2015-12-29 2016-04-20 深圳先进技术研究院 Navigation method and apparatus for robot
CN105913489A (en) * 2016-04-19 2016-08-31 东北大学 Indoor three-dimensional scene reconstruction method employing plane characteristics
CN105987693A (en) * 2015-05-19 2016-10-05 北京蚁视科技有限公司 Visual positioning device and three-dimensional surveying and mapping system and method based on visual positioning device
CN106091921A (en) * 2015-04-28 2016-11-09 三菱电机株式会社 For the method determining the size in scene
CN106384383A (en) * 2016-09-08 2017-02-08 哈尔滨工程大学 RGB-D and SLAM scene reconfiguration method based on FAST and FREAK feature matching algorithm
CN106529838A (en) * 2016-12-16 2017-03-22 湖南拓视觉信息技术有限公司 Virtual assembling method and device
CN106596557A (en) * 2016-11-07 2017-04-26 东南大学 Three-dimensional scanning mobile type platform carrying Kinect and method thereof
CN106780297A (en) * 2016-11-30 2017-05-31 天津大学 Image high registration accuracy method under scene and Varying Illumination
CN106780590A (en) * 2017-01-03 2017-05-31 成都通甲优博科技有限责任公司 The acquisition methods and system of a kind of depth map
CN106803267A (en) * 2017-01-10 2017-06-06 西安电子科技大学 Indoor scene three-dimensional rebuilding method based on Kinect
CN106952299A (en) * 2017-03-14 2017-07-14 大连理工大学 A kind of 3 d light fields Implementation Technology suitable for Intelligent mobile equipment
CN107025661A (en) * 2016-01-29 2017-08-08 成都理想境界科技有限公司 A kind of method for realizing augmented reality, server, terminal and system
CN107123138A (en) * 2017-04-28 2017-09-01 电子科技大学 Based on vanilla R points to rejecting tactful point cloud registration algorithm
CN107274440A (en) * 2017-06-26 2017-10-20 赵红林 A kind of image matching algorithm
CN107577451A (en) * 2017-08-03 2018-01-12 中国科学院自动化研究所 More Kinect human skeletons coordinate transformation methods and processing equipment, readable storage medium storing program for executing
CN107610212A (en) * 2017-07-25 2018-01-19 深圳大学 Scene reconstruction method, device, computer equipment and computer-readable storage medium
CN107833270A (en) * 2017-09-28 2018-03-23 浙江大学 Real-time object dimensional method for reconstructing based on depth camera
CN108055456A (en) * 2017-12-07 2018-05-18 中煤航测遥感集团有限公司 Texture collection method and device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030137508A1 (en) * 2001-12-20 2003-07-24 Mirko Appel Method for three dimensional image reconstruction
CN101976455A (en) * 2010-10-08 2011-02-16 东南大学 Color image three-dimensional reconstruction method based on three-dimensional matching

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030137508A1 (en) * 2001-12-20 2003-07-24 Mirko Appel Method for three dimensional image reconstruction
CN101976455A (en) * 2010-10-08 2011-02-16 东南大学 Color image three-dimensional reconstruction method based on three-dimensional matching

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PETER HENRY等: "RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments", 《THE 12TH INTERNATIONAL SYMPOSIUM ON EXPERIMENTAL ROBOTICS》 *
刘鑫等: "基于GPU和Kinect的快速物体重建", 《自动化学报》 *

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103268729A (en) * 2013-05-22 2013-08-28 北京工业大学 Mobile robot cascading type map creating method based on mixed characteristics
CN103325142A (en) * 2013-05-29 2013-09-25 南京大学 Computer three-dimensional model establishing method based on Kinect
CN103325142B (en) * 2013-05-29 2016-02-17 南京大学 A kind of electronic 3-D model modeling method based on Kinect
CN103260015A (en) * 2013-06-03 2013-08-21 程志全 Three-dimensional visual monitoring system based on RGB-Depth camera
CN103260015B (en) * 2013-06-03 2016-02-24 程志全 Based on the three-dimensional visible supervisory control system of RGB-Depth camera
CN103413352A (en) * 2013-07-29 2013-11-27 西北工业大学 Scene three-dimensional reconstruction method based on RGBD multi-sensor fusion
CN103456038A (en) * 2013-08-19 2013-12-18 华中科技大学 Method for rebuilding three-dimensional scene of downhole environment
CN105335399B (en) * 2014-07-18 2019-03-29 联想(北京)有限公司 A kind of information processing method and electronic equipment
CN105335399A (en) * 2014-07-18 2016-02-17 联想(北京)有限公司 Information processing method and electronic device
CN104126989A (en) * 2014-07-30 2014-11-05 福州大学 Foot surface three-dimensional information obtaining method based on multiple RGB-D cameras
CN104517287A (en) * 2014-12-10 2015-04-15 广州赛意信息科技有限公司 Image matching method and device
CN104517289B (en) * 2014-12-12 2017-08-08 浙江大学 A kind of indoor scene localization method based on hybrid camera
CN104517289A (en) * 2014-12-12 2015-04-15 浙江大学 Indoor scene positioning method based on hybrid camera
CN106091921B (en) * 2015-04-28 2019-06-18 三菱电机株式会社 Method for determining the size in scene
CN106091921A (en) * 2015-04-28 2016-11-09 三菱电机株式会社 For the method determining the size in scene
CN105987693B (en) * 2015-05-19 2019-04-30 北京蚁视科技有限公司 A kind of vision positioning device and three-dimensional mapping system and method based on the device
WO2016184255A1 (en) * 2015-05-19 2016-11-24 北京蚁视科技有限公司 Visual positioning device and three-dimensional mapping system and method based on same
CN105987693A (en) * 2015-05-19 2016-10-05 北京蚁视科技有限公司 Visual positioning device and three-dimensional surveying and mapping system and method based on visual positioning device
CN105222789A (en) * 2015-10-23 2016-01-06 哈尔滨工业大学 A kind of building indoor plane figure method for building up based on laser range sensor
CN105319991A (en) * 2015-11-25 2016-02-10 哈尔滨工业大学 Kinect visual information-based robot environment identification and operation control method
CN105509748A (en) * 2015-12-29 2016-04-20 深圳先进技术研究院 Navigation method and apparatus for robot
CN105513128A (en) * 2016-01-13 2016-04-20 中国空气动力研究与发展中心低速空气动力研究所 Kinect-based three-dimensional data fusion processing method
CN107025661A (en) * 2016-01-29 2017-08-08 成都理想境界科技有限公司 A kind of method for realizing augmented reality, server, terminal and system
CN105913489A (en) * 2016-04-19 2016-08-31 东北大学 Indoor three-dimensional scene reconstruction method employing plane characteristics
CN105913489B (en) * 2016-04-19 2019-04-23 东北大学 A kind of indoor three-dimensional scenic reconstructing method using plane characteristic
CN106384383B (en) * 2016-09-08 2019-08-06 哈尔滨工程大学 A kind of RGB-D and SLAM scene reconstruction method based on FAST and FREAK Feature Correspondence Algorithm
CN106384383A (en) * 2016-09-08 2017-02-08 哈尔滨工程大学 RGB-D and SLAM scene reconfiguration method based on FAST and FREAK feature matching algorithm
CN106596557A (en) * 2016-11-07 2017-04-26 东南大学 Three-dimensional scanning mobile type platform carrying Kinect and method thereof
CN106780297A (en) * 2016-11-30 2017-05-31 天津大学 Image high registration accuracy method under scene and Varying Illumination
CN106780297B (en) * 2016-11-30 2019-10-25 天津大学 Image high registration accuracy method under scene and Varying Illumination
CN106529838A (en) * 2016-12-16 2017-03-22 湖南拓视觉信息技术有限公司 Virtual assembling method and device
CN106780590A (en) * 2017-01-03 2017-05-31 成都通甲优博科技有限责任公司 The acquisition methods and system of a kind of depth map
CN106780590B (en) * 2017-01-03 2019-12-24 成都通甲优博科技有限责任公司 Method and system for acquiring depth map
CN106803267A (en) * 2017-01-10 2017-06-06 西安电子科技大学 Indoor scene three-dimensional rebuilding method based on Kinect
CN106952299A (en) * 2017-03-14 2017-07-14 大连理工大学 A kind of 3 d light fields Implementation Technology suitable for Intelligent mobile equipment
CN106952299B (en) * 2017-03-14 2019-07-16 大连理工大学 A kind of 3 d light fields Implementation Technology suitable for Intelligent mobile equipment
CN107123138B (en) * 2017-04-28 2019-07-30 电子科技大学 Based on vanilla-R point to the point cloud registration method for rejecting strategy
CN107123138A (en) * 2017-04-28 2017-09-01 电子科技大学 Based on vanilla R points to rejecting tactful point cloud registration algorithm
CN107274440A (en) * 2017-06-26 2017-10-20 赵红林 A kind of image matching algorithm
CN107610212A (en) * 2017-07-25 2018-01-19 深圳大学 Scene reconstruction method, device, computer equipment and computer-readable storage medium
CN107577451A (en) * 2017-08-03 2018-01-12 中国科学院自动化研究所 More Kinect human skeletons coordinate transformation methods and processing equipment, readable storage medium storing program for executing
CN107833270A (en) * 2017-09-28 2018-03-23 浙江大学 Real-time object dimensional method for reconstructing based on depth camera
CN108055456A (en) * 2017-12-07 2018-05-18 中煤航测遥感集团有限公司 Texture collection method and device

Also Published As

Publication number Publication date
CN103106688B (en) 2016-04-27

Similar Documents

Publication Publication Date Title
Kar et al. Learning a multi-view stereo machine
Sinha et al. Efficient high-resolution stereo matching using local plane sweeps
Wöhler 3D computer vision: efficient methods and applications
KR101554241B1 (en) A method for depth map quality enhancement of defective pixel depth data values in a three-dimensional image
Klingner et al. Street view motion-from-structure-from-motion
CN105205858B (en) A kind of indoor scene three-dimensional rebuilding method based on single deep vision sensor
Henry et al. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments
Lhuillier et al. A quasi-dense approach to surface reconstruction from uncalibrated images
Agarwal et al. A survey of planar homography estimation techniques
Zhao et al. Alignment of continuous video onto 3D point clouds
Borrmann et al. Globally consistent 3D mapping with scan matching
Košecká et al. Extraction, matching, and pose recovery based on dominant rectangular structures
Fitzgibbon et al. Automatic 3D model acquisition and generation of new images from video sequences
Pizarro et al. Toward large-area mosaicing for underwater scientific applications
CN103247075B (en) Based on the indoor environment three-dimensional rebuilding method of variation mechanism
Xiao et al. Uncalibrated perspective reconstruction of deformable structures
Coughlan et al. Manhattan world: Orientation and outlier detection by bayesian inference
CN100559398C (en) Automatic deepness image registration method
Liebowitz Camera calibration and reconstruction of geometry from images
CN105245841B (en) A kind of panoramic video monitoring system based on CUDA
US7509241B2 (en) Method and apparatus for automatically generating a site model
EP2731075B1 (en) Backfilling points in a point cloud
CN103345736B (en) A kind of virtual viewpoint rendering method
Teller et al. Calibrated, registered images of an extended urban area
Quan Image-based modeling

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant