CN107292956A - A kind of scene reconstruction method assumed based on Manhattan - Google Patents
A kind of scene reconstruction method assumed based on Manhattan Download PDFInfo
- Publication number
- CN107292956A CN107292956A CN201710563682.0A CN201710563682A CN107292956A CN 107292956 A CN107292956 A CN 107292956A CN 201710563682 A CN201710563682 A CN 201710563682A CN 107292956 A CN107292956 A CN 107292956A
- Authority
- CN
- China
- Prior art keywords
- normal
- normal vector
- mrow
- vector
- msubsup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/38—Registration of image sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention discloses a kind of scene reconstruction method assumed based on Manhattan.This method gives accurate estimation based on the scheme that Manhattan is assumed；The normal direction of all 3D points of estimated record frame first；Then the normal vector direction of three main orthogonal planes is estimated by PCA.The normal vector of plane is estimated by then passing through all depth points, random noise is greatly filtered out so that the normal vector of estimation is very accurate；Principal plane coordinate is further determined that by position of each pixel in three-dimensional coordinate；Finally, by the transformation matrix that is obtained by plane coordinates and normal vector to estimate the posture of camera, and scene threedimensional model is spliced into using the camera posture of each two field picture.Robustness is had more because the plane information of the present invention is obtained by largely putting calculating, therefore than the feature point methods of only a single point.
Description
Technical field
The invention belongs to computer vision field, especially for scene three-dimensional reconstruction, and in particular to one kind is based on Manhattan
The scene reconstruction method of hypothesis.
Background technology
In recent years, with the development of depth perception technology, realize that the real-time 3D scenes scanning of 3D indoor scenes is possibly realized.
Industry proposes several systems, and generates promising result.On the other hand, as augmented reality (AR) turns into academic and row
The hot issue of industry, in the urgent need to real-time 3D scan because the recovery of the 3D geometry of our real scenes be make virtual objects without
Stitch the key of alignment.In the head-mounted display Hololens of Microsoft, many application programs based on AR, which need to scan, works as anterior chamber
Between 3D geometry.
Using depth camera, directly record 3D information, the key for realizing 3D scannings be the continuous input frame of estimation each two it
Between camera motion.It is the correspondence put between cloud for estimating to be obtained by two depth frames using iteration closest approach (ICP) first
Relation.Then, 2 clouds can be merged by the camera motion of estimation.
However, it is necessary to abundant textural characteristics in the method based on ICP, by the geometric properties in scene, robustly
Estimate correct camera motion.For two pure 2D planes in 3d space, immediate point may not be that correct correspondence is closed
System.In this case, ICP may produce the camera motion of mistake.In addition, ICP needs a large amount of sampled points, it is necessary to iteration
Final corresponding relation can be converged to, it means that relatively heavy calculating cost.Even if some systems based on ICP are used
GPU realizes real-time performance, but is not still suitable for many practical applications, because GPU may be taken by other tasks, so as to lead
The system based on ICP of cause can not be calculated in time.
The content of the invention
The purpose of the present invention is that there is provided a kind of scene rebuilding side assumed based on Manhattan in view of the shortcomings of the prior art
Method.
When carrying out three-dimensional reconstruction to indoor scene, because most scene structure meets Manhattan it is assumed that i.e. scene
It is made up of multiple mutually orthogonal planes, such as ceiling, metope, floor.When the image sequence that depth camera is gathered includes foot
During enough orthogonal planes, it is understood that there may be big plane domain, because plane generally has consistent color (such as metope, ceiling
Deng), so seldom feature can only be extracted.In this case, the scheme assumed based on Manhattan gives accurate motion
Estimation.The normal direction of all 3D points of estimated record frame first；Then three are estimated mainly by principal component analysis (PCA)
The normal vector direction of orthogonal plane (big plane domain in such as wall, ceiling, the scene such as floor).By then passing through all depths
Spend point to estimate the normal vector of plane, random noise is greatly filtered out so that the normal vector of estimation is very accurate；By each
Position of the pixel in three-dimensional coordinate further determines that principal plane coordinate；Finally, we pass through by plane coordinates and normal vector
The transformation matrix of acquisition is spliced into scene threedimensional model to estimate the posture of camera using the camera posture of each two field picture.
The inventive method comprises the following steps：
Step (1), the image sequence by depth camera shooting, collecting acquisition indoor scene, calculate each pixel in image
Point normal vector, be specifically：
First, all pixels point for recording frame in image sequence is converted into 3D by the camera model of depth camera to sit
Mark；Then calculated by the 3D coordinates of adjacent 4 pixels of certain pixel and obtain the normal vector.
D1(u, v)=D (u+k, v)-D (u-k, v) (1)
D2(u, v)=D (u, v+k)-D (u, v-k) (2)
Wherein, k is to represent distance between two pixels, is adjustable parameters；D1、D2For by pixel D's (u, v)
Vector；
By D1、D2Bring into formula (3), obtain D (u, v) normal vector n (u, v)；
N (u, v)=ψ (D1(u,v)×D2(u,v)) (3)
Wherein, × and it is crossed product, Ψ is the function that normal is converted to unit vector：ψ (normal)=normal |
|normal||-1；
Step (2), by setting up spherical coordinate normal vector statistics is carried out, extracted from the normal vector of all pixels point near
Like the normal direction duration set of principal plane normal vector, it is specifically：
2.1 unit spherical coordinates are set up
The normal n=(x, y, z) that step (1) is obtained is converted into spherical coordinates form n=(α, beta, gamma), and wherein α is normal n
Angle between axle X, β is the angle between normal n and axis Y, and γ is the angle between normal n and axis Z.
2.2 normal vectors are counted
Each base in unit spherical coordinate is subdivided into 180 intervals, i.e., it is each interval for 1 degree of scope；Then by three
Each normal vector n is stored in its corresponding container by the individual interval combinations for being belonging respectively to each base into a container.
Described base refers to α, the coordinate where beta, gamma, i.e. α, and beta, gamma is 0~180 degree；
The extraction of 2.3 principal normal vector set
Due to having the pixel for largely belonging to a certain principal plane, therefore the corresponding container of plane normal vector in a two field picture
Middle normal vector quantity will be much larger than other containers, therefore will be set to first comprising all normal vectors in the most container of normal vector quantity
The normal direction quantity set L of individual principal plane1；
According to the normal vector vertical relation of principal plane, the normal direction quantity set L of second principal plane2See formula (4)：
θ1＜ Θ (α 1, α 2)+Θ (β 1, β 2)+Θ (γ 1, γ 2) ＜ θ2 (4)
Wherein Θ (a, b)=cos (a) * cos (b), θ1=π * 100/180, θ2=π * 80/180；
The normal direction quantity set L of 3rd principal plane3For simultaneously apart from L1And L280 ° to 100 ° of normal direction quantity set；
Step (3), principal component analysis
By 3 normal direction quantity set L1、L2、L3In all normal vectors as PCA input, then PCA extract these normal vectors
In Main way, three mutually orthogonal characteristic vectors are obtained, while these vectors to be set to the normal vector n of principal plane1、
n2、n3；Characteristic vector wherein corresponding to the minimal eigenvalue of PCA methods output is the normal vector of first principal plane, maximum on the contrary
Characteristic vector corresponding to characteristic value is the normal vector of the 3rd principal plane.
Step (4), each 3D points are calculated first in normal vector n1、n2、n3On projected position, then utilize projected position
Extract position d of the principal plane on normal vector1,d2,d3, then each principal plane can be expressed as a normal vector form (n, d),
Specifically：
4.1, by the way that depth camera optical centre is set into the origin of coordinates, calculate each pixel 3D coordinate D (u, v) point
Projected position p that Wei Yu be on three principal plane normal vectorsi f=D (u, v) ni, i=1,2,3.
4.2 in perfect condition, is n for normal vectoriPlane, being equal to apart from d for plan range camera opticses center be flat
The projected position on plane normal vector, i.e. D (u, v) n are put on facei=d；But it is due to depth camera noise itself and precision
The reason for, projected position of the partial dot on plane normal vector in plane be not equal to plan range camera opticses center away from
From, therefore peak value extracted using one-dimensional mean shift algorithm to the projected positions of all each pixels, corresponding to peak value peak
Projected position exactly principal plane is to camera opticses center apart from d.Therefore we by the step obtained three principal planes away from
From camera opticses center distance d1,d2,d3。
Step (5), pass through the normal vector and range information of three principal planes obtained before, calculate camera posture changing square
Battle array；And the camera posture changing matrix and the 3D coordinates of all pixels by each frame carry out the reconstruction of scene, are specifically
The camera posture changing matrix of f two field pictures to g two field pictures can be expressed as form：
Wherein R is the spin matrix that size is 3 × 3, and t is the translation vector that size is 3 × 1.
The camera coordinates of first frame are set to the world coordinates of world coordinates, i.e. scene so that each picture in frame f
Vegetarian refreshments 3D coordinates Df(u, v) is reverted in world coordinates by camera posture changing matrix：
Df' (u, v)=T1,2·T2,3…Tf-1,f·Df(u,v) (8)
The pixel 3D coordinates D after world coordinates will be reverted in all framesf' (u, v) add up, obtain the 3D points of scene
Cloud, and the threedimensional model finally given after scene rebuilding is rendered with OpenGL.
The beneficial effects of the invention are as follows：
This planar approach assumed based on Manhattan, can rebuild interior in the case where geometry and textural characteristics are less
Scene and amount of calculation is small, this is all important for the method based on characteristics of image and the method based on ICP and with challenge
Property.Main idea is that being related to Manhattan it is assumed that it efficiently estimates principal plane by setting up unit soccer star's coordinate
Normal vector, and quickly and accurately calculate using one-dimensional mean shift algorithm the distance for obtaining camera opticses center and then pass through
The information of 3 principal planes calculates camera motion.Because the plane information of the present invention is obtained by largely putting calculating, therefore ratio only has
The feature point methods of a single point have more robustness.
Brief description of the drawings
Fig. 1 is the areal model for calculating normal；
Fig. 2 is the unit sphere in 3D cartesian coordinate systems；
Fig. 3 is l1,l2,l3Main shaft orthogonal graph；
Fig. 4 is the inventive method flow chart.
Embodiment
It is for further analysis to the present invention with reference to specific embodiment.
A kind of scene reconstruction method assumed based on Manhattan, is comprised the following steps as shown in Figure 4：
Step (1)：The image sequence for obtaining indoor scene is gathered by depth camera, each pixel method in image is calculated
Vector.
First, all pixels point for recording frame in image sequence is converted into 3D by the camera model of depth camera to sit
Mark；Then calculated by the 3D coordinates of adjacent 4 pixels of certain pixel and obtain the normal vector.Specific manifestation form is as schemed
O-UV is pixel coordinate system in 1, wherein Fig. 1, and O-XYZ is camera coordinates system.
D1(u, v)=D (u+k, v)-D (u-k, v) (1)
D2(u, v)=D (u, v+k)-D (u, v-k) (2)
Wherein, k is to represent distance between two pixels, is adjustable parameters；D1, D2For by pixel D (u, v) to
Amount, D (u, v) is the 3D coordinates of the D (u, v) of pixel on pixel coordinate system (u, v) position.
By D1, D2Bring into formula (3), obtain normal vector
N (u, v)=ψ (D1(u,v)×D2(u,v)) (3)
Wherein, × and it is crossed product, Ψ is the function that normal is converted to unit vector：ψ (normal)=normal |
|normal||-1。
Step (2)：Normal vector statistics is carried out by setting up spherical coordinate, is extracted from the normal vector of all pixels point near
Like the normal direction duration set of principal plane normal vector, specific manifestation form such as Fig. 2 of the step.
2.1 unit spherical coordinates are set up
There to be the point for largely belonging to a certain principal plane (such as wall) in captured image, these points are counted in step 1
The normal vector direction calculated is very close, and all very close with the principal plane normal vector.Simultaneously as irregularly shaped object and biography
To have most of flat with master in interference and noise that sensor is produced in itself, the normal vector that a certain two field picture all pixels point is calculated
Face normal vector direction is not close.So setting up unit spherical coordinate on the basis of three-dimensional coordinate.
The normal n=(x, y, z) calculated is converted into spherical coordinates form n=(α, beta, gamma), wherein α is normal n and axle X
Between angle, β is the angle between normal n and axis Y, and γ is the angle between normal n and axis Z.
2.2 normal vectors are counted
Each base in unit spherical coordinate is subdivided into 180 intervals first, i.e., each interval is 1 degree of scope, wherein
Base herein refers to α, the coordinate where beta, gamma, i.e. α, and beta, gamma is 0~180 degree.Then it is belonging respectively to each base by three
Interval combinations are into a container, and container total quantity is the product of quantity between three bases, i.e., 5832000, each of which container
Storage belongs to its interval normal vector respectively, and such as certain interval storage α is at 90 °~91 °, and β is at 90 °~91 °, and γ is at 90 °~91 °
Normal vector.Finally each normal vector n is stored in its corresponding container.
The extraction of 2.3 principal normal vector set
Due to having the pixel for largely belonging to a certain principal plane, therefore the corresponding container of plane normal vector in a two field picture
Middle normal vector quantity will be much larger than other containers.Then the present invention sets normal vector all in the container comprising most normal vectors
It is set to the normal direction quantity set L of first principal plane1, according to Manhattan it is assumed that the principal plane in image is orthogonality relation, i.e., two masters
The normal vector of plane is vertical.Then when we regard each container as a vector, i.e., each container is expressed as its correspondence
The vectorial l=(α, beta, gamma) of interval intermediate value angle, for example, will deposit α at 90 °~91 °, β is at 90 °~91 °, and γ is at 90 °~91 °
The container of normal vector is expressed as vectorial l=(90.5,90.5,90.5).According to the normal vector vertical relation of principal plane, we can be with
Judge the corresponding container L of normal vector of second principal plane2, its vector representation l2With the l of first principal plane1Between angle
About 90 ° of degree.In view of error range, we will be apart from l1Most normal vectors are included in 80 ° to 100 ° all containers of scope
All normal vectors are set to the normal direction quantity set L of second principal plane in container2, wherein container l2Extraction formula see formula
(4).。
θ1＜ Θ (α 1, α 2)+Θ (β 1, β 2)+Θ (γ 1, γ 2) ＜ θ2 (4)
Wherein Θ (a, b)=cos (a) * cos (b), θ1=π * 100/180, θ2=π * 80/180
Finally apart from l1And l2L is found in container in the range of 80 ° to 100 °3.And by l3In all normal vector set
It is set to the normal direction quantity set L of the 3rd principal plane3
Step (3)：Principal component analysis
Because the normal vector of three principal planes is mutually orthogonal, but only by calculating 3 normal direction quantity set L1, L2, L3Interior method
The average value of line can not obtain most accurate value to extract the normal vector of principal plane, and the vector that they are extracted is from each other
It is not orthogonality relation in maximum probability very much.Therefore the present invention is using the linear independence characteristic of characteristic vector in PCA (PCA),
Principal plane normal vector is extracted from PCA methods.
By 3 normal direction quantity set L1, L2, L3In all normal amounts as PCA input, then PCA will extract these normal direction
Main way in amount.Finally, we obtain three mutually orthogonal characteristic vectors, and these vectors are set into principal plane
Normal vector n1, n2, n3, the characteristic vector wherein corresponding to the minimal eigenvalue of PCA methods output is the normal direction of first principal plane
Amount, on the contrary the characteristic vector corresponding to eigenvalue of maximum is the normal vector of the 3rd principal plane.
Step (4)：By calculating each 3D points in normal vector n1, n2, n3On coordinate extract principal plane on normal vector
Position d1,,d2,d3, then each principal plane can be expressed as a normal vector form (n, d)：
By the way that depth camera optical centre is set into the origin of coordinates, each pixel 3D coordinate D (u, v) position respectively is calculated
Projected position p on three principal plane normal vectorsi f=D (u, v) ni, i=1,2,3；In perfect condition, it is for normal vector
niPlane, plan range camera optics center apart from d be equal to projected position of the Plane-point on plane normal vector, i.e.,
D(u,v)·ni=d；But the partial dot in the reason for being due to depth camera noise itself and precision, plane is in plane normal vector
On projected position be not equal to the distance at plan range camera opticses center；Therefore the projected position of all each pixels is used
One-dimensional mean shift algorithm extracts peak value, the projected position exactly principal plane corresponding to peak value peak to camera opticses center away from
From d.
L in Fig. 31, l2, l3Respectively three orthogonal principal plane normal vectors, l1Curve on axle represents l1It is different on axle to throw
The quantity of 3D points on shadow position, the wherein exactly principal plane of the projected position corresponding to peak peak to camera opticses center away from
From d.Some less peaks are can also be observed that in Fig. 3, they represent the cluster containing a small amount of sample, and they are also represented and master
The parallel facet of plane.However, the information that the present invention only needs to principal plane can rebuild scene, therefore obtained by the step
Three principal planes are apart from camera opticses center apart from d1,d2,d3。
Step (5)：The step calculates the appearance of camera by the normal vector and range information of three principal planes obtained before
State transformation matrix, and the reconstruction of the camera posture changing matrix and the 3D coordinates progress scene of all pixels by each frame.
Camera posture changing matrix from frame f to frame g can be expressed as form：
Wherein R is the spin matrix that size is 3 × 3, and t is the translation vector that size is 3 × 1..
The present invention obtains the information of three vertical principal planes by above-mentioned 1~4 step.In this step, we make
Use plane normal vector, i.e. liTo calculate the spin matrix R from frame g to f：
And translation vector t is obtained by calculating the planar offset of consecutive frame, and it is converted into camera coordinates：
The camera coordinates of first frame are set to the world coordinates of world coordinates, i.e. scene by the present invention so that in frame f
Each pixel 3D coordinate D (u, v) are reverted in world coordinates by camera posture changing matrix：
Df' (u, v)=T1,2·T2,3…Tf-1,f·Df(u,v) (8)
Calculated by formula (8) and obtained in all image sequences of depth camera photographed scene each 3D point in the world
Position in coordinate, the pixel 3D coordinates D after world coordinates will be reverted to by then passing through in all framesf' (u, v) add up, obtain
To the 3D point cloud of scene, and the threedimensional model finally given after scene rebuilding is rendered with OpenGL.
Above-described embodiment is not the limitation for the present invention, and the present invention is not limited only to above-described embodiment, as long as meeting
Application claims, belong to protection scope of the present invention.
Claims (6)
1. a kind of scene reconstruction method assumed based on Manhattan, it is characterised in that this method comprises the following steps：
Step (1), the image sequence by depth camera shooting, collecting acquisition indoor scene, calculate each pixel method in image
Vector；
Step (2), by set up spherical coordinate carry out normal vector statistics, approximate master is extracted from the normal vector of all pixels point
The normal direction duration set of plane normal vector；
Step (3), according to all normal vectors in the normal direction duration set of approximate principal plane normal vector, carried using PCA
The Main way in these normal vectors is taken, mutually orthogonal principal plane normal vector is obtained；
Step (4), the projected position of each pixel 3D coordinates on above-mentioned principal plane normal vector is obtained, then using projecting position
Put the position for extracting principal plane on normal vector；
The positional information of step (5), the principal plane normal vector and principal plane obtained according to above-mentioned steps on normal vector, is calculated
To camera posture changing matrix；And the camera posture changing matrix by each frame and the 3D coordinates of all pixels point carry out scene
Reconstruction.
2. a kind of scene reconstruction method assumed based on Manhattan as claimed in claim 1, it is characterised in that step (1), logical
The image sequence that depth camera shooting, collecting obtains indoor scene is crossed, each pixel normal vector in image is calculated, is specifically：
First, all pixels point of record frame in image sequence is converted into 3D coordinates by the camera model of depth camera；So
Calculated afterwards by the 3D coordinates of adjacent 4 pixels of certain pixel and obtain the normal vector；
D1(u, v)=D (u+k, v)-D (u-k, v) (1)
D2(u, v)=D (u, v+k)-D (u, v-k) (2)
Wherein, k is to represent distance between two pixels, is adjustable parameters；D1、D2For by pixel D (u, v) vector；
By D1、D2Bring into formula (3), obtain D (u, v) normal vector n (u, v)；
N (u, v)=ψ (D1(u,v)×D2(u,v)) (3)
Wherein, × and it is crossed product, Ψ is the function that normal is converted to unit vector：ψ (normal)=normal | |
normal-1。
3. a kind of scene reconstruction method assumed based on Manhattan as claimed in claim 2, it is characterised in that step (2), logical
Cross and set up spherical coordinate progress normal vector statistics, the method for approximate principal plane normal vector is extracted from the normal vector of all pixels point
Vector set, be specifically：
2.1 unit spherical coordinates are set up
The normal n=(x, y, z) that step (1) is obtained is converted into spherical coordinates form n=(α, beta, gamma), and wherein α is normal n and axle
Angle between X, β is the angle between normal n and axis Y, and γ is the angle between normal n and axis Z；
2.2 normal vectors are counted
Each base in unit spherical coordinate is subdivided into 180 intervals, i.e., it is each interval for 1 degree of scope；Then by three points
Do not belong to the interval combinations of each base into a container, each normal vector n is stored in its corresponding container；
Described base refers to α, the coordinate where beta, gamma, i.e. α, and beta, gamma is 0~180 degree；
The extraction of 2.3 principal normal vector set
Due to there is method in the pixel for largely belonging to a certain principal plane, therefore the corresponding container of plane normal vector in a two field picture
Vectorial quantity will be much larger than other containers, therefore will be set to first master comprising all normal vectors in the most container of normal vector quantity
The normal direction quantity set L of plane1；
According to the normal vector vertical relation of principal plane, the normal direction quantity set L of second principal plane2See formula (4)：
θ1＜ Θ (α 1, α 2)+Θ (β 1, β 2)+Θ (γ 1, γ 2) ＜ θ2 (4)
Wherein Θ (a, b)=cos (a) * cos (b), θ1=π * 100/180, θ2=π * 80/180；
The normal direction quantity set L of 3rd principal plane3For simultaneously apart from L1And L280 ° to 100 ° of normal direction quantity set.
4. a kind of scene reconstruction method assumed based on Manhattan as claimed in claim 3, it is characterised in that step (3) is specific
It is by 3 normal direction quantity set L1、L2、L3In all normal vectors as PCA input, then PCA extract the master in these normal vectors
Direction is wanted, three mutually orthogonal characteristic vectors are obtained, while these vectors to be set to the normal vector n of principal plane1、n2、n3；
Characteristic vector wherein corresponding to the minimal eigenvalue of PCA methods output is the normal vector of first principal plane, maximum feature on the contrary
The corresponding characteristic vector of value is the normal vector of the 3rd principal plane.
5. a kind of scene reconstruction method assumed based on Manhattan as claimed in claim 4, it is characterised in that step (4) is first
Each pixel 3D coordinates are calculated in normal vector n1、n2、n3On projected position, then using projected position extract principal plane exist
Position d on normal vector1,d2,d3, then each principal plane can be expressed as a normal vector form (n, d), be specifically：
4.1, by the way that depth camera optical centre is set into the origin of coordinates, calculate each pixel 3D coordinate D (u, v) position respectively
Projected position on three principal plane normal vectors
4.2 in perfect condition, is n for normal vectoriPlane, plan range camera optics center apart from d be equal to plane on
Projected position of the point on plane normal vector, i.e. D (u, v) ni=d；But it is due to the original of depth camera noise itself and precision
Projected position of the partial dot on plane normal vector in cause, plane is not equal to the distance at plan range camera opticses center；
Therefore peak value extracted using one-dimensional mean shift algorithm to the projected positions of all each pixels, corresponding to peak value peak
Projected position exactly principal plane is to camera opticses center apart from d.
6. a kind of scene reconstruction method assumed based on Manhattan as claimed in claim 5, it is characterised in that step (5) is specific
It is：
The camera posture changing matrix of f two field pictures to g two field pictures can be expressed as form：
<mrow>
<msub>
<mi>T</mi>
<mrow>
<mi>f</mi>
<mo>,</mo>
<mi>g</mi>
</mrow>
</msub>
<mo>=</mo>
<mfenced open = "[" close = "]">
<mtable>
<mtr>
<mtd>
<mi>R</mi>
</mtd>
<mtd>
<mi>t</mi>
</mtd>
</mtr>
<mtr>
<mtd>
<mn>0</mn>
</mtd>
<mtd>
<mn>1</mn>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>5</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein R is the spin matrix that size is 3 × 3, and t is the translation vector that size is 3 × 1；
<mrow>
<mi>R</mi>
<mo>=</mo>
<mrow>
<mo>(</mo>
<msubsup>
<mi>n</mi>
<mn>1</mn>
<mi>f</mi>
</msubsup>
<mo>,</mo>
<msubsup>
<mi>n</mi>
<mn>2</mn>
<mi>f</mi>
</msubsup>
<mo>,</mo>
<msubsup>
<mi>n</mi>
<mn>3</mn>
<mi>f</mi>
</msubsup>
<mo>)</mo>
</mrow>
<mo>&CenterDot;</mo>
<msup>
<mrow>
<mo>(</mo>
<msubsup>
<mi>n</mi>
<mn>1</mn>
<mi>g</mi>
</msubsup>
<mo>,</mo>
<msubsup>
<mi>n</mi>
<mn>2</mn>
<mi>g</mi>
</msubsup>
<mo>,</mo>
<msubsup>
<mi>n</mi>
<mn>3</mn>
<mi>g</mi>
</msubsup>
<mo>)</mo>
</mrow>
<mrow>
<mo>-</mo>
<mn>1</mn>
</mrow>
</msup>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>6</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<mi>t</mi>
<mo>=</mo>
<munder>
<mo>&Sigma;</mo>
<mi>i</mi>
</munder>
<mrow>
<mo>(</mo>
<msubsup>
<mi>d</mi>
<mi>i</mi>
<mi>f</mi>
</msubsup>
<mo>-</mo>
<msubsup>
<mi>d</mi>
<mi>i</mi>
<mi>g</mi>
</msubsup>
<mo>)</mo>
</mrow>
<msubsup>
<mi>n</mi>
<mi>i</mi>
<mi>g</mi>
</msubsup>
<mo>,</mo>
<mi>i</mi>
<mo>=</mo>
<mn>1</mn>
<mo>,</mo>
<mn>2</mn>
<mo>,</mo>
<mn>3</mn>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>7</mn>
<mo>)</mo>
</mrow>
</mrow>
The camera coordinates of first frame are set to the world coordinates of world coordinates, i.e. scene so that all 3D points D in frame ff
(u, v) reverts to world coordinates P by camera posture changing matrixglobalIn：
Df' (u, v)=T1,2·T2,3···Tf-1,f·Df(u,v) (8)
By the 3D points D of all framesf' (u, v) add up, and obtains the 3D point cloud of scene, and rendered with OpenGL and finally give scene rebuilding
Threedimensional model afterwards.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710563682.0A CN107292956B (en) | 2017-07-12 | 2017-07-12 | Scene reconstruction method based on Manhattan hypothesis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710563682.0A CN107292956B (en) | 2017-07-12 | 2017-07-12 | Scene reconstruction method based on Manhattan hypothesis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107292956A true CN107292956A (en) | 2017-10-24 |
| CN107292956B CN107292956B (en) | 2020-09-22 |
Family
ID=60100633
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710563682.0A Active CN107292956B (en) | 2017-07-12 | 2017-07-12 | Scene reconstruction method based on Manhattan hypothesis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107292956B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108648224A (en) * | 2018-05-18 | 2018-10-12 | 杭州电子科技大学 | A method of the real-time scene layout identification based on artificial neural network and reconstruction |
| CN108805972A (en) * | 2018-04-11 | 2018-11-13 | 杭州电子科技大学 | A kind of three-dimensional rebuilding method based on ground normal direction and two-dimentional intersection |
| CN110782524A (en) * | 2019-10-25 | 2020-02-11 | 重庆邮电大学 | Indoor three-dimensional reconstruction method based on panoramic image |
| WO2020042970A1 (en) * | 2018-08-29 | 2020-03-05 | 广景视睿科技（深圳）有限公司 | Three-dimensional modeling method and device therefor |
| CN113096185A (en) * | 2021-03-29 | 2021-07-09 | Oppo广东移动通信有限公司 | Visual positioning method, visual positioning device, storage medium and electronic equipment |
| CN114463406A (en) * | 2022-01-25 | 2022-05-10 | 北京工业大学 | Camera rotation estimation method based on indoor environment under Manhattan hypothesis |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0633550B1 (en) * | 1993-06-29 | 2001-10-04 | Canon Kabushiki Kaisha | Image processing method and apparatus thereof |
| CN103247045A (en) * | 2013-04-18 | 2013-08-14 | 上海交通大学 | Method of obtaining artificial scene main directions and image edges from multiple views |
| CN103714420A (en) * | 2013-12-11 | 2014-04-09 | 深圳先进技术研究院 | Object three-dimensional reconstruction method and device |
| CN104123746A (en) * | 2014-07-10 | 2014-10-29 | 上海大学 | Calculating method for three-dimension scanning point cloud real-time normal vectors |
| CN105205858A (en) * | 2015-09-18 | 2015-12-30 | 天津理工大学 | Indoor scene three-dimensional reconstruction method based on single depth vision sensor |
| CN106327576A (en) * | 2016-08-10 | 2017-01-11 | 周口师范学院 | Urban scene reconstruction method and system |
| CN106530342A (en) * | 2016-10-31 | 2017-03-22 | 武汉大学 | Measurable panorama image generation method assisted by laser point cloud |
| CN106570507A (en) * | 2016-10-26 | 2017-04-19 | 北京航空航天大学 | Multi-angle consistent plane detection and analysis method for monocular video scene three dimensional structure |
-
2017
- 2017-07-12 CN CN201710563682.0A patent/CN107292956B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0633550B1 (en) * | 1993-06-29 | 2001-10-04 | Canon Kabushiki Kaisha | Image processing method and apparatus thereof |
| CN103247045A (en) * | 2013-04-18 | 2013-08-14 | 上海交通大学 | Method of obtaining artificial scene main directions and image edges from multiple views |
| CN103714420A (en) * | 2013-12-11 | 2014-04-09 | 深圳先进技术研究院 | Object three-dimensional reconstruction method and device |
| CN104123746A (en) * | 2014-07-10 | 2014-10-29 | 上海大学 | Calculating method for three-dimension scanning point cloud real-time normal vectors |
| CN105205858A (en) * | 2015-09-18 | 2015-12-30 | 天津理工大学 | Indoor scene three-dimensional reconstruction method based on single depth vision sensor |
| CN106327576A (en) * | 2016-08-10 | 2017-01-11 | 周口师范学院 | Urban scene reconstruction method and system |
| CN106570507A (en) * | 2016-10-26 | 2017-04-19 | 北京航空航天大学 | Multi-angle consistent plane detection and analysis method for monocular video scene three dimensional structure |
| CN106530342A (en) * | 2016-10-31 | 2017-03-22 | 武汉大学 | Measurable panorama image generation method assisted by laser point cloud |
Non-Patent Citations (4)
| Title |
|---|
| TAGUCHI Y, JIAN Y D, RAMALINGAM S, ET AL.: "Point-plane SLAM for hand-held 3D sensors", 《 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION》 * |
| 王伟， 高伟等: "快速鲁棒的城市场景分段平面重建", 《自动化学报》 * |
| 王思洁，方莉娜等: "基于结构化场景的单幅图像建筑物三维重建", 《地球信息科学》 * |
| 缪君，储珺: "基于稀疏点云的多平面场景稠密重建", 《自动化学报》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108805972A (en) * | 2018-04-11 | 2018-11-13 | 杭州电子科技大学 | A kind of three-dimensional rebuilding method based on ground normal direction and two-dimentional intersection |
| CN108648224A (en) * | 2018-05-18 | 2018-10-12 | 杭州电子科技大学 | A method of the real-time scene layout identification based on artificial neural network and reconstruction |
| CN108648224B (en) * | 2018-05-18 | 2021-07-13 | 杭州电子科技大学 | Real-time scene layout recognition and reconstruction method based on artificial neural network |
| WO2020042970A1 (en) * | 2018-08-29 | 2020-03-05 | 广景视睿科技（深圳）有限公司 | Three-dimensional modeling method and device therefor |
| CN110782524A (en) * | 2019-10-25 | 2020-02-11 | 重庆邮电大学 | Indoor three-dimensional reconstruction method based on panoramic image |
| CN110782524B (en) * | 2019-10-25 | 2023-05-23 | 重庆邮电大学 | Indoor three-dimensional reconstruction method based on panoramic image |
| CN113096185A (en) * | 2021-03-29 | 2021-07-09 | Oppo广东移动通信有限公司 | Visual positioning method, visual positioning device, storage medium and electronic equipment |
| WO2022206255A1 (en) * | 2021-03-29 | 2022-10-06 | Oppo广东移动通信有限公司 | Visual positioning method, visual positioning apparatus, storage medium and electronic device |
| CN114463406A (en) * | 2022-01-25 | 2022-05-10 | 北京工业大学 | Camera rotation estimation method based on indoor environment under Manhattan hypothesis |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107292956B (en) | 2020-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107292956A (en) | A kind of scene reconstruction method assumed based on Manhattan | |
| CN104463108B (en) | A kind of monocular real time target recognitio and pose measuring method | |
| CN105631861B (en) | Restore the method for 3 D human body posture from unmarked monocular image in conjunction with height map | |
| CN104331924B (en) | Three-dimensional rebuilding method based on single camera SFS algorithms | |
| CN102999942B (en) | Three-dimensional face reconstruction method | |
| CN103810744B (en) | It is backfilled a little in cloud | |
| CN113012293B (en) | Stone carving model construction method, device, equipment and storage medium | |
| CN104835144B (en) | The method for solving camera intrinsic parameter using the picture and orthogonality of the centre of sphere of a ball | |
| CN106023303B (en) | A method of Three-dimensional Gravity is improved based on profile validity and is laid foundations the dense degree of cloud | |
| CN102129708A (en) | Fast multilevel imagination and reality occlusion method at actuality enhancement environment | |
| CN112001926B (en) | RGBD multi-camera calibration method, system and application based on multi-dimensional semantic mapping | |
| CN104992441A (en) | Real human body three-dimensional modeling method specific to personalized virtual fitting | |
| CN104574432B (en) | Three-dimensional face reconstruction method and three-dimensional face reconstruction system for automatic multi-view-angle face auto-shooting image | |
| CN108401461A (en) | Three-dimensional mapping method, device and system, cloud platform, electronic equipment and computer program product | |
| CN103607584B (en) | Real-time registration method for depth maps shot by kinect and video shot by color camera | |
| CN109887030A (en) | Texture-free metal parts image position and posture detection method based on the sparse template of CAD | |
| CN110148217A (en) | A kind of real-time three-dimensional method for reconstructing, device and equipment | |
| CN103400409A (en) | 3D (three-dimensional) visualization method for coverage range based on quick estimation of attitude of camera | |
| CN110910431B (en) | Multi-view three-dimensional point set recovery method based on monocular camera | |
| CN103489214A (en) | Virtual reality occlusion handling method, based on virtual model pretreatment, in augmented reality system | |
| CN105513063B (en) | Veronese maps the method that Throwing thing catadioptric video cameras are determined with chessboard case marker | |
| CN102902355A (en) | Space interaction method of mobile equipment | |
| CN107977996A (en) | Space target positioning method based on target calibrating and positioning model | |
| CN105913488B (en) | A kind of three-dimensional point cloud fast reconstructing method based on three-dimensional mapping table | |
| CN109920000B (en) | Multi-camera cooperation-based dead-corner-free augmented reality method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Yan Chenggang Inventor after: Zhu Zunjie Inventor after: Xu Feng Inventor after: Ning Ruixin Inventor before: Zhu Zunjie Inventor before: Yan Chenggang Inventor before: Xu Feng Inventor before: Ning Ruixin |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |