KR102494552B1 - a Method for Indoor Reconstruction - Google Patents

Abstract

Disclosed is a method of restoring a 3D interior using a video captured by a depth camera. A 3D indoor restoration method according to an embodiment of the present invention includes acquiring a video of a room photographed using a depth measuring camera, creating initial data for 3D restoration using the acquired video, and the initial data. estimating a layout from, and performing global 3D data matching using the estimated layout.

Description

Indoor restoration method {a Method for Indoor Reconstruction}

The present invention relates to a technique for restoring a room in three dimensions. Specifically, it relates to an indoor 3D reconstruction technique using joint layout estimation and global multi-view registration.

The background technology of the present invention is a technology for restoring a room in 3D using a low-cost depth camera, and corresponds to Simultaneous localization and mapping (SLAM), 3D reconstruction, and the like.

In the past, many studies have been conducted on SLAM and 3D reconstruction using RGB cameras, but when RGB images are used, the algorithm does not work well in scenes with insufficient texture, and the distance measurement is not accurate. there is

Recently, as low-cost depth measurement cameras have been commercialized, 3D reconstruction studies that can compensate for the above-mentioned disadvantages using them have been actively conducted.

Prior Art 1 is the most representative study for performing 3D reconstruction using a depth measuring camera. The posture of the camera is continuously estimated using the iterative closest point (ICP) algorithm, and the depth information of each pixel obtained from each frame is projected onto the 3D volume space using the truncated signed distance function (TSDF) to obtain a 3D restoration result. Acquired in real time.

However, prior art 1 is vulnerable to drift errors generated when estimating a camera attitude, and can only restore a narrow space due to limitations of GPU memory. Therefore, various studies are being conducted to solve these problems.

However, if the existing algorithm still doesn't work well (e.g. loop closure is not detected/falsely detected), then loop closure: detects the camera coming back to where it was previously shot, and based on that detects the camera pose An algorithm that minimizes the cumulative error generated during estimation) occurs, and therefore, the following describes a method that can solve problems in the existing technology and improve the performance of the indoor 3D reconstruction technology.

R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In ISMAR, 2011. S. Choi, Q.-Y. Zhou, and V. Koltun. Robust reconstruction of indoor scenes. In CVPR, 2015. R. B. Rusu, N. Blodow, and M. Beetz. Fast point feature histograms (fpfh) for 3d registration. In ICRA, 2009. Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. TPAMI, 23(11):1222??1239, 2001. R. B. Rusu and S. Cousins. 3D is here: Point Cloud Library (PCL). In ICRA, 2011. M. Iwanowski. Morphological boundary pixel classification. In International Conference on "Computer as a Tool", 2007. Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing, 10(3):145??155, 1992. T. Whelan, M. Kaess, M. Fallon, H. Johannsson, J. Leonard, and J. McDonald. Kintinuous: Spatially extended Kinect-Fusion. In RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras, 2012. T. Whelan, S. Leutenegger, R. S. Moreno, B. Glocker, and A. Davison. Elasticfusion: Dense slam without a pose graph. In RSS, 2015. J. Xiao, A. Owens, and A. Torralba. SUN3D: A database of big spaces reconstructed using sfm and object labels. In ICCV, 2013.

In the present invention, it is intended to improve 3D reconstruction accuracy by estimating the layout of an indoor structure to be restored and utilizing it.

A 3D indoor restoration method according to an embodiment of the present invention includes acquiring a video of a room photographed using a depth measuring camera, creating initial data for 3D restoration using the acquired video, and the initial data. estimating a layout from, and performing global 3D data matching using the estimated layout.

The indoor restoration method according to an embodiment of the present invention is vulnerable to cumulative errors generated when estimating the camera posture, solves the problem of being able to restore only a narrow space due to the limitation of GPU memory, and improves the performance of indoor 3D restoration technology. can

1 shows the overall flow of an indoor 3D reconstruction technology algorithm according to an embodiment of the present invention.
Figure 2 shows the results of hierarchical cohesive clustering and planar clustering by energy-based multi-model matching.
3 shows a layout estimation process.
4 is an associative optimization algorithm of layout estimation and global matching according to an embodiment of the present invention.
FIG. 5 shows the improvement of restoration quality according to the result of iterative optimization according to the algorithm of FIG. 4 .

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the following embodiments, and those skilled in the art who understand the spirit of the present invention can easily find other embodiments included in the scope of the same spirit by adding, changing, deleting, and adding elements. It may be suggested, but it will be said that this is also included within the scope of the spirit of the present invention.

In the accompanying drawings, in describing the overall structure in order to easily understand the spirit of the invention, minute parts may not be specifically expressed, and in describing minute parts, the overall structure may not be specifically reflected. In addition, even if the specific parts such as the installation location are different, when the action is the same, the same name is given to increase the convenience of understanding. In addition, when there are a plurality of the same configuration, only one configuration is described, and the same description is applied to the other configurations, and the description is omitted.

As described above, in order to solve the existing problem, in the present invention, the layout of an indoor structure to be restored is estimated, and 3D reconstruction accuracy is improved by using the estimated layout. That is, in the present invention, the interior is restored by constraining the interior structure so that the floor and the ceiling are parallel and perpendicular to the wall.

1 shows the overall flow of an indoor 3D reconstruction technology algorithm according to an embodiment of the present invention.

The algorithm shown in FIG. 1 may be implemented by modules in which each step is named an initial matching unit, a layout estimation unit, and a global matching unit using layout. Also, the algorithm shown in FIG. 1 may be performed by an application in which each step is programmed, or a system in which the application is installed. Here, the system may refer to a hardware device capable of installing and executing applications, and each of the above-described modules may be provided in one hardware device or may be provided in a plurality of different hardware devices.

Briefly explaining the indoor 3D restoration technology algorithm according to an embodiment of the present invention, first, using the existing algorithm (prior art 2), 3D volume data for each part of the room is formed, and global 3D data matching (Global ICP ) to obtain incomplete 3D reconstruction data due to accumulated errors. In addition, supplementary data is obtained by repeatedly performing layout estimation and layout constraint-based global ICP on incomplete 3D reconstruction data. Hereinafter, the global 3D data matching method will be sequentially described.

1. Initial global 3D data registration (Initial registration)

When a video taken indoors using a depth measuring camera is given, the algorithm according to the prior art 2 is used to create initial data for 3D reconstruction. The algorithm according to the prior art 2 consists of three steps: 1) partial 3D reconstruction, 2) loop joint detection, and 3) posture graph optimization.

(1) Partial 3D reconstruction

The interior of the building is first partially restored. Specifically, 3D reconstruction is performed by dividing a video captured using a depth camera into 50 frames, and the restored scene fragments are registered with a 3D global coordination. The reason for this is that 3D reconstruction using a small number of frames can be somewhat accurate, and then the large-scale indoor 3D reconstruction problem can be simplified to a problem of registering scene fragments to the global coordinate system. am.

)을 만들기 위해서, 선행기술 1에서 소개하는 KinectFusion 알고리즘이 사용된다. KinectFusion 알고리즘을 이용하면 연속된 프레임 간 카메라의 자세 변화를 추정해주기 때문에, 연속된 장면 조각들 간의 좌표계(혹은 자세) 변환 행렬(T_{i, i+1})을 알 수 있으며, 이 변환 행렬의 순차적인 곱을 통해 각 장면 조각들을 전역 좌표계에 정합시킬 수 있다. 이러한 과정을 통해 장면 조각들의 집합(

)에 대해, 장면 조각들을 전역 좌표계로 정합시키는 변환 행렬(

)을 구할 수 있다.scene fragments (

), the KinectFusion algorithm introduced in Prior Art 1 is used. Since the KinectFusion algorithm estimates the change in the pose of the camera between consecutive frames, it is possible to know the coordinate system (or pose) transformation matrix (T _{i, i+1} ) between successive scene pieces, and the sequential sequence of this transformation matrix Through multiplication, each scene fragment can be registered to the global coordinate system. Through this process, a set of scene fragments (

), a transformation matrix that aligns the scene fragments to the global coordinate system (

) can be obtained.

(2) loop closure detection

When global registration is performed using a transformation matrix (T _{i, i+1} ) between scene pieces, an accumulated error for pose estimation occurs, and thus an inaccurate indoor reconstruction result may be provided. Therefore, in order to compensate for this, a step of detecting loop coupling and optimizing a transformation matrix based on the detection result is performed.

Matching between scene fragments is performed using the FPFH descriptor (prior art 3) to detect loop coupling. If the ratio of the overlapping area when matching a pair of scene pieces F _i and F _j is 30% or more, the pair of scene pieces are considered to be loop-combined successfully, and the transformation matrix (T _i,j ) from the matching result ) is defined.

(3) Optimization of pose graph

)이 최적화된다. T_i는 장면 조각 F_i를 전역 좌표계로 변환하는 행렬로 정의되며,

는 j번째 장면 조각 좌표들 F_j를 i번째 장면 조각 좌표계로 변환하는 행렬로 정의된다. 그리고 난뒤, 아래 수학식 1에 따라 에너지 함수를 최소화함으로써, 최적의 변환 행렬(

)과 라인 프로세스(line process) (

)가 추정될 수 있다.When loop coupling is detected, the pose graph, that is, a set of transformation matrices (

) is optimized. T _i is defined as a matrix that transforms a scene fragment F _i into the global coordinate system,

is defined as a matrix that transforms the j-th scene fragment coordinates F _j into the i-th scene fragment coordinate system. And then, by minimizing the energy function according to Equation 1 below, the optimal transformation matrix (

) and line process (

) can be estimated.

는 부분적 3차원 복원에 의해 계산된 변환 행렬과 T_a, T_b로부터 계산된 변환 행렬간의 차이를 측정한다.

는 0 ~ 1의 값을 갖는 라인 프로세스의 매개 변수이다. 함수

는 올바른 루프 결합(inlier loop closure)의 수를 최대화 해주며, l_ij가 일정 임계치 이상이면 i와 j번째 조각이 올바른 루프 결합을 이룸을 의미한다.Here, the function

measures the difference between the transformation matrix calculated by partial 3D reconstruction and the transformation matrix calculated from T _{a and} T _b .

is a parameter of the line process with a value of 0 to 1. function

maximizes the number of correct loop closures (inlier loop closure), and if l _ij is above a certain threshold value, it means that the i and j th pieces form the correct loop closure.

2. Layout estimation

The layout estimation problem is a problem of finding a plane representing the interior. In order to estimate the layout of a room composed of a set of planes, such as a floor, ceiling, and wall, a dominant plane (P _dominant ) is first estimated, and then a set of layout planes (P _layout ) is selected from the representative plane.

(1) Dominant plane extraction

Figure 2 shows the results of hierarchical cohesive clustering and planar clustering by energy-based multi-model matching.

)으로 나눈다. 그리고 임의의 세개의 인접한 수퍼복셀을 선택하고, 그 수퍼복셀들의 중심점을 이용하여 평면 파라미터 추측치(plane parameter hypothesis,

)를 계산한다. 이렇게 하는 이유는 선택된 수퍼복셀의 모든 점을들 이용하는 것보다 빠르게 평면 파라미너를 계산할 수 있기 때문이다. 벽과 같이 넓은 평면을 갖는 영역은 유사한 다수의 평면 파라미터가 존재하는바, 이들을 군집화(clustering)하여 하나의 평면 파라미터로 표현할 필요가 있다. 이를 위해 본 발명에서는 두 단계의 군집화를 통해 대표 평면을 추출하는 방법을 제안한다.First, a scene fragment (Fi ) is a set of _supervoxels (

) divided by Then, randomly select three adjacent supervoxels, and use the center points of the supervoxels to obtain a plane parameter hypothesis.

) is calculated. The reason for doing this is that the planar parameters can be calculated faster than using all the points of the selected supervoxel. A region having a wide plane, such as a wall, has a plurality of similar plane parameters, and it is necessary to cluster them and express them as one plane parameter. To this end, the present invention proposes a method of extracting a representative plane through two-step clustering.

) 사이의 거리를 측정하는 함수를 아래 수학식 2와 같이 정의한다.In the first clustering step, planar parameters are merged through hierarchical agglomerative clustering (HAC). For this purpose, an arbitrary supervoxel S _k and an arbitrary plane parameter estimate (

) A function for measuring the distance between is defined as in Equation 2 below.

는 수퍼복셀 내 3차원 점의 개수를 의미하며, 거리함수 d()는 수학식 3과 같이 정의된다.From here

denotes the number of 3D points in the supervoxel, and the distance function d() is defined as in Equation 3.

로 정의되며,

는 수퍼복셀 S_k에 속하는 3차원 점 P의 동차 좌표(homogeneous coordinate), 즉

를 의미한다. 군집화된 수퍼복셀들은 병합된 하나의 평면 파라미터를 계산하는데 사용된다.Here, the plane parameter is

is defined as,

is the homogeneous coordinate of the 3-dimensional point P belonging to the supervoxel S _k , that is,

means The clustered supervoxels are used to calculate the merged one plane parameter.

However, there are many cases where the region is distorted and reconstructed at the scene fragment boundary, so parts that should be clustered on the same plane still remain unclustered. Therefore, in order to enable clustering of these parts, the present invention proposes an energy-based multi-model matching method as a second clustering step.

)과 평면 파라미터 추측치들(

)이 주어질 때, 본 발명에서 해결하고자 문제는 3차원 점(p)를 평면 파라미터에 매핑하는 함수

를 구하는 것으로 정의된다. 이 문제를 해결하기 위한 에너지 함수 E_P는 아래 수학식 4와 같이 정의된다.3D points in a scene fragment (

) and planar parameter estimates (

) is given, the problem to be solved in the present invention is a function that maps a three-dimensional point (p) to a plane parameter.

is defined as finding An energy function E _P to solve this problem is defined as in Equation 4 below.

Here, the data term (D _p ) is defined as in Equation 5 that measures the distance between the 3D point (p) and the plane parameter.

는 상수이다.

는 평면 파라미터가 할당되지 않은 상태를 의미하는데, 이는 비평면 위에 놓여진 점들 혹은 잡음이 큰 점들에

를 할당하기 위함이다. 스무드 항(smoothness term) V_p,q는 Pott 모델(선행기술 4)이 사용된다. Pott 모델은

로 정의되며, 여기에서

는 패널티 파라미터(penalty weight)이고, 함수(T)는 함수 안의 내용이 참이면 1, 그렇지 않으면 0을 출력하는 함수이다.From here

is a constant.

means a state in which no planar parameter is assigned, which is for points lying on a non-planar surface or points with large noise.

is to assign The smoothness term V _p,q uses the Pott model (prior art 4). Pott's model is

is defined as, where

is a penalty weight, and function T is a function that outputs 1 if the content in the function is true and 0 otherwise.

N _p is a set of points neighboring the point p, which is estimated using the k-nearest neighbor (k-NN) algorithm (Prior Art 5). The energy function E _p estimates the optimal mapping h using graph cuts.

As a result, it is possible to obtain a small number of merged plane parameters, and if all the plane parameters estimated in each piece F _i are converted to the global coordinate system and projected, it is expressed as (a).

Next, as in (a), the planes projected in the global coordinate system are merged and the planes are merged using the HAC algorithm again to represent them as representative planes as in (b). In one embodiment, a merged plane such as (a) can be obtained using the energy-based multi-model matching method according to Equation 4, and a representative plane such as (b) using the HAC algorithm according to Equation 2 can obtain them.

(2) Layout plane estimation

3 shows a layout estimation process.

Given the previously estimated representative planes (P _dominant ) and clustered points, layout planes including floors, ceilings, and walls can be estimated. In this case, in an embodiment, a weak Manhattan world assumption may be used as an algorithm for estimating the layout plane. Through this, in the process of estimating the layout plane, it is possible to improve the accuracy of the layout estimation by forcibly making the planes almost vertical to be vertical.

As shown in FIG. 3 , in the present invention, a set of layout planes (P _layout ) is found through two steps. The first step is to find a base plane having the largest area among representative planes assumed to be a ceiling or a floor. As shown in the occupancy grid generation of FIG. 3, an occupancy grid map parallel to each plane is created, and points belonging to the plane are projected onto the occupancy grid map. Then, by counting the number of occupied lattices, the plane occupying the most area is determined as the base plane.

), 즉, 경계면에 해당하는 위치를 알 수 있다. 최종적으로 수학식 6에 따라 계산된 값이 1인 평면을 레이아웃 평면으로 결정한다.The second step is to find planes perpendicular to the base plane, i.e. walls. First, an occupancy grid map parallel to the basal plane is formed. Then, all points belonging to all scene fragments projected in the global coordinate system are projected into the occupancy grid map. If the number of projected points in a cell of the occupied grid map exceeds a certain threshold, the cell is determined to be occupied. The occupied lattice map thus created uses the morphological boundary detection method (prior art 6) to determine the boundary of the occupied lattice (

), that is, the position corresponding to the boundary surface can be known. Finally, a plane having a value of 1 calculated according to Equation 6 is determined as a layout plane.

와

는 선택된 대표 평면과 기저 평면의 법선 벡터를 나타낸다. 첫 번째 판단 기준은 두 평면의 수직성을 검사하는 것이며, 두 번째 판단 기준은 해당 대표 평면(

)과 경계면(

) 사이의 거리를 측정한다.

과

는 사용자에 의해 정해진 상수 파라미터이다. 거리함수 g()는 아래 수학식 7과 같이 정의된다.From here,

and

represents the normal vector of the selected representative plane and the basal plane. The first criterion is to check the perpendicularity of the two planes, and the second criterion is to check the corresponding representative plane (

) and the boundary (

) to measure the distance between them.

class

is a constant parameter determined by the user. The distance function g() is defined as in Equation 7 below.

는 평면(

)에 속하는 점들의 집합을 의미하고,

는 점들의 집합(

)에 속하는 점들의 수를 나타낸다.Here, p _proj is the projection of point p onto the base plane,

is the plane (

) is the set of points belonging to

is the set of points (

) represents the number of points belonging to

3. Layout-constrained global registration

Global matching refers to restoring a room in 3D by matching a set of scene fragments to a global coordinate system using a previously estimated layout. In one embodiment of the present invention, the global 3D data matching proposes a combined approach in which the layout estimation problem and the global matching problem alternately solve each other using the mutually dependent characteristics. This will be described in detail below.

4 is an associative optimization algorithm of layout estimation and global matching according to an embodiment of the present invention.

FIG. 5 shows the improvement of restoration quality according to the result of iterative optimization according to the algorithm of FIG. 4 .

는 장면 조각의 쌍으로 이루어진 집합으로 정의하며, 겹쳐진 영역이 존재하는 한 쌍의 조각(

)에 대하여 겹쳐진 영역에 대한 대응점들의 집합을

로 정의한다. 이와 유사하게, 조각

와 레이아웃 사이의 대응되는 점들의 집합을

로 나타낸다. 레이아웃은 평면 파라미터 형태로 존재하기 때문에, 가상적으로 3차원 점을 만들어주어야 한다 이를 위해

의 점을 레이아웃 평면에 투영하여 그 점을 레이아웃의 점으로 정의하고, 두 점간의 거리가 작은 경우, 그 주점을 대응점으로써 정의한다. 조각에 속하는 임의의 3차원 점(

)과 임의의 장면 조각으로부터 전역 좌표계로 변환시키는 행렬(T)에 대하여 전역 좌표계로 변환된 좌표는

로 구해지며,

은 회전 행렬(rotation matrix),

는 병진 벡터(translation vector)를 나타낸다.From here,

is defined as a paired set of scene fragments, and a pair of fragments with overlapping regions (

), a set of correspondence points for the overlapped region

is defined as Similarly, fragment

the set of corresponding points between and the layout

represented by Since the layout exists in the form of a plane parameter, it is necessary to virtually create a 3-dimensional point.

The point of is projected onto the layout plane, the point is defined as a point of the layout, and when the distance between the two points is small, the principal point is defined as a corresponding point. Any 3D point belonging to the piece (

) and the coordinates converted to the global coordinate system for the matrix T that transforms from an arbitrary scene fragment to the global coordinate system are

is saved by

is the rotation matrix,

represents a translation vector.

For global matching of all pieces, an energy function is defined as in Equation 8.

와

는 가중치 파라미터(weighting parameter)이며,

는 계산되는 점들의 개수에 따라,

는 계산되는 장면 조각 쌍들의 개수에 따라 결정된다. 수학식 8에서 첫번째 항은 레이아웃과 한 장면 조각 간 대응점의 거리를 최소화하는 역할을 한다. 이를 위한 거리 척도로써, point-to-plane(선행기술 7)를 이용하며 이는 수학식 9와 같이 정의된다.From here,

and

is a weighting parameter,

depends on the number of points to be calculated,

is determined according to the number of scene piece pairs to be calculated. In Equation 8, the first term serves to minimize the distance of the corresponding point between the layout and one scene fragment. As a distance scale for this, point-to-plane (prior art 7) is used, which is defined as in Equation 9.

Here, p denotes a 3D point belonging to a scene fragment, n _p denotes a normal vector of p, and q denotes a point belonging to a layout. The second term of Equation 8 serves to minimize the distance of the corresponding point between the two scene pieces, and is defined as Equation 10 using the point-to-plane scale in the same way as above.

The last term in Equation 8 is defined as in Equation 11 as a term limiting the estimated transformation matrix from being significantly different from the previously calculated transformation matrix.

는 ICP 알고리즘(선행기술 7)에 의해 추정된 두 장면 조각 간 변환 행렬을 의미하고,

는 행렬 요소 값들의 L1-norm 값들의 합을 나타낸다. 수학식 8을 최적화하기 위해 Gauss-Newton 방법이 이용될 수 있다.From here,

Means a transformation matrix between two scene fragments estimated by the ICP algorithm (prior art 7),

represents the sum of L1-norm values of matrix element values. A Gauss-Newton method may be used to optimize Equation 8.

)와 변환 행렬에 대한 초기값(

)이 주어지는 것으로 가정하며, 첫 번째 장면 조각의 좌표계를 전역 좌표계로 설정하고, T₀는 항등 행렬(identity matrix)로 고정시킨다.Since it is difficult to solve the layout estimation problem and the global matching problem at once, the present invention alternately optimizes the two problems, as shown in FIG. 4 . scene fragment (

) and the initial value for the transformation matrix (

) is given, the coordinate system of the first scene fragment is set to the global coordinate system, and T ₀ is fixed as an identity matrix.

Further, when the algorithm of FIG. 4 is repeated, as shown in FIG. 5 , it can be seen that as the number of iterations increases, the result of restoring the curved interior is correctly restored to a straightened form.

The table below compares the restoration performance evaluation of the algorithm proposed in the present invention and the existing algorithm.

As shown in Table 1, it can be confirmed that the interior can be restored more accurately than other recent methods. In addition, it can be confirmed that the restoration accuracy is improved numerically in the quantitative evaluation using virtual data.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet).

Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (4)

Obtaining a video of a room photographed using a depth measuring camera;
Creating initial data for 3D reconstruction using the acquired video;
estimating a layout from the initial data; and
Performing global 3D data matching using the estimated layout;
The step of estimating the layout from the initial data,
Extracting a representative plane representing the interior using hierarchical agglomerative clustering (HAC) and an energy-based multi-model matching method; and
Estimating a layout plane from the estimated representative planes
3D interior restoration method. According to claim 1,
The step of creating initial data for the 3D reconstruction,
Creating 3D reconstructed scene fragments by dividing the acquired video by a certain frame, acquiring a transformation matrix by matching the restored scene fragments to a 3D global coordinate system, and detecting loop combinations to complement the obtained transformation matrix optimizing the transformation matrix based on
3D interior restoration method. According to claim 1,
Estimating a layout plane from the estimated representative planes,
Finding a base plane among the representative planes and a plane perpendicular to the base plane
3D interior restoration method. According to claim 1,
The step of performing global 3D data matching using the estimated layout
Iterating the layout estimation and global matching alternately a predetermined number of times.
3D interior restoration method.

Images