KR20190025267A - Apparatus for detecting of inside wall frame in single image using orthogonal vanishing points and method thereof - Google Patents

Abstract

An apparatus and a method for detecting an inner wall structure using a single image photographing an indoor space are provided. The method for detecting an inner wall structure comprises the steps of: extracting a line segment from an input single image; extracting a reference vanishing point for an image based on the line segment; detecting an optimal vanishing point structure based on the reference vanishing point and the line segment; generating a wall direction map for the image based on the optimal vanishing point structure and the line segment; determining a spatial layout in the image based on the wall direction map and the optimal vanishing point structure; and outputting information of the spatial layout as inner wall structure detection information.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for detecting an inner wall structure of a single image using a quadrature vanishing point,

The present invention relates to an apparatus and method for recognizing and detecting an inner wall structure in a two-dimensional single image.

A single image of an indoor space contains only surface information (ie, color information), and thus an auxiliary sensor such as a light dectection and ranging (leader) sensor or depth camera is required to recognize the shape of the interior wall structure of the room Do. The three-dimensional coordinate data of the indoor space can be detected using the auxiliary sensor.

The technique of modeling the indoor space based on the three-dimensional coordinate data is highly accurate but requires special equipment. Accordingly, there has been a limit in that a picture taken in real time via a photograph or a smart phone can not recognize the inner wall structure of the indoor space.

Japanese Patent Registration No. 4153761 (entitled " 3-dimensional model space generation device, 3-dimensional model space generation method, and 3-dimensional model space generation program ") discloses a method in which the same reference point and the same subject are photographed in at least two different directions A pair of stereo pair images is acquired and the points that are distinguishably arranged on the subject are extracted and corresponded on the stereo pair image and two corresponding points on the same plane among the corresponding points on the already obtained stereo pair image are selected, A color on a straight line including the selected two points is identified by image processing and a point closest to either one of the two points at which the identified color on the straight line changes is specified as a corresponding point on the stereo pair image , A combination of the extracted point and a point at which a predetermined expression accuracy can be obtained at a specific point is specified , Thus using the point that is included in the particular combination is determined by the start of calculating the three-dimensional information of the object, three-dimensional model space generating unit to a stereo image pair.

In order to detect the indoor wall structure (i.e., spatial layout) based on only a single image without using the auxiliary sensing means, it is necessary to identify a plurality of objects included in the image or visible portions of the inner wall, It is necessary to derive the estimate for To do this, it is important to define how the scene space should be parameterized for a single image.

In order to solve the problems of the related art described above, an embodiment of the present invention detects three mutually orthogonal vanishing points found in an image of an indoor space, and uses the detected vanishing points to infer the relationship between the shooting point and the indoor space Thereby detecting an inner wall structure in a single image, and a method thereof.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to an aspect of the present invention, there is provided an apparatus for detecting an inner wall structure using a single image of an indoor space, the apparatus comprising: a memory for storing an inner wall structure detection program; And a processor for executing the program stored in the memory. At this time, in accordance with execution of the inner wall structure detection program, the processor extracts line segments from the input single image, detects a reference vanishing point with respect to the image based on the line segment, and based on the reference vanishing point and line segment Wherein the spatial layout is determined based on the wall surface direction map and the optimal vanishing point structure, and the method further comprises the steps of: determining an optimal vanishing point structure based on the optimal vanishing point structure and the optimal vanishing point structure; generating a wall surface direction map for the image based on the optimal vanishing point structure and line segments; And outputs the information of the spatial layout as inner wall structure detection information. The optimal vanishing point structure may include a reference vanishing point having a highest vote value for all line segments in the image among the intersections between the line segments as a reference vanishing point and one of intersections perpendicular to the reference vanishing point, A first vanishing point and a second vanishing point perpendicular to both the reference vanishing point and the first vanishing point. Also, the voting value has a higher value as the angle between the center point of the line segment and the intersection is smaller, and has a value proportional to the length of the line segment.

According to another aspect of the present invention, there is provided a method of detecting an inner wall structure using a single image of an indoor space, comprising: extracting a line segment from a single input image; Extracting a reference vanishing point for the image based on the line segment; Detecting an optimal vanishing point structure based on the reference vanishing point and the line segment; Generating a wall orientation map for the image based on the optimal vanishing point structure and the line segment; Determining a spatial layout in the image based on the wall surface direction map and the optimal vanishing point structure; And outputting the information of the spatial layout as inner wall structure detection information. The optimal vanishing point structure may include a reference vanishing point having a highest vote value for all line segments in the image among the intersections between the line segments as a reference vanishing point and one of intersections perpendicular to the reference vanishing point, A first vanishing point and a second vanishing point perpendicular to both the reference vanishing point and the first vanishing point. Also, the voting value has a higher value as the angle between the center point of the line segment and the intersection is smaller, and has a value proportional to the length of the line segment.

According to one of the above-mentioned objects of the present invention, it is possible to detect an indoor wall structure without an additional device such as a stereo camera, a lidar, etc., and to provide an indoor structure (i.e., a space layout) And provides inner wall structure detection information for generating the recognized indoor structure as a three-dimensional modeled object.

1 is a configuration diagram of an inner wall structure detecting apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating an inner wall structure detection method according to an embodiment of the present invention.
3 is an example of edge detection result and line segment detection result in an image according to an embodiment of the present invention.
4 is a view for explaining a reference vanishing point detection method according to an embodiment of the present invention.
5 is a view for explaining a vanishing point candidate elimination method according to an embodiment of the present invention.
FIGS. 6 and 7 are diagrams for explaining the criteria of the vanishing point candidate suppression process applied to the embodiment of the present invention, respectively.
8 is a diagram for explaining a vanishing point structure candidate generating method according to an embodiment of the present invention.
9 is a diagram for explaining an optimal vanishing point structure detection method according to an embodiment of the present invention.
10 is a diagram for explaining an optimal vanishing point structure detection method using an equation according to an embodiment of the present invention.
11 and 12 are views for explaining a method of detecting a wall surface direction map for an image according to an embodiment of the present invention, respectively.
13 is a view illustrating a wall direction map for an image according to an embodiment of the present invention.
14 is a diagram for explaining a corner detection method in an image based on a wall surface direction map according to an embodiment of the present invention.
15 is a diagram for explaining an intra-image spatial layout determination method based on a corner detection result according to an embodiment of the present invention.
16 is a diagram illustrating spatial layout detection results according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as " comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Refers to a unit realized by hardware or software, a unit realized by using both hardware and software, and a unit includes two or more hardware Or two or more units may be realized by one hardware.

1 is a configuration diagram of an inner wall structure detecting apparatus according to an embodiment of the present invention.

The inner wall structure detecting apparatus 100 according to an embodiment of the present invention detects an inner wall structure in an image based on a single image. At this time, the inner wall structure detection apparatus 100 can detect the inner wall structure in the image by recognizing the spatial layout of indoor scenes, which is a boundary between the wall and the wall, from the image taken in the room. For reference, a single image may refer to a single photograph or the like, or may be a single image photographed using a monocular lens camera.

1, the inner wall structure detecting apparatus 100 includes a video input / output module 110, a memory 120, and a processor 130. The video input /

The video input / output module 110 receives the video of the inner wall structure detection target and provides the input video to the processor 130. The video input / output module 110 may output the input image to a display (not shown), and may output an image including the inner wall structure detection information transmitted from the processor 130 to a display (not shown).

The memory 120 stores an inner wall structure detection program for detecting a spatial layout in an image based on a single image and outputting inner wall structure detection information. This memory 120 is collectively referred to as a non-volatile storage device that keeps stored information even when power is not supplied, or a volatile storage device that requires power to maintain stored information.

The processor 130 executes a program stored in the memory 120, and performs processes according to the execution of the inner wall structure detection program.

Hereinafter, with reference to FIG. 2, a description will be given of the procedures by which the processor 130 processes according to the execution of the inner wall structure detection program.

2 is a flowchart illustrating an inner wall structure detection method according to an embodiment of the present invention.

Referring to FIG. 2, the processor 130 extracts a line segment from the input image (S210), detects a reference vanishing point in the image based on the extracted line segments (S220), and detects the reference vanishing point and the line segment (S230), and generates a wall surface direction map for the image based on the vanishing points and the line segment included in the detected vanishing point structure (S240). Then, the generated wall surface direction map The spatial layout is determined based on the vanishing points of the vanishing point structure (S250), and the inner wall structure detection information based on the spatial layout information is output (S260).

First, a description will be given of a step S210 of the processor 130 extracting a line segment with reference to FIG.

3 is an example of edge detection result and line segment detection result in an image according to an embodiment of the present invention.

The processor 130 detects a plurality of edges in an input image (i.e., a single image). The processor 130 can express an edge by emphasizing a change in an image (change in color or brightness) and binarize the contour of the object and the distribution of the image signal. At this time, a result of detecting the edge pixels in the input image as shown in FIG. 3 (a) can be displayed as shown in FIG. 3 (b). For example, the processor 130 may detect an edge pixel in an image using a Canny Edge detection algorithm, and may detect a prewitt, LoG (Laplacian of Gaussian), or DoG (Difference of Gaussians) can be applied.

Next, the processor 130 performs gradient classification processing for detecting the direction with respect to the detected edge pixels. At this time, the processor 130 detects the direction of brightness gradient of the position where each edge pixel is detected in the input image original, and classifies the plurality of edges according to the direction. For example, the processor 130 may sample one edge image into eight directional edge images by sampling the edge direction in eight directions.

Then, the processor 130 detects line segments for edge images (i.e., directional edge images) including edge pixels classified by a plurality of directions. At this time, the processor 130 detects a cluster of consecutive edge pixels in each directional edge image, and generates a covariance matrix for the x-axis and the y-axis of the edge pixel. The processor 130 detects a line segment using an eigenvalue λ and an eigenvector v for the generated covariance matrix. The result of detecting the line segments in the input image as shown in FIG. 3 (a) can be displayed as shown in FIG. 3 (c).

Hereinafter, the step (S220) in which the processor 130 detects the reference vanishing point will be described with reference to FIG.

For reference, a vanishing point is a point where a parallel line is converged on an image plane in an actual environment in which an image is captured, and represents the relationship between the position (i.e., shooting point) at which the image was captured and the indoor space. In general, most interior structures are rectangular parallelepiped, and these cuboids have three vanishing points.

4 is a view for explaining a reference vanishing point detection method according to an embodiment of the present invention.

The processor 130 detects an intersection between all line segments (hereinafter, abbreviated as "lines"), and uses the unit sphere to re-image the intersection of the normal line projected on the unit sphere in the image To detect a vanishing point.

Referring to FIG. 4A, the processor 130 detects the intersection between lines using a Gaussian sphere. The processor 130 detects the intersection using the outer product of the normal vectors of the two lines (i.e., the line segments), and detects an intersection point with respect to all the lines in the image. That is, the processor 130 finds a vector in which two line segments face the image plane from the intersection point of the circles projected on the Gaussian sphere, and detects a point extending as far as the vector reaches the image plane as an intersection. The intersection points (P) between all these lines become vanishing point candidates.

In FIG. 4A, O is a coordinate center (i.e., a perspective center point), 1 is a straight line (i.e., a line) on an image plane, and p is an intersection point between straight lines. At this time, the line on the image plane and the line projected on the Gaussian sphere can be expressed by the following Equation 1, respectively.

&Quot; (1) "

및

는 각각 라인의 시작점 및 끝점의 벡터이고, z는 가우시안 구의 중심과 영상 평면 간의 거리이며,

는 라인 l과 원점을 지나는 평면의 법선 벡터이다. 이러한 법선 벡터

를 외적(Cross Product)하여 교점 방향의 단위 벡터(unit vector)

를 구할 수 있다.In Equation (1) above,

And

Is the vector of the start and end points of the line, z is the distance between the center of the Gaussian sphere and the image plane,

Is the normal vector of the plane passing through the line l and the origin. These normal vectors

A unit vector of the intersection direction is obtained by cross-

Can be obtained.

The processor 130 sets the detected intersection point for all lines as a candidate of a vanishing point in the image. Then, the processor 130 can measure (i.e., vote) the degree of contribution of the line l on the image plane to the vanishing point candidate p through the voting function v (l, p) as shown in Equation (2) below.

&Quot; (2) "

Referring to Figure 4 (b), and from equation 2 above, l ^m is the center point of the line l, p is a vanishing point candidates (that is, the point of intersection), d is the distance between l ^m and p, α is the line l And the vanishing point candidate (that is, the intersection). According to Equation (2), as the line segment connecting the center point of the line and the vanishing point candidate and the angle formed by the line are parallel, the degree of contribution of the line to the vanishing point candidate is measured (i.e. According to Equation (2), if the distance between the center point l ^m of the line and the vanishing point candidate p is less than half of the line length, 'o' is voted. As such, the processor 130 votes a higher value for the intersection (i.e., the vanishing point candidate) so that the angle formed by the intersection with the center point of the line segment is smaller and proportional to the length of the line segment.

Then, the processor 130 calculates a sum S of votes for each vanishing point candidate, as shown in Equation (3) below, and detects a vanishing point candidate having a maximum sum of the votes.

&Quot; (3) "

The vanishing point candidate p _n having the largest sum of the detected voices is detected as the reference vanishing point vp through Equation (3).

At this time, the processor 130 may remove a line that votes above the threshold value set for the reference vanishing point candidate through Equation (4) below.

&Quot; (4) "

Hereinafter, with reference to FIGS. 5 to 10, a description will be made of the step (S230) of the processor 130 detecting the vanishing point structure corresponding to the image.

First, the processor 130 removes unnecessary vanishing point candidates among vanishing point candidates other than the reference vanishing point, thereby preventing an unnecessary vanishing point structure candidate generation and an operation procedure for detecting the optimal vanishing point structure.

5 is a view for explaining a vanishing point candidate elimination method according to an embodiment of the present invention.

)과 그 외에 교점들(즉, 소실점 후보들

)을 표시한 도면이다. 5A is a graph showing the relationship between a vanishing point selected through Equation (3)

) And other intersections (i.e., vanishing point candidates

Fig.

Referring to FIG. 5B, the processor 130 detects a case where the angle between the reference vanishing point and the vanishing point candidate is acute with respect to the center point of the image plane, and removes the vanishing point candidate accordingly. The process of eliminating vanishing point candidates that form an acute angle with this reference vanishing point is based on the theory that projected line segments obtuse each other when projecting vanishing points perpendicular to each other on the image plane.

Specifically, referring to FIGS. 6 and 7, when projecting the vanishing points orthogonal to each other on the image plane, it is proved that the projected line segments form an obtuse angle with respect to each other.

FIGS. 6 and 7 are diagrams for explaining the criteria of the vanishing point candidate suppression process applied to the embodiment of the present invention, respectively.

In FIGS. 6 and 7, the three vectors on the image plane are represented by ①, ②, and ③, respectively, and the vector toward the opposite side of the image plane corresponding to the vectors ①, ②, ③ '. For reference, in an embodiment of the present invention, it is assumed that an indoor space is assumed to be a rectangular parallelepiped, and three vanishing points in an image of the indoor space are included. Since there are faces facing each other in the rectangular parallelepiped, there are opposite direction vectors 1 ', 2', and 3 'corresponding to the vectors 1, 2, and 3 respectively. Then, the three vectors ①, ②, and ③ facing the image plane are converged to the line segments ①, ②, and ③ on the image plane.

As shown in FIG. 6, assume initial states of vectors that are orthogonal to each other.

At this time, the three vectors ①, ②, and ③ facing the image plane are orthogonal to each other, and the line segments ①, ②, and ③ on the image plane are generated by projecting the vectors ①, ②, and ③ onto the image plane to be. And, the vectors ① ', ②', and ③ 'are vectors facing the opposite side of the image plane which are opposite to the directions of the vectors ①, ② and ③. Vector ① ', ②' and ③ 'and vectors ①, ② and ③ are vanishing points in the interior space forming a rectangular parallelepiped and perpendicular to each side of the interior space.

Referring to FIG. 7A, when the vectors in the initial state in FIG. 6 are rotated by an arbitrary angle in the counterclockwise direction about the vector 1, the vector 2 approaches the center of the image plane, Is far from the center of the image plane.

7B, when the vectors of the state shown in FIG. 7A are further rotated by an arbitrary angle in the counterclockwise direction about the vector 1, the locus of the vector 2 (that is, ') And the locus of the vector 1 (that is, the line segment 1' ') are parallel to each other. In this case, the vector ③ is parallel to the image plane but both directions (i.e., the vectors ③ and ③ ') are parallel to the image plane, and the locus (i.e., the line segment ③' 'And < 2 > "). This means that the line segments (i.e., line segments) on the image plane for detecting the vector ③ are in parallel with the locus of the vector ③ (i.e., the line segment ③ '').

 Referring to FIG. 7C, when the vectors in the state shown in FIG. 7B are further rotated by an arbitrary angle in the counterclockwise direction about the vector 1, the vector 2 is moved away from the center of the image plane again It can be seen that the lag vector ③ again approaches the center of the image plane. In this case, the vector? Directed toward the image plane in FIG. 6 (i.e., in the initial state) is directed to the opposite side of the image plane and is represented by vector? In FIG. 7 (c) 3 " is oriented toward the image plane and is represented by vector 3 in Fig. 7 (c).

In other words, the vanishing point that can be detected in the image plane can be seen to be exchanged between the vector pointing toward the opposite side of the image plane and the vector pointing toward the image plane according to the rotation of the indoor space. Such a vector direction swap that occurs when the indoor space rotates occurs in the same way even when the direction in which the camera (i.e., photographing point) is viewed changes. As described above, since the vectors directed to the image plane are exchanged with each other, it can be seen that the condition that the trajectories projected onto the image plane are not acute angles are maintained.

6 and 7, when the projection points orthogonal to each other are projected onto the image plane, based on the condition that the projected line segments obtuse each other, the processor 130 determines whether the center point of the image plane If the angle between the vanishing point and the vanishing point candidate is acute, remove the corresponding vanishing point candidates.

Then, the processor 130 determines a vanishing point perpendicular to the vanishing point candidates that have not been removed, and creates a vanishing point structure for each vanishing point candidate. At this time, the corresponding vanishing point structure for each vanishing point candidate is a vanishing point structure candidate, and each vanishing point structure candidate is a vanishing point candidate composed of a reference vanishing point, a vanishing point candidate, and one remaining vanishing point (i.e., vanishing point perpendicular to both the reference vanishing point and vanishing point candidate) Lt; / RTI > At this time, the three vanishing points included in the vanishing point structure candidate satisfy the mutually orthogonal property.

FIG. 8 is a view for explaining a method for generating a vanishing point structure candidate according to an embodiment of the present invention, and FIG. 9 is a view for explaining an optimal vanishing point structure detecting method according to an embodiment of the present invention.

) 별로 수직하는 점(Orthogonal Point)

를 생성한다.Referring to FIG. 8 (a), the processor 130 determines whether the remaining vanishing point candidates

(Orthogonal Point)

.

,

,

) 를 선택한다.Referring to FIG. 8B, the processor 130 detects a vanishing point structure candidate satisfying the following equation (5) and calculates an optimal vanishing point structure for the input image

,

,

).

Equation (5)

및

는, 각각 앞서 수학식 2에서와 같이 해당 소실점들에 대해 라인 세그먼트들이 기여하는 정도를 측정(즉, 투표)하는 투표 함수이다. 즉, 프로세서(130)는 소실점 구조체 후보 별로 기준 소실점을 제외한 나머지 두 소실점에 대해 라인 세그먼트 투표를 실시하여 가장 큰 투표 값을 갖는 소실점 구조체를 검출한다. 이때, 투표에 참여하는 라인 세그먼트들은 앞서 수학식 4를 통해 기준 소실점에 대해 일정 임계값 이상 투표한 라인 세그먼트는 제외시킨 나머지 라인 세그먼트일 수 있다. 이와 같이 기준 소실점에 임계값 이상 관계된 라인 세그먼트는 제외시킴으로써, 나머지 두 소실점에 관계된 라인 세그먼트들에 대해서만 투표를 진행할 수 있어 효율적인 연산 처리가 가능하다.In Equation (5)

And

Is a voting function that measures (i. E., Votes) the extent to which the line segments contribute to the vanishing points, as in Equation 2 above. That is, the processor 130 performs a line segment vote on the other two vanishing points excluding the vanishing point for each vanishing point structure candidate, and detects the vanishing point structure having the largest voting value. In this case, the line segments participating in the voting may be the remaining line segments excluding the line segments that have been voted higher than the predetermined threshold value with respect to the reference vanishing point through Equation (4). By excluding the line segments related to the reference vanishing point by more than the threshold value, voting can be performed only on the line segments related to the remaining vanishing points, thereby enabling efficient calculation processing.

,

,

)를 검출하고, 검출된 소실점 구조체를 최적 소실점 구조체로 결정한다.The processor 130 calculates the number of the vanishing point structure candidates having the largest number of vanishing point structure candidates (i.e., the largest voting value)

,

,

), And determines the detected vanishing point structure as the optimal vanishing point structure.

Then, the processor 130 determines the forward, horizontal, and vertical vanishing points based on the distance from the center point of the image plane to the vanishing point elements of the determined optimal vanishing point structure. At this time, the processor 130 sets the vertical vanishing point that is the y-axis distance from the center point of the image plane among the vanishing point elements of the optimal vanishing point structure as the vertical vanishing point, sets the farthest x-axis distance from the center point of the image plane as the horizontal vanishing point , And the one with the smallest distance from the center point of the image plane is set as the front vanishing point.

,

,

)에서 영상 평면의 중심점과의 거리가 가장 작은

가 전방 소실점

로 설정되고, x축 방향으로 거리가 가장 먼

가 수평 소실점

로 설정되며, y축 방향으로 거리가 가장 먼

가 수직 소실점

으로 설정된 것을 알 수 있다. 참고로, x축 방향 및 y축 방향은 각각 영상 내 실내 환경의 조건에 대응하는 수평 방향 및 수직 방향을 의미할 수 있다.Referring to FIG. 9, an optimal vanishing point structure (FIG. 8

,

,

), The distance from the center point of the image plane is the smallest

The front vanishing point

And the distance is the longest in the x-axis direction

Horizontal Vanishing Point

And the distance is the longest in the y-axis direction

Vertical Vanishing Point

. ≪ / RTI > For reference, the x-axis direction and the y-axis direction may mean a horizontal direction and a vertical direction corresponding to the conditions of the indoor environment in the image, respectively.

Referring to FIG. 10, a description will be given of how the processor 130 detects the vanishing point structure through the equation.

10 is a diagram for explaining an optimal vanishing point structure detection method using an equation according to an embodiment of the present invention.

Since the vanishing points included in the vanishing point structure must satisfy the conditions that are orthogonal to each other, the first vanishing point (that is, the vanishing point) and the vanishing point candidate z must be satisfied when the vanishing point structure is created.

과 소실점 후보

의 내적과 직교 조건을 이용하여 z를 구한다.Accordingly, the processor 130 determines whether the reference vanishing point &

And Vanishing Point Candidates

Z is obtained by using the inner product and the orthogonal condition.

과 소실점 후보

는 각각 아래의 수학식 6과 같이 표현할 수 있으며, 이 두 벡터의 내적은 수학식 7과 같다.Standard vanishing point

And Vanishing Point Candidates

Can be expressed by Equation (6) below, and the inner product of these vectors is expressed by Equation (7).

&Quot; (6) "

&Quot; (7) "

At this time, since the two vectors are orthogonal to each other, theta is 90 degrees, and the equation (7) can be calculated by applying Equation (8) as follows.

&Quot; (8) "

Referring to FIG. 10, since z-axis distances from the origin of the Gaussian sphere to the image plane are the same, z ₁ = z _i .

Therefore, the condition of z is expressed by Equation (9) below.

&Quot; (9) "

를 구한다. 이때, 소실점

는 아래의 수학식 10을 통해 산출할 수 있다.Next, the processor 130 uses the condition of z, which can be orthogonal to the reference vanishing point and the vanishing point candidate, as the vanishing point

. At this time,

Can be calculated by the following equation (10).

&Quot; (10) "

에 수직하는 소실점

을

라하고, 소실점

에 수직하는 소실점

을

라고 하면, 최적 소실점 구조체의 요소들은 아래의 수학식 11을 통해 전방, 수평 및 수직 소실점으로 결정된다.Then, the processor 130 detects the n-th vanishing point structure that maximizes the voting result of the line segment with respect to the N vanishing point structure candidates through Equation (5) as the optimal vanishing point structure. At this time,

The vanishing point perpendicular to

of

And the vanishing point

The vanishing point perpendicular to

of

, The elements of the optimal vanishing point structure are determined as forward, horizontal and vertical vanishing points by the following equation (11).

Equation (11)

Hereinafter, with reference to FIGS. 11 to 13, a description will be given of the step (S240) in which the processor 130 generates a wall orientation map for the image based on the optimal vanishing point structure and the intra-image line segment.

11 and 12 are views for explaining a method of detecting a wall surface direction map for an image according to an embodiment of the present invention, respectively.

에 속하는 라인을 전방 소실점

에 따른 방향으로 이동시킬 때 생기는 궤적면과, 전방 소실점

에 속하는 라인을 수직 소실점

에 따른 방향으로 이동시킬 때 생기는 궤적면 간의 교집합이 수평 소실점

를향하는 면(R_H)인 것을 알 수 있다.First, referring to FIG. 11 (a), the vertical vanishing point of the optimal vanishing point structure

Lt; RTI ID = 0.0 >

A locus plane generated when moving in a direction along the front side,

Lt; RTI ID = 0.0 >

The intersection between the trajectory planes generated when moving in the direction along the horizontal dis-

(R _H ) that is the direction facing the surface of the wafer _W.

Such a wall surface detection method can be expressed by the following equation (12) with reference to FIG. 11 (b).

&Quot; (12) "

L _{x, i} in Equation (12) indicates a line segment belonging to one of the vanishing point vp _x of the elements of a perfect vanishing point structure and, if S is in which α moves by a l _{x, i} in the direction of the other elements of the vanishing point vp _y Area. That is, each line segment is swept to the vanishing point direction and the reverse direction to which it does not belong.

This will be described in more detail with reference to FIG.

과 수직 소실점

에 기초하여 수직 소실점 소속 라인을 전방 소실점 방향으로 스윕 처리하여 벽면을 검출하는 것을 나타냈다.12 (a) is a diagram showing an example of an element of the optimal vanishing point structure,

And vertical vanishing point

, The vertical vanishing point belonging line is swept in the direction of the front disappearing point to detect the wall surface.

The processor 130 can detect the wall surface by sweeping the line belonging to the vertical vanishing point in the direction of the front vanishing point according to the following Algorithm 1.

<Algorithm 1>

12A shows that in the algorithm 1, x is vertical (that is, V) and y is forward (that is, M), and the vertical vanishing point belonging segment is swept in the forward vanishing point direction and the reverse direction respectively by ? Respectively.

The processor 130 integrates the planes generated by all the line segments in the image by the vanishing point directions.

과 수직 소실점

에 기초하여, 수직 소실점 소속 선분을 전방 소실점 방향으로 스윕 처리하고, 전반 소실점 소속 라인을 수직 소실점 방향으로 스윕 처리하여, 두 스윕 처리의 결과의 교집합을 수평 방향 벽면으로서 검출하는 것을 알 수 있다.Referring to FIG. 12 (b), among the elements of the optimal vanishing point structure,

And vertical vanishing point

, It is found that the intersection of the results of the two sweep processes is detected as the horizontal wall surface by sweeping the line belonging to the vertical vanishing point to the forward vanishing point direction and sweeping the line belonging to the vanishing vanishing point to the direction of the vertical vanishing point.

At this time, the processor 130 may process the vertical vanishing point sweep segment belongs according to the algorithm (2) below a vanishing point in the forward direction, and to sweep across the vanishing point to the processing belonging to the line in the vertical direction of the vanishing point, to detect a horizontal wall (R- _H) .

<Algorithm 2>

The wall surface detection process described with reference to Figs. 11 and 12 can be also performed on other vanishing points to detect the vertical wall surface and the front wall surface.

에 속하는 라인을 전방 소실점

에 따른 방향으로 이동시킬 때 생기는 궤적면과, 전방 소실점

에 속하는 라인을 수평 소실점

에 따른 방향으로 이동시킬 때 생기는 궤적면 간의 교집합을 수직 소실점

를 향하는 수직 방향 벽면(R_V)으로서 검출할 수 있다. 그리고, 최적 소실점 구조체의 수평 소실점

에 속하는 라인을 수직 소실점

에 따른 방향으로 이동시킬 때 생기는 궤적면과, 수직 소실점

에 속하는 라인을 수평 소실점

에 따른 방향으로 이동시킬 때 생기는 궤적면 간의 교집합을 전방 소실점

를 향하는 전방 방향 벽면(R_M)으로서 검출할 수 있다.That is, the horizontal vanishing point of the optimal vanishing point structure

Lt; RTI ID = 0.0 &gt;

A locus plane generated when moving in a direction along the front side,

Line to a horizontal vanishing point

The intersection between the trajectory planes generated when moving in the direction along the vertical vanishing point

As a vertically oriented wall surface R _V facing toward the center. Then, the horizontal vanishing point of the optimal vanishing point structure

Lt; RTI ID = 0.0 &gt;

, A locus plane generated when moving in the direction along the vertical vanishing point

Line to a horizontal vanishing point

The intersection between the trajectory planes generated when the vehicle moves in the direction corresponding to the front vanishing point

Direction wall surface R _M facing the front wall surface R _M.

At this time, the processor 130 recognizes the indoor structure according to the vanishing point in the direction of each wall face, recognizes the vertical wall face facing the vertical vanishing point direction as a ceiling or floor (hereinafter referred to as "vertical wall face"), The horizontal wall surface facing the vanishing point direction is recognized as at least one of the left and right wall surfaces (hereinafter referred to as " horizontal wall surface "), and the front wall surface facing the front vanishing point direction is defined as a front wall surface Can be recognized.

 13 is a view illustrating a wall direction map for an image according to an embodiment of the present invention.

The processor 130 detects walls facing each direction from the image, and then encodes pixels corresponding to wall surfaces in each direction. For example, in Fig. 13, the front direction wall surface is encoded in blue, the vertical direction wall surface is encoded in green, and the horizontal direction wall surface is encoded in red.

Hereinafter, with reference to FIGS. 14 to 16, a description will be given of the step (S250) in which the processor 130 determines the spatial layout based on the wall surface direction map and the vanishing points of the vanishing point structure.

14 is a diagram for explaining a corner detection method in an image based on a wall surface direction map according to an embodiment of the present invention. And FIG. 15 is a diagram for explaining a spatial layout determination method in an image based on the corner detection result according to an embodiment of the present invention. 16 is a diagram illustrating spatial layout detection results according to an embodiment of the present invention.

The processor 130 detects four corner (i.e., corner) points of the front wall surface based on the generated wall surface direction map.

Referring to FIG. 14, a wall surface direction map encoded with colors different from each other on the wall surface pixels is divided into four parts based on the front side vanishing point position, and corner points are detected in each of the divided areas. At this time, the processor 130 projects the encoded wall orientation map vertically and horizontally to obtain a cumulative histogram, and detects a portion having the smallest difference in each cumulative histogram as a corner point.

라하고 역방향을

라고 할 경우, h_k 및 H_k는 아래의 수학식 13과 같이 표현할 수 있다.Specifically, the vertical histogram is defined as h _k for each direction wall surface of the wall surface direction map, and the cumulative histogram is _denoted by H _k ,

And reverse direction

H _k and H _k can be expressed by the following Equation (13).

&Quot; (13) &quot;

At this time, the four corner points C of the front wall surface can be expressed by Equation (14) below.

&Quot; (14) &quot;

In Equation 14, C _LT is the left upper corner point, C _RT is the upper right corner point, C _LB is the lower left corner point, and C _RB is the lower right corner point.

과 및 수직 방향 벽면을 전치시킨

의 누적 히스토그램 차가 가장 적은 부분에 기초하여 각 코너 점을 검출할 수 있다. At this time, the x position (i.e., the horizontal position) of the corner point uses the front wall direction direction map and the horizontal wall direction direction map, and the y position (i.e., the vertical position) is detected using the front wall direction direction map and the vertical wall direction direction map . That is, in Equation (14), the portion having the smallest cumulative histogram difference between the front wall surface direction map and the horizontal wall surface direction map and the portion with the smallest cumulative histogram difference are obtained by transposing the front wall surface

And a vertical wall surface

It is possible to detect each corner point based on the portion where the cumulative histogram difference is smallest.

Next, the processor 130 detects a spatial layout (i.e., an inner wall structure) in the image based on the detected corner points of the front wall surface and the optimal vanishing point structure.

Referring to FIG. 15 (a), four corner points for the front wall are detected through the above Equation (14) with respect to the input image, and then, through the straight lines connecting these four corner points, , The bottom, the side wall, and the right wall.

 15B, line segments are radially drawn from the front vanishing point of the optimal vanishing point structure to the front wall in the direction passing through the four corner points of the front wall surface, and the ceiling, the right wall, and the left wall You can create a boundary line between the bottom and right and left walls.

That is, as shown in FIG. 15 (b), when a line segment extending from the front vanishing point to the upper right corner and the lower corner corner is drawn, the boundary line between the ceiling and the right wall and the bottom and right walls You can create a boundary line.

As described above, the spatial layout detected based on the edge of the front wall surface and the optimal vanishing point structure can be displayed as inner wall structure information in the image as shown in Fig.

As described above, according to the apparatus and method for detecting an inner wall structure in an image according to an embodiment of the present invention, an indoor wall structure can be detected without an additional device such as a stereo camera, a lidar, etc., It is possible to recognize the indoor structure (that is, the spatial layout) without providing the indoor structure and to provide the inner wall structure detection information for generating the recognized indoor structure as the three-dimensional modeled object.

An apparatus and method for detecting an inner wall structure according to an embodiment of the present invention can also be implemented in the form of a recording medium including instructions executable by a computer such as a program module executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: inner wall structure detection device in single image
110: Video input / output module
120: Memory
130: Processor

Claims (14)

An apparatus for detecting an inner wall structure using a single image of an indoor space,
A memory for storing an inner wall structure detection program; And
And a processor for executing a program stored in the memory,
Wherein the processor is configured to extract line segments from a single input image in accordance with execution of the inner wall structure detection program, to detect a reference vanishing point for the image based on the line segment, A spatial directional map for the image based on the optimal vanishing point structure and the line segment, determines a spatial layout in the image based on the wall surface direction map and the optimal vanishing point structure, Outputting layout information as inner wall structure detection information,
Wherein the optimal vanishing point structure comprises:
A first vanishing point which is one of an intersection point having a highest vote value for all line segments in the image among the intersections between the line segments and one of intersections perpendicular to the reference vanishing point among the intersections, And a second vanishing point that is both perpendicular to the first vanishing point,
Wherein the voting value has a higher value as the angle between the center point of the line segment and the intersection is smaller and has a value proportional to the length of the line segment. The method according to claim 1,
The processor comprising:
Setting the intersection points other than the reference vanishing point among the intersections between the line segments as vanishing point candidates,
Detecting a first vanishing point perpendicular to the reference vanishing point among the vanishing point candidates,
Generating a second vanishing point perpendicular to both the reference vanishing point and the first vanishing point,
Generating a vanishing point structure candidate including the reference vanishing point, the first vanishing point and the second vanishing point for each of the first vanishing points,
Detecting, as the optimal vanishing point structure, the highest voting value for the line segments in the image among the vanishing point structure candidates,
Detecting the optimal vanishing point structure according to a result of summing a first vote value for the first vanishing point and a second vote value for the second vanishing point for each vanishing point structure candidate,
Calculating the first vote value having a value that is higher as the angle between the center point of the line segment and the first vanishing point becomes smaller and has a value proportional to the length of the line segment,
And calculates the second vote value having a value that is higher as the angle between the center point of the line segment and the second vanishing point becomes smaller and has a value proportional to the length of the line segment. 3. The method of claim 2,
The processor comprising:
And calculates the first vote value and the second vote value for a remainder excluding a line segment having a vote value of a threshold value or more with respect to the reference vanishing point among all line segments in the image. 3. The method of claim 2,
The processor comprising:
Detecting a vanishing point candidate having an acute angle between the reference vanishing point and the vanishing point candidate with respect to the center point in the image,
And detects the first vanishing point after removing the detected vanishing point candidate. 3. The method of claim 2,
The processor comprising:
A first vanishing point, a first vanishing point and a second vanishing point of the optimal vanishing point structure are set as a front vanishing point and a horizontal vanishing point is set as a horizontal vanishing point, A vertical vanishing point which is the farthest from the center point in the vertical direction is set as a vertical vanishing point,
Detecting a locus plane in which a line segment corresponding to one vanishing point is shifted in a direction along the other vanishing point with respect to each of the selected two vanishing points, The intersection of the locus surfaces of the two vanishing points is calculated, and the wall facing the other vanishing point is detected,
And generates the wall surface direction map by encoding the pixels corresponding to the front vanishing point, the vertical vanishing point, and the wall surface facing the horizontal vanishing point in the image with different colors. 6. The method of claim 5,
The processor comprising:
The wall surface direction map is divided into four parts based on the position of the front vanishing point,
The four divided regions are projected vertically and horizontally to obtain a cumulative histogram,
A portion having the smallest difference between the accumulated histograms is detected as a corner point,
Detecting straight lines connecting the corner points as boundary lines of vertical and horizontal wall surfaces with respect to the front wall surface facing the front vanishing point,
And detects line segments from the front vanishing point to each of the corner points as a boundary line between the vertical wall surface and the horizontal wall surface facing the vertical and horizontal vanishing points. The method according to claim 1,
The processor comprising:
Projecting a pair of two line segments onto a unit sphere to obtain a vector for an intersection of normal lines and projecting a vector for an intersection of the normal lines onto a plane of the image to detect the intersection. A method for detecting an inner wall structure through an inner wall structure detecting apparatus using a single image of an indoor space,
Extracting a line segment from the input single image;
Extracting a reference vanishing point for the image based on the line segment;
Detecting an optimal vanishing point structure based on the reference vanishing point and the line segment;
Generating a wall orientation map for the image based on the optimal vanishing point structure and the line segment;
Determining a spatial layout in the image based on the wall surface direction map and the optimal vanishing point structure; And
And outputting the information of the spatial layout as inner wall structure detection information,
Wherein the optimal vanishing point structure comprises:
A first vanishing point which is one of an intersection point having the highest vote value for all the line segments in the image among the intersections between the line segments as a reference vanishing point and an intersection perpendicular to the reference vanishing point among the intersections, And a second vanishing point that is both perpendicular to the first vanishing point,
Wherein the voting value has a higher value as the angle between the center point of the line segment and the intersection is smaller, and has a value proportional to the length of the line segment. 9. The method of claim 8,
Wherein the step of detecting the optimal vanishing point structure comprises:
Setting the intersection points among the intersections between the line segments as vanishing point candidates excluding the reference vanishing point;
Detecting a first vanishing point perpendicular to the reference vanishing point among the vanishing point candidates;
Generating a second vanishing point perpendicular to the first vanishing point;
Generating a vanishing point structure candidate including the reference vanishing point, the first vanishing point and the second vanishing point for each of the first vanishing points; And
Detecting, as the optimal vanishing point structure, the highest voting value for the line segments in the image among the vanishing point structure candidates,
Detecting the optimal vanishing point structure according to a result of summing a first vote value for the first vanishing point and a second vote value for the second vanishing point for each vanishing point structure candidate,
Calculating the first vote value having a value that is higher as the angle between the center point of the line segment and the first vanishing point becomes smaller and has a value proportional to the length of the line segment,
And calculating the second vote value having a value that is higher as the angle between the center point of the line segment and the second vanishing point becomes smaller and has a value proportional to the length of the line segment. 10. The method of claim 9,
Wherein the first voting value and the second voting value are calculated for all but the line segment having a voting value of a threshold value or more with respect to the reference vanishing point among all line segments in the image. 10. The method of claim 9,
Detecting a vanishing point candidate having an acute angle between the reference vanishing point and the vanishing point candidate with respect to the center point in the image,
And detecting the first vanishing point after removing the detected vanishing point candidate. 10. The method of claim 9,
Wherein the step of generating the wall-
A first vanishing point, a first vanishing point and a second vanishing point of the optimal vanishing point structure are set as a front vanishing point and a horizontal vanishing point is set as a horizontal vanishing point, Setting a vertical vanishing point that is the greatest distance in the vertical direction from the center point as a vertical vanishing point;
Detecting a locus plane in which a line segment corresponding to one vanishing point is shifted in a direction along the other vanishing point with respect to each of the selected two vanishing points, Calculating an intersection of the locus surfaces of the two vanishing points and detecting a wall facing the other vanishing point;
And generating the wall surface direction map by encoding pixels corresponding to the front vanishing point, the vertical vanishing point, and the horizontal vanishing point in the image with different colors. 13. The method of claim 12,
Wherein the step of determining spatial layout in the image comprises:
Dividing the wall surface direction map with respect to the position of the front vanishing point and projecting the divided regions vertically and horizontally to obtain a cumulative histogram;
Detecting a portion having a smallest difference in each cumulative histogram as a corner point;
Detecting straight lines connecting the corner points as boundary lines of vertical and horizontal wall surfaces with respect to the front wall surface facing the front vanishing point; And
Detecting line segments from the front vanishing point to each of the corner points as a boundary line between a vertical wall surface and a horizontal wall surface facing the vertical and horizontal vanishing points. 9. The method of claim 8,
Projecting a pair of two line segments onto a unit sphere to obtain a vector for an intersection of normal lines and projecting a vector for an intersection of the normal lines onto a plane of the image to detect the intersection.

Images