KR20160031967A - Muti-projection system and method for projector calibration thereof - Google Patents

Abstract

The present invention relates to a multi-projection system and a projector calibration method thereof to automatically calibrate, by using a single camera, each of multiple projectors that constitute the multi-projection system. The method includes the steps of: generating a structured light pattern image based on a perfect submap and then providing each of the projectors with the structured light pattern image with a hue which is different from those of the rest; projecting simultaneously the structured light pattern images via the projectors and acquiring at least one shot image by shooting, by using the camera, a superposed area of the structured light pattern images; restoring the structured light pattern images included in each of the shot image, simultaneously identifying, based on the hue, the projector that corresponds to each of the restored images and calculating, based on correspondence between the restored images, homography between the camera and the projectors; and calculating a vertical vanishing point and a horizontal vanishing point of each of at least one plane that constitutes a display environment based on the homography between the camera and the projectors and edge information included in the shot image and determining a large-scale display area based on the calculated vertical and horizontal vanishing points.

Description

[0001] The present invention relates to a multi-projection system and a method of calibrating the projector,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-projection system and a method of calibrating the same, which can minimize the cost and space required to construct a large-sized display when a large display system is constructed using a plurality of projectors.

Space-immersive displays can provide more immersive and realistic experiences to users, and their applications are rapidly improving in various fields such as entertainment, scientific visualization, medical imaging, aerospace engineering, and virtual reality.

Since immersion is mainly generated by a wide viewing angle and a high resolution display, a space immersion display requires the use of a multi-projection system that implements one large display through multiple projectors.

However, there is a disadvantage that complicated projector calibration process is required to create one virtual display through a plurality of projectors. The projector calibration process requires a geometric calibration process and a color calibration process, and the user performing these two calibration steps must have expertise to perform the configuration and maintenance of the display.

Recently, an automatic calibration method using a camera has been proposed in order to minimize user intervention, but it is necessary to establish a relationship between the camera and the projector through a camera and an active structured light technique and a camera feedback system , There is a disadvantage that the camera must observe the entire projection area.

Further, as the number of cameras required increases, it is necessary to expand the display. In such a case, there arises a problem that it is difficult to install a plurality of cameras in a small space such as a room without blocking a part of the multi-plane display.

Recent researchers have proposed a way to reduce the number of cameras required through a computer-controlled PTU camera, but the manufacture of PTU cameras has the added disadvantage of incurring additional hardware and implementation costs. In addition, the PTU camera must be disposed at a position where the entire projection area can be observed within a camera-rotatable angle, so that the installation of the apparatus is problematic. In other words, building a multi-projection system and calibrating a multi-projector in a small space where it is difficult to ensure sufficient distance between the camera and the screen is still more difficult than with a typical single projector installation.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and an object of the present invention is to provide a multi-projection system and a multi-projection display system, which can more effectively correct geometric distortion of a plurality of projectors forming a large display, Thereby providing a method of calibrating the projector.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

As a means for solving the above problems, a method of calibrating a projector in a multi-projection system having a plurality of projectors and a camera according to an embodiment of the present invention includes generating a structured light pattern image based on a perfect submap, Providing each of the plurality of projectors with a different color of the light pattern image; Projecting the structured light pattern image simultaneously through the plurality of projectors and photographing an overlapping area of the structured light pattern image through the camera to obtain at least one shot image; A plurality of structured light pattern images included in each of the photographed images are reconstructed, a projector corresponding to each of the reconstructed images is color-based, and based on the correspondence between the reconstructed images, Calculating homogenization between projectors; And calculating vertical and horizontal vanishing points for each of the at least one plane constituting the display environment based on the homography between the camera and the plurality of projectors and the edge information included in the photographed image, And determining a large display area based on the large display area.

The structured light pattern image is generated by generating a code pattern image composed of a plurality of symbols reflecting the mathematical property of each of a plurality of elements included in the perfect submap and then performing an exclusive OR operation on the code pattern image and the checkered pattern image And the like.

로 결정되는 것을 특징으로 하며, 상기 ρ은 가장 바깥쪽에 위치되는 동심원의 반경이며, 상기

는 상기 엘리먼트의 값인 것을 특징으로 한다. The plurality of symbols may have at least one concentric circle in which white and black concentric circles are alternately arranged if the value of the element is "0 ", and the radius of the concentric circles

, Where r is the radius of a concentric circle located at the outermost position,

Is a value of the element.

"의 식에 따라 결정되며, 상기 r는 퍼펙트 서브맵의 행 크기, 상기 s는 퍼펙트 서브맵의 열 크기, 상기 u는 퍼펙트 서브맵에 대응되는 서브 어레이의 행 크기, 상기 v는 퍼펙트 서브맵에 대응되는 열 크기, 상기 σ_h는 프로젝터 화면의 가로 비율, 상기 σ_v는 프로젝터 화면의 세로 비율, 상기 t_a(양의 실수)와 t_s(양의 정수, 2 ≤ ts ≤ 2min(r,s))는 패턴 해상도와 각 대응점의 식별을 위해 필요한 최소 패턴 영역을 결정하기 위해 사용자로부터 입력되는 스케일링 팩터인 것을 특징으로 하며, 상기 바둑판 패턴 이미지는 상기 퍼퍽트 서브맵의 크기에 따라 능동 조절된 개수의 바둑판 패턴을 가지는 것을 특징으로 한다. The size of the perfert submap is "

, Where r is the row size of the perfect submap, s is the column size of the perfect submap, u is the row size of the sub-array corresponding to the perfect submap, v is the column size corresponding to the perfect submap the σ _h is the horizontal ratio, wherein σ _v is ratio, wherein t _a (a positive real number) and t _s (positive integer number, 2 ≤ ts ≤ 2min (r , s)) of the projector screen, the projector screen pattern And a scaling factor input from a user to determine a minimum pattern area required for identifying the corresponding points, wherein the checkered pattern image includes a number of checkered patterns having a number of active checkered patterns according to the size of the perpendrite submap .

And the restored image is obtained by performing an exclusive OR operation on the photographed image and the restoration checkerboard pattern image.

The restoration checkerboard pattern is obtained by acquiring, as an initial point, a point at which four quadrangles meet in the photographed image, determining neighboring points adjacent to the initial point, extracting edge components of the photographed image to generate an edge image And determining the valid neighbor points by selecting only the neighboring points located on the bridge line of the edge image among the neighboring points and analyzing the checkered shape of the photographed image based on the valid neighboring points .

Wherein determining the large display area comprises: extracting a plurality of line segments based on an edge component of the camera images; Computing a vanishing point, a vertical and a horizontal vector of each of the planes constituting the large display area based on the intersection information of the line segments and the line segments; Determining a display area of each plane based on a vanishing point, a vertical and a horizontal vector of each plane; And determining a height and an area of the large display area with a height and an area of the smallest display area after evaluating the size of the display area of each of the planes.

As a means for solving the above problems, a multi-projection system according to another embodiment of the present invention includes: a plurality of projectors for projecting an image into a large display area; A camera for photographing an image projected on the large display area; And projecting the structured light pattern image based on the perfect submap simultaneously through the plurality of projectors while photographing an overlapping area of the structured light pattern image through the camera, Restoring, analyzing, and calculating homography between the camera and the plurality of projectors, and calculating a homography between the camera and the plurality of projectors based on edge information of the photographed image and at least one plane constituting a display environment And calculating a vertical and horizontal vanishing point, and then determining a large display area based on the calculated vertical and horizontal vanishing points.

Thus, the present invention proposes a structured light pattern of a new type, and enables to calibrate the geometrical distortion of a plurality of projectors constituting the multi-projection system.

Therefore, the projector calibration operation can be performed using a general portable camera instead of the PTU camera, so that the cost and space required for the multi-projection system can be drastically reduced. In addition, since the calibration of a plurality of projectors can be performed collectively, the time required for calibration can be drastically reduced.

1 is a view for explaining a multi-projection system according to an embodiment of the present invention.
2 is a schematic view for explaining a method of calibrating a projector of a multi-projection system according to an embodiment of the present invention.
FIG. 3 illustrates a process of generating a structured light pattern image according to an exemplary embodiment of the present invention. Referring to FIG.
4 is a diagram for explaining a principle of converting each element of a perfect submap into a symbol of a code pattern image (Icode) according to an embodiment of the present invention.
5 is a diagram for explaining a principle of restoring a code pattern image from a camera image according to an embodiment of the present invention.
FIG. 6 is a view for explaining the principle of corresponding point identification from a camera image according to an embodiment of the present invention.
7 is a diagram for explaining the principle of homography calculation between a camera and a projector according to an embodiment of the present invention.
8 and 9 are views for explaining a method of determining a large display area according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like.

The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art to which the present invention pertains. Only. Therefore, the definition should be based on the contents throughout this specification.

1 is a view for explaining a multi-projection system according to an embodiment of the present invention.

1, the large display system of the present invention may include a plurality of projectors 21 and 22, a single camera 30, and a processor 40 disposed in front of the large display area 10 .

The large display area 10 to which the present invention is applied may be a general shape realized as a single large plane, but may also be realized as a three-dimensional shape having a central area and both side areas. Of course, it will be appreciated that such a large display implementation can be varied in various ways according to the needs of the user in the future.

The plurality of projectors 21 and 22 projects the image provided from the processor 40 to the large display area 10. [

The single camera 30 allows the large display area 10 to be photographed under the control of the processor 40. However, the single camera 30 of the present invention may be a general portable small camera for general use, or a camera that does not require a separate PTU function.

The processor 40 performs a calibration operation of the plurality of projectors 21 and 22 on the basis of the single camera 30 and then transmits the contents required by the user through the plurality of calibrated projectors 21 and 22 to the large display area .

In particular, in order to simultaneously calibrate multiple projectors, the processor 40 of the present invention allows to generate structured light pattern images having different colors based on the perfect submap. While the structured light pattern image is simultaneously projected through the plurality of projectors 21 and 22, the overlapping area of the structured light pattern image is photographed and analyzed through the camera 30 to grasp the structured light pattern image corresponding to each of the projectors And calculates the homography between the camera and the plurality of projectors based on their correspondence. Then, based on the homography between the plurality of projectors and the edge information included in the photographed image, the vertical and horizontal vanishing points for each of the at least one plane constituting the display environment are calculated. Then, based on the calculated vertical and horizontal vanishing points, The display area is finally determined.

That is, the present invention newly proposes a structured light pattern image generated based on a perfect submap, so that it is possible to perform software calibration of a plurality of projectors that construct a multi-projection system using only one general portable camera It will help.

As a result, the cost and space required for the construction of the multi-projector system are drastically reduced, and the application fields of the multi-projector system can be further expanded.

2 is a schematic view for explaining a method of calibrating a projector of a multi-projection system according to an embodiment of the present invention.

As shown in FIG. 2, the projector calibration method of the present invention includes: generating a structured light pattern image based on a perfect submap; and providing the structured light pattern image to each of the plurality of projectors with different colors of the structured light pattern image (S20) of photographing a superimposed area of the structured light pattern image through the camera while simultaneously projecting a structured light pattern image through the plurality of projectors to acquire at least one taken image, A plurality of structured light pattern images included in each of the plurality of structured light pattern images are reconstructed and at the same time a projector corresponding to each of the plurality of structured light pattern images is identified on a color basis, A step (S30) of calculating the homogenization among the plurality of projectors, and Based on the number of projector-to-projector homographies and edge information included in the shot image, calculates vertical and horizontal vanishing points for each of at least one plane constituting the display environment, and calculates the vertical and horizontal vanishing points based on the calculated vertical and horizontal vanishing points (S40), and correcting the brightness difference between the projectors through an alpha blending algorithm based on the geometric information (S50).

For reference, the structured light pattern image to be used in the present invention should satisfy the following requirements.

(1) Perfect Submap Based on Spatial Patterns: A perfect submap based on a spatial pattern is mainly used in the fields of pattern occlusion and object shading. However, in the present invention, in order to calibrate a multi-projection system through a general- It employs a perfect submap-based pattern.

This is because, when the mathematical characteristic of the perfect submap is utilized, homography between the projector and the camera can be estimated even if only a part of the pattern is captured in the camera view, and the encoded codeword of the perfect submap- There are features to include. Thus, this spatial structuring technique allows to estimate the entire pattern using a partially captured image.

In addition, the perfect map theory is proposed by M-arrays, pseudorandom arrays or De Bruijin tori, which has unique mathematical properties, It is widely used in schemes (spatial codification schemes). A perfect map is a two-dimensional periodic array of r ㅧ s dimensions (where r is the width value of the perfect map and s is the vertical value of the perfect map) with symbols of the alphabet size k, u ㅧ v (u is the width of the subarray Value, r is the vertical value of the sub-array) sub-arrays of size occur only once within the entire array. The perfect map contains all sorts of sub-arrays except for the sub-arrays filled with zeros, and the perfect sub-matrix is an aperiodic array in which all sub-arrays of size u v in the array have a unique r ss size.

(2) Geometric primitive based on binary pattern: In a multi-projection system, the projection areas of a plurality of projectors are overlapped with each other to form a large display. Conventional projector calibration operation causes each of the plurality of projectors to sequentially project a pattern image, thereby sequentially performing calibration of each of a plurality of projectors. However, the present invention allows all projectors to simultaneously project an image having a unique pattern on the screen, and then allows all projectors to be temporarily calibrated.

That is, the present invention uses binary pattern images having different geometric shapes according to the alphabet. By varying the colors of the projected patterns of the respective projectors, it is possible to distinguish adjacent projectors from each other on a pattern basis.

(3) Flexible Perfect Submaps: When designing a structured light pattern, the resolution of the pattern is generally determined based on considerations such as the resolution of the light emitting device, the resolution of the camera, and the capture distance.

In the case of a perfect submap based on a structured light pattern, the minimum pattern area that needs to be captured in the camera view for pattern resolution and corresponding point identification is determined by the size r ㅧ s of the perfect submap and the size u ㅧ v of the sub- Can be determined.

An irregular perfect map satisfying the following condition is converted into a pattern image.

또한 상기의 조건으로부터 획득될 수 있으며, 많은 퍼펙트 서브맵은 이 조건을 만족시킴으로써, 다양한 환경 변수를 가지는 파라메터 r, s, u, v, k를 어렵게 결정할 수 있도록 한다. Conditions for a Perfect Submap

Also, many perfect submaps can satisfy this condition, making it difficult to determine the parameters r, s, u, v, k having various environmental variables.

And the structured light pattern is mainly used to use the depth of the object in the scene like the 3D scanner. The shooting distance for this is somewhat constant. Many people create a perfect submap based on a spatial pattern with preset k alphabet sizes and preset parameters r, s, u, v depending on the display environment.

As in the present invention, a technique for calibrating a multi-projector through a perfect submap based on a structured light pattern is the first.

The conventional method of using a parameter having a fixed value has a limitation that it can not be applied to a display environment in which a distance between a screen and a camera is changed frequently as the pattern is photographed through various small cameras. For example, if a low resolution camera captures a high resolution pattern, the recognition accuracy may be lowered. When a camera placed too close to the screen acquires a low resolution pattern, the number of corresponding points may become insufficient. Therefore, in the present invention, a codification scheme for a matching pattern is required regardless of a size change of a perfect submap determined according to a display environment.

Hereinafter, each step of the method of calibrating the projector of the multi-projection system will be described in more detail with reference to FIG. 3 to FIG.

3 is a diagram for explaining a process of generating a structured light pattern image according to an embodiment of the present invention.

3, the present invention generates a code pattern image (I _code ) reflecting the mathematical property of the perfect submap, and then generates a code pattern image generated according to Equation (1) with a white square and a black square intersecting And performs an exclusive OR operation with the arranged _checker pattern image (I _checker ) to generate a structured light pattern image (I _combined ). The color of the structured light pattern image is also varied according to the total number of projectors, and then a plurality of structured light pattern images having different colors are provided to each of the plurality of projectors.

[Equation 1]

The code pattern image I _code provides a mathematical attribute for point correspondence and the _checker pattern image I _checker provides the exact location of the corresponding point with vertices shared by the adjoining checkerboard squares.

)를 계산하도록 한다. The size of the perfect submap determines the number of checkerboard patterns in the _checker pattern image (I _checker ). In the present invention, r s is determined taking into account the aspect ratio of the projector (σ _h : σ _v ) U u v and the alphabet size (k =

).

&Quot; (2) "

At this time, r is the row of perfect submap size, s is the perfect submap column size, u is the row size, v of the sub-array corresponding to perfect submap is the column size, σ _h of the sub-array corresponding to perfect submap is the projector screen width ratio, σ _v is a ratio, t _a (a positive real number) and t _s (positive integer, 2≤ ts ≤ 2min (r, s)) of the projector screen is minimum necessary for the identification of the pattern resolution and the respective corresponding points It is a scaling factor that adjusts the pattern area.

Therefore, in the present invention, the user confirms the correct recognition of the pattern instead of the parameters r, s, u, v, and k, and adjusts only the t _a and t _s to generate the optimal pattern image. r ㅧ s), sub array size (u ㅧ v), and alphabet size (k) can be calculated automatically.

That is, when the user determines the pattern resolution and the minimum area size for identifying each corresponding point by the user parameters t _a and t _s , a perfect submap is generated using the brute-force algorithm or the like (RA Morano, C. Ozturk, R. Conn, S. Dubin, S. Zietz, and J. Nissanov, Structured light using pseudorandom codes, IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 22 (3): 322 327, 1998).

로 표현된다. FIG. 4 is a diagram for explaining the principle of converting each element of a perfect submap according to an embodiment of the present invention into a symbol of a code pattern image (I _code ). At this time, each element of the perfect submap

Lt; / RTI >

가 "0" 보다 크다면,

로 결정되며, 이때 ρ은 가장 바깥쪽에 위치되는 동심원의 반경이다.

가 "0" 인 경우에는 블랭크(blank) 표시되고, 인접된 심볼들이 오버랩핑되는 것을 방지하기 위해

로 설정하도록 한다. 이때, W/H는 프로젝터 해상도의 너비/ 높이이다. 더하여 심볼은 도4에서와 같이 흰색 및 검은색 동심원이 교대로 배치되는 형태인 것이 바람직할 것이다. 이 방식은 k 값에 따라 다양한 요소들을 유연하게 설명할 수 있도록 한다. As shown in FIG. 4, in the present invention, a spatial pattern includes a concentric circle around a corresponding point. These circles represent symbols of the code pattern image (I _code ). The concentric circles

Is greater than "0 &

, Where rho is the radius of the concentric circle located at the outermost position.

Is "0 ", a blank is displayed, and in order to prevent overlapping of adjacent symbols

. At this time, W / H is the width / height of the projector resolution. In addition, it is preferable that the symbol is a form in which white and black concentric circles are alternately arranged as shown in FIG. This method allows flexible explanation of various factors according to k value.

5 is a diagram for explaining a principle of restoring a code pattern image from a camera image according to an embodiment of the present invention.

를 적용함으로써 복원될 수 있다. Referring to Equation (3) described above, the code pattern image (I _code )

. ≪ / RTI >

&Quot; (3) "

This means that the code pattern image (I _code ) based on the perfect submap can be restored by _performing an exclusive OR operation on the image photographed by the camera and the checkerboard pattern image (I _checker ). However, since it is necessary to be able to _recover from the photographed pattern image even without knowing the checkerboard pattern image I _checker , the following expression (4) can be derived.

&Quot; (4) "

In this case, I _capture is an image captured through a camera, I ' _checker is an unknown checkerboard pattern image, and I' _code is a code pattern image reconstructed using I _capture .

In the "After obtaining the _checker, I _capture and I 'the I from the I _capture the invention two images based on the I" restore _code and the restored code pattern image (I' _code) from the _checker, to project the pattern image The correlation between the projector and the camera view including the pattern image can be grasped.

)을 형성한다. 다만, 본 발명에서는 계산 부하를 감소하기 위해,

를 임계 거리 d로 선택함으로써, i번째 점(p_i)로부터 너무 가깝거나 먼 점들은 이웃 점(p_j)에서 제외시키도록 한다. It is possible to obtain a restoration checkerboard pattern image (I ' _checker ) from point correspondences located at the center of the symbol. After recalling centers matching the square corners of the checkerboard pattern, we use the Shi-Tomasi corner detection algorithm to find the point at which the four rectangles encountered in the restoration _checker pattern image (I ' _checker ) meet, . 6 (a), neighboring points (p _j ) located adjacent to the i-th point (p _i ) belonging to the initial point set (P)

). However, in the present invention, in order to reduce the calculation load,

Is selected as the threshold distance d, the points that are too close or far from the i-th point (p _i ) are excluded from the neighboring point (p _j ).

Considering that the structured light pattern is a binary image, an edge image (I _edge ) is obtained through a morphological image operation such as Equation (5).

&Quot; (5) "

In order to determine whether two points are connected by the edge of the edge image as shown in FIG. 6 (b), based on the Bresenham line mainly used for line drawing with two end points, only the selected set (N _i) i-th point _(i p) and the neighboring point (p _j) is connected to the edge line will be in the neighborhood to obtain a valid set of points (M _i).

Equation (6) is the Bregenz line calculation formula.

&Quot; (6) "

의 조건을 만족시키는 점만을 M_i로 유지하도록 한다. 이때, 임계값 λ은 0.8으로 설정될 수 있을 것이다. In Equation (6), Ω _ij denotes a pixel located on the path between the i-th point (p _i ) and the neighboring point (p _j ), and n denotes the number of pixels located on the path. Among p _j belonging to N _i ,

And M _i , respectively. At this time, the threshold value? Can be set to 0.8.

Then, the corresponding point P 'is extracted from P based on the effective function V (p _i ) expressed by Equation (7).

&Quot; (7) "

If all the points in the valid neighbor point set (M _i ) are in a straight line, then all vectors from p _i to M _i in-point are on the same straight line and the sum of the collinear functions (C (p _i , p _j ) do. Otherwise, the vector has two different orientations, and p _i can be regarded as the corresponding point. The error value (e) is set to 0.1 to check the collinearity.

The restoration _checker pattern image (I ' _checker ) can be obtained by confirming that the four corresponding points in the captured image I _capture form a white or black square. The midpoint between two diagonal points determines the white or black color of each square of the checkerboard.

Then, the decoded code pattern image (I ' _code ) is generated by decrypting Equation (4) through the obtained restoration checkerboard pattern image (I' _checker ). However, the erroneous edge caused by the misalignment between the restoration _checker pattern image (I ' _checker ) and the captured image (I _capture ) remains in the reconstructed code pattern image (I' _code ) and can be easily removed by a morphological opening operation.

Then, a contour image (I _contour ) is calculated using the restored code pattern image (I ' _code ). Since each element c _ij of the perfect submap is converted into a concentric symbol as described above, these elements are derived according to the number of circular contours surrounding each corresponding point. Then, the point correspondence between the camera and the projector is calculated by comparing the u and v sub-arrays of these elements using the code words of the perfect submap used in the code pattern image (I _code ).

Using the correspondence of the two patterns observed in a single camera view, the present invention produces homography between the camera and the projector to create a pseudo-single view and determine the display area.

Equation (9) below is a formula for calculating a homography matrix H _{i, j} using RANSAC robustness consensus robust estimation method.

&Quot; (9) "

Here, i is the index of the camera view and j is the index of the projector. The camera view acts as a link to convert the projector coordinates to the next projector coordinates. Conversely, the pattern can be used as a link to enable conversion between two camera views as in Equation (9).

As shown in FIG. 7, the transformation between all the projector nodes and the view node can be represented by one homography matrix H, and the reverse direction can be represented by H ^-1 .

을 만족시키는

의 라인의 변환은 l'= H ^-?? l로 표현될 수 있다. 이때, a, b, c :　2차원 직선 방정식의 계수이다. H, the transformation of the point x can be expressed as x ' = Hx , and the linear equation

Satisfying

The conversion of the line of l '= H ^{- ??} l. < / RTI > Here, a, b, and c are the coefficients of the two-dimensional linear equation.

8 and 9 are views for explaining a method of determining a large display area according to an embodiment of the present invention.

으로 설정하면, 상단 에지와 하단에지를 통과하는 벡터는

와

로 표현되고, 이들 두 벡터 사이의 각도는

로 표현된다. 동일하게, 수직 소실점이

이며, 좌측 에지와 우측 에지를 통과하는 벡터는 각각

및

로, 이들 두 벡터 사이의 각도는

로 표현될 수 있다.For reference, edges to walls, ceiling, and floor provide clues to extract the necessary geometric information. Can be estimated using the vanishing point obtained from the edge set of the pseudo-single image. Each wall is assumed to be a square, and the horizontal vanishing point where the upper edge and the lower edge meet

, The vector passing through the top edge and the bottom edge is

Wow

, And the angle between these two vectors is

Lt; / RTI > Similarly, the vertical vanishing point

, And the vectors passing through the left and right edges are respectively

And

, The angle between these two vectors is

. ≪ / RTI >

및

는 추출된 라인 세그먼트들과 이들의 교차 정보를 기반으로 계산될 수 있다. Accordingly, the present invention extracts edges corresponding to the wall intersections and the patterns using a Canny edge detector or the like. Then, the subtraction pattern area extracted from each channel image has essential edges having a geometric structure. Then, a complete line segment corresponding to these edge components is extracted by using Hough transform. The vanishing point ω of each plane, vector

And

May be computed based on the extracted line segments and their intersection information.

In particular, the present invention eliminates the need to align the X-axis of the camera image parallel to the paper plane, and can be easily applied to a multi-plane display without placing reference markers or attaching a tilt sensor due to the vanishing point extracted from the geometry of the display environment .

및

을 통해 결정한 후, 이하의 수학식 10을 이용하여 각 평면에 생성할 수 있는 최대 디스플레이 영역을 설정한다. The display area of each plane is defined as the vanishing point omega, vector

And

And then sets the maximum display area that can be generated in each plane by using the following expression (10).

&Quot; (10) "

를 포함하는 벡터 집합으로, 이는 소실점 ω에서 투사 이미지 상단 에지의 끝점을 향하는 벡터들의 집합이다. 끝 점 중 인접한 투사 이미지 안에 놓여져 있거나, 벽면 바깥 쪽에 위치한 점은 인접한 투사 이미지 혹은 벽면의 경계선과의 교차점(도9에서 주황색 점으로 표시됨)으로 대체 된다. 벡터 집합 V는 벡터

와 투사 이미지의 바닥 에지의 끝점을 이용하여 같은 방식으로 정의된다.At this time, U is a vector

, Which is a set of vectors from the vanishing point ω to the end point of the top edge of the projected image. A point located within an adjacent projection image of the end points, or located outside the wall surface, is replaced with an intersection point (indicated by an orange dot in FIG. 9) of the adjacent projection image or the boundary of the wall surface. Vector set V is vector

And the end point of the bottom edge of the projection image.

After evaluating the size of the maximum display area of each plane, the flat display height is set to the height of the smallest display area. Next, using the transformation matrix H _{L, R} , the widths of the displays are compared with each other, and the smallest one is set to the width of the large display area (V).

The foregoing description is merely illustrative of the technical idea of the present invention and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (8)

A projector calibration method for a multi-projection system having a plurality of projectors and a camera,
Generating a structured light pattern image based on the perfect submap, and providing the structured light pattern image to each of the plurality of projectors with different colors of the structured light pattern image;
Projecting the structured light pattern image simultaneously through the plurality of projectors and photographing an overlapping area of the structured light pattern image through the camera to obtain at least one shot image;
A plurality of structured light pattern images included in each of the photographed images are reconstructed, a projector corresponding to each of the reconstructed images is color-based, and based on the correspondence between the reconstructed images, Calculating homogenization between projectors; And
Calculating vertical and horizontal vanishing points for each of at least one plane constituting a display environment based on the homography between the camera and the plurality of projectors and the edge information included in the photographed image, And determining a large display area on the basis of the calculated result. The method of claim 1, wherein the structured light pattern image
Generating a code pattern image composed of a plurality of symbols reflecting the mathematical property of each of a plurality of elements included in the perfect submap, and then performing an exclusive-OR operation on the code pattern image and the checkerboard pattern image Wherein the projector is a multi-projection system. 2. The method of claim 1, wherein the plurality of symbols
A blank is displayed if the value of the element is "0 ", and a concentric circle in which white and black concentric circles are alternately arranged, and the radius of the concentric circle is

, ≪ / RTI >
Is a radius of a concentric circle located at the outermost position,

Is a value of the element. 3. The method of claim 2, wherein the size of the perfert submap is
"

"And "
Wherein r is a row of perfect submap size, wherein s is the perfect submap column size, wherein u is a row of sub-arrays corresponding to a perfect submap size, the v is a column corresponding to a perfect submap size, the σ _h is the projector screen width ratio, the σ _v is to identify the ratio, the t _a (a positive real number) and t _s (positive integer number, 2 ≤ ts ≤ 2min (r , s)) of the projector screen, the pattern resolution and the respective corresponding points And a scaling factor input from a user to determine a minimum pattern area required,
Wherein the checkerboard pattern image has a number of checkered patterns that are actively controlled according to the size of the perfert submap. 3. The method of claim 2,
And performing an exclusive OR operation on the captured image and the restoration checkerboard pattern image. The method according to claim 5, wherein the restoration checkerboard pattern
Acquiring, as an initial point, a point at which four quadrangles meet in the captured image, determining neighboring points adjacent to the initial point,
Extracting an edge component of the photographed image to generate an edge image, selecting only a neighboring point located on a border line of the edge image among the neighboring points to identify valid neighboring points,
And analyzing a checkered shape of the photographed image based on the valid neighbor points. 2. The method of claim 1, wherein determining the large display area comprises:
Extracting a plurality of line segments based on an edge component of the camera images;
Computing a vanishing point, a vertical and a horizontal vector of each of the planes constituting the large display area based on the intersection information of the line segments and the line segments;
Determining a display area of each plane based on a vanishing point, a vertical and a horizontal vector of each plane; And
And determining a height and an area of the large display area with the height and the width of the smallest display area after evaluating the sizes of the display areas of the respective planes. A plurality of projectors for projecting images into a large display area;
A camera for photographing an image projected on the large display area; And
Projecting a structured light pattern image based on a perfect submap simultaneously through the plurality of projectors while photographing an overlapping area of the structured light pattern image through the camera and restoring a plurality of structured light pattern images included in each of the captured images And calculating a homography between the camera and the plurality of projectors by analyzing the image, and calculating a homography between the camera and the plurality of projectors based on the homography between the camera and the plurality of projectors and the edge information of the photographed image, And a processor for calculating a horizontal vanishing point and then determining a large display area based on the calculated vertical and horizontal vanishing points.

Images