KR102160839B1 - Muti-projection system and method for projector calibration thereof - Google Patents

Abstract

The present invention relates to a multi-projection system capable of automatically calibrating each of a plurality of projectors constructing a multi-projection system using one camera, and to a projector calibration method thereof,
The method includes generating a structured light pattern image based on a perfect submap, and then providing the structured light pattern image to each of the plurality of projectors by varying all colors of the structured light pattern image; Simultaneously projecting a structured light pattern image through the plurality of projectors and capturing an overlapping area of the structured light pattern image through the camera to obtain at least one captured image; While restoring a plurality of structured light pattern images included in each of the captured images, a projector corresponding to each of the restored images is identified based on color, and the camera and the plurality of Calculating homography between projectors; And calculating vertical and horizontal vanishing points for each of at least one plane constituting the display environment based on the homography between the camera and the plurality of projectors and edge information included in the captured image, and the calculated vertical and horizontal vanishing points. It may include determining a large display area based on.

Description

Multi-projection system and its projector calibration method TECHNICAL FIELD

The present invention relates to a multi-projection system and a projector calibration method for minimizing the cost and space required for building a large display when a large-sized display system is constructed using a plurality of projectors.

Spatial immersive displays can provide users with improved immersion and reality, and their application fields are rapidly improving in various fields such as entertainment, scientific visualization, medical imaging, aerospace engineering, and virtual reality.

Since immersion is mainly generated by a wide viewing angle and high resolution display, a space immersive display requires the use of a multi-projection system that implements one large display through multiple projectors.

However, there is a disadvantage in that a complicated projector calibration process is involved in order to create one virtual display through a plurality of projectors. The projector calibration process requires a geometric calibration process and a color calibration process, and the user performing these two calibration steps must have expertise to perform the configuration and maintenance of the display.

Accordingly, in recent years, an automatic calibration technique using a camera has been proposed to minimize user intervention, but this requires establishing a relationship between the camera and the projector through the camera and active structured light technique and camera feedback system. However, there is a drawback that the camera must observe the entire projection area.

In addition, as the number of cameras required increases, the display needs to be expanded. In this case, it is difficult to install a plurality of cameras without blocking a part of the multi-flat display in a small space such as a room.

Recently, researchers have proposed a method of reducing the required number of cameras through a computer-controlled pan-tilt device (PTU) camera, but manufacturing a PTU camera has additional disadvantages that incur additional hardware and implementation costs. In addition, since the PTU camera must be placed in a position where the entire projection area can be observed within the camera rotatable angle, installation of the device has a related problem. In other words, it is still difficult to establish a multi-projection system and calibrate a multi-projector in a small space where it is difficult to secure a sufficient distance between the camera and the screen, compared to a typical single projector installation.

Accordingly, as to solve the above problems, the present invention more efficiently corrects geometric distortion of a plurality of projectors forming a large display, thereby minimizing the cost and space required for large display construction, and We would like to provide a method for calibrating its projector.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

As a means for solving the above problem, the projector calibration method of a multi-projection system including a plurality of projectors and cameras according to an embodiment of the present invention is, after generating a structured light pattern image based on a perfect submap, Providing each of the plurality of projectors with different colors of the light pattern image; Simultaneously projecting a structured light pattern image through the plurality of projectors and capturing an overlapping area of the structured light pattern image through the camera to obtain at least one captured image; While restoring a plurality of structured light pattern images included in each of the captured images, a projector corresponding to each of the restored images is identified based on color, and the camera and the plurality of Calculating homography between projectors; And calculating vertical and horizontal vanishing points for each of at least one plane constituting the display environment based on the homography between the camera and the plurality of projectors and edge information included in the captured image, and the calculated vertical and horizontal vanishing points. It may include determining a large display area based on.

The structured light pattern image generates a code pattern image composed of a plurality of symbols reflecting the mathematical properties of each of the plurality of elements included in the perfect submap, and then performs an exclusive OR operation on the code pattern image and the checkerboard pattern image. It is characterized by being created by performing.

로 결정되는 것을 특징으로 하며, 상기 ρ은 가장 바깥쪽에 위치되는 동심원의 반경이며, 상기

는 상기 엘리먼트의 값인 것을 특징으로 한다. The plurality of symbols may include at least one concentric circle in which white and black concentric circles are alternately arranged if the value of the element is “0”, and otherwise, the radius of the concentric circle is

It characterized in that it is determined as, wherein ρ is the radius of the concentric circle located at the outermost, the

Is characterized in that the value of the element.

"의 식에 따라 결정되며, 상기 r는 퍼펙트 서브맵의 행 크기, 상기 s는 퍼펙트 서브맵의 열 크기, 상기 u는 퍼펙트 서브맵에 대응되는 서브 어레이의 행 크기, 상기 v는 퍼펙트 서브맵에 대응되는 열 크기, 상기 σ_h는 프로젝터 화면의 가로 비율, 상기 σ_v는 프로젝터 화면의 세로 비율, 상기 t_a(양의 실수)와 t_s(양의 정수, 2 ≤ ts ≤ 2min(r,s))는 패턴 해상도와 각 대응점의 식별을 위해 필요한 최소 패턴 영역을 결정하기 위해 사용자로부터 입력되는 스케일링 팩터인 것을 특징으로 하며, 상기 바둑판 패턴 이미지는 상기 퍼퍽트 서브맵의 크기에 따라 능동 조절된 개수의 바둑판 패턴을 가지는 것을 특징으로 한다. The size of the Percent submap is "

Is determined according to the formula, where r is the row size of the perfect submap, s is the column size of the perfect submap, u is the row size of the subarray corresponding to the perfect submap, and v is the column size corresponding to the perfect submap , The σ _h is the aspect ratio of the projector screen, the σ _v is the vertical ratio of the projector screen, the t _a (positive real number) and t _s (a positive integer, 2 ≤ ts ≤ 2min(r,s)) are patterns It is characterized in that it is a scaling factor input from the user to determine the resolution and the minimum pattern area required for identification of each corresponding point, and the checkerboard pattern image has a number of checkerboard patterns that are actively adjusted according to the size of the perfoct submap. It features.

The restored image is characterized in that it is obtained by performing an exclusive OR operation on the photographed image and the restoration checkerboard pattern image.

The restoring checkerboard pattern obtains a point where four squares meet in the photographed image as an initial point, identifies neighboring points adjacent to the initial point, and extracts an edge component of the photographed image to generate an edge image. Thereafter, it is characterized in that it is obtained by selecting only neighboring points located on the Brezenham line of the edge image among the neighboring points to identify effective neighbors, and analyzing the checkerboard shape of the photographed image based on the effective neighboring points. .

The determining of the large display area may include extracting a plurality of line segments based on edge components of the camera images; Calculating vanishing points and vertical and horizontal vectors of each plane constituting a large display area based on intersection information of the line segments and the line segments; Determining a display area of each plane based on a vanishing point of each plane and a vertical and horizontal vector; And determining the height and width of the large display area based on the height and width of the smallest display area after evaluating the size of the display area of each plane.

As a means for solving the above problems, a multi-projection system according to another embodiment of the present invention includes a plurality of projectors for projecting an image onto a large display area; A camera for photographing an image projected on the large display area; And simultaneously projecting a structured light pattern image based on a perfect submap through the plurality of projectors, photographing an overlapping area of the structured light pattern image through the camera, and storing a plurality of structured light pattern images included in each of the captured images. For each of at least one plane constituting a display environment based on the homography between the camera and the plurality of projectors by restoration and analysis and the homography between the camera and the plurality of projectors and edge information of the captured image After calculating the vertical and horizontal vanishing points, it may include a processor that determines a large display area based on the calculated vertical and horizontal vanishing points.

As described above, the present invention proposes a new type of structured light pattern, and through this, it is possible to calibrate the geometric distortion of a plurality of projectors constructing a multi-projection system.

Accordingly, since the projector calibration operation can be performed using a portable general camera instead of the PTU camera, the cost and space required for the multi-projection system can be drastically reduced. In addition, by allowing the calibration of multiple projectors to be performed at once, the time required for calibration can be drastically reduced.

1 is a view for explaining a multi-projection system according to an embodiment of the present invention.
2 is a diagram schematically illustrating a projector calibration method of a multi-projection system according to an embodiment of the present invention.
3 is a view for explaining a process of generating a structured light pattern image according to an embodiment of the present invention.
4 is a diagram for explaining a principle of converting each element of a perfect submap into a symbol of a code pattern image (Icode) according to an embodiment of the present invention.
5 is a diagram illustrating a principle of restoring a code pattern image from a camera image according to an embodiment of the present invention.
6 is a diagram for explaining a principle of identifying a corresponding point from a camera image according to an embodiment of the present invention.
7 is a diagram illustrating a principle of calculating a homography between a camera and a projector according to an embodiment of the present invention.
8 and 9 are diagrams for explaining a method of determining a large display area according to an embodiment of the present invention.

Objects and effects of the present invention, and technical configurations for achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. In describing the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intention or custom of users or operators.

However, the present invention is not limited to the embodiments disclosed below, and may be implemented in various different forms. These embodiments are provided only to make the disclosure of the present invention complete, and to fully inform the scope of the invention to those skilled in the art to which the present invention pertains, and the present invention is defined by the scope of the claims. It just becomes. Therefore, the definition should be made based on the contents throughout this specification.

1 is a view for explaining a multi-projection system according to an embodiment of the present invention.

Referring to FIG. 1, the large display system of the present invention may include a plurality of projectors 21 and 22 disposed in front of the large display area 10, a single camera 30, and a processor 40. .

The large display area 10 to which the present invention is applied may be a general shape implemented with one large flat surface, but may also be implemented in a three-dimensional shape having a center area and both side areas. Of course, it will be natural that the implementation form of such a large display can be changed in various ways according to user needs in the future.

A number of projectors 21 and 22 project an image provided from the processor 40 onto the large display area 10.

The single camera 30 allows photographing of the large display area 10 under the control of the processor 40. However, the single camera 30 of the present invention may be a general portable camera that is used for general use, and a camera that does not require a separate PTU function.

The processor 40 performs a calibration operation of a plurality of projectors 21 and 22 based on a single camera 30, and then displays the content required by the user through the calibrated plurality of projectors 21 and 22 in a large display area. To project it.

In particular, in order to calibrate multiple projectors simultaneously, the processor 40 of the present invention generates structured light pattern images having different colors based on the perfect submap. And while simultaneously projecting the structured light pattern image through a plurality of projectors (21, 22), by photographing and analyzing the overlapping area of the structured light pattern image through the camera 30 to grasp the structured light pattern image corresponding to each projector. , Based on their correspondence, a homography between the camera and the plurality of projectors is calculated. And after calculating the vertical and horizontal vanishing points for each of at least one plane constituting the display environment based on the homography between multiple projectors and the edge information included in the photographed image, the large size is based on the calculated vertical and horizontal vanishing points. Finalize the display area.

That is, the present invention newly proposes a structured light pattern image that is generated based on a perfect submap, so that a plurality of projectors that build a multi-projection system can be calibrated in software using only one universally used portable general camera. Allows.

As a result, the cost and space required to build a multi-projector system are drastically reduced, and accordingly, the application field of the multi-projector system can be further expanded.

2 is a diagram schematically illustrating a projector calibration method of a multi-projection system according to an embodiment of the present invention.

As shown in Figure 2, the projector calibration method of the present invention largely generates a structured light pattern image based on a perfect submap, and then provides the structured light pattern image to each of the plurality of projectors by varying all colors of the structured light pattern image. (S10), while simultaneously projecting a structured light pattern image through the plurality of projectors, capturing an overlapping area of the structured light pattern image through the camera to obtain at least one photographed image (S20), the photographed image While restoring a plurality of structured light pattern images included in each, a projector corresponding to each of the plurality of structured light pattern images is identified based on a color, and the camera and the camera are connected based on correspondence between the plurality of structured light pattern images. Calculating a homography between the plurality of projectors (S30), and vertical and horizontal for each of at least one plane constituting a display environment based on the homography between the plurality of projectors and edge information included in the captured image Computing a vanishing point, determining a large display area based on the calculated vertical and horizontal vanishing points (S40), and correcting the difference in brightness values between projectors through an alpha blending algorithm based on geometric information (S50). Can include.

For reference, the structured light pattern image to be used in the present invention must satisfy the following adjustment.

(1) Spatial pattern-based perfect submap: Spatial pattern-based perfect submap is mainly used in the fields of pattern occlusion and object shading, but in the present invention, in order to calibrate a multi-projection system through a general-purpose small camera It adopts a perfect submap-based pattern.

When using the mathematical characteristics of the perfect submap, homography between the projector and the camera can be estimated even if only part of the pattern is captured in the camera view, and the encoded codeword of the pattern based on the perfect submap uses the location information of the projector coordinates. There are features to include. Therefore, this spatial structuring technique makes it possible to estimate the entire pattern using a partially captured image.

In addition, the perfect map theory has been proposed by M-arrays, pseudorandom arrays, or De Bruijin tori, etc., which have unique mathematical properties, so that the spatial coordination Widely used in spatial codification schemes. A perfect map is a two-dimensional periodic arrangement of r×s size (r is the horizontal value of the perfect map, s is the vertical value of the perfect map) with symbols of alphabet size k, and u × v (u is the horizontal value of the sub array). Value, r is the vertical value of the sub-array) The size of the sub-array should occur only once in the entire array. The perfect map includes all types of sub-arrays except for sub-arrays filled with zeros, and the perfect sub-map is an aperiodic array in which all u×v-sized sub-arrays in the array have a unique r×s size.

(2) Geometric primitive based on binary pattern: In a multi-projection system, projection areas of multiple projectors are made to overlap each other to create a large display. Accordingly, in the conventional projector calibration operation, each of a plurality of projectors sequentially project a pattern image, and accordingly, a calibration of each of the plurality of projectors is performed sequentially. However, the present invention allows all projectors to simultaneously project an image having a unique pattern on the screen, and then all the projectors to be calibrated temporarily.

That is, the present invention uses binary pattern images having different geometric shapes depending on the alphabet. By changing the color of the pattern projected by each projector, it is possible to distinguish adjacent projectors from each other based on the pattern.

(3) Flexible perfect submap: When designing a structured light pattern, generally, the resolution of the pattern is determined based on considerations such as the resolution of the light emitting device, the resolution of the camera, and the capture distance.

In the case of a structured light pattern-based perfect submap, the minimum pattern area that needs to be captured in the camera view to identify the pattern resolution and corresponding point is determined by the size of the perfect submap r×s and the size of the sub array within the perfect submap u×v. Can be determined.

An irregular perfect map that satisfies the following conditions is converted into a pattern image.

또한 상기의 조건으로부터 획득될 수 있으며, 많은 퍼펙트 서브맵은 이 조건을 만족시킴으로써, 다양한 환경 변수를 가지는 파라메터 r, s, u, v, k를 어렵게 결정할 수 있도록 한다. Conditions for perfect submap

It can also be obtained from the above conditions, and many perfect submaps satisfy this condition, making it difficult to determine parameters r, s, u, v, and k having various environmental variables.

And structured light patterns are mainly used to use the depth of objects in the scene, such as 3D scanners. The shooting distance for this is somewhat constant. Accordingly, many people create a perfect submap based on a spatial pattern having a fixed k alphabet size and parameters r, s, u, and v set in advance according to the display environment.

This is the first technology to calibrate a multi-projector through a perfect submap based on a structured light pattern as in the present invention.

The conventional method of using a parameter having a fixed value photographs a pattern through various small cameras, and has a limitation that cannot be applied in a display environment where the distance between the screen and the camera changes frequently. For example, when a low-resolution camera captures a high-resolution pattern, recognition accuracy may be lowered. When a camera placed too close to the screen acquires a low-resolution pattern, the number of corresponding points may become insufficient. Therefore, in the present invention, a codification scheme for matching patterns is required regardless of a change in the size of the perfect submap determined according to the display environment.

Hereinafter, each step of the projector calibration method of the multi-projection system will be described in more detail with reference to FIGS. 3 to 9.

3 is a view for explaining a process of generating a structured light pattern image according to an embodiment of the present invention.

As shown in Fig. 3, the present invention generates a code pattern image (I _code ) that reflects the mathematical characteristics of the perfect submap, and then the white square and the black square cross the code pattern image generated according to Equation 1 The structured light pattern image (I _combined ) is generated by _{performing an} exclusive OR operation with the arranged checkerboard pattern image (I _checker ). In addition, the colors of the structured light pattern image are also diversified according to the total number of the plurality of projectors, and then a plurality of structured light pattern images having different colors are provided to each of the plurality of projectors.

[Equation 1]

The code pattern image (I _code ) provides a mathematical property for point correspondence, and the checkerboard pattern image (I _checker ) provides the exact location of the corresponding point having vertices shared by adjacent checkerboard squares.

)를 계산하도록 한다. The size of the perfect submap determines the number of checkerboard patterns in the checkerboard pattern image (I _checker ), and in the present invention, after determining r×s in consideration of the aspect ratio (σ _h :σ _v ) of the projector, a small sub-array is U×v and the size of the alphabet (k=

) To calculate.

[Equation 2]

Here, r is the row size of the perfect submap, s is the column size of the perfect submap, u is the row size of the subarray corresponding to the perfect submap, v is the column size of the subarray corresponding to the perfect submap, and σ _h is the size of the projector screen. The aspect ratio, σ _v is the aspect ratio of the projector screen, t _a (positive real number) and t _s (positive integer, 2 ≤ ts ≤ 2min(r,s)) are the minimum required for pattern resolution and identification of each corresponding point. It is a scaling factor that adjusts the pattern area.

Accordingly, in the present invention, instead of the parameters r, s, u, v, and k, the user confirms the correct recognition of the pattern and adjusts only t _a and t _s for generating the optimal pattern image, thereby the size of the perfect submap ( r×s), the size of the sub-array (u×v), and the alphabet size (k) can be automatically calculated.

In other words, if the pattern resolution and the minimum area size for identification of each corresponding point are determined by the user by the user parameters t _a and t _s , the perfect submap can be generated using a brute-force algorithm. (RA Morano, C. Ozturk, R. Conn, S. Dubin, S. Zietz, and J. Nissanov. Structured light using pseudorandom codes.IEEE Transactions on Pattern Analysis and Machine Intelligence, 21,22(3):322) -327, 1998. Reference).

로 표현된다. 4 is a diagram for explaining a principle of converting each element of a perfect submap into a symbol of a code pattern image (I _code ) according to an embodiment of the present invention. At this time, each element of the perfect submap

Is expressed as

가 "0" 보다 크다면,

로 결정되며, 이때 ρ은 가장 바깥쪽에 위치되는 동심원의 반경이다.

가 "0" 인 경우에는 블랭크(blank) 표시되고, 인접된 심볼들이 오버랩핑되는 것을 방지하기 위해

로 설정하도록 한다. 이때, W/H는 프로젝터 해상도의 너비/ 높이이다. 더하여 심볼은 도4에서와 같이 흰색 및 검은색 동심원이 교대로 배치되는 형태인 것이 바람직할 것이다. 이 방식은 k 값에 따라 다양한 요소들을 유연하게 설명할 수 있도록 한다. As shown in Fig. 4, in the present invention, the spatial pattern includes a concentric circle around a corresponding point. These circles represent symbols of the code pattern image (I _code ). The concentric circles' radii are

Is greater than "0",

Is determined, where ρ is the radius of the outermost concentric circle.

When is "0", a blank is displayed, and to prevent overlapping of adjacent symbols

Set it to. In this case, W/H is the width/height of the projector resolution. In addition, it is preferable that the symbol is a form in which white and black concentric circles are alternately arranged as in FIG. 4. This method allows flexible explanation of various factors depending on the value of k.

5 is a diagram illustrating a principle of restoring a code pattern image from a camera image according to an embodiment of the present invention.

를 적용함으로써 복원될 수 있다. Referring to Equation 3 described above, the code pattern image (I _code ) is on both sides of Equation 1

Can be restored by applying.

[Equation 3]

This means that the code pattern image (I _code ) based on the perfect submap can be reconstructed through an exclusive OR operation on the image captured by the camera and the checkerboard pattern image (I _checker ). However, since the checkerboard pattern image (I _checker ) must be restored from the captured combined pattern image even without knowing the checkerboard pattern image (I _checker ), the following Equation 4 can be derived.

[Equation 4]

At this time, I will _capture the image, I 'is a _checker board pattern image that is known, I' _code is a code pattern image restoration using a _capture I taken by the camera.

In the "After obtaining the _checker, I _capture and I 'the I from the I _capture the invention two images based on the I" restore _code and the restored code pattern image (I' _code) from the _checker, to project the pattern image It is possible to grasp the correlation between the projector and the field of view of the camera including the pattern image.

)을 형성한다. 다만, 본 발명에서는 계산 부하를 감소하기 위해,

를 임계 거리 d로 선택함으로써, i번째 점(p_i)로부터 너무 가깝거나 먼 점들은 이웃 점(p_j)에서 제외시키도록 한다. A checkerboard pattern image (I' _checker ) for restoration may be obtained from point correspondences located at the center of the symbol. After recalling the centers that match the square corners of the checkerboard pattern, the initial point set (P) is found by finding the point where the four squares that meet in the restoration checkerboard pattern image (I' _checker ) meet through the Shi-Tomasi corner detection algorithm. To form. And, as shown in Fig. 6(a), by extracting neighboring points (p _j ) located adjacent to the i-th point (p _i ) belonging to the initial point set (P), the neighboring point set (

) To form. However, in the present invention, in order to reduce the computational load,

By selecting D as the critical distance d, points that are too close or far from the i-th point p _i are excluded from the neighboring point p _j .

Considering that the structured light pattern is a binary image, an edge image (I _edge ) is obtained through a morphological image operation such as Equation (5).

[Equation 5]

And in order to determine whether the two points are connected by the edge of the edge image as shown in (b) of FIG. 6, based on the Bresenham line mainly used for line drawing with two end points, neighboring points In the set (N _i ), only the i-th point (p _i ) and the neighboring point (p _j ) connected to the edge line are selected to obtain an effective neighboring point set (M _i ).

Equation 6 is a Bressenham line calculation equation.

[Equation 6]

의 조건을 만족시키는 점만을 M_i로 유지하도록 한다. 이때, 임계값 λ은 0.8으로 설정될 수 있을 것이다. In Equation 6, Ω _ij denotes a pixel located on a path between the i-th point p _i and a neighboring point p _j , and n denotes the number of pixels located on the path. Among p _j belonging to N _i ,

Only points that satisfy the condition of are kept as M _i . In this case, the threshold value λ may be set to 0.8.

And, based on the effective function V(p _i ) represented by Equation 7, the corresponding point P'is extracted from P.

[Equation 7]

If all points in the set of effective neighbors (M _i ) are on a straight line, all vectors from p _i to points in M _i are on the same line and the sum of the collinear functions (C(p _i ,p _j )) must be zero. do. Otherwise, the vector has two different orientations, and p _i can be considered as the corresponding point. The error value (e) is set to 0.1 to check collinearity.

The restoration checkerboard pattern image (I' _checker ) may be obtained by checking whether four corresponding points in the photographed image (I _capture ) form a white or black square. The midpoint between the two diagonal points determines the white or black color of each square on the checkerboard.

In addition, Equation 4 is decoded through the obtained restoration checkerboard pattern image (I′ _checker ) to generate a reconstructed code pattern image (I′ _code ). However, incorrect edges caused by misalignment between the restoration checkerboard pattern image (I' _checker ) and the photographed image (I _capture ) remain in the restored code pattern image (I' _code ), but these edges are morphologically open. It can be easily removed by (morphological opening operation).

Then, a contour image (I _contour ) is calculated using the reconstructed code pattern image (I' _code ). Since each element (c _ij ) of the perfect submap is converted into a concentric circle shape as described above, these elements are derived according to the number of circular outlines surrounding each corresponding point. And by comparing the u×v sub-array of these elements using the codeword of the perfect submap used in the code pattern image (I _code ), the point correspondence between the camera and the projector is calculated.

Using the correspondence of the two patterns observed in a single camera view, the present invention produces a pseudo-single view and produces a homography between the camera and the projector to determine the display area.

Equation 9 below is an equation for calculating the homography matrix H _i,j using a RANSAC (RANdom SAmple Consensus) robust estimation method.

[Equation 9]

In this case, i is the index of the camera view, and j is the index of the projector. The camera view acts as a link to convert the projector coordinates to the next projector coordinates. Conversely, the pattern can be used as a link that enables conversion between two camera views as in Equation 9.

As shown in FIG. 7, transformations between all projector nodes and view nodes can be expressed by one homography matrix H, and the reverse direction can be expressed by H- ¹ .

을 만족시키는

의 라인의 변환은 l'=

로 표현될 수 있다. 이때, a, b, c 는　2차원 직선 방정식의 계수이다. By H, the transformation of the point x can be expressed as x′ = Hx , and the linear equation

Satisfying

The conversion of the line of l'=

It can be expressed as In this case, a, b, and c are coefficients of a two-dimensional linear equation.

8 and 9 are diagrams for explaining a method of determining a large display area according to an embodiment of the present invention.

으로 설정하면, 상단 에지와 하단에지를 통과하는 벡터는

와

로 표현되고, 이들 두 벡터 사이의 각도는

로 표현된다. 동일하게, 수직 소실점이

이며, 좌측 에지와 우측 에지를 통과하는 벡터는 각각

및

로, 이들 두 벡터 사이의 각도는

로 표현될 수 있다.For reference, edges on walls, ceilings, and floors provide clues to extracting the necessary geometric information. The spatial geometry can be estimated using a vanishing point obtained from an edge set of a pseudo-single image. Each wall is assumed to be square, and the horizontal vanishing point where the top edge and bottom edge meet is

If set to, the vectors passing through the top and bottom edges are

Wow

And the angle between these two vectors is

Is expressed as Equally, the vertical vanishing point

And the vectors passing through the left and right edges are respectively

And

As, the angle between these two vectors is

It can be expressed as

및

는 추출된 라인 세그먼트들과 이들의 교차 정보를 기반으로 계산될 수 있다. Accordingly, the present invention uses a Canny edge detector or the like to extract an edge corresponding to an intersection point of the wall and each pattern. Then, the subtraction pattern region extracted from each channel image has essential edges having a geometrical structure. And, by using the Hough transform, perfect line segments corresponding to these edge components are extracted. Vanishing point ω of each plane, vector

And

May be calculated based on the extracted line segments and their intersection information.

In particular, in the present invention, the X axis of the camera image does not need to be aligned parallel to the ground, and thanks to the vanishing point extracted from the geometry of the display environment, it can be easily applied to a multi-flat display without placing reference markers or attaching a tilt sensor. Has an advantage.

및

을 통해 결정한 후, 이하의 수학식 10을 이용하여 각 평면에 생성할 수 있는 최대 디스플레이 영역을 설정한다. The display area of each plane is the vanishing point ω, vector

And

After determining through Equation 10, the maximum display area that can be generated in each plane is set using Equation 10 below.

[Equation 10]

를 포함하는 벡터 집합으로, 이는 소실점 ω에서 투사 이미지 상단 에지의 끝점을 향하는 벡터들의 집합이다. 끝 점 중 인접한 투사 이미지 안에 놓여져 있거나, 벽면 바깥 쪽에 위치한 점은 인접한 투사 이미지 혹은 벽면의 경계선과의 교차점(도9에서 주황색 점으로 표시됨)으로 대체 된다. 벡터 집합 V는 벡터

와 투사 이미지의 바닥 에지의 끝점을 이용하여 같은 방식으로 정의된다.Where U is a vector

It is a vector set containing ω, which is a set of vectors from the vanishing point ω toward the end point of the top edge of the projection image. Among the end points, the points placed in the adjacent projection image or located outside the wall are replaced by the adjacent projection images or intersections with the boundary lines of the wall (indicated by orange dots in Fig. 9). Vector set V is vector

It is defined in the same way using the end point of the bottom edge of the projected image and.

After evaluating the size of the maximum display area of each plane, the flat display height is set to the height of the smallest display area. Subsequently, the widths of the displays are compared with each other using the transformation matrices H _{L and R} , and then the smallest one is set as the width of the large display area V.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (8)

In the projector calibration method of a multi-projection system equipped with a plurality of projectors and cameras,
Generating a structured light pattern image based on the perfect submap, and then providing the structured light pattern image to each of the plurality of projectors by varying all colors of the structured light pattern image;
Simultaneously projecting a structured light pattern image through the plurality of projectors and capturing an overlapping area of the structured light pattern image through the camera to obtain at least one captured image;
While restoring a plurality of structured light pattern images included in each of the captured images, a projector corresponding to each of the restored images is identified based on color, and the camera and the plurality of Calculating homography between projectors; And
Based on the homography between the camera and the plurality of projectors and edge information included in the captured image, vertical and horizontal vanishing points are calculated for each of at least one plane constituting the display environment, and the calculated vertical and horizontal vanishing points are calculated. Determining a large display area based on,
The structured light pattern image is
Generated by generating a code pattern image composed of a plurality of symbols reflecting the mathematical properties of each of the plurality of elements included in the perfect submap, and then performing an exclusive OR operation on the code pattern image and the checkerboard pattern image.
How to calibrate a projector in a multi-projection system.
delete The method of claim 1, wherein the plurality of symbols
If the value of the element is "0", a blank is displayed; otherwise, at least one concentric circle in which white and black concentric circles are alternately arranged may be provided, and the radius of the concentric circle is

Characterized in that it is determined as,
The ρ is the radius of a concentric circle located at the outermost side, and

Is the value of the element, the projector calibration method of the multi-projection system. The method of claim 1, wherein the size of the perfect submap is
"

"It is determined by consciousness,
Where r is a row size of the perfect submap, s is a column size of the perfect submap, u is a row size of a subarray corresponding to the perfect submap, v is a column size corresponding to the perfect submap, and σ _h is the projector screen The aspect ratio, σ _v is the vertical ratio of the projector screen, and t _a (positive real number) and t _s (positive integer, 2 ≤ ts ≤ 2min(r,s)) are for pattern resolution and identification of each corresponding point. It is characterized in that it is a scaling factor input from a user to determine a required minimum pattern area,
The projector calibration method of a multi-projection system, wherein the checkerboard pattern image has a number of checkerboard patterns that are actively adjusted according to the size of the perfect submap. The method of claim 1, wherein the restored image
A projector calibration method of a multi-projection system, characterized in that obtained by performing an exclusive OR operation on the captured image and the restoration checkerboard pattern image. The method of claim 5, wherein the restoration checkerboard pattern
After acquiring a point where four squares meet in the photographed image as an initial point, neighboring points located adjacent to the initial point are identified,
After generating an edge image by extracting the edge component of the photographed image, only neighboring points located on the Brezenham line of the edge image are selected from among the neighboring points to identify effective neighboring points,
A projector calibration method of a multi-projection system, characterized in that obtained by analyzing a checkerboard shape of the captured image based on the effective neighboring points. The method of claim 1, wherein determining the large display area
Extracting a plurality of line segments based on edge components of the camera images;
Calculating vanishing points and vertical and horizontal vectors of each plane constituting a large display area based on intersection information of the line segments and the line segments;
Determining a display area of each plane based on a vanishing point of each plane and a vertical and horizontal vector; And
And determining the height and width of the large display area based on the height and width of the smallest display area after evaluating the size of the display area of each plane. A plurality of projectors projecting an image onto a large display area;
A camera for photographing an image projected on the large display area; And simultaneously projecting a structured light pattern image based on a perfect submap through the plurality of projectors, photographing an overlapping area of the structured light pattern image through the camera, and storing a plurality of structured light pattern images included in each of the captured images. For each of at least one plane constituting a display environment based on the homography between the camera and the plurality of projectors by restoration and analysis and the homography between the camera and the plurality of projectors and edge information of the captured image After calculating vertical and horizontal vanishing points, including a processor for determining a large display area based on the calculated vertical and horizontal vanishing points,
The structured light pattern image is
Generated by generating a code pattern image composed of a plurality of symbols reflecting the mathematical properties of each of the plurality of elements included in the perfect submap, and then performing an exclusive OR operation on the code pattern image and the checkerboard pattern image.
Multi projection system.

Images