JP6717488B2 - Projection system, projection method, pattern generation method and program - Google Patents

Description

The present invention relates to a projection system, a projection method, a pattern generation method and a program.

2. Description of the Related Art In recent years, video projectors are widely used due to the widespread use of projection matching. For example, it is disclosed that a projected image of a single or a plurality of projector groups is projected on a lenticular screen to form an image with perspective (see, for example, Non-Patent Documents 1 and 2). Further, a system has been proposed in which a plurality of rows of water drops are dropped so as to block the optical path of the projection light of a single projector, and an image is displayed on the falling water drops (for example, see Non-Patent Document 3). According to this system, it is possible to drop the water droplet array at a plurality of positions on the optical path of the projection light and display different images at the respective positions.

"Prototyping a light field display involving direct observation of a video projector array.," Jurik, J., Jones, A., Bolas, M., & Debevec, P. (2011, June)., Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on, IEEE, 2011. "An autostereoscopic projector array optimized for 3D facial display.," Nagano, K., Jones, A., Liu, J., Busch, J., Yu, X., Bolas, M., & Debevec, P. (2013 , July). InACM SIGGRAPH 2013 Emerging Technologies (p.3). ACM. "A multi-layered display with water drops," Barnum, Peter C., Srinivasa G. Narasimhan, and Takeo Kanade, ACM Transactions on Graphics (TOG) 29.4 (2010): 76.

However, the system described in Non-Patent Document 3 can display different images at a plurality of positions only by a special method of dropping a water droplet array on the optical path of projection light. In addition, in this system, only the projection light that passes through the optical path of the projection light and does not hit the front water droplet array can reach the rear water droplet array, so the displayed images are extremely limited. It is a matter of fact that they are not independent of each other.

The present invention has been made in view of the above circumstances, and a projection system, a projection method, a pattern generation method, and a program capable of displaying a plurality of different independent images on the optical path of projection light by a simpler method. The purpose is to provide.

In order to achieve the above object, the projection system according to the first aspect of the present invention is
Equipped with a plurality of projectors that are installed so that the irradiation areas of the projected projection light intersect,
The plurality of projectors are
By projecting the pattern images so that they are superimposed on each other, different composite images are formed at a plurality of positions depending on how the pattern images overlap ,
The composite image formed at each of the plurality of positions is
It is an image displaying a pattern indicating a position where the composite image is formed .

The pattern image is
The background color is an intermediate color of the multiple types of colors that make up the pattern,
It may be that.

A projection system according to a second aspect of the present invention is
It is installed so that the irradiation areas of the emitted projection light intersect, and by projecting the pattern images so that they overlap each other, different composite images are formed at a plurality of positions depending on how the pattern images overlap. Multiple projectors,
A pattern image displayed on the plurality of projectors each, based on the information indicating the positional relationship between the projected pattern image on the plurality of positions, each pixel of the displayed pattern image in the plurality of projectors each element And a conversion matrix generation unit that generates a conversion matrix that converts the first matrix into a second matrix having each pixel of the composite image projected at each of the plurality of positions as an element,
Based on the second matrix corresponding to the composite image having a pattern to be displayed, and the transformation matrix generating unit The generated transformation matrix, generating a pattern image corresponding to the first matrix for each projector A pattern generation unit that
Equipped with .

The pattern generation unit obtains a solution of the first matrix based on the second matrix and the conversion matrix under the constraint of the range of brightness of the generated pattern image,
It may be that.

The conversion matrix generation unit,
The pattern having the positional information of the pixel of the displayed pattern image to each projector, the projected onto the plurality of positions from the projector, images the projection shadow pattern projected on each of the plurality of positions, it is captured Obtaining information indicating the positional relationship by decoding the projection pattern,
It may be that.

The conversion matrix generation unit limits the information indicating the positional relationship to epipolar lines of the plurality of projectors, and generates the conversion matrix for the epipolar lines.
The pattern generation unit,
Generation of a pattern image corresponding to the first matrix is executed for all epipolar lines based on the second matrix and the conversion matrix generated by the conversion matrix generation unit, and the generated pattern image is generated. Generate a composite image as an image to be projected for each projector,
It may be that.

A color calibration unit configured to project a color test pattern image on each of the projectors and to calculate color adjustment information of the pattern image based on an imaging result of the projected test pattern image,
A projection control unit configured to project the pattern image generated by the pattern generation unit onto each of the projectors based on color adjustment information adjusted by the color calibration unit;
With
It may be that.

By placing translucent screens at a plurality of positions where the composite image is projected, a different composite image is projected on each translucent screen,
It may be that.

A projection method according to a third aspect of the present invention is
Installed multiple projectors so that the projected light emitted would cross each other and proceed while intersecting each other.
By causing the plurality of projectors to respectively project, with projection light, a plurality of pattern images each having at least one of a strengthening pattern and a weakening pattern when they are superimposed on each other, different composition depending on how the pattern images overlap at a plurality of positions. Forming an image ,
Based on the information indicating the positional relationship between the pattern image displayed on each of the plurality of projectors and the pattern images projected on the plurality of positions, each pixel of the pattern image displayed on each of the plurality of projectors is set as an element. To generate a conversion matrix for converting the first matrix into a second matrix having each pixel of the composite image projected at each of the plurality of positions as an element,
A pattern image corresponding to the first matrix is generated for each projector based on the second matrix corresponding to the composite image having the displayed pattern and the generated conversion matrix .

A pattern generation method according to a fourth aspect of the present invention is
A pattern generation method executed by a projection system including a plurality of projectors so that emitted projection light intersects with each other and travels while intersecting each other.
A pattern image displayed on the projector each multiple, based on the information indicating the positional relationship between the projected pattern image into a plurality of positions, said to each pixel of each projector each displayed pattern image elements a first matrix, the geometric calibration step of generating a transformation matrix for converting the second matrix with the pixels of the synthetic image that will be projected on the plurality of positions each element,
Said second matrix corresponding to the composite image having a pattern to be displayed, based on the transformation matrix generated by the geometric calibration process, generating a pattern image corresponding to the first matrix for each projector A pattern generation process for
including.

Before SL is projected test pattern image colors each projector, based on the imaging result of the projected test pattern image, the color calibration process of calculating a color adjustment information of the pattern image,
A projection step of the pattern image generated in the previous SL pattern generating step, based on the adjustment information of the adjusted color by the color calibration process, is projected to the each projector,
including,
It may be that.

A program according to a fifth aspect of the present invention is
On the computer,
Based on the information indicating the positional relationship between the pattern images displayed on each of the plurality of projectors and the pattern images projected on the plurality of positions, each pixel of the pattern image displayed on each of the projectors is set as an element. 1 matrix, and the geometric calibration procedure for generating a transform matrix for converting the second matrix with the pixels of the synthetic image that will be projected on the plurality of positions each element,
Said second matrix corresponding to the composite image having a pattern to be displayed, based on the transformation matrix generated by the geometric calibration procedure, generating a pattern image corresponding to the first matrix for each projector Pattern generation procedure,
To run.

According to the present invention, the pattern image formed by the projection light emitted from each of the plurality of projectors has a plurality of strengthening portions and weakening portions that are overlapped with each other. It is an image in which the strengthening part and the weakening part change. In other words, in the present invention, the projection light emitted from each projector intersects with each other, and at a plurality of positions in a section where the projection light advances, a portion that actually strengthens and a portion that weakens depending on the overlapping position of the pattern images. By adjusting and, the composite images of the pattern images at a plurality of positions can be made completely independent and completely different. Therefore, according to the present invention, it is possible to form a plurality of different independent images on the optical path of the projection light by a simpler method of intersecting the plurality of projection lights by the plurality of projectors.

1 is a perspective view showing a schematic configuration of a projection system according to Embodiment 1 of the present invention. 2(A), 2(B), 2(C) and 2(D) are diagrams showing an example of a pattern image and a composite image projected by the projection system of FIG. It is a figure which shows the positional relationship of the projection surface of the pattern image in a projector, and the to-be-projected surface of the pattern image in a some position. It is a block diagram which shows the structure of a control part. It is a figure which shows geometric calibration. FIGS. 6A and 6B are diagrams showing an example of a test pattern projected on each projector. 7A and 7B are diagrams showing an example of a test pattern imaged by a camera. It is a graph which shows an example of the estimated response curve. 6 is a flowchart showing an image projection process in the projection system of FIG. 1. 10A and 10B are examples of images projected by each projector. 10C and 10D are examples of images projected on each screen. It is a perspective view which shows the schematic structure of the projection system which concerns on Embodiment 2 of this invention. It is a perspective view which shows the schematic structure of the projection system which concerns on Embodiment 3 of this invention. It is a figure which shows the pattern image in the projection system which concerns on Embodiment 4 of this invention. It is a figure which shows the projection system which concerns on Embodiment 5 of this invention. It is a figure which shows the projection system which concerns on Embodiment 6 of this invention. It is a figure which shows the projection system which concerns on Embodiment 7 of this invention. It is a figure which shows a mode that a different synthetic image is projected on each translucent screen by arranging two translucent screens and placing a translucent screen in the some position where the synthetic image of two projectors is projected. It is a figure which shows the synthetic image projected on two screens. Epipolar plane (optical centers C _1, C _2, light rays proceed to the screen 1, epipolar lines, including epipoles e1, e2) is a diagram showing a. It is a figure which shows a mode that a light beam interferes in an epipolar plane. 21(A), FIG. 21(B), FIG. 21(C), FIG. 21(D), FIG. 21(E), and FIG. 21(F) are diagrams showing an example of a projected image in which the target object is a plane. Is. 22(A), FIG. 22(B), FIG. 22(C), and FIG. 22(D) are diagrams showing the results of numerical evaluation of the projected image. FIGS. 23A and 23B are views showing a state in which a plurality of horizontal striped patterns are projected by gray code projection on an arbitrarily shaped object. FIG. 23C is a diagram showing an example of a map image obtained by gray code projection of a horizontal striped pattern. FIGS. 24A and 24B are views showing a state in which a plurality of vertical striped patterns are projected by gray code projection on an arbitrary shaped object. FIG. 24C is a diagram showing an example of a map image obtained by gray code projection of a vertical striped pattern. FIGS. 25A and 25B are views showing a state in which a plurality of horizontal striped patterns are projected by gray code projection on another arbitrary shaped object. FIG. 25C is a diagram showing an example of a map image obtained by gray code projection of a horizontal striped pattern. FIGS. 26A and 26B are diagrams showing a state in which a plurality of vertical striped patterns are projected by gray code projection on another arbitrary shaped object. FIG. 26C is a diagram showing an example of a map image obtained by gray code projection of a vertical striped pattern. 27A and 27B are diagrams illustrating an example of an image projected on an arbitrary shaped object. FIG. 28A is a map image based on a horizontal striped pattern obtained when an arbitrary shaped object is placed at one depth. FIG. 28B is a map image based on a vertical striped pattern obtained when an arbitrarily shaped object is placed at one depth. FIG. 29A is a map image based on a horizontal striped pattern obtained when an arbitrary shaped object is placed at the other depth. FIG. 29B is a map image based on a vertical striped pattern obtained when an arbitrarily shaped object is placed at the other depth. 30A and 30B are examples of pattern images projected by each projector. FIG. 31A is a diagram showing an example of an image projected on an arbitrarily shaped object placed at one depth. FIG. 31B is a diagram showing an example of an image projected on an arbitrarily shaped object placed at the other depth. 32A and 32B are diagrams showing an example of an image projected on each object when objects having different shapes are placed at two positions.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Embodiment 1.
First, the first embodiment of the present invention will be described.

FIG. 1 shows a schematic configuration of a projection system 100 according to the first embodiment. As shown in FIG. 1, the projection system 100 includes two projectors 1 and 2. The projectors 1 and 2 emit the projection lights L1 and L2, respectively, and the emitted projection lights L1 and L2 spread as they progress.

The projectors 1 and 2 are installed so that the emitted projection lights L1 and L2 intersect with each other and proceed while intersecting each other. That is, the projectors 1 and 2 are provided so as to face substantially the same direction, and the optical axes of the respective projection optical systems intersect at a shallow angle.

The projection system 100 includes a control unit 10. The control unit 10 is a computer and includes a communication interface, a CPU (Central Processing Unit), and a memory. The control unit 10 realizes its function by the CPU executing the program stored in the memory and communicating with the projectors 1 and 2 through the communication interface.

The control unit 10 controls the projectors 1 and 2 by, for example, projecting the pattern images P1 and P2 projected by the projectors 1 and 2 on the projectors 1 and 2. The control unit 10 has a pointing device such as a mouse as an input interface. The cursor by the pointing device is displayed on the projection surfaces of the projectors 1 and 2.

Under the control of the control unit 10, the projector 1 displays the projected pattern image P1 on the internal projection surface (projection surface PL1 in FIG. 3) and projects the pattern image P1 to the outside with the projection light L1. Under the control of the control unit 10, the projector 2 displays the projected pattern image P2 on the internal projection surface (projection surface PL2 in FIG. 3) and projects the pattern image P2 to the outside with the projection light L2. In the case of this example, the pattern images P1 and P2 gradually overlap with each other as the projection lights L1 and L2 travel, and most of them overlap at the position of the depth d2 with respect to the projectors 1 and 2, and at the position of the depth d1. , They are supposed to overlap. In actual installation, it is preferable that the projectors are brought as close as possible to each other, because the overlapping portion becomes large regardless of the depth.

The projectors 1 and 2 project the pattern images P1 and P2, which have at least one of a strengthening pattern and a weakening pattern when they are overlapped with each other, with the projection lights L1 and L2. In this embodiment, the pattern images P1 and P2 have the background color as an intermediate color of a plurality of types of colors forming the pattern. For example, if the image is grayscale, the background color is gray, which is an intermediate color between white and black. In a normal image, the background color is the color of the darkest part (black) in the scene, whereas in the present invention, a negative color can be realized by defining it as an intermediate color. As a result, in the pattern images P1 and P2, if the overlapping patterns are patterns of a similar color (for example, white), they become mutually intensifying patterns (this is the same in a normal image), but complementary colors (opposite colors). ) Relationship (for example, white and black), they become patterns that weaken each other, and when both patterns have a completely complementary color relationship, these patterns can cancel each other.

For example, as shown in FIG. 2A, pattern images P1 and P2 projected by the projectors 1 and 2 have a pattern image of number "1" having a high brightness with respect to the background color. In this case, at the position of the depth d2 where the pattern images of "1" of both pattern images overlap, a composite image in which the pattern of "1" is emphasized is displayed. However, in this case, at the position of the depth d1, two light “1” pattern images, each of which has a luminance slightly higher than the background color, are displayed.

In response to this, for example, as shown in FIG. 2B, a pattern “1” having a lower brightness than the background color is newly added to the pattern image projected by the projector 2. The pattern image of "1" cancels out with the pattern of "1" on the right side at the position of the depth d1, and the pattern of "1" disappears. However, in this case, at the position of the depth d2, a thin "1" pattern whose brightness is slightly lower than the background color is displayed.

In order to cancel the "1" pattern in which the brightness displayed at the position of the depth d2 is slightly lower than the background color, for example, as shown in FIG. 2C, the pattern image P1 projected by the projector 1 has a brightness higher than the background color. The pattern of "1" having a high value is newly added. This "1" pattern cancels out with the "1" pattern on the right side at the position of the depth d2, and the "1" pattern disappears. However, at the position of the depth d1, the pattern of "1" having higher brightness than the background color is displayed on the right side.

That is, by adding a plurality of "1" patterns to appropriate positions of the pattern images P1 and P2 in this manner, it is possible to form a "1" pattern at arbitrary positions of the depths d2 and d1. For example, as shown in FIG. 2D, by appropriately combining a pattern of "1" having a luminance higher than that of the background color and a pattern of "1" having a luminance lower than the background color, the depths d1 and d2 are respectively different. As a result of strengthening and weakening each other, a pattern of "1" can be formed as a composite image at that position.

In order to generate pattern images P1 and P2 in which appropriate patterns are combined and form a desired composite image at the positions of the depths d1 and d2, it is necessary to accurately determine the positions of the patterns on the projection surfaces of the projectors 1 and 2. There is. For that purpose, as a premise, it is necessary to grasp the positional relationship between the projection surface of the pattern image in the projectors 1 and 2 and the projection surface of the pattern image at the positions of the depths d1 and d2.

FIG. 3 shows projection surfaces (video display surfaces) PL1 and PL2 of the pattern images P1 and P2 in the projectors 1 and 2, and projected surfaces (patterns of the screens I1 and I2) of the pattern images P1 and P2 at positions of the depths d1 and d2. It shows the positional relationship with the surface on which the image is projected). The screens I1 and I2 are also referred to as projection surfaces I1 and I2.

The projection planes PL1 and PL2 are image planes on which the pattern images P1 and P2 are displayed in the projectors 1 and 2, respectively. In the projectors 1 and 2, the light from each point light source is applied to the projection surfaces PL1 and PL2, and the projection light including the pattern images P1 and P2 is emitted to the outside via the projection optical system. Pixels forming the pattern images P1 and P2 on the projection planes PL1 and PL2 are defined as p _j,1 , p _j,2 ,..., P _j,m ,... (j=1 or 2).

The pattern images P1 and P2 displayed on the projection planes PL1 and PL2 are projected on the screen I1 or I2 which is the projection surface. Each pixel when the pattern images P1 and P2 projected on the projection surface on the screens I1 _and I2 are divided into pixels is _defined as i _k,1 , i _k,2 ,..., I _k,n,. k=1 or 2).

また、スクリーンＩ１_、Ｉ２上の位置におけるパターン画像Ｐ１、Ｐ２の各画素を要素とする行列Ｉ₁、Ｉ₂を以下のように定義する。

上述のように投影面ＰＬ１、ＰＬ２の各画素ｐ_j,1、ｐ_j,2、…、ｐ_j,m、…から出た光は、スクリーンＩ１_、Ｉ２上の位置座標ｉ_k,1、ｉ_k,2、…、i_k,n、…に入射するため、投影面ＰＬ１、ＰＬ２の各画素ｐ_j,1、ｐ_j,2、…、ｐ_j,m、…の行列Ｐ₁、Ｐ₂と、スクリーンＩ１_、Ｉ２上の位置座標の行列Ｉ₁、Ｉ₂とは、以下の式（１）、式（２）のように射影変換の式で定義することができる。

ここで、行列Ａ_1,1、Ａ_1,2、Ａ_2,1、Ａ_2,2は、ホモグラフィ行列から生成される変換行列である。ホモグラフィ行列とは、画像上のある点が、射影変換後の画像上のどこに移動するかを、行列の積の形で計算可能とするものである。そこで、ホモグラフィ行列を用いて、移動前後の座標のセットを計算しておけば、行列Ａ_1,1、Ａ_1,2、Ａ_2,1、Ａ_2,2は、各行のいずれか１つの要素が１であり、残りは０である行列として即座に変換行列を生成できる。また、各行の１つの要素のみ１で残りは０のため、この行列は疎行列である。疎行列は、そのアトリビュートにより、逆行列計算と言った計算量の大きな処理を、効率的に解くことができる場合があり、その性質を利用できるメリットがある。 Here, matrices P ₁ and P ₂ each having pixels of the pattern images P1 and P2 on the projection planes PL1 and PL2 as elements are defined as follows.

Further, matrices I ₁ and I ₂ having pixels of the pattern images P1 and P2 at the positions on the screens I1 _and I2 as elements are defined as follows.

As described above, the light emitted from the pixels p _j,1 , p _j,2 ,..., P _j,m ,... Of the projection planes PL1, PL2 is the position coordinates i _k,1 , i on the screens I 1 _, I 2. _Since they are incident on _k,2 ,..., I _k,n ,..., Matrixes P ₁ , P _{2 of} pixels p _j,1 , p _j,2 ,..., P _j,m ,... When, the matrix I _1, I ₂ coordinates on the screen I1, _I2, the following equation (1) can be defined by the formula of projective transformation as in equation (2).

Here, the matrices A _1,1 , A _1,2 , A _2,1 and A _2,2 are conversion matrices generated from the homography matrix. The homography matrix makes it possible to calculate where a certain point on the image moves on the image after the projective transformation in the form of a matrix product. Therefore, if a set of coordinates before and after the movement is calculated using a homography matrix, the matrix A _1,1 , A _1,2 , A _2,1 , A _{2,2 will} be one of the rows. A transformation matrix can be immediately generated as a matrix whose elements are 1 and the rest are 0. Also, since only one element in each row is 1 and the rest are 0, this matrix is a sparse matrix. A sparse matrix may be able to efficiently solve a process that requires a large amount of calculation, such as an inverse matrix calculation, depending on its attributes, and has the advantage of being able to utilize its properties.

上記式（１）、（２）は、以下の式（３）のようにまとめられる。

本実施の形態では、スクリーンＩ１、Ｉ２上に所望の合成画像を形成するために、プロジェクタ１、２にそれぞれ投影させるパターン画像Ｐ１、Ｐ２を投影面ＰＬ１、ＰＬ２に表示させる必要があり、そのためには、行列Ｐ₁、Ｐ₂、すなわち行列Ｐを求める必要がある。行列Ｐを求めるには、まず、変換行列Ａを求める幾何キャリブレーション処理を行う必要がある。幾何キャリブレーション処理やプロジェクタ１、２へのパターン画像Ｐ１、Ｐ２の投影指示を行うのは制御部１０である。 Here, the matrices P ₁ and P ₂ , the matrices I ₁ and I _2, and the conversion matrices A _1,1 , A _1,2 , A _2,1 and A _2,2 are the following matrices P and I: , A. This matrix P corresponds to the first matrix, and the matrix I corresponds to the second matrix.

The above equations (1) and (2) are summarized as the following equation (3).

In the present embodiment, in order to form a desired composite image on the screens I1 and I2, it is necessary to display the pattern images P1 and P2 to be projected on the projectors 1 and 2 on the projection planes PL1 and PL2. Needs to find the matrices P ₁ and P ₂ , that is, the matrix P. In order to obtain the matrix P, it is first necessary to perform geometric calibration processing to obtain the conversion matrix A. It is the control unit 10 that performs the geometric calibration process and the projection instruction of the pattern images P1 and P2 to the projectors 1 and 2.

FIG. 4 is a block diagram showing the configuration of the control unit 10. As shown in FIG. 4, the control unit 10 includes a conversion matrix generation unit 11, a pattern generation unit 12, a color calibration unit 13, and a projection control unit 14.

The conversion matrix generation unit 11 uses, for example, a homography matrix, the projection planes PL1 and PL2 of the pattern images P1 and P2 of the projectors 1 and 2, and the pattern images P1 at the depths (positions) d1 and d2. Information indicating the positional relationship between the projection planes I1 and I2 of P2 is acquired by calculation, and the pixels p _j,1 and p _j,2 of the pattern images P1 and P2 on the projection planes PL1 and PL2 of the projectors 1 and 2 are acquired. , ..., p _j,m , ..., a matrix P having pixels as elements i _k,1 , i _k,2 , ..., i _{k of} a composite image formed at each of a plurality of depths (positions) d1 and d2. A conversion matrix A for converting into a matrix I having elements such as _{, n} ,... Is generated.

As shown below, the conversion matrix generation unit 11 obtains the conversion matrix A. First, as shown in FIG. 5, the control unit 10 causes the projectors 1 and 2 to project a test pattern image in which the background color is white and the black patterns are formed at the four corners. Then, the user operates the pointing device provided in the control unit 10 to place the cursor on the position of the black pattern in the pattern images P1 and P2 projected on the screens I1 and I2 and click. By this click operation, the positions of the pixels p _j,1 , p _j,2 ,..., P _j,m ,... And the position coordinates i _k,1 , i _k,2 ,..., i _k,n ,. The relationship becomes clear, and the conversion matrix A can be obtained from the positional relationship. The conversion matrix generation unit 11 thus obtains the conversion matrix A. The conversion matrix generation unit 11 is realized by operations of a CPU, a memory, a pointing device, and a communication interface.

The pattern generation unit 12 generates pattern images P1 and P2 corresponding to the matrix P based on the matrix I corresponding to the composite image having the desired pattern and the conversion matrix A generated by the conversion matrix generation unit 11. .. Specifically, the pattern images P1 and P2 are generated by multiplying the matrix I by the inverse matrix of the conversion matrix A to obtain the matrix P. The pattern generation unit 12 is realized by the operations of the CPU and the memory. Here, although the size of the conversion matrix is very large, by utilizing the fact that the conversion matrix is a sparse matrix, it is possible to realize extremely high-speed calculation using the numerical operation library for the sparse matrix.

The color calibration unit 13 causes each of the projectors 1 and 2 to project a color test pattern image. For example, a test pattern in which the color changes uniformly is used for the color test pattern image. 6A and 6B show an example of the test pattern projected on each of the projectors 1 and 2. The color calibration unit 13 calculates the color adjustment information of the pattern image based on the imaging result of the test pattern image captured by the camera. The image data of the camera can be taken in through a normal interface of a computer.

The color test pattern images projected on the screens I1 and I2 are captured by a camera. 7(A) and 7(B) show an example of the test pattern imaged by the camera. The color calibration unit 13 non-linearly sets the brightness value input to the projectors 1 and 2 and the brightness value actually projected on the screens I1 and I2 based on the imaging data of the color test pattern image captured by the camera. A brightness curve (response curve) showing the above relationship is calculated. FIG. 8 shows an example of the estimated response curve. In addition to the test pattern curves (Proj1, Proj2) projected on the projectors 1 and 2, FIG. 8 also shows the superimposed image curves (sum, Add) of the images projected by the projectors 1 and 2. .. The color calibration unit 13 is realized by a CPU, a memory, and a communication interface.

The projection control unit 14 adjusts the pattern images P1 and P2 using the luminance curve calculated by the color calibration unit 13, that is, based on the color adjustment information adjusted by the color calibration unit 13. Then, the projection control unit 14 causes the projectors 1 and 2 to project the pattern images P1 and P2 generated and adjusted by the pattern generation unit 12. The projection control unit 14 is realized by a CPU, a memory, and a communication interface.

FIG. 9 is a flowchart of the image projection process executed by the control unit 10 using the projectors 1 and 2. As shown in FIG. 9, first, the control unit 10 (the conversion matrix generation unit 11) sets the matrix P corresponding to the pattern images P1 and P2 projected by the projectors 1 and 2 to a plurality of depths (positions) d1 and d2. A transformation matrix A that transforms into a matrix I corresponding to the composite image projected on is calculated (step S1; geometric calibration step). In the geometrical calibration process, the conversion matrix A described above is obtained.

Here, for example, as shown in FIG. 5, on the screens I1 and I2 installed at the positions of the depths d2 and d1 onto which the pattern images P1 and P2 are projected, the pattern images P1 and P2 in which check patterns are formed at the four corners are formed. Is displayed and the position is clicked using a pointing device to obtain the position coordinates on the projection planes PL1 and PL2 corresponding to the check pattern. The control unit 10 (the conversion matrix generation unit 11) calculates the conversion matrix A based on the relationship between the position coordinates of the check pattern in the pattern image and the position coordinates of the check pattern in the imaged result.

Then, the control unit 10 (pattern generation unit 12) calculates the matrix P by multiplying the matrix I by the inverse matrix A ⁻¹ (by the inverse matrix operation), and the pattern images P1 of the projectors 1 and 2, P2 is generated (step S2; pattern generation process). Here, if the inverse matrix cannot be obtained, the matrix P may be obtained by performing a fitting operation that minimizes P-IA ^-1 .

The control unit 10 adjusts the color of the pattern image based on the imaging result of the camera of the pattern images P1 and P2 formed on the screens I1 and I2 to the color test pattern images for the projectors 1 and 2 (step S3). : Color calibration process). As a result, a brightness curve (response curve) showing non-linearity between the specified brightness value and the actual brightness value is generated.

The control unit 10 projects the pattern images P1 and P2 adjusted by the color adjustment information generated in the color calibration process onto the projectors 1 and 2 (step S4: projection process). As a result, the composite image of the pattern images P1 and P2 is projected on the screens I1 and I2.

As described above in detail, according to the present embodiment, the pattern images P1 and P2 formed by the projection lights L1 and L2 emitted from each of the plurality of projectors 1 and 2 are strengthened when they are overlapped with each other. Further, a weakened portion is formed, and the image is such that the actually strengthened portion and the weakened portion change depending on the overlapping position. In other words, in this embodiment, the projection images L1 and L2 emitted from the respective projectors 1 and 2 intersect and the pattern images P1 and P2 at a plurality of depths (positions) d1 and d2 in a section where the projection lights L1 and L2 proceed while intersecting each other. By adjusting the actually strengthening portion and the weakening portion in accordance with the overlapping position, the composite images of the pattern images P1 and P2 at a plurality of depths (positions) d1 and d2 are made completely different independently. You can Therefore, according to this embodiment, it is possible to form a plurality of different independent images on the optical paths of the projection lights L1 and L2 by a simpler method of intersecting the plurality of projection lights with the plurality of projectors 1 and 2. You can

For example, the pattern image shown in FIG. 10(A) is projected on the projector 1, and the pattern image shown in FIG. 10(B) is projected on the projector 2. These images are completely independent images. In this case, the pattern images shown in FIGS. 10C and 10D are projected on the screens I1 and I2.

Embodiment 2.
Next, a second embodiment of the present invention will be described.

As shown in FIG. 11, this embodiment includes three projectors 1, 2, and 3. The projectors 1, 2, and 3 are installed so that the emitted projection lights intersect with each other and proceed while intersecting each other. The projection system 100 forms different pattern images at three positions having different depths (three layers LA1, LA2, LA3). The layer LA1 displays the composite image in which the matrix of the pattern of the number "1" is formed, the layer LA2 displays the composite image in which the matrix of the pattern of the number "2" is formed, and the layer LA3. , A composite image in which a matrix of the pattern of the number “3” is formed is displayed.

According to the present invention, as in this embodiment, the number of projectors is not limited to two, and the number of composite images formed is not limited to two.

Embodiment 3.
Next, a third embodiment of the present invention will be described.

As shown in FIG. 12, the present embodiment includes a projector group 5 including 16 pico projectors. The projector group 5 includes pico projectors arranged in 4 rows and 4 columns. The respective pico projectors are installed so that the emitted projection lights intersect with each other and proceed while intersecting each other. The projection system 100 forms different pattern images at the positions of the layers LA1 to LA5.

The layer LA1 displays the composite image in which the matrix of the pattern of the number "1" is formed, the layer LA2 displays the composite image in which the matrix of the pattern of the number "2" is formed, and the layer LA3. , A composite image in which a matrix of the pattern of the number “3” is formed is displayed. On the layer LA4, a composite image in which a matrix of the pattern of the number "4" is formed is displayed. On the layer LA5, a composite image in which a matrix of the pattern of the number "5" is formed is displayed.

As in this embodiment, the number of projectors is not limited to two or three, and the number of composite images formed is not limited to two or three. As in this embodiment, many small projectors can be used, and the number of composite images is not limited.

Fourth Embodiment
Next, a fourth embodiment of the invention will be described.

As shown in FIG. 13, this embodiment includes three projectors 1, 2, and 3. The projectors 1, 2, and 3 are installed so that the emitted projection lights intersect with each other and proceed while intersecting each other. The projection system 100 forms different pattern images at three positions (layers LA1 and LA2). The layer LA1 displays the composite image in which the matrix of the pattern of the number “1” is formed, and the layer LA2 displays the composite image in which the matrix of the pattern of the number “2” is formed. In this embodiment, the pattern image projected by each of the projectors 1, 2, and 3 is not a pattern such as "1" but a point image pattern. Even if a pattern image including such a point image pattern is used, a composite image including a matrix of “1” patterns and a composite image including a matrix of “2” patterns can be displayed in different depths (layers). ..

The pattern images shown in FIGS. 10A and 10B are images generated from such a viewpoint.

Embodiment 5.
Next, a fifth embodiment of the invention will be described.

As shown in FIG. 14, the present embodiment includes a projector group 5 including 16 pico projectors. The projector group 5 includes pico projectors arranged in 4 rows and 4 columns. The respective pico projectors that form the projector group 5 are installed so that the projected light beams that are emitted intersect with each other and proceed while intersecting each other. In this projection system 100, translucent screens I1, I2, I3, and I4 are installed at four positions, and different independent images are displayed by the projector group 5 on each of the screens I1 to I4.

With this projection system 100, projection mapping can be performed on different images at different positions at the same time.

Sixth Embodiment
Next, a sixth embodiment of the present invention will be described.

As shown in FIG. 15, the present embodiment includes a projector group 5 including 16 pico projectors. The projector group 5 includes pico projectors arranged in 4 rows and 4 columns. The pico projectors are installed so that the emitted projection lights intersect with each other and proceed while intersecting each other. The projection system 100 forms different pattern images at a plurality of positions. At the position of depth 1 m, the composite image in which the matrix of the pattern of number “1” is formed is displayed, and at the position of depth 2 m, the composite image in which the matrix of the pattern of number “2” is formed is displayed. Also, a composite image in which a matrix of the pattern of the number "3" is formed is displayed at the position of depth 3m, and a composite image in which the matrix of the pattern of number "4" is formed is displayed at the position of the depth 4m. ..

This projection system 100 can be installed in an industrial facility as a surveying system (three-dimensional ruler). For example, when a baggage is placed at a depth of 2 m in the measurement area 40, a pattern matrix of "2" is displayed on the baggage. Thereby, the place where the luggage is placed is measured to be 2 m. As described above, according to the projection system 100, the distance information in the depth direction can be used for displaying on the luggage.

Embodiment 7.
Next, a seventh embodiment of the invention will be described.

The projection system 100 according to this embodiment is used as a system that projects a pattern onto the measurement target 51 in a shielded environment in nature. The projected pattern image is information indicating the position (depth) of the measurement target 51 onto which the pattern is projected, as in the sixth embodiment.

As shown in FIG. 16, the present embodiment includes a projector group 5 including 16 pico projectors. The projector group 5 includes pico projectors arranged in 4 rows and 4 columns. The respective pico projectors that form the projector group 15 are installed so that the projected light beams that are emitted intersect with each other and proceed while intersecting each other.

The pattern image projected on the measurement target 51 by the projector group 5 varies depending on the depth of the measurement target 51. Therefore, by observing the pattern of the synthetic image formed on the measurement target 51, the three-dimensional shape of the measurement target 51 can be measured.

If this projector group 5 is used, even if the light emitted from one of the pico projectors is blocked by the obstacle 50, if the light incident on the same position from another pico projector is incident on that position, the measurement target A pattern image can be formed at 51. That is, in this projection system, a plurality of pico projectors project the pattern image of the same portion, that is, the image of the strengthening portion, so that the angle of view can be substantially increased to realize a virtual wide aperture. it can. Therefore, even if there is an obstacle 50 (eg, hedge) in front of the projector group 5, a synthetic image can be formed in front of it and the pattern can be projected at the position of the measurement target 51. As a result, many shape measurement algorithms by the projector/camera system can be used even in an obstacle scene.

In FIG. 17, two translucent screens I1 and I2 are arranged, and the translucent screens I1 and I2 are placed at a plurality of positions where the composite images of the projectors 1 and 2 are projected. Shows how different composite images are projected. As shown in FIG. 18, the image shown in FIG. 10C is displayed on the semitransparent screen I1, and the image shown in FIG. 10D is displayed on the semitransparent screen I2.

Eighth embodiment.
Next, an eighth embodiment of the present invention will be described.

The pattern generation algorithm according to this embodiment is an independent calculation method using a limited number of pixels. That is, as shown in FIG. 19, all the rays that mutually compensate the luminance are on the epipolar plane. Therefore, the pattern generation calculation can be performed independently for each epipolar plane. Further, as shown in FIG. 20, even in the epipolar plane, the light rays that interfere with each other are scattered. For example, in FIG. 20, only a total of 8 points, that is, 4 points on the screen 1 and 4 points on the screen 2, interfere with each other, but nothing else is involved. Further, the projection patterns necessary for projecting these 8 points are 4 points in the projector 1 and 5 points in the projector 2, which is a total of 9 points, and the other patterns are not involved at all. Therefore, a solution (luminance of 9 points in the projection pattern) can be obtained by simultaneously solving 9 points (unknown parameters) on the projection pattern and 8 points (8 linear equations) on the screen. As a result, the pattern images displayed on each of the plurality of projectors and the information indicating the positional relationship between the pattern images projected at the plurality of positions are displayed on the plurality of projectors without solving the huge matrix as in the first embodiment. It is possible to obtain a projection pattern by solving independent small simultaneous linear equations generated only on the epipolar line for each limited pixel set for all epipolar lines. This is a method suitable for parallel processing (SIMD; Single Instruction Multiple Data), which is good at many independent processes.

In other words, the conversion matrix generation unit 11 (see FIG. 4) stores a plurality of pieces of information indicating the positional relationship between the pattern images displayed on each of the plurality of projectors 1 and 2 and the pattern images projected at the plurality of positions. The matrix P′ having the pixels of the pattern images displayed on the projectors 1 and 2 as elements is limited to the epipolar lines of the projectors 1 and 2 and the pixels of the composite image projected at each of a plurality of positions are used as elements. A conversion matrix A′ that is converted into a matrix I′ is generated for the epipolar line. Here, the matrix P′ is a part of the matrix P of the above formula (3), the matrix I′ is a part of the matrix I of the above formula (3), and the conversion matrix A′ is a simultaneous linear equation. Is a part of the matrix A of Expression (3) having each coefficient of as an element. Furthermore, the pattern generation unit 12 executes the generation of the pattern image corresponding to the matrix P′ for all epipolar lines based on the matrix I and the conversion matrix A′ generated by the conversion matrix generation unit 11. Further, the pattern generation unit 12 synthesizes the pattern images of all epipolar lines and generates the synthesized image as an image to be projected for each projector.

As is clear from FIG. 20, in the above simultaneous equations, when the number of screens and the number of projectors are the same, the unknowns and the number of constraint equations are usually the same. However, since it is possible that no intersection is created at the edge of the screen, the unknown number may increase by one or two. Therefore, this equation always has a solution and, if there are many unknowns, has several degrees of freedom. By the way, if the simultaneous linear equations are solved without any restrictions on the solution, it is possible in calculation that the range becomes wider. However, since the brightness value that can be projected by the projector has a finite range, in order to actually project, it has to be normalized within the range of the brightness that can be projected. This reduces the contrast (gradation) of the projection pattern. As a result, the contrast of the pattern finally projected on the screen is also reduced. Therefore, when the pattern generation unit 12 solves the simultaneous equations, for example, the solution (the solution of the simultaneous equations, that is, the matrix I′) is set under the constraint that the range is 0 to 255 (the range of the pixel luminance value in the normal projector). If the solution of the matrix P′ based on the transformation matrix A′) is calculated, the contrast is not reduced. However, in that case, since the constraint number exceeds the unknown number, it is necessary to solve the optimization problem that minimizes the error between the final projection image and the projection target image in the calculation.

21A to 21F show an example of a projected image in which the target object is a plane. The images shown in FIGS. 21A and 21D are images generated without any restriction on the brightness value. In addition, the images shown in FIGS. 21B and 21E, 21C, and 21F are applied with a constraint on the luminance value by using the pattern generation method according to the eighth embodiment. It is an image generated by 21(A), FIG. 21(B) and FIG. 21(C) are images projected on a plane of depth 1, and FIGS. 21(D), 21(E) and 21(F) are depth 1 It is an image projected on a plane having a depth 2 different from. Comparing the image shown in FIG. 21(A), which is the image projected at the depth 1, with the images shown in FIGS. 21(B) and 21(C), FIG. It can be seen that the images shown in FIGS. 21B and 21C have a significantly larger dynamic range than the image shown in FIG. 21A, which is generated by limiting the brightness value.

The results of numerical evaluation of the projected image are shown in FIGS. 22(A) to 22(D). The horizontal axis represents a combination of two projected images. The combinations of the two images are Camerara/jetplane, Lena/Mandril, Lena/Peppers, Peppers/House, Lena/Cameraman, Peppers/Lena, respectively. The combination of Lena/Mandril is the image shown in FIGS. 21(A) to 21(F).

FIG. 22(A) (actual image) and FIG. 22(B) (composite image) show the evaluation results by PSNR (Peak Signal-to-Noise Ratio) which is often used when evaluating the image quality. Evaluation results by SSIM (Structural SIMilarity), which is said to be close to feeling, are shown in FIG. 22(C) (actual image) and FIG. 22(D) (composite image). Here, the actual image is an image obtained by actually projecting the projector on the screen, and the synthetic image is an image synthesized by simulating projection on a computer. Further, LF is an abbreviation of Linear Factorization without any restriction on the solution, and EO is an abbreviation of Epipolar Optimization with a restriction (a range of brightness values represented by [*,*]) added to the solution. Is. 22(A) to 22(D), the dynamic range is expanded by restricting the range of luminance values such as LF, EO[-100,255], and EO[0,255]. , You can see that the image quality has improved.

Ninth Embodiment
Next, a ninth embodiment of the invention will be described.

The pattern generation algorithm according to this embodiment is a method of simultaneously projecting different patterns on a plurality of arbitrarily shaped objects. A plurality of arbitrarily shaped objects may be prepared literally, or the same arbitrarily shaped object may be moved. The details of the pattern generation algorithm will be described below.

It is necessary to know which position of the object the projection light of each pixel on the projector irradiates in order to generate an arbitrary pattern by overlapping multiple patterns projected by each projector on an arbitrary shaped object. There is. In the present embodiment, the correspondence is obtained by projecting the gray code pattern from each projector. The gray code pattern is a pattern having positional information of pixels on the pattern image displayed on the projector. FIG. 23A, FIG. 23B, FIG. 24A, FIG. 24B, FIG. 25A, FIG. 25B, FIG. 26A, FIG. ), a plurality of striped patterns are projected on an arbitrarily shaped object, and vertical and horizontal map images (luminance) shown in FIGS. 23(C), 24(C), 25(C), and 26(C) are projected. By creating values (coordinates), correspondence (information indicating the positional relationship between the pattern image displayed on each projector and the pattern images projected at a plurality of positions) can be obtained.

The images finally projected on the arbitrarily shaped object are the images shown in FIGS. 27A and 27B. Further, the arbitrarily shaped object is the head of the human body model. Then, when the conversion matrix generation unit 11 projects the gray code image on the arbitrarily shaped object placed at a plurality of positions, each camera captures the projection pattern projected on the arbitrarily shaped object. Then, the conversion matrix generation unit 11 decodes the captured projection pattern. By doing so, information indicating the positional relationship between the pattern image displayed on each projector and the pattern images projected at a plurality of positions, that is, FIG. 28A, FIG. 28B, and FIG. ) And the map image shown in FIG. 29(B) are obtained. The conversion matrix generation unit 11 and the pattern generation unit 12 generate the pattern images projected by the projectors 1 and 2 by the method according to the present embodiment using the obtained information indicating the positional relationship. An example of the generated projection pattern projected by each projector is shown in FIGS. 30(A) and 30(B). By projecting this pattern image onto the arbitrary shape object from each of the two projectors 1 and 2, as shown in FIG. 31A, an arbitrary shape object placed at one depth is shown in FIG. 27A. An image corresponding to the image is projected, and as shown in FIG. 31B, the image corresponding to the image shown in FIG. 27B is projected on the arbitrary shape object placed at the other depth.

Note that, as shown in FIGS. 32A and 32B, the shape of the object onto which the pattern is projected may be changed according to the depth.

In each of the above-described embodiments, the case of using a projector that projects a grayscale image has been described, but the present invention is not limited to this. For example, a projector that projects a color image may be used. In this case, the background color of the pattern image is an intermediate color of a plurality of types of colors used for the pattern in the image.

In the projection system 100 of each of the above-described embodiments, since the camera is not used to control the projection image, it is possible to simplify the configuration of the entire system and prevent the system response from being delayed. However, the image data captured by the camera may be used for the geometric calibration and the color calibration. Even in this case, since the image data captured by the camera is not processed to actually project the pattern image, no delay occurs, the algorithm is simplified, and high-speed calculation is possible.

In addition, the hardware configuration and software configuration of the control unit 10 are examples, and can be arbitrarily changed and modified.

The central part of the processing of the control unit 10 can be realized by using a normal computer system instead of a dedicated system. For example, a computer program for executing the above operation is stored in a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.) and distributed, and the computer program is installed in the computer. The control unit 10 that executes the above process may be configured as described above. Alternatively, the control unit 10 may be configured by storing the computer program in a storage device included in a server device on a communication network such as the Internet, and downloading the program by an ordinary computer system.

When the function of the control unit 10 is realized by the sharing of the OS (operating system) and the application program or the cooperation of the OS and the application program, even if only the application program portion is stored in the recording medium or the storage device. Good.

It is also possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, the computer program may be posted on a bulletin board (BBS, Bulletin Board System) on the communication network, and the computer program may be distributed via the network. Then, the computer program may be started up and executed under the control of the OS in the same manner as other application programs so that the above-described processing can be executed.

The present invention allows various embodiments and modifications without departing from the broad spirit and scope of the present invention. Further, the above-described embodiments are for explaining the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is shown not by the embodiments but by the claims. Various modifications made within the scope of the claims and within the scope of the meaning of the invention equivalent thereto are regarded as within the scope of the present invention.

1, 2 and 3 projectors, 5 projector groups, 10 control unit, 11 conversion matrix generation unit, 12 pattern generation unit, 13 color calibration unit, 14 projection control unit, 40 measurement region, 50 obstacle, 51 measurement target, 100 Projection system, d1, d2 depth (position), I1, I2, I3, I4 screen (projected surface), P1, P2 pattern image, PL1, PL2 projection surface, L1, L2 projection light, LA1, LA2, LA3, LA4 , LA5 layer.

Claims (12)

Equipped with a plurality of projectors that are installed so that the irradiation areas of the projected projection light intersect,
The plurality of projectors are
By projecting the pattern images so that they are superimposed on each other, different composite images are formed at a plurality of positions depending on how the pattern images overlap ,
The composite image formed at each of the plurality of positions is
It is an image displaying a pattern indicating the position where the composite image is formed,
Projection system. The pattern image is
The background color is an intermediate color of the multiple types of colors that make up the pattern,
The projection system according to claim 1 . It is installed so that the irradiation areas of the emitted projection light intersect, and by projecting the pattern images so that they overlap each other, different composite images are formed at a plurality of positions depending on how the pattern images overlap. Multiple projectors,
A pattern image displayed on the plurality of projectors each, based on the information indicating the positional relationship between the projected pattern image on the plurality of positions, each pixel of the displayed pattern image in the plurality of projectors each element And a conversion matrix generation unit that generates a conversion matrix that converts the first matrix into a second matrix having each pixel of the composite image projected at each of the plurality of positions as an element,
Based on the second matrix corresponding to the composite image having a pattern to be displayed, and the transformation matrix generating unit The generated transformation matrix, generating a pattern image corresponding to the first matrix for each projector A pattern generation unit that
With
Projected shadow system. The pattern generation unit obtains a solution of the first matrix based on the second matrix and the conversion matrix under the constraint of the range of brightness of the generated pattern image,
The projection system according to claim 3 . The conversion matrix generation unit,
The pattern having the positional information of the pixel of the displayed pattern image to each projector, the projected onto the plurality of positions from the projector, images the projection shadow pattern projected on each of the plurality of positions, it is captured Obtaining information indicating the positional relationship by decoding the projection pattern,
The projection system according to claim 3 or 4 . The conversion matrix generation unit limits the information indicating the positional relationship to epipolar lines of the plurality of projectors, and generates the conversion matrix for the epipolar lines.
The pattern generation unit,
Generation of a pattern image corresponding to the first matrix is executed for all epipolar lines based on the second matrix and the conversion matrix generated by the conversion matrix generation unit, and the generated pattern image is generated. Generate a composite image as an image to be projected for each projector,
The projection system according to any one of claims 3 to 5 . A color calibration unit configured to project a color test pattern image on each of the projectors and to calculate color adjustment information of the pattern image based on an imaging result of the projected test pattern image,
A projection control unit configured to project the pattern image generated by the pattern generation unit onto each of the projectors based on color adjustment information adjusted by the color calibration unit;
With
The projection system according to claim 3 . Wherein said plurality of positions composite image is projected by placing a translucent screen, each translucent screen, projecting the different synthetic images,
A projection system according to any one of claims 1 to 7 . Installed multiple projectors so that the projected light emitted would cross each other and proceed while intersecting each other.
By causing the plurality of projectors to respectively project, with projection light, a plurality of pattern images each having at least one of a strengthening pattern and a weakening pattern when they are superimposed on each other, different composition depending on how the pattern images overlap at a plurality of positions. Forming an image ,
Based on the information indicating the positional relationship between the pattern image displayed on each of the plurality of projectors and the pattern images projected on the plurality of positions, each pixel of the pattern image displayed on each of the plurality of projectors is set as an element. Generate a conversion matrix for converting the first matrix into a second matrix having each pixel of the composite image projected on each of the plurality of positions as an element,
A pattern image corresponding to the first matrix is generated for each projector based on the second matrix corresponding to the composite image having a pattern to be displayed and the generated conversion matrix,
Projection method. A pattern generation method executed by a projection system including a plurality of projectors so that emitted projection light intersects with each other and travels while intersecting each other.
A pattern image displayed on the plurality of projectors each, based on the information indicating the positional relationship between the projected pattern image into a plurality of positions, said to each pixel of each projector each displayed pattern image elements a first matrix, the geometric calibration step of generating a transformation matrix for converting the second matrix with the pixels of the synthetic image that will be projected on the plurality of positions each element,
Said second matrix corresponding to the composite image having a pattern to be displayed, based on the transformation matrix generated by the geometric calibration process, generating a pattern image corresponding to the first matrix for each projector A pattern generation process for
A pattern generation method including. Before SL is projected test pattern image colors each projector, based on the imaging result of the projected test pattern image, the color calibration process of calculating a color adjustment information of the pattern image,
A projection step of the pattern image generated in the previous SL pattern generating step, based on the adjustment information of the adjusted color by the color calibration process, is projected to the each projector,
The pattern generation method according to claim 10 , which includes: On the computer,
Based on the information indicating the positional relationship between the pattern images displayed on each of the plurality of projectors and the pattern images projected on the plurality of positions, each pixel of the pattern image displayed on each of the projectors is set as an element. 1 matrix, and the geometric calibration procedure for generating a transform matrix for converting the second matrix with the pixels of the synthetic image that will be projected on the plurality of positions each element,
Said second matrix corresponding to the composite image having a pattern to be displayed, based on the transformation matrix generated by the geometric calibration procedure, generating a pattern image corresponding to the first matrix for each projector Pattern generation procedure,
A program to execute.

Images