WO2005084017A1

WO2005084017A1 - Multiprojection system

Info

Publication number: WO2005084017A1
Application number: PCT/JP2005/001337
Authority: WO
Inventors: Takeyuki Ajito; Kazuo Yamaguchi; Takahiro Toyama; Takafumi Kumano
Original assignee: Olympus Corporation
Priority date: 2004-02-27
Filing date: 2005-01-31
Publication date: 2005-09-09
Also published as: JP2005244835A

Abstract

A multiprojection system for projecting a single large image by patching images projected on a screen (2) by image projectors (1A, 1B). The multiprojection system comprises image acquiring means (3A, 3B) for capturing the images projected on the screen (2) by the image projectors (1A, 1B) from different positions their relative positional relationship of which is known and for acquiring parallax image data, image correction data calculating means (4) for estimating the three-dimensional position of each point of the image projected on the screen (2) on the basis of the acquired parallax image data and the information on the relative positional relationship and calculating correction data for correcting the images inputted into the image projectors (1A, 1B), and image correcting means (5) for correcting the images inputted into the image projectors (1A, 1B) according to the calculated correction data.

Description

Specification

Manorechi Projection System

Technical field

The present invention relates to a multi-projection system for forming one large image by pasting images projected on a screen by an image projection device such as a plurality of projectors, and in particular, a position shift of an image projected on a screen. The present invention relates to a multi-projection system in which distortion and color shift are detected by a digital camera and automatically corrected.

Background art

[0002] In recent years, in projectors, theaters and planetariums, and VR systems in museums and exhibitions, in order to construct large-screen, high-definition image display systems, multiple projectors are used to display images on the screen. A multi-projection system that realizes high-resolution image display on a large screen by pasting together is applied.

[0003] In such a multi-projection system, it is important to adjust the positional shift and color shift of the images by the individual projectors and to bond the images neatly on the screen. The system automatically detects and corrects image misregistration, distortion, and color misregistration from images captured by digital cameras.

[0004] As a conventional image correction method, for example, even when a screen having a curved surface such as a dome or an arch is used, a curved distortion due to the screen shape can be corrected by using a cylindrical transformation or a spherical transformation of an image. A method of correcting an image without distortion at a predetermined observation position is known (for example, see Patent Document 1).

[0005] Further, there is also known an arrangement in which a marker projector is arranged at a desired observation position, a marker is projected on a screen, and image correction is performed using the projected marker as a reference position (for example, Patent Document 2).

(Patent Document 1) Japanese Patent Application Laid-Open No. 2002-72359

(Patent Document 2) JP 2003-18503 [0007] In the image correction method disclosed in Patent Document 1 described above, an image taken by a digital camera is converted into a cylinder or a sphere according to the screen shape, so that distortion is accurately corrected. However, there is a restriction that the three-dimensional shape of the screen is known in advance, and that the digital camera must be installed at the same Cf position as the observation position.

[0008] On the other hand, in the image correction method disclosed in Patent Document 2, image correction is performed using a marker projected from the marker projector onto the screen as a reference position, so that the shape of the screen is not known. Even if the digital camera is not installed at the observation position, it is possible to correct the image without distortion at the observation position.

However, on the other hand, it is an essential condition to place the marker projector at the observation position. Therefore, if the equipment cannot be placed at the observation position due to the system configuration, the distortion cannot be accurately corrected. . In addition, when the observation position changes arbitrarily, it is necessary to install a marker projector at the observation position and project the marker on the screen each time. There is a problem that cannot be corrected.

[0010] Therefore, an object of the present invention made in view of the power is to correct the displacement and distortion of the projected image at an arbitrary observation position and the color shift in real time even if the shape of the screen is not known. It is an object of the present invention to provide a multi-projection system. Disclosure of the invention

[0011] The invention according to claim 1, which achieves the above object, provides a multi-projection system for forming one large image by combining images projected on a screen by a plurality of image projection devices,

Image acquisition means for acquiring an image projected on the screen by the image projection device from a different position having a known relative positional relationship to acquire parallax image data; and a parallax image acquired by the image acquisition means. Correction data for estimating the three-dimensional position of each point of the image projected on the screen based on the data and the information on the relative positional relationship, and correcting the image input to the image projection device Image correction data calculating means for calculating

Image correction means for correcting an image input to the image projection device based on the correction data calculated by the image correction data calculation means. It is.

[0012] The invention according to claim 2 is the multi-projection system according to claim 1, wherein the image acquisition means has two digital cameras installed at different positions having a known relative positional relationship. In addition, parallax image data is obtained by photographing an image projected on the screen by the two digital cameras.

[0013] The invention according to claim 3 is the multi-projection system according to claim 1, wherein the image acquisition unit has one digital camera and a moving mechanism that translates the digital camera in parallel. Moving the digital camera in parallel by the moving mechanism to capture parallax image data by capturing an image projected on the screen from a different position having a known relative positional relationship. It is.

[0014] The invention according to claim 4 is a multi-projection system for forming one large image by pasting images projected on a screen by a plurality of image projection devices,

Marker projecting means for projecting a marker at a predetermined angle on the screen, and an image projected on the screen by the image projection device and an image obtained by photographing the marker projected on the screen by the marker projecting means, respectively. Image acquisition means for acquiring data;

Support means for fixing a relative positional relationship between the marker projection means and the image acquisition means;

The three-dimensional position of each point of the image projected on the screen is estimated based on the image data acquired by the image acquiring means and the information on the relative positional relationship, and is input to the image projection device. Image correction data calculating means for calculating correction data for correcting an image to be corrected;

An image correcting means for correcting an image input to the image projection device based on the correction data calculated by the image correction data calculating means.

[0015] The invention according to claim 5 provides a multi-projection system for forming one large image by pasting images projected on a screen by a plurality of image projection devices. And

A plurality of image acquisition means for acquiring an image data by photographing an image projected on the screen by the image projection device from a location whose relative positional relationship with the image projection device is known,

The three-dimensional position of each point of the image projected on the screen is estimated based on the image data acquired by the plurality of image acquisition means and the information on the relative positional relationship, and the image projection device Image correction data calculating means for calculating correction data for correcting an image input to the

[0016] The invention according to claim 6 is a multi-projection system for forming one large image by pasting images projected on a screen by a plurality of image projection devices,

Different positional forces whose relative positional relationship is known; marker projecting means for projecting a marker on the screen;

Image acquisition means for taking an image projected on the screen by the image projection device and a marker projected on the screen by the marker projection means to acquire image data,

[0017] The invention according to claim 7 is the multi-projection system according to claim 6, wherein the two marker projection units are installed at different positions with a known relative positional relationship. Characterized in that it has a laser pointer.

[0018] The invention according to claim 8 is the multi-projection system according to claim 6, wherein the marker projecting means has one laser pointer and a moving mechanism that translates the laser pointer in parallel. The laser pointer is moved in parallel by the moving mechanism to project a marker onto the screen with a different positional force whose relative positional relationship is known.

[0019] In a ninth aspect of the present invention, in the multi-projection system according to any one of the first to eighteenth aspects, the correction data calculating means performs correction based on observer position information with respect to the screen. It is characterized by calculating data.

[0020] The invention according to claim 10 is the multi-projection system according to claim 9, further comprising a visual inspection sensor for detecting a viewpoint position of the observer and obtaining position information of the observer. It is characterized by the following.

Brief Description of Drawings

FIG. 1 is a diagram showing an overall configuration of a multi-projection system according to a first embodiment of the present invention.

FIG. 2 is a view showing a test pattern image input to the projector shown in FIG. 1.

FIG. 3 is a block diagram showing a configuration of a correction data calculation unit shown in FIG. 1.

FIG. 4 is a block diagram showing a configuration of an image conversion unit shown in FIG. 1.

FIG. 5 is a flowchart for explaining a geometric correction data calculation process in the first embodiment.

FIG. 6 is a view showing marker projection by the projector shown in FIG. 1 and marker imaging by two cameras.

FIG. 7 is a conceptual diagram showing a screen three-dimensional shape estimation method according to the first embodiment.

FIG. 8 is a conceptual diagram showing a method for estimating a screen shape with a small number of markers.

FIG. 9 is a diagram illustrating a state in which a marker projection image centering on the observation viewpoint is created from the estimated screen shape.

FIG. 10 is a diagram showing a state in which a marker projection image is created on a projection plane having a wide viewing angle centered on an observation viewpoint from an estimated screen shape. FIG. 11 is a diagram showing an overall configuration of a multi-projection system according to a second embodiment of the present invention.

FIG. 12 is a block diagram showing a configuration of a correction data calculation unit shown in FIG.

FIG. 13 is a diagram showing an overall configuration of a multi-projection system according to a third embodiment of the present invention.

FIG. 14 is a block diagram showing a configuration of a correction data calculation unit shown in FIG.

[15] This is a flowchart for explaining the geometric correction data calculation process in the third embodiment.

FIG. 16 is a conceptual diagram showing a screen shape estimation method in the third embodiment.

FIG. 17 is a diagram showing an overall configuration of a multi-projection system according to a fourth embodiment of the present invention.

FIG. 18 is a block diagram showing a configuration of a correction data calculation unit shown in FIG.

FIG. 19 is a diagram showing an overall configuration of a multi-projection system according to a fifth embodiment of the present invention.

FIG. 20 is a block diagram showing a configuration of a correction data calculation unit shown in FIG. 19.

FIG. 21 is a diagram illustrating an overall configuration of a multi-projection system according to a sixth embodiment of the present invention.

[22] FIG. 22 is a diagram for explaining an example of a screen shape calculation method according to the sixth embodiment.

FIG. 23 is a diagram showing an overall configuration of a multi-projection system according to a seventh embodiment of the present invention.

FIG. 24 is a diagram showing an overall configuration of a multi-projection system according to an eighth embodiment.

FIG. 25 is a diagram showing an overall configuration of a multi-projection system according to a ninth embodiment.

[26] FIG. 26 is a diagram for describing the screen shape estimation processing in the ninth embodiment.

FIG. 27 is a flowchart for explaining the same screen shape estimation process.

FIG. 28 is a flowchart for explaining the same geometric correction data calculation process. FIG. 29 is a block diagram illustrating a configuration of a correction data calculation unit illustrated in FIG. 25.

FIG. 30 is a diagram showing an overall configuration of a multi-projection system according to a tenth embodiment of the present invention.

FIG. 31 is a perspective view showing observation glasses used in the tenth embodiment.

FIG. 32 is a view showing a modification of the present invention.

FIG. 33 is a diagram showing details of each marker shown in FIG. 2 (a).

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of a multi-projection system according to the present invention will be described with reference to the drawings.

(First Embodiment)

Fig. 1 Fig. 10 and Fig. 33 show the first embodiment, Fig. 1 shows the overall configuration of a multi-projection system, and Figs. 2 (a) and (b) show test patterns input to the projector. FIG. 3 is a block diagram showing the configuration of the correction data calculation unit shown in FIG. 1, FIG. 4 is a block diagram showing the configuration of the image conversion unit shown in FIG. 1, and FIG. Flowchart for explanation, Fig. 6 shows marker projection by the projector and marker shooting by two cameras, Fig. 7 is a conceptual diagram showing the method of estimating the three-dimensional shape of the screen, and Fig. 8 is the screen with a small number of markers. Fig. 9 is a conceptual diagram showing the method of estimating the shape, Fig. 9 is a diagram showing how to create a marker projection image centered on the observation viewpoint from the estimated screen shape (marker position), and Figs. 10 (a) and (b) are Estimated Figure 33 (a) and (b) show how a marker projection image is created from a screen shape (marker position) on a wide viewing angle projection plane centered on the observation viewpoint.Figure 2 (a) FIG. 3 is a diagram showing details of each marker.

In the present embodiment, as shown in FIG. 1, an image is partially overlapped and projected onto a dome-shaped or arch-shaped screen 2 by projectors 1 A and IB, which are image projection devices, respectively. Thus, a single large image is displayed on the screen 2 by pasting the projected images together. In addition, the projectors as the image projection device include a transmission type liquid crystal projector, a reflection type liquid crystal projector, a DLP type projector using a DMD (digital micromirror element), a CRT projection tube display, and a laser projector. A can projector or the like can be used. Here, the images projected by the projector 1A and the projector 1B are not stuck together as they are due to the difference in the color characteristics of the projectors 1A and 1B and the displacement of the installation positions.

[0025] Therefore, in the present embodiment, first, test pattern image data is input to projector 1A and projector 1B, and a test pattern image is projected on screen 2, and the projected test pattern image is obtained by image acquisition means. A camera (digital camera) 3A, 3B captures an image of the test pattern. Note that the test pattern images projected here include marker images regularly arranged on the screen as shown in Fig. 2 (a) and the color characteristics of projectors 1A and 1B as shown in Fig. 2 (b). There are color signal images with different signal levels for each color of R (red), G (green), and B (blue) to acquire. Here, each marker shown in Fig. 2 (a) is point-symmetric on the XY space coordinates as shown in Fig. 33 (a), and as shown in Fig. 33 (b), the force is applied to the marker center. So that the signal value becomes higher. By using such a marker, the position of the center of the marker can always be detected with constant accuracy even if the marker image is significantly inclined or rotated due to the influence of the screen shape. Similar effects can be obtained when the camera is tilted or rotated. As a digital camera as an image acquisition means, a monochrome or multi-band digital camera can be applied, and a CCD or CMOS type image sensor can be applied.

These acquired test pattern images are sent to a correction data calculation section 4, where correction data for correcting the images input to the projectors 1A and 1B is calculated based on the shot images of the test patterns. . The calculated data is sent to the image conversion unit 5, where the correction data calculated by the correction data calculation unit 4 corrects the content image data input to the projectors 1A and 1B.

[0027] By inputting the content image data corrected as described above to the projector 1A and the projector 1B, a single image that is seamlessly bonded at the seam is displayed on the screen 2.

Here, the camera 3A and the camera 3B are fixed to the support member 6 at a predetermined distance d as shown in FIG. In addition, camera 3A and camera 3B can both use a wide-angle lens to capture the entire area of the screen, or a fish lens to provide a wider angle of view. Use an eye lens.

[0029] In this way, the parallax image of the test pattern can be obtained by photographing the test pattern images projected on the screen 2 with the cameras 3A and 3B at different viewpoint forces. Further, since the relative positional relationship between the viewpoints is known, it is possible to estimate the three-dimensional position of each point of the image projected on the screen 2 from the parallax image.

Next, with reference to FIG. 3, a detailed block of the correction data calculation unit 4 in the present embodiment will be described.

[0031] The correction data calculation unit 4 in the present embodiment includes a camera photographed image data storage unit 11A and a camera photographed image data storage unit 11B, a marker position detection storage unit 12, a screen shape / camera position estimation storage unit 13, an observation position. It has a setting unit 14, a marker position coordinate conversion unit 15, a projector geometric correction data calculation unit 16, a projector gamma correction data calculation unit 17, and a projector color correction matrix calculation unit 18.

[0032] The camera photographed image data storage unit 11A and the camera photographed image data storage unit 11B store photographed images of various test patterns photographed by the camera 3A and the camera 3B.

[0033] The marker position detection storage unit 12 detects the position of the marker projected from each of the test patterns captured by the cameras 3A and 3B on the captured image and projected by each projector on the captured image. Stores information.

[0034] The screen shape 'camera position estimation storage unit 13 estimates the three-dimensional position of each marker from the position information of each marker on the captured image corresponding to the camera 3A and the camera 3B. Calculate the 3D shape and the camera position with respect to the screen 2.

The observation position setting unit 14 stores the three-dimensional position information of the observation position set in advance or the observation position arbitrarily set by the user, and sends the information to the marker position coordinate conversion unit 15 in the subsequent stage.

In the marker position coordinate conversion unit 15, the three-dimensional marker position calculated in the screen shape′camera position calculation storage unit and the three-dimensional position information of the observation position set in the observation position setting unit 14 Based on the above, the two-dimensional coordinate position of the marker when the marker is projected on the two-dimensional plane with the observation position as the viewpoint is calculated.

[0037] In the projector geometric correction data calculation unit 16, the marker position coordinate conversion unit 15 creates Using the two-dimensional coordinates of the marker with the observed observation position as the viewpoint, the geometric coordinate relationship between the projection plane centered on the observation position and the image planes of the projectors 1A and 1B is derived, and the derived geometric coordinate relation Based on the calculated geometric correction data for correcting the displacement and distortion of the projector image, the calculated geometric correction data is output to the image conversion unit 5 at the subsequent stage.

[0038] The projector gamma correction data calculation unit 17 corrects color unevenness and gamma characteristic unevenness in the screens of the projectors 1A and 1B based on various color signal images captured by the camera 3A (or camera 3B). Gamma correction data is calculated, and the calculated gamma correction data is output to the image conversion unit 5 at the subsequent stage.

[0039] The projector color correction matrix calculator 18 calculates a color correction matrix for correcting a color difference between the projectors 1A and 1B based on various color signal images captured by the camera 3A (or the camera 3B). Is calculated, and the calculated color correction matrix is output to the image conversion unit 5 at the subsequent stage.

Next, with reference to FIG. 4, detailed blocks of the image conversion unit 5 in the present embodiment will be described.

The image conversion unit 5 is roughly divided into a correction data storage unit 21 and a correction data operation unit 22. The correction data storage unit 21 includes a geometric correction data storage unit 23, a gamma correction data storage unit 24, and a color correction matrix storage unit 25, and the geometric data calculated by the projector geometric correction data calculation unit 16 of the correction data calculation unit 4. The correction data is stored in the geometric correction data storage unit 23, the projector gamma correction data calculated in the projector gamma correction data calculation unit 17 is stored in the gamma correction data storage unit 24, and the color correction matrix calculated in the projector color correction matrix calculation unit 18 Are stored in the color correction matrix storage unit 25, respectively.

The correction data operation unit 22 includes a gamma conversion unit 26, a geometric correction data operation unit 27, a color correction matrix operation unit 28, a gamma correction unit 29, and a gamma correction data operation unit 30. In the correction data operation unit 22, first, the gamma conversion unit 26 corrects the nonlinear gamma characteristic of the input image (content image data), and then the geometric correction data operation unit 27 inputs the data from the geometric correction data storage unit 23. The geometric correction of the input image is performed using the geometric correction data for each projector. After that, the color correction matrix The color matrix conversion of the RGB signal of the input image is performed using the color correction matrix for each projector input from the positive matrix storage unit 24. Next, the gamma correction section 29 corrects the uniform gamma characteristics over the entire projector screen, and then the gamma correction data action section 30 executes the gamma correction based on the gamma correction data stored in the gamma correction data storage section 25. In addition, uniform gamma force deviation (difference) is corrected for each pixel of the projector screen. In this manner, the gamma correction unit 29 performs a rough gamma correction to some extent on the entire screen, and then, if the gamma correction data for each pixel used in the gamma correction data operation unit 30 can be provided as a difference, the correction data can be obtained. Since the amount of data and the amount of memory can be compressed, costs can be reduced.

As described above, the content image data corrected by the image conversion unit 5 is output to the subsequent projector 1A and projector 1B. The test pattern image is output to projector 1A and projector 1B without correction.

Next, with reference to FIG. 5, a description will be given of the geometric correction data calculation processing in the present embodiment.

First, in the image conversion unit 5, no correction is made to the input image !, a state (through state) is set (step S1), and FIG. The marker image shown in is input and displayed on projectors 1A and 1B (step S2). Thereafter, the images projected on the screen 2 by the projectors 1A and 1B are photographed by the cameras 3A and 3B respectively (steps S3 and S4), and the photographed parallax images are stored in the camera photographed image data storage unit 11A of the correction data calculation unit 4. , 11B. FIG. 6 shows this state, in which a marker is projected by one of the projectors 1B and a marker is photographed by the two cameras 3A and 3B.

Next, based on the captured parallax image, the marker position detection storage unit 12 detects a manual force position. Thereafter, the screen shape 'camera position calculation storage unit 13 detects the position of a marker point (corresponding point) on the detected captured image plane corresponding to the same marker between the parallax images (step S5), and the detected position is detected. As for the three-dimensional position force of a plurality of markers, the overall screen shape is estimated by interpolation and the camera position is estimated (step S6). Figure 7 illustrates this situation, with two cameras 3A, 3 This figure conceptually shows a method for estimating the three-dimensional shape of the screen 2 with the marker image captured by B. As shown in Fig. 8, when the number of marker points is small to some extent, the screen shape is estimated with the same accuracy as that estimated at many marker points by giving rough screen shape foresight information. It is also possible.

Next, the user sets the position where the projected image is actually observed (step S7).

The observation position may be, for example, the center position of the dome without being specified by the user, or may be determined in advance by default. Thereafter, based on the estimated three-dimensional position of the marker and the set observation position, the marker position coordinate conversion unit 15 firstly outputs the marker on the two-dimensional projection plane centered on the viewpoint at the observation position. Calculate the position coordinates (step S8). Figure 9 illustrates this situation. At this time, the angle of view of the projection plane is the same as the angle of view of the content image input to the multi-projection system. For example, if the content image is a wide-field image captured by a fish-eye lens, as shown in FIG. 10A, the projected image is also represented by a coordinate system with a wide viewing angle (for example, 110 degrees to 360 degrees). What is necessary is just to calculate the marker position coordinates on the two-dimensional projection plane. Further, as shown in FIG. 10B, when the observation position changes arbitrarily, the projection point at the observation position may be calculated.

Next, based on the calculated marker position coordinates, the projector geometric correction data calculation unit 16 calculates geometric correction data for each projector (step S9). Specifically, from the correspondence between the marker position coordinates on the viewpoint image plane at the observation position and the marker position coordinates on the test pattern image input to the projectors 1A and 1B, the coordinates on the viewpoint image plane and the positions on the projector image plane are obtained. The geometric relationship with the coordinates is obtained, and geometric correction data for correcting the input image is calculated based on the geometric relationship so that the image is output without any displacement or distortion on the viewpoint image plane. After that, the calculated geometric correction data for each projector is output to the image conversion unit 5 (step S10), and the processing of calculating the geometric correction data is terminated.

[0049] If the content image is geometrically corrected by the image conversion unit 5 using the correction data calculated as described above, an image with no distortion is displayed at the set observation position. It comes out. Regarding the gamma correction data and the color correction matrix, various color signal image data shown in FIG. 2 (b) is displayed by a projector as a test pattern image, and this is displayed on a camera in the same manner as in the case of a marker image. Photographs are taken with 3A and 3B, and the unevenness of each color characteristic and the gamma characteristic of projectors 1A and 1B are calculated from the photographed image data, and correction data for making these uniform over the entire screen is obtained.

In calculating these gamma correction data and color correction matrices, it is not necessary to take parallax images. Therefore, as the shooting data of various color signal images, either camera 3A or camera 3B can be used. All you have to do is take a picture. The color signal image is not limited to the case where the single color signal data of R (red), G (green), and B (blue) shown in Fig. 2 (b) is used as a test pattern. By using color signal data by color (gray scale) V, and by inserting filters with different spectral characteristics into cameras 3A and 3B at the same time, for example, camera 3A can convert the color signal image of blue component into camera 3B. , A color signal image of the red component can be obtained at the same time. As described above, if the color signal images are shared and obtained by the cameras 3A and 3B, the shooting time can be reduced.

(Second Embodiment)

11 and 12 show the second embodiment. FIG. 11 is a diagram showing the overall configuration of the multi-projection system, and FIG. 12 is a block diagram showing the configuration of the correction data calculation unit shown in FIG.

In the present embodiment, instead of the two cameras 3A and 3B used in the first embodiment, one camera (digital camera) is provided so as to be able to move in parallel to a moving stage 31 which is a moving mechanism. The camera 3 is supported, the camera 3 is moved in parallel, and the relative position is sequentially photographed at both ends of a known distance d, thereby obtaining a parallax image as in the first embodiment.

For this reason, the correction data calculation unit 4 is provided with a switching switch 32 for switching and supplying the image projected by the camera 3 to the camera photographed image data storage units 11A and 11B. When the camera is taken at the left end of the camera, the captured image is stored in the camera-captured image data storage unit 11A via the switch 32, and when the camera 3 is taken at the right end of the distance d, the image is taken. Image switching data via camera switch 32 Storage 1 Store in IB. Other configurations and operations are the same as those of the first embodiment.

According to the present embodiment, as compared with the first embodiment, the shooting time is increased by the sequential shooting while moving the camera 3, but the same correction can be realized with less equipment, and the cost can be reduced. Down can be achieved.

(Third Embodiment)

13 to 16 show the third embodiment. FIG. 13 is a diagram showing the overall configuration of the multi-projection system, and FIG. 14 is a block diagram showing the configuration of the correction data calculation unit shown in FIG. FIG. 15 is a flowchart for explaining the geometric correction data calculation processing, and FIG. 16 is a conceptual diagram showing a method for estimating a screen shape (marker position).

In the present embodiment, as shown in FIG. 13, instead of camera 3A and camera 3B in the first embodiment, one camera 3 projects markers at an equal angle over the entire screen. The screen shape is estimated using a laser pointer 35 as marker projection means. The camera 3 and the laser pointer 35 are fixed to a support member 36 as a support means, with their relative positional relationship fixed.

As shown in FIG. 14, in addition to the camera captured image data storage unit 11, the correction data calculation unit 4 includes a marker position detection storage unit 12, a screen shape 'camera position estimation storage unit as in FIG. 13, an observation position setting unit 14, a marker position coordinate conversion unit 15, a projector geometric correction data calculation unit 16, a projector gamma correction data calculation unit 17, and a projector color correction matrix calculation unit 18 The functions of the camera captured image data storage unit 11, the marker position detection storage unit 12, the screen shape 'camera position calculation storage unit 13, and the marker position coordinate conversion unit 15 are different from those of the first embodiment. The functions are the same as in the first embodiment.

[0059] That is, the camera photographed image data storage unit 11 stores the photographed image of the marker projected on the screen 2 by the laser pointer 35 and the test pattern of the marker image and the color signal image projected by the projectors 1A and 1B. The captured image is stored. In the marker one position detection storage unit 12, the position of the marker projected on the screen 2 by the laser pointer 35 and the position of the marker projected by the projectors 1A and 1B are respectively shown. From the captured image.

Further, the screen shape / camera position calculation storage unit 13 estimates the screen shape and the camera position from the marker position detected by the laser pointer 35. Further, in the marker position coordinate conversion unit 15, the projector detected in the marker position detection storage unit 12 from the estimated screen shape and camera position and the observation position set in the observation position setting unit 14. The marker position coordinates by 1A and 1B are converted to the marker position coordinates viewed from the observation viewpoint.

[0061] Based on the marker position coordinates of projectors 1A and 1B converted as described above, projector geometric correction data calculation unit 16 calculates geometric correction data in the same manner as in the first embodiment.

Hereinafter, the geometric correction data calculation processing according to the present embodiment will be described with reference to FIG. First, a marker is projected on the screen 2 by the laser pointer 35 (step S11), the projected marker image is photographed by the camera 3 (step S12), and stored in the camera photographed image data storage unit 11. Next, the marker images are projected onto the screen 2 by the projectors 1A and 1B (step S13), and the projected marker images are similarly photographed by the camera 3 (step S14). To be stored. Next, the marker position by the laser pointer 35 and the marker position by the projectors 1A and 1B are detected in the marker position detection storage unit 12 based on each of the captured marker images (steps S15 and S16). .

Next, based on the detected marker position by the laser pointer 35, the shape of the screen 2 and the position of the camera 3 are estimated in the screen shape / camera position calculation storage unit 13 (step S17). . Specifically, as shown in FIG. 16, the three-dimensional position of the marker is calculated from the projection angle of the marker by a predetermined laser pointer 35 and the position of the marker on the captured image, and the screen shape and the camera are calculated. Estimate the location.

Next, after setting the observation position (Step S18), the marker position coordinate conversion unit 15 first calculates the coordinate position of the laser pointer 35 on the projection image from the observation position as a viewpoint (Step S18). 19) Then, from the marker coordinate position at the observation position and the marker coordinate position on the camera image, the coordinate relationship between the observation position and the camera image (Step S20), and then, using the calculated coordinate relationships, the coordinate positions of the markers projected by the projectors 1A and 1B on the captured image are used as the projectors 1A and 1B on the projection plane with the observation position as the viewpoint. (Step S21).

Using the marker position information of the projector 35 at the observation position converted as described above, the projector geometric correction data calculation unit 16 calculates the geometric correction data for each projector as in the first embodiment (step S22). ), And outputs the calculated geometric correction data to the image conversion unit 5 (step S23), and performs image conversion of the input image based on the geometric correction data. This allows the projectors 1A and 1B to display the input image on the image without distortion while also viewing the observation position.

(Fourth Embodiment)

17 and 18 show a fourth embodiment of the present invention. FIG. 17 is a diagram showing the overall configuration of the multi-projection system, and FIG. 18 is a block diagram showing the configuration of the correction data calculation unit shown in FIG. FIG.

The present embodiment is different from the first embodiment in that, in addition to cameras 3A and 3B for acquiring a parallax image of the entire screen, a camera (digital Cameras) 3C and 3D are provided, and the geometric correction data for correcting the displacement and distortion of the projectors 1A and 1B is the same as in the first embodiment using parallax images taken by the cameras 3A and 3B. For the color correction matrices and gamma correction data of the projectors 1A and 1B, the color signal image powers captured by the cameras 3C and 3D are also calculated.

[0068] Here, in general, even if the resolution of the camera is low to some extent, it is possible to obtain the position shift and the distortion of the projector with a precision finer than the resolution of the camera in the subsequent marker position detection processing. A certain force It is difficult to detect the color unevenness that occurs at the pixel level of the projector with a precision smaller than the resolution of the camera.

Thus, in the present embodiment, color unevenness is corrected with finer accuracy by capturing a color signal image for detecting color unevenness in a small area on screen 2 by cameras 3C and 3D.

[0070] For this reason, as shown in Fig. 18, the correction data calculation unit 4 includes the camera photographed image data storage units 11A and 11B, the marker position detection storage unit 12, the screen shape "camera position" shown in Fig. 3. Estimation storage unit 13, observation position setting unit 14, marker position coordinate conversion unit 15, projector In addition to the geometric correction data calculation unit 16, camera image data storage units 11C and 11D and a projector color correction matrix / gamma correction data calculation unit 19 are provided.

[0071] Camera photographed image data storage units 11C and 11D store photographed images of test patterns of marker images and color signal images photographed by cameras 3C and 3D. In addition, the marker one position detection storage unit 12 detects the marker one coordinate position of the marker image captured by the cameras 3A and 3B, and also detects the marker coordinate position of the marker image captured by the cameras 3C and 3D. The detected marker positions in the captured images of the cameras 3C and 3D are supplied to the projector color correction matrix / gamma correction data calculation unit 19.

[0072] The projector color correction matrix 'gamma correction data calculation unit 19 stores the marker positions in the captured images of the cameras 3C and 3D detected by the marker position detection storage unit 12 and the camera captured image data storage units 11C and 11D. The color unevenness corresponding to each pixel of the projectors 1A and 1B is detected using the obtained color signal image, and a color correction matrix and gamma correction data for correcting the color unevenness are calculated, and the calculated color is calculated. The correction matrix and the gamma correction data are output to the image conversion unit 5 to convert the input image. As a result, color unevenness can be corrected with finer accuracy, and an image without color unevenness can be displayed on the screen 2. Note that the geometric correction data of the projectors 1A and 1B are the same as in the first embodiment.

(Fifth Embodiment)

19 and 20 show a fifth embodiment of the present invention. FIG. 19 shows the overall configuration of a multi-projection system, and FIG. 20 shows the configuration of the correction data calculation unit shown in FIG. It is a block diagram.

This embodiment is different from the third embodiment in that the camera 3 is rotatably supported by the support 36, and the camera 3 is rotated by the rotation control unit 41 for each small area on the screen 2. The test pattern images are sequentially photographed, and the correction data is calculated using the test pattern images photographed separately for each small area.

For this reason, in the present embodiment, rotation angle information corresponding to each image captured by the camera 3 is supplied from the rotation control unit 41 to the correction data calculation unit 4. Further, the correction data calculation unit 4 includes a rotation angle storage unit 20 for storing rotation angle information of the camera 3 from the rotation control unit 41. In addition, the rotation angle information corresponding to each captured image stored in the rotation angle storage unit 20 is supplied to the screen shape 'camera position calculation storage unit 13 and the marker position coordinate conversion unit 15, and the screen shape' camera position It is used in correspondence with each photographed image data when estimating and converting the observation position to the marker position coordinates, thereby calculating respective correction data in the same manner as in the third embodiment.

As described above, by calculating the correction data using the test pattern images shot separately for each of the small regions on the screen 2, correction with finer spatial accuracy can be performed as in the fourth embodiment. It becomes possible.

(Sixth Embodiment)

FIGS. 21 and 22 show a sixth embodiment of the present invention. FIG. 21 is a diagram showing the overall configuration of a multi-projection system, and FIG. 22 is an example of a screen shape calculating method according to the present embodiment. FIG.

This embodiment is different from the fifth embodiment in that the laser pointer 35 and the camera 3 are fixed to the support jig 36, and the support 36 is rotated by the rotation The marker projection by the laser pointer 35 and the imaging by the camera 3 are performed for each small area, and the correction data is calculated using the test pattern image photographed separately for each small area. Is the same as in the fifth embodiment. With such a configuration, the entire screen can be covered even when the marker projection angle by the laser pointer 35 is narrow and in a range, so that the configuration of the laser pointer ₃₅ can be simplified.

Here, when obtaining the entire shape of the screen 2 from the marker images taken for each small area of the screen 2, each image output from the rotation control unit 41 is obtained in the same manner as in the fifth embodiment. Using the rotation angle information corresponding to the shadow image, the shape of the entire screen can be estimated from the relative positional relationship between the rotation angles of the captured images.

Alternatively, without using the rotation angle information, for example, as shown in FIG. 22, the projectors 1A and 1B project the markers onto the overlapping portions of the photographing areas, and photograph the markers. Based on the marker positions (same point) of projectors 1A and 1B in the image, the screen shape estimated from each captured image may be synthesized. Yes. In this way, even if there is some error in the rotation angle information, the positional relationship can be accurately synthesized.

(Seventh Embodiment)

FIG. 23 is a diagram showing the overall configuration of the multi-projection system according to the seventh embodiment of the present invention.

In the present embodiment, a plurality of sets of a camera and a laser pointer whose positions are fixed to each other are used to overlap a small area of the screen 2 with an adjacent small area to form a test pattern image. The projection and its photographing are performed, and correction data is calculated using a test pattern image photographed separately for each of the small areas. The other configurations are the fifth and sixth embodiments. Is the same as Fig. 23 shows two sets, one with camera 3A and laser pointer 35A fixed to support 36A at a distance dl, and the other with camera 3B and laser pointer 35B fixed at support d at a distance d2. This shows the case where is used. Here, the distances dl and d2 can be set arbitrarily, even if they are equal or different.

With this configuration, a test pattern image for each small area of the screen 2 can be photographed simultaneously, so that the photographing time can be reduced and the correction data can be calculated in a short time. Note that the screen shape estimation according to the present embodiment estimates the shape of the small area of the screen 2 from each marker captured image as described in the sixth embodiment, and then synthesizes as shown in FIG. Then estimate the overall screen shape!

(Eighth Embodiment)

FIG. 24 is a diagram showing the overall configuration of the multi-projection system according to the eighth embodiment of the present invention.

[0085] The present embodiment uses the same number of cameras 3A and 3B as the projectors 1A and 1B without providing a plurality of sets of cameras and laser pointers as in the seventh embodiment. Is estimated. For this reason, the projector 1A and the camera 3A are fixed to the support 36A at a distance dl, the projector 1B and the camera 3B are fixed to the support 36B at a distance d2, and the screen 3 Projection area, that is, the image projected by projector 1A and the image projected by projector 1B overlapping it Part of the image can be captured so that the camera 3B can capture the projection area of the screen 2 by the projector 1B, that is, the image projected by the projector 1B and a part of the image projected by the projector 1A that overlaps it. . Note that the distances dl and d2 can be set arbitrarily or differently, as long as the cameras 3A and 3B can shoot the projection areas by the corresponding projectors 1A and 1B.

[0086] As described above, when estimating the screen shape, the marker images are projected by the projectors 1A and 1B and photographed by the cameras 3A and 3B, and the photographed images are combined with the projectors 1A and 1B and the cameras 3A and 3B. The screen shape is estimated using the information indicating the relative positional relationship between the two. Other configurations and operations are the same as those of the seventh embodiment.

According to the present embodiment, it is possible to estimate a screen shape and correct an image by using a simple configuration without using a laser pointer.

(Ninth Embodiment)

FIGS. 25 to 29 show a ninth embodiment of the present invention. FIG. 25 is a diagram showing the overall configuration of a multi-projection system, and FIGS. 26 (a) to (c) explain screen shape estimation processing. FIG. 27 is a flowchart for explaining the screen shape estimation process, FIG. 28 is a flowchart for explaining the geometric correction data calculation process, and FIG. 29 is a block diagram showing the configuration of the correction data calculation unit shown in FIG. FIG.

In the present embodiment, the two laser pointers 35A and 35B are fixed to the support 36 at a distance d, and different positional forces can be applied by the laser pointers 35A and 35B whose relative positional relationship is fixed. Markers are projected on the entire screen 2, and each projected marker is shot separately for each small area of the screen 2 by cameras 3A and 3B, and the shape of the screen 2 is estimated from these shot images. To do.

Hereinafter, a specific method for estimating the screen shape using the two laser pointers 35A and 35B and the cameras 3A and 3B will be described with reference to FIGS. 26 (a) and 26 (c). I do.

[0091] First, as shown in Fig. 26 (a), two laser pointers 35A and 35B are used, and markers are set at predetermined angles Θ Ak and Θ Bk (k = l-k, where k is the number of angle samples). Are projected at the same time, and this is photographed by the camera 3A. Next, for example, focusing on the i-th marker point projected by the laser pointer 35B, the marker point (k = j, j + 1) projected by the laser pointer 35A adjacent thereto is photographed by a camera. Extract from the image.

Next, one point (B) of the i-th marker of the laser pointer 35B on the captured image shown in FIG. 26 (b), and j and j + 1-th marker points (A, A) of the laser pointer 35A Ijj + 1 distance ratio (β Ζ α) of the laser pointer 35 Α j and j + 1st marker projection angles 0 Α and 0 、 Fig. 26 (c)

j j + l

As shown in the figure, the projection angle Θ'A when the marker is projected from the laser pointer 35A to the same position as the marker projection position by the laser pointer 35B is calculated by interpolation. For example, for linear interpolation, θ 'Α = (β / α) · (Θ Α — 0 A) + 0 Α

i j + l j j

Calculate. Further, when the screen shape is a curved surface, it can be obtained with higher accuracy by a higher-order interpolation formula.

[0094] Thereafter, using the calculated 0'A, the i-th projection angle 0A of the i-th power by the laser pointer 35B set in advance, and the distance d between the laser pointers 35A and 35B, the laser Calculate the three-dimensional position of the i-th marker by one pointer 35B. The screen shape is estimated by obtaining this processing for all points of i = l−1K.

FIG. 27 summarizes the above processing, and a detailed description thereof will be omitted because it is redundant with the description of the drawings. In order to estimate the screen shape more accurately, after calculating the projection angle Θ'A when projecting the marker from the laser pointer 35A to the same position as the marker projection position by the laser pointer 35B, the projection angle is actually calculated. When the marker is projected with the laser pointer 35A and the camera is shot again, as a result, if it is misaligned with the marker with the laser pointer 35B, it is corrected again and the more accurate projection angle with the laser pointer 35A is obtained. You can estimate!

FIG. 28 shows a process of calculating geometric correction data in the present embodiment, and a detailed description thereof will be omitted because it is the same as that in the drawings. Note that, in FIG. 28, the process indicated by reference symbol S corresponds to the process in FIG.

FIG. 29 is a diagram showing detailed blocks of the correction data calculation unit 4 in the present embodiment. This correction data calculation unit 4 is the correction data calculation unit of the third embodiment shown in FIG. 4, instead of the camera captured image data storage unit 11, a camera captured image data storage unit 11 A for storing captured image data from the camera 3A and a camera captured image data storage unit for storing captured image data from the camera 3B 11B is provided, and the other configuration is the same as that of the third embodiment.

[0098] In the present embodiment, the correction data calculation unit 4 uses the image for each small area of the screen 2 stored in the camera captured image data storage unit 11A and the camera captured image data storage unit 11B. In addition to calculating the geometric correction data by performing the processing shown in FIGS. 27 and 28, the color characteristic unevenness and the gamma characteristic unevenness of the projectors 1A and 1B are calculated in the same manner as described in the first embodiment. Then, a color correction matrix and gamma correction data that make these uniform over the entire screen are calculated, and the correction data is output to the image conversion unit 5.

[0099] According to the present embodiment, markers are projected onto the entire screen 2 from different positions by laser pointers 35A and 35B having a fixed relative positional relationship, and each projected marker is projected by a camera. 3A and 3B were used to take pictures of each small area of screen 2, and the shape of screen 2 was estimated from the captured images.Therefore, even if the positions of cameras 3A and 3B were not known, they were fixed. If not, the three-dimensional position of the screen 2 can be estimated, and the degree of freedom in installing the cameras 3A and 3B can be increased.

[0100] In the present embodiment, one laser pointer using two laser pointers 35A and 35B is supported on the moving stage so as to be able to move in parallel, and this one laser pointer is used. The correction data can be calculated in the same manner by projecting a marker on the entire screen 2 at both ends of a distance d whose relative position is known by moving in parallel.

[0101] (Tenth embodiment)

FIGS. 30 and 31 show a tenth embodiment of the present invention. FIG. 30 is a diagram showing the overall configuration of a multi-projection system, and FIG. 31 is a perspective view showing an observation scope used in the present embodiment. FIG.

[0102] In the present embodiment, observation position detection sensors 45 are provided at a plurality of positions on the screen 2, and the viewpoint position of the observer 46 is detected by the observation position detection sensors 45. The position is used as the observation position in the observation position setting unit 14 of the correction data calculation unit 4. A distortion is automatically corrected at the observation point based on the set observation position, and an image without distortion is displayed on the screen 2.

For this reason, in the present embodiment, the observer 46 is put on observation glasses 48 equipped with an infrared LED 47 as shown in FIG. 31, for example, and the observation position detection sensor 45 is an infrared detection sensor or the like. The observation position detection sensor 45 detects infrared rays from the infrared LED 47 to detect the viewpoint position of the observer 46. Note that the infrared rays emitted from the infrared LED 47 have directivity in the direction of the viewpoint of the observer 46.

[0104] In the correction data calculation unit 4 and the image conversion unit 5, the shape of the screen 2 and the projector using the camera may be set according to any one of the first to ninth embodiments. The projection position relationship between 1A and 1B is calculated in advance, and distortion correction at an arbitrary observation position is always performed.

With this configuration, the observation position can be detected in real time according to the movement of the observer 46 and automatically corrected to an image without distortion. Even in a display system or the like, an image can always be observed without distortion due to the screen shape.

[0106] The present invention is not limited to the above-described embodiment, but can be variously modified or changed. For example, in the above-described embodiment, a curved screen such as an arch screen or a spherical screen in which all 360 ° directions are covered with screens is used to project and display an image on the hemispherical dome screen 2. When projecting and displaying an image, as shown in Fig. 32 (a), when projecting an image by front projection on a flat screen 2a, or as shown in Fig. 32 (b), a flat screen The present invention can also be effectively applied to the case where an image is displayed by rear projection on 2a. Also, the number of projectors is not limited to two, and the present invention can be applied to a case where there are three or more projectors.

Industrial applicability

According to the present invention, the three-dimensional position of each point of the image projected on the screen, that is, the screen position 'shape is estimated, and the positional deviation and distortion of the projected image and the color deviation are corrected. Therefore, an image can be displayed well even if the shape of the screen is not known. Once the image data has been acquired by the image acquisition means, Can freely set the observation position and change the distortion correction, so that it is possible to correct the change in the observation position in real time.

Claims

The scope of the claims

[1] In a multi-projection system in which images projected on a screen by a plurality of image projection devices are combined to form one large image,

A multi-projection system comprising: an image correction unit that corrects an image input to the image projection device based on the correction data calculated by the image correction data calculation unit.

[2] The image acquisition means has two digital cameras installed at different positions with a known relative positional relationship, and captures an image projected on the screen by the two digital cameras. 2. The multi-projection system according to claim 1, wherein parallax image data is acquired.

[3] The image acquisition means has one digital camera and a moving mechanism for translating the digital camera in parallel, and the digital camera is translated by the moving mechanism so that the relative positional relationship is known. 2. The multi-projection system according to claim 1, wherein parallax image data is obtained by photographing an image projected on the screen from a different position.

[4] In a multi-projection system in which images projected onto a screen by a plurality of image projection devices are combined to form one large image,

To fix the relative positional relationship between the marker projection means and the image acquisition means Support means of

[5] In a multi-projection system in which images projected on a screen by a plurality of image projection devices are combined to form one large image,

[6] In a multi-projection system for forming one large image by pasting images projected on a screen by a plurality of image projection devices,

The image data acquired by the image acquiring means and the information on the relative positional relationship. Image correction data calculating means for estimating the three-dimensional position of each point of the image projected on the screen based on the information and calculating correction data for correcting the image input to the image projection device. When,

7. The multi-projection system according to claim 6, wherein the marker projecting means has two laser pointers installed at different positions whose relative positional relationship is known.

[8] The marker projecting means has one laser pointer and a moving mechanism for moving the laser pointer in parallel. The moving mechanism moves the laser pointer in parallel so that the relative positional relationship is known. 7. The multi-projection system according to claim 6, wherein a marker is projected on the screen.

9. The multi-projection system according to claim 11, wherein the correction data calculation means calculates correction data based on position information of an observer with respect to the screen.

10. The multi-projection system according to claim 9, further comprising a viewpoint detection sensor for detecting a viewpoint position of the observer and obtaining position information of the observer.