JP4751084B2 - Mapping function generation method and apparatus, and composite video generation method and apparatus - Google Patents

Description

  The present invention relates to a mapping function generation method and a composite image generation method used for parallel image generation by computer graphics, to these devices, and to applications thereof.

A high resolution and a wide field of view are required to obtain an immersive and realistic feeling in training simulators and research simulators. As a method for realizing this with good cost performance, a projector mosaic or a multi-projection technique is anticipated in which images with reverse distortion are projected from a number of projectors onto a flat screen and integrated into a continuous image. Conventionally, a method of optically correcting distortion using an electronic circuit or a lens has been widely used. However, there are many problems such as a fixed screen size and complicated adjustment when a projector is installed. In addition, since multiple screens are used to form a large screen, an expensive projector with a dedicated circuit that relaxes the boundary is required, and a great deal of time and effort is required for alignment, adjustment of projector characteristics, and brightness shift due to aging. Cost.
Further, as shown in Non-Patent Document 1, there are three lines of an image projected from a projector j (1... N) whose installation position is unknown and a distortion image on the screen imaged by a camera whose installation position is unknown. There is a matrix that is associated with a matrix of three columns and uses this matrix to apply reverse distortion to the image of the projector coordinates to correct the distortion of the image, but this is limited to a flat screen.
Information Processing Society of Japan, “FIT2003 Information Science and Technology Forum General Lecture Collection Vol. 3”, published on August 25, 2003, p. 401-403

  The present invention provides a method and means for facilitating the generation of real-time video by displaying a high-resolution large screen on a quadratic curved screen such as a dome using a plurality of relatively low-resolution projects for projecting partial images. It is to provide.

In order to solve the above-described problem, a mapping function generation method according to claim 1 includes a quadratic curved screen, a plurality of n projectors positioned in front of the screen and arranged to project onto the screen, and in front of the screen. A camera, a computer, and a storage device, which are positioned so as to photograph a screen, and is a process by the computer, which is a transformation matrix (M) from a reference coordinate system to a viewpoint coordinate system, and camera coordinates By performing matrix calculation on the transformation matrix (S) from the reference coordinate system to the reference coordinate system, the transformation matrix (from the viewpoint coordinate system to the camera coordinate system) for a predetermined point of the test pattern projected from the projector onto the quadric curved screen ( a first step of obtaining H), using an inverse matrix of the transformation matrix (H) and the transformation matrix (H), quadrics parameter determined in the camera coordinate system By performing the matrix calculation to data (Q), a second process of obtaining a quadric surface parameter (Qv) in the viewpoint coordinate system assumptions viewpoint from quadric parameters determined by the camera coordinate system (Q) as the origin And using the quadric surface parameter (Qv) given by the matrix under the constraint that the test pattern is on the quadric surface, the points projected on the screen by the matrix calculation are obtained from the projector i coordinate system. a point viewed, a third process of obtaining a matrix mapping function that indicates the correspondence between a point as viewed from an arbitrary assumption viewpoint, using the mapping function given by a matrix, the more the inverse matrix calculation A fourth process to be obtained, and a first process to a fourth process are repeated n times for the plurality of projectors to obtain an inverse function of a mapping function for each projector, and stored in the storage device. And extent and is characterized in that it consists of.

  In order to solve the above-mentioned problem, the composite image generating method according to claim 2 is such that the quadratic curved screen and the regions projected in front of the screen and overlapping each region have overlapping portions and are continuous. A plurality of n projectors arranged, a computer provided for each of the projectors, and a storage device, and each computer generates an image for each area projected by each projector in a pixel buffer, and in advance. A first process of generating, in a pixel buffer, an image obtained by correcting the distortion of the generated image using the inverse function of the mapping for each projector obtained in step 2, and the quadratic curved surface It consists of the 2nd process of projecting on a screen.

In order to solve the above-mentioned problem, the composite image generating method according to claim 3 is continuous so that the quadric curved screen and the areas projected in front of the screen and overlapping each area have overlapping portions. A plurality of n projectors arranged; a camera located in front of the screen and arranged to photograph the screen; a computer; and a storage device. By performing matrix calculation on the transformation matrix (M) to the coordinate system and the transformation matrix (S) from the camera coordinates to the reference coordinate system, predetermined points of the test pattern projected from the projector onto the quadric surface screen a first step of obtaining transformation matrices from the viewpoint coordinate system to the camera coordinate system (H), by using an inverse matrix of the transformation matrix (H) and the transformation matrix (H), Ca By performing the matrix calculation to quadric parameters determined La coordinate system (Q), an origin either assumptions perspective right or left from the quadric parameters determined by the camera coordinate system (Q) A quadratic surface parameter (Qv) is obtained in the viewpoint coordinate system of either the right eye or the left eye and the viewpoint coordinate system of the right eye or the left eye with the other assumed viewpoint as the origin. and 2 processes, by using under the constraint of the obtained right and left eyes of the viewpoint coordinate quadric parameters given by a matrix of system (Qv) of the test pattern is on a quadric surface, the matrix calculation the obtaining a point viewed points projected on the screen from the projector i coordinate system, the mapping function that indicates the correspondence between a point as viewed from an arbitrary right and left eyes of the assumptions viewpoint as a matrix Comprising the steps of, using said mapping function given by a matrix, and a fourth process of obtaining a more thereof inverse matrix calculation, the fourth step from the first step is repeated n times of the multi-projector, And a fifth step of obtaining an inverse function of the mapping function for the right eye and the left eye for each projector and storing it in the storage device.

  In order to solve the above-mentioned problem, the composite image generation method according to claim 4 is continuous so that the quadric curved screen and the regions projected in front of the screen and overlapping each region have overlapping portions. A plurality of n projectors arranged; a computer provided for each projector; and a storage device. The computers generate right-eye and left-eye images for each projector in a pixel buffer and request in advance. A first process of generating, in a pixel buffer, an image obtained by correcting distortion of the generated image by using an inverse function of right eye mapping and left eye mapping for each projector obtained in item 3, and distortion in each pixel buffer from each projector A second step of projecting the corrected right-eye and left-eye images onto the quadric curved screen; And it is characterized in and.

  In order to solve the above-mentioned problem, the composite image generation method according to claim 5 is arranged to have a quadratic curved screen and a portion overlapping in common with an area projected in front of the screen and projected onto the screen. A plurality of n projectors, a computer, and a storage device, wherein the computer generates an image for each area projected by each projector, and the inverse function of the mapping for each projector previously obtained in claim 1 A first process of generating a distortion-corrected image of the generated image in a pixel buffer; and a brightness of a common overlapping portion by projecting the distortion-corrected image from each projector onto the quadric curved screen. It consists of the 2nd process which enlarges thickness.

  In order to solve the above-mentioned problem, the composite image generating method according to claim 6 is continuous so that the quadratic curved screen and the regions projected in front of the screen and overlapping each region have overlapping portions. Each of the projectors obtained in advance according to claim 1, comprising: a plurality of n projectors arranged; and a video device, wherein the images from the video devices are divided and generated so as to be projected onto the screen onto the plurality of n projectors. And a second process of projecting the distortion-corrected video image from each projector onto the quadric curved screen. It is characterized by.

  In order to solve the above-mentioned problem, the composite image generating method according to claim 7 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. Each of the projectors obtained by dividing and generating a plurality of n projectors arranged and a video device so as to project an image from the video device onto the screen onto the plurality of n projectors. A first process of generating a distortion-corrected image of the divided and generated image using an inverse function of right-eye and left-eye mapping, and the distortion-corrected right-eye and left-eye images from each projector It consists of the 2nd process of projecting on a curved screen.

  In order to solve the above problem, the composite image generating method according to claim 8 is arranged to have a quadratic curved screen and an overlapping portion located in front of the screen and having a common area projected on the screen. A plurality of n projectors and video devices, wherein the video from the video devices is divided and generated so as to be projected onto the screen onto the n projectors, and mapping for each projector previously obtained in claim 1 is performed. A first process of generating a distortion-corrected image of the divided and generated image by using an inverse function, and a brightness of a common overlapping portion by projecting the distortion-corrected image from each projector onto the quadric curved screen It consists of the 2nd process which enlarges thickness.

  In order to solve the above problem, a composite video generation method according to claim 9 is the method according to claim 6 or claim 7 or claim 8, wherein the video equipment is a television device, a video playback device, or a personal computer. It is characterized by this.

  In order to solve the above-mentioned problem, a composite video generation method according to claim 10 is the one according to claim 2, claim 4, claim 5, claim 6, claim 7, claim 8, or claim 9. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged to project onto the screen, a computer, and a camera located in front of the screen and arranged to photograph the screen; A first process of shooting with a camera in a state in which none of the projectors projects onto the screen, and shooting a black image from the projector onto the screen and shooting with a camera, and a plurality of n The second process of the projector repeated n times and the brightness of the image for each projector by the second process in the computer To obtain the maximum value by the sum of the average value of the screen by each projector and the average value based on the difference from the luminance of the image obtained by the first process, and obtain the offset value by the difference between the maximum value and the average value. And a fourth process of generating an image by lowering the contrast of the overlapping portion of the image of each projector by the offset value and increasing the luminance by the offset value in the computer.

In order to solve the above-mentioned problem, the composite image generation method according to claim 11 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. A plurality of n projectors arranged, a polarizing plate provided in front of each projector, a computer provided for each projector, and a storage device, and an image for each area projected by each projector by each computer Is generated in the pixel buffer, and a first process of generating, in the pixel buffer, an image obtained by correcting the distortion of the generated image according to the inverse function of the mapping for each projector previously obtained in claim 1; a second step of projecting the image to distortion correction in the secondary curved screen in the buffer, distortion To represent a night scene with an image obtained by correcting the one in which an image obtained by the distortion correction to be projected from the projectors, characterized in that it consists of a third process of dimmed by the polarizing plate.

In order to solve the above-described problem, a mapping function generation device according to a twelfth aspect includes a quadric curved screen, a plurality of n projectors disposed in front of the screen and arranged to project onto the screen, and in front of the screen. A camera, a computer, and a storage device, which are positioned so as to capture a screen, a conversion matrix (M) from a reference coordinate system to a viewpoint coordinate system, and conversion from a camera coordinate to a reference coordinate system Conversion matrix calculation for obtaining a conversion matrix (H) from the viewpoint coordinate system to the camera coordinate system for a predetermined point of the test pattern projected from the projector onto the quadric surface screen by performing matrix calculation on the matrix (S) and means, using the inverse matrix of the transformation matrix (H) and the transformation matrix (H), a matrix calculation with respect to quadric parameters determined by the camera coordinate system (Q) By performing a quadric surface parameter (Qv) calculating means for calculating the quadric surface parameter (Qv) in the viewpoint coordinate system with its origin at the hypothetical viewpoint from quadric parameters determined by the camera coordinate system (Q), a matrix Using the given quadric surface parameter (Qv) under the constraint that the test pattern is on the quadric surface, the point projected on the screen by matrix calculation is the point seen from the projector i coordinate system; A mapping function and an inverse function calculating means for obtaining a mapping function indicating a correspondence with a point viewed from an arbitrary hypothetical viewpoint as a matrix and obtaining the inverse function by matrix calculation using the mapping function given by the matrix ; It comprises a storage device for storing a mapping function and its inverse function for each of the plurality of projectors.

  In order to solve the above-mentioned problem, the composite image generating apparatus according to claim 13 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. 13. A plurality of n projectors arranged, a computer provided for each projector, and a storage device, wherein each computer generates an image for each area projected by each projector in a pixel buffer, and is preliminarily defined. And an image generation means for generating an image obtained by correcting distortion of the generated image in a pixel buffer according to an inverse function of the mapping for each projector obtained in the above, and each distortion corrected image from the pixel buffer of each computer A plurality of projectors that project onto a curved screen. .

In order to solve the above-described problem, the mapping function generation device according to claim 14 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. A plurality of n projectors arranged; a camera arranged in front of the screen to shoot a screen; a computer; and a storage device, and a conversion matrix (from a reference coordinate system to a viewpoint coordinate system) M) and a matrix calculation for the transformation matrix (S) from the camera coordinates to the reference coordinate system, a predetermined point of the test pattern projected from the projector onto the quadric surface screen is determined from the viewpoint coordinate system to the camera coordinates. a transformation matrix calculation means for obtaining the transformation matrix to systems with (H), by using an inverse matrix of the transformation matrix (H) and the transformation matrix (H), determined in the camera coordinate system By performing the matrix calculation for the next surface parameter (Q), either from the quadric parameters determined by the camera coordinate system (Q) of the right or left as the origin either assumptions perspective right or left Quadratic surface parameter (Qv) calculation for obtaining a quadric surface parameter (Qv) in one of the viewpoint coordinate systems of the right eye or the left eye with the origin point of either one of the viewpoint coordinate system and the right eye or the left eye as the origin. The screen is calculated by matrix calculation using a quadratic surface parameter (Qv) given by a matrix in the viewpoint coordinate system of the right eye and the left eye obtained under the constraint that the test pattern is on the quadric surface determined and in that the points projected viewed from the projector i coordinate system, the mapping function that indicates the correspondence between a point as viewed from an arbitrary right and left eyes of the assumptions viewpoint as matrix And use the mapping function given by a matrix, the matrix and the mapping function and the inverse function calculating means for calculating the inverse function by calculation, storage for the mapping function and the inverse function thereof for the right eye and the left eye for each of the plurality projectors It consists of an apparatus.

  In order to solve the above-mentioned problem, the composite image generating apparatus according to claim 15 has a quadratic curved screen and a continuous portion having a portion where the regions projected in front of the screen overlap each other. A plurality of n projectors arranged; a computer provided for each projector; and a storage device. The computers generate right-eye and left-eye images for each projector in a pixel buffer and request in advance. According to the inverse function of the mapping for the right eye and the left eye for each projector obtained in item 14, image generation means for generating an image obtained by correcting the generated image in a pixel buffer, and distortion correction from each projector in the pixel buffer A plurality of n projectors that project right-eye and left-eye images onto the quadric curved screen. And it is characterized in that comprising the data.

  In order to solve the above-described problem, the composite image generation device according to claim 16 is arranged to have a portion that overlaps a quadric curved screen and a region that is located in front of the screen and that is projected onto the screen in common. 13. A projector having a plurality of n projectors, a computer, and a storage device, wherein each computer generates an image for each region projected by each projector in a pixel buffer and reverses the mapping for each projector previously obtained in claim 12. A function for generating distortion-corrected video in the pixel buffer by a function, and projecting each distortion-corrected video from the pixel buffer of each computer onto the quadric curved screen From a plurality of n projectors arranged to increase the brightness of overlapping portions And it is characterized in Rukoto.

  In order to solve the above problem, the composite image generating apparatus according to claim 17 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. 13. Each projector having a plurality of n projectors arranged and a video device, each of which is divided and generated so that an image from the video device is projected onto the screen onto the plurality of n projectors. An image generation means for generating a distortion-corrected image of the divided and generated image, and a plurality of n projectors for projecting the distortion-corrected images onto the quadric curved screen. It is a feature.

  In order to solve the above-mentioned problem, the composite image generating apparatus according to claim 18 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. 15. Each projector having a plurality of n projectors arranged and a video device is divided and generated so that an image from the video device is projected onto the screen onto the plurality of n projectors. Image generating means for generating a distortion corrected image of the divided and generated image by an inverse function of right eye and left eye mapping, and projecting the distortion corrected right eye and left eye images on the quadric curved screen The projector includes a plurality of n projectors.

  In order to solve the above problem, the composite image generating device according to claim 19 is arranged to have a quadratic curved screen and an overlapping portion located in front of the screen and having a common area projected on the screen. 13. A projector having a plurality of n projectors and a video device, wherein the video from the video device is divided and generated so as to be projected onto the screen onto the plurality of n projectors, and mapping for each projector previously obtained in claim 12 is performed. By using an inverse function, image generation means for generating an image obtained by correcting distortion of the divided and generated image, and projecting the image subjected to distortion correction onto the quadric curved screen to increase the brightness of the common overlapping portion. It is characterized by comprising a plurality of n projectors arranged as described above.

  In order to solve the above problem, the composite video generation device according to claim 20 is the one according to claim 17 or claim 18 or claim 19, wherein the video equipment is a television device, a video playback device, or a personal computer. It is characterized by.

  In order to solve the above-mentioned problem, a composite video generation device according to claim 21 is the one according to claim 9 or claim 11 or claim 12 or claim 17 or claim 18 or claim 19 or claim 20. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged to project on the screen, and a camera arranged in front of the screen and arranged to photograph the screen A camera that shoots a screen in a first state that is not projected from any of the projectors, and shoots a second state of the screen that projects a black state from each of the projectors, and the second that is captured by the camera. Each projector has a difference between the brightness of the image projected for each projector and the brightness of the image captured for the first state. Offset calculating means for obtaining a maximum value by a sum of the average value of the screen by the projector and the average value and obtaining an offset value by a difference between the maximum value and the average value, and for each overlapping portion of the image of each projector by the computer It is characterized by comprising image generation means for generating an image by decreasing the contrast by the offset value and increasing the luminance by the offset value.

In order to solve the above-described problem, the composite image generating apparatus according to claim 22 is continuous so that a quadric curved screen and a region projected in front of the screen and overlapping each region have overlapping portions. A plurality of n projectors arranged, a polarizing plate, a computer provided for each projector, and a storage device, and each computer generates an image of each area projected by each projector in a pixel buffer; 13. Image generation means for generating, in a pixel buffer, an image obtained by correcting the distortion of the generated image according to an inverse function of the mapping for each projector previously obtained in claim 12, and each image after distortion correction from the pixel buffer of each computer A plurality of projectors that project the image onto the quadric curved screen, and the front of each projector The provided and is characterized by comprising a polarizing plate for dimming the quantity of light from the projector to represent a night scene.
Composite video generator.

  In order to solve the above-mentioned problem, the composite image generating apparatus according to claim 23 is characterized in that each projector is attached to the base by a flexible arm having a degree of freedom. It is what.

  In order to solve the above-mentioned problems, a composite video generator according to claim 24 is the one according to claim 13 or claim 15 to 22, wherein each projector is attached to the base by a flexible arm having a degree of freedom. It is characterized by that.

  According to the mapping function generation method according to claim 1, the shape of the curved surface at the center as the reference point can be determined by using the assumption that the surface is a quadric surface. The quadric surface parameter (Qv) in the viewpoint coordinate system can be obtained mathematically, the accuracy can be improved because it does not involve distance calculation, and it can be realized at low cost without using a special device. it can.

  According to the composite video generation method according to claim 2, since the video to be projected onto one quadratic curved screen is generated and projected by a plurality of projectors and a computer equipped therewith, the resolution can be increased, Detailed portions can be displayed, and the processing of the computer provided in the plurality of projectors can reduce the rendering load by rendering different videos. Further, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved without depending on the resolution of the database video as in the past. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the mapping function generation method according to claim 3, the shape of the curved surface at the center as the reference point can be determined by using the assumption that the surface is a quadric surface, and thereby the assumed viewpoint is the origin. The quadric surface parameters (Qv) for the right eye and left eye in the viewpoint coordinate system can be mathematically obtained, and accuracy is improved because no distance calculation is performed, and no special device is used. It can be realized at low cost.

  According to the composite video generation method according to claim 4, since the video to be projected on one quadric curved screen is generated and projected by a plurality of projectors and a computer equipped with them, the resolution can be increased, A detailed portion can be displayed, and the processing of the computer provided in the plurality of projectors can reduce the rendering load by rendering the video of different areas for the right eye and the left eye, respectively. Further, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved without depending on the resolution of the database video as in the past. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the composite image generation method of the fifth aspect, since the brightness of the projection area is improved by using several projectors and overlapping the projection areas, the necessary brightness can be obtained without using an expensive projector. In addition, the brightness of the projection area can be adjusted by changing the number of projectors, and the brightness at a fixed position can be improved by combining the projector overlay and distortion correction.

  According to the composite video generation method according to claim 6, since the video to be projected onto one quadric curved screen is generated and projected by the video equipment, the resolution can be increased, and a fine portion of the video is displayed. In addition, the video processing of the video equipment can reduce the rendering load by rendering a different video for each divided projector. In addition, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the composite video generation method according to claim 7, since the video to be projected onto one quadric curved screen is generated and projected by the video equipment, the resolution can be increased and a fine portion of the video can be displayed. In addition, the video processing of the video equipment can reduce the rendering load by rendering the divided video for each projector for the right eye and the left eye. In addition, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the composite video generation method according to the eighth aspect, since the brightness of the projection area is improved by using several projectors and overlapping the projection areas, the necessary brightness can be obtained without using an expensive projector. In addition, the brightness of the projection area can be adjusted by changing the number of projectors, and the brightness at a fixed position can be improved by combining the projector overlay and distortion correction.

  According to the composite video generation method of the ninth aspect, it can be easily realized by using a household device such as a television device, a video playback device, a personal computer, or the like as the video device.

  According to the composite video generation method according to the tenth aspect, it is possible to reduce the conspicuousness of the luminance in the overlapping portion of the projector projection area in a state where black is produced, and without using a special blending apparatus, for example, a graphic It can be realized using the function of the board.

  According to the composite image generation method according to the eleventh aspect, since it is possible to perform the dimming step by step, the optimum dimming rate can be adjusted, so that the overlapping portions of the projection areas by the projectors become less noticeable. It becomes possible.

  According to the mapping function generation device according to the twelfth aspect, it is possible to determine the shape of the curved surface at the center as the reference point by using the assumption that the surface is a quadratic curved surface, and thus the assumed viewpoint is the origin. The quadric surface parameter (Qv) in the viewpoint coordinate system can be obtained mathematically, the accuracy can be improved because it does not involve distance calculation, and it can be realized at low cost without using a special device. it can.

  According to the composite video generation device of the thirteenth aspect, since the video to be projected onto one quadric curved screen is generated and projected by a plurality of projectors and a computer equipped therewith, the resolution can be increased, Detailed portions can be displayed, and the processing of the computer provided in the plurality of projectors can reduce the rendering load by rendering different videos. Further, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved without depending on the resolution of the database video as in the past. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the mapping function generation device of the fourteenth aspect, it is possible to determine the shape of the curved surface at the center as the reference point by using the assumption that the surface is a quadratic curved surface, and thus the assumed viewpoint is the origin. The quadric surface parameters (Qv) for the right eye and left eye in the viewpoint coordinate system can be mathematically obtained, and accuracy is improved because no distance calculation is performed, and no special device is used. It can be realized at low cost.

  According to the composite image generation device of the fifteenth aspect, since the image to be projected onto one quadric curved screen is generated and projected by a plurality of projectors and a computer equipped with them, the resolution can be increased, A detailed portion can be displayed, and the processing of the computer provided in the plurality of projectors can reduce the rendering load by rendering the video of different areas for the right eye and the left eye, respectively. Further, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved without depending on the resolution of the database video as in the past. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the composite video generation device of the sixteenth aspect, since the brightness of the projection area is improved by using several projectors and overlapping the projection areas, the necessary brightness can be obtained without using an expensive projector. In addition, the brightness of the projection area can be adjusted by changing the number of projectors, and the brightness at a fixed position can be improved by combining the projector overlay and distortion correction.

  According to the composite video generation device of the seventeenth aspect, since the video to be projected onto one quadratic curved screen is generated and projected by the video device, the resolution can be increased and a fine portion of the video can be displayed. In addition, the video processing of the video equipment can reduce the rendering load by rendering a different video for each divided projector. In addition, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the composite image generation device according to claim 18, since an image to be projected onto one quadratic curved screen is generated and projected by a video device, the resolution can be increased and a detailed portion of the image can be displayed. In addition, the video processing of the video equipment can reduce the rendering load by rendering the divided video for each projector for the right eye and the left eye. In addition, since partial videos in different projection areas can be rendered with high resolution, the resolution of the video projected on the screen can be improved. Thereby, the resolution of the image on the screen can be improved by increasing the number of projectors.

  According to the composite video generation device of the nineteenth aspect, since the brightness of the projection area is improved by using several projectors and overlapping the projection areas, the necessary brightness can be obtained without using an expensive projector. In addition, the brightness of the projection area can be adjusted by changing the number of projectors, and the brightness at a fixed position can be improved by combining the projector overlay and distortion correction.

  According to the composite video generation device of the twentieth aspect, it can be easily realized by using a household device such as a television device, a video reproduction device, or a personal computer as the video device.

  According to the composite video generation device according to claim 21, it is possible to reduce the conspicuousness of the luminance in the overlapping portion of the projector projection area in a state where black is produced, and without using a special blending device, for example, a graphic It can be realized using the function of the board.

  According to the composite image generation device of the twenty-second aspect, since the light can be gradually reduced, the adjustment can be made to the optimum light reduction rate, so that the overlapping portion of the projection area by each projector can be made less noticeable. It becomes possible.

  According to the mapping function generation device of the twenty-third aspect, since the adjustment range is wide by the flexible arm without strictly setting the positional relationship between the screen and the base on which the projector is mounted, the display system can be configured in a short time. Can do.

  According to the composite image generation device of the twenty-fourth aspect, since the adjustment range is wide by the flexible arm without strictly setting the positional relationship between the screen and the base on which the projector is mounted, the display system can be configured in a short time. Can do.

([1] Mapping function related to quadratic curved screen, inverse function thereof and image generation)
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1A is a flowchart for explaining an embodiment of a mapping function generating method related to a quadric curved screen, and FIG. 1B explains an embodiment of a video generating method related to a quadric curved screen. FIG. FIGS. 2A, 2B, and 2C are functional block diagrams illustrating an embodiment of a correction function generation device and a video generation device related to a quadratic curved screen. In FIG. 2, 101 is a quadric curved screen, 102j (j = 1, 2, 3,... N) is a projector provided in front of the screen, and 103j is a plurality of computers provided corresponding to each projector 102j. . Reference numerals 1041 and 1042 denote digital cameras, 105 denotes a computer connecting the digital camera 104, 106 denotes transformation matrix calculation means in the computer 105, 107 denotes quadric surface parameter (Qv) calculation means in the computer 105, and 108 denotes in the computer 105 109j is a mapping function and inverse function calculation means in each computer 103j, 110j is a storage device in each computer 103j, 111j is an image generation means provided in each computer 103j, and 112aj and 112bj are provided in each computer 103j. The image buffer is a pixel buffer and constitutes a video generator together with the computer 103j and the image generator 111j. A network 113 connects the computers 103j and 105. 114j is an offset calculation means of each computer 103j.

(Projector mounting)
The surface of the quadric curved screen 101 forms a quadric curved surface, for example, a spherical surface, and the periphery is formed in a circular shape. As shown in FIG. 3A, the secondary curved screen 101 is fixed on the base 301 by the support frame 302. Support columns 303 and 304 are provided on the left and right ends of the base 301, and two flexible arms 3051, 3052 and 3053 and 3054 are attached to the support columns 303 and 304, respectively. Projectors 1021, 1022, 1023, and 1024 are attached to the flexible arms 3051, 3052, 3053, and 3054, respectively, and are positioned forward in the periphery so as to face the concave surface of the secondary curved screen 101. FIG. 3B shows a connected state of the flexible arm 3051 and the projector 1021, for example. When light is projected from the projectors 1021, 1022, 1023, and 1024 onto the quadric curved screen 101, it will be described when viewed from the quadric curved screen 101 as shown in FIG. The flexible arms 3051, 3052, 3053, and 3054 are adjusted so that there is no portion where light does not hit the central portion. At this time, each projection area from each projector 1021, 1022, 1023, 1024 (each area is indicated by a one-dot chain line, two-dot chain line, three-dot chain line, and four-dot chain line) only needs to cover the screen. The shape and area need not be the same.
In the embodiment, the quadric curved screen 101 is a spherical surface. However, the present invention is not limited to this, and it may be formed by a hyperboloid, an elliptical surface, or the like. Furthermore, although the case where the number of projectors is four will be described, the present invention can be applied without limitation to the number.

(Principle of distortion correction)
When the viewpoint and the projection center match, there is no video distortion. However, distortion generally occurs because the viewpoint and the projection center do not match. In order to correct this distortion, the original image to be projected may be projected in a reverse distortion in advance so that the distortion disappears when viewed from the viewpoint.
FIG. 4 shows a schematic diagram of distortion correction by reverse distortion. FIG. 4 shows that if distortion f generated by the projection of the projector is projected with reverse distortion at f ⁻¹ at the time of image generation, the distortion is canceled out by f (f ⁻¹ ). That is, a mapping function f that relates the projector image and the distortion on the screen is obtained. The inverse function f ⁻¹ is obtained from the mapping function. In FIG. 4, it is assumed that the camera position and direction to be installed at the stage of detecting distortion are the same as the viewpoint and the line-of-sight direction.

The mapping function is obtained by the following procedure.
([1-1] Measurement of 3D coordinate positions of multiple points on the screen)
A light spot is projected onto the quadric curved screen 101 from any one of the projectors 102j, for example. The light spot indicates a predetermined coordinate point constituting a predetermined test pattern. By imaging the light spot in the digital camera 1041 and 1042 constituting the stereo camera obtains a three-dimensional position P _i from the principle of triangulation. i indicates a score. At this time, the stereo camera is premised on that an internal parameter indicating the internal characteristics of the focal length and the number of pixels of each camera lens and an external parameter indicating the external positional relationship are known by a predetermined calibration process. The matrices representing the internal and external parameters of the left camera 1041 are A _c1 and M _c1 . Similarly, let A _c2 and M _c2 be matrices representing internal and external parameters of the right camera 1042. At this time, the relationship between the three-dimensional position P _i on the screen, the position p _i ^c1 of the light spot on the left camera image, and the position p _i ^c2 of the light spot on the right camera image is p _i ^c1 = A _c1 M _c1 P _i
p _i ^c2 = A _c2 M _c2 P _i (1)
It is represented by Here, M _c1 and M _c2 are M _c1 = [I | 0]
M _c2 = [R _c2 | T _c2 ] (2)
It is.
To a projected center of the left camera 1041 as the origin of coordinates, external parameters M _c1 of the left camera 1041 is represented using a unit matrix I. Further, since the projection center of the right camera 1042 is a position obtained by rotating R _c2 and translation T _c2 from the projection center of the left camera 1041, the external parameter M _c2 of the right camera 1042 is expressed using R _c2 and T _c2. Can do. In FIGS. 5 and 6, the projection center (Center Of Projection) of the left camera 1041 (hereinafter sometimes referred to as COP) is COP _c1 , and the COP of the right camera 1042 is COP _c2 . Also, if the straight lines connecting the COP of each camera and the position of the light spot on the image are l _c1 and l _c2 , the three-dimensional position P _i on the screen is estimated at these intersections using l _c1 and l _c2. Can do. If no intersection point is found, a common normal that minimizes the distance between l _c1 and l _c2 is obtained, and the midpoint of the normal is determined as P _i . Also, assuming that the position of the light spot on the projector image coordinates is p _i ^pj and the matrix representing the internal parameters of the projector is A _pj , when p _i ^pj , A _pj , P _i , and the COP of the left camera 1041 are the origin. The rotation matrix R _pj and the translation matrix T _pj of the projector have the following relationship.
p _i ^pj = A _pj [R _pj | T _pj ] P _i (3)
Here, j represents the number of projectors. Also, the position p _i ^pj on the projector image coordinates is known because it is the position of the point of the test pattern. _Apj is known because it is obtained by a predetermined calibration process. Therefore, R _pj and T _pj can be obtained using p _i ^pj , A _pj and P _i . R _pj and T _pj are repeatedly calculated using all the corresponding p _i ^pj and P _i , and the values when the error is minimized are used. These calculations are performed in the computer 105 to which the cameras 1041 and 1042 are connected, and the calculation results are stored in the storage device 108.
Next, using the internal parameters of the projector obtained by the above method, a quadric surface parameter Q, which is information representing the type and shape of the quadric surface (spherical surface, hyperboloid, ellipsoid, etc.) and shape, is obtained.

であり、この関数をＱの要素を並べたベクトル《Ｖ》を用いて変形すれば、

（ここで、例えば《ｈ′》はベクトルｈ′を示す表記である。なお、この明細書、図面においてベクトル表記を、《 》を用いた場合と、矢印を上部に付した場合と白抜き又は肉太の場合とで併用して表す場合がある。）
Ｎ点（Ｎ≧９）用いれば、以下の連立方程式が成立する。
Ｘ《Ｖ》 ＝ 《０》
但し ＸはＮ行１０列のマトリクスである。
この連立方程式を解けば、《Ｖ》、即ちＱが求まる。 ([1-2] Calculation of quadric surface parameter Q)
In order to obtain the mapping function f (), it is necessary to obtain the quadric surface Q. Therefore, first, ^assuming that the projection matrix of the left camera 1041 is P ^c1 and the projection matrix of the projector is P _j ^prj , the projection matrix, the light spot position p _i ^c1 on the left camera image, and one of the test patterns on the projector image coordinates The three-dimensional position P _i is obtained using the corner position p _i ^pj which is a light spot. As a method for obtaining, a method similar to that for obtaining a three-dimensional position with a stereo camera is used. The calculation is performed in the computer 105 to which the cameras 1041 and 1042 are connected. Here, P ^c1 and P _j ^prj are expressed as follows.
P ^c1 = A _c1 [I | 0]
P _j ^prj = A _pj [R _pj | T _pj ] (4)
When the obtained three-dimensional position P _i is on the quadric surface Q,

If this function is transformed using a vector << V >> in which elements of Q are arranged,

(Here, for example, << h '>> is a notation indicating a vector h'. In this specification and drawings, the vector notation is indicated by using <<>>, the case where an arrow is attached at the top, and white or (It may be used in combination with meat thickness.)
If N points (N ≧ 9) are used, the following simultaneous equations are established.
X << V >> = << 0 >>
X is a matrix of N rows and 10 columns.
By solving these simultaneous equations, << V >>, that is, Q can be obtained.

を、異なる視点即ちプロジェクタ１０２ｊ及びカメラ１０４１又は１０４２から見たときの像ｐ_ｉ及びｐ_ｉ′の関係を示すマッピング関数ｆを求める。以下、図７をもとに説明する。すなわち、カメラの位置及び方向が視点及び視線方向と同じである場合を考える。
今、射影空間Ｐ^２における点を

、射影空間ｐ^３における対応する点を

とすれば、

の対応がある。但し、式中のｔは転置を示す。
また、Ｐ^３の点

をプロジェクタの座標系Ψから見たｐ^２の点を

、カメラ座標系Ψ′から見た点を

とすれば、

ここで、

はΨ′におけるエピポールであり、Ｈはホモグラフィー行列である。
今、

が二次曲面上にあるとの制約を課すと、

より

これを展開して、次のλに関する二次方程式を得る。（λは図７の

と

との関係を示す倍率

、Ｑ_３３は二次曲面規定マトリクスＱの上から３×３部を抜き出したもので回転を示す。また、《ｑ》は平行移動量、
である。）

ｑ_４４＝１として良いので、これを解き、λを求めると、

式（８）に、このλを代入して整理すれば、

となる。
右辺は

でまとめられ、次式を得る。

と置けば次式となる。

これが、カメラ設置位置と視点が一致する場合における、スクリーン１０１に投影された点

をプロジェクタ座標Ψから見た点

と、カメラ座標系Ψ′から見た点

の対応を示すマッピング関数

である。 ([1-3] Principle of Derivation of Mapping Function f ())
Quadratic surface, for example, one point on the screen

Is a mapping function f indicating the relationship between the images p _i and p _i ′ when viewed from different viewpoints, that is, from the projector 102j and the camera 1041 or 1042. Hereinafter, a description will be given with reference to FIG. That is, consider the case where the camera position and direction are the same as the viewpoint and line-of-sight direction.
Now, the point in the projective space P ²

, The corresponding point in the projective space p ³

given that,

There is correspondence. However, t in a formula shows transposition.
In addition, in terms of ^{P 3}

P ² point from the projector coordinate system Ψ

, The point seen from the camera coordinate system Ψ ′

given that,

here,

Is the epipole in ψ ′ and H is the homography matrix.
now,

Imposes the constraint that is on a quadric surface

Than

This is expanded to obtain a quadratic equation for the following λ. (Λ is shown in FIG.

When

Magnification showing the relationship with

, Q ₃₃ represents the rotation in an extract of the 3 × 3 parts from the top of the quadric surface defining a matrix Q. Moreover, << q >> is the amount of translation,
It is. )

Since q ₄₄ = 1 is good, if this is solved and λ is obtained,

Substituting this λ into equation (8) and organizing it,

It becomes.
The right side is

And the following formula is obtained.

The following formula is obtained.

This is the point projected on the screen 101 when the camera installation position matches the viewpoint

From the projector coordinate Ψ

And the point seen from the camera coordinate system Ψ ′

Mapping function to show the correspondence of

It is.

([1-4] Mapping function by diagonalized virtual camera method)
([1-4-1] Principle)
Even when the camera installation position and the viewpoint do not match, it will be described that calibration can be performed by virtually moving the camera to the viewpoint using the screen shape information.
In a display system that requires accuracy such as a training apparatus, the viewpoint position is usually defined by a displacement from a reference point. For example, in the case of a dome screen, it is defined by 1 m behind the base line passing through the center of the dome and the displacement from the reference point. FIG. 8 shows the relationship between the reference point 801, the installation position and orientation of the camera 802, and the viewpoint and line-of-sight direction referenced by 803, that is, the position and orientation of the virtual camera. Without thinking of finding the amount of rotation / translation to move directly to the specified viewpoint position / orientation from a camera with an uncertain installation position / orientation, move to the reference point accurately and finally use the displacement from this point. Thinking in two stages when moving to the specified viewpoint position / posture. Based on this idea, when the screen is a quadric surface, the reference point is generally the center of the screen, so moving the camera from any position / posture to the position / posture of the reference point is a quadric surface Corresponds to diagonalization of prescribed matrix If this correspondence is used, the amount of movement from the ambiguous position / posture to the reference point can be accurately obtained without actually measuring the posture position of the camera, and it is possible to move to the position / posture of the designated viewpoint. This corresponds to the fact that the camera is virtually placed at the designated viewpoint position and orientation for calibration.

このＳは式（１９）のように分解することができる。

ここで、Ｒ_３３は式（２０）に示すＱ_３３を対角化する回転行列であり、Ｑ_３３及びｑは式（２０）で示されたＱの成分である。また、ΛはＱ_３３を対角化した行列、すなわちＲ_３３ ^−１Ｑ_３３Ｒ_３３である。この回転行列Ｒ_３３によりＱ_３３が対角化されることは数学的に保証されている。

式（１７）の導出を説明する。
座標変換によるＱの対角化Ｓ^ｔＱＳを、Ｑを対角化する相似変換Ｓ^−１ＱＳと捉えて直接的にＳを求められない。なぜならば、Ｓは回転行列Ｒと平行移動行列Ｔの積からなるがＴによりＳが正規直交行列とならないからである。
しかしＳの３行３列の回転成分Ｒ２２は正規直交行列となるため座標変換による対角化問題を相似変換の問題へおきかえることが可能である。すなわち、座標変換によるＱ_３３の対角化Ｒ^−１ _３３Ｑ_３３Ｒ_３３＝Λは相似変換の問題に帰着できる。これにより、未知のＲ_３３がＱ_３３の固有ベクトルを並べたものとして決まり、

となる。展開すると

ここで対角行列Ｄは、

の関係から次の連立する方程式ができる。

上の２式より、

Ｒ_３３及びＴを用いてＳを求めれば、

Ｍは設計時に指定される既知の変換行列であるのでＨが決まり、これによりＱｖが求まり、最終的にマッピング関数ｆ（）が式（１５）から計算できる。発明者はこの方法を「対角化仮想カメラ法」と呼ぶ。 ([1-4-2] Moving to the center by diagonalization)
The quadric surface parameter is information that represents the quadric surface type and shape determined by the camera coordinate system Q, a quadric surface parameter at the viewpoint coordinate system with its origin at the viewpoint and Q _v. The relationship between the two is given by the following equation using H as a transformation (movement, rotation) matrix from the viewpoint coordinate system to the camera coordinate system.
Q _v = H ^t QH (16)
The quadric surface parameter Q is obtained by the method described above ([1-2] Calculation of quadric surface parameter Q).
Considering a two-step transformation where S is the transformation matrix from the camera coordinate system to the reference coordinate system with the center of the quadric surface as the origin, and M is the transformation matrix from the reference coordinate system to the viewpoint coordinate system, H is It is expressed by a formula.
H = MS (17)
S diagonalizes Q as shown in equation (18).

This S can be decomposed as shown in equation (19).

Here, R ₃₃ is a rotation matrix that diagonalizes Q ₃₃ shown in equation (20), and Q ₃₃ and q are components of Q shown in equation (20). Λ is a matrix obtained by diagonalizing Q ₃₃ , that is, R ₃₃ ⁻¹ Q ₃₃ R ₃₃ . It is mathematically guaranteed that Q ₃₃ is diagonalized by this rotation matrix R ₃₃ .

Derivation of Expression (17) will be described.
The diagonalization S ^t QS of Q by coordinate transformation is regarded as a similarity transformation S ⁻¹ QS that diagonalizes Q, and S cannot be obtained directly. This is because S is a product of the rotation matrix R and the translation matrix T, but S does not become an orthonormal matrix due to T.
However, since the rotation component R22 of S in 3 rows and 3 columns is an orthonormal matrix, it is possible to replace the diagonalization problem by coordinate transformation with the problem of similarity transformation. That is, the diagonalization of Q ₃₃ by coordinate transformation R ⁻¹ ₃₃ Q ₃₃ R ₃₃ = Λ can be reduced to the problem of similarity transformation. As a result, the unknown R ₃₃ is determined as an array of Q ₃₃ eigenvectors,

It becomes. When expanded

Where the diagonal matrix D is

From this relationship, the following simultaneous equations can be obtained.

From the above two formulas,

If S is calculated using R ₃₃ and T,

Since M is a known transformation matrix designated at the time of design, H is determined, whereby Qv is obtained, and finally the mapping function f () can be calculated from the equation (15). The inventor calls this method the “diagonalized virtual camera method”.

  That is, since M is a known transformation matrix designated at the time of design, H is determined by the computation of the transformation matrix computing means 106 (step P101 in FIG. 1A), and thereby the quadric surface parameter (Qv) computing means. Qv is obtained in the calculation of 107 (step P102 in FIG. 1A). These results are stored in the storage device 108 as a table. In the mapping function and inverse function calculation means 109j of each computer 103j, the mapping function f () can be calculated from the equation (15) (step P103 in FIG. 1A). The mapping function and inverse function calculation means 109j obtains the inverse function from the mapping function f () (step P104 in FIG. 1A). Step P101 to Step P102, Step P103, and Step P104 are repeated n times (in the embodiment, n = 4) for the projector 102j, and then the inverse function obtained in Step P104 is stored in the storage device 110j of each computer 103j ( Step P105 in FIG. Mapping functions and inverse functions obtained in the series of processes are obtained in advance for a suitable number of virtual viewpoints and stored as a table.

([2] Load distribution by parallel processing)
The circumference of the screen is an ellipse unless it is looked down from a directly facing position. Since the diagonal viewpoint described in “[1-4-2] Moving to the center by diagonalization” described above makes it possible to face the virtual viewpoint position directly on the screen, the rectangle outlined in the circle is calculated. The field of view can be obtained directly and overlapped.

([2-1] Division of Viewing Angle Using Rectangle FIG. 9A shows the relationship between the rectangle circumscribing the circle representing the edge of the screen and the division and overlap of the field of view using the rectangle. (B) shows the viewing angle division method, which will be described below assuming that each division is included in the projectable region in the figure.

([2-2] Transformation matrix for combining)
In the following, the calculation of the transformation matrix is shown by taking 4 fields of view (4 parallel) as an example. However, the discussion here does not lose generality other than the four divisions. The mapping function f ^j () from the projector coordinates of the projector 102j to the virtual camera coordinates (a region divided into four as shown in FIG. 9A) is obtained as follows.
(1) _Obtain the radius r of the screen and use this to calculate W _rct and H _rct shown in FIG.
(2) A rectangular matrix circumscribing the circle is divided into four so as to have overlapping portions H _over and W _over , and a transformation matrix H ^j _rct from the circumscribed rectangular display area to the corresponding area divided into four is calculated.
(3) The mapping function f ^j () is calculated in the mapping function and inverse function calculating means 109j of the computer 103j provided corresponding to the projector 102j using H ^j _rct .
Hereinafter, it demonstrates in order.

を用いた

はカメラ座標でのスクリーンの縁を表す楕円を示す式である。従って、カメラ座標系における二次曲面行列Ｑを、視点座標系に変換したＱ_ｖを用いて計算し直したＥに対応するＥ_ｖを用いた以下の式

は、スクリーンに正対する視点座標系でのスクリーンの縁に対応する円を示す式となる。
また、Ｅｖで規定される円と任意の点

の距離ｄは、

の関係がある。
ここで、

として展開すれば、
ｅ_１１ｘ^２ _ｉ＋２ｅ_１２ｘ_ｉｙ_ｉ
＋２ｅ_１３ｘ_ｉ＋２ｅ_２３ｙ_ｉ
＋ｅ_２２ｙ^２ _ｉ＋ｅ_３３
＝ｄ^２ （２８）
この距離ｄが最大となる点が、円の中心であり、また、このときの距離が半径となる。ｄを最大とする点

は次の関係を満足する。

上式で示した連立方程式を解けば、

この座標点は、円の中心であるので、（ｘ_ｃ，ｙ_ｃ）と表記する。この中心点（ｘ_ｃ，ｙ_ｃ）を用いれば、半径ｒは、
ｒ＝√（ｅ_１１ｘ^２ _ｃ＋２ｅ_１２ｘ_ｃｙ_ｃ＋２ｅ_１３ｘ_ｃ
＋２ｅ_２３ｙ_ｃ＋ｅ_２２ｙ^２ _ｃ＋ｅ_３３） （３１）
となる。 ([2-2-1] Circle radius)
In the square root of the above equation (15)

Used

Is an equation representing an ellipse representing the edge of the screen in camera coordinates. Therefore, the following equation using E _v corresponding to E recalculated using Q _v converted from the quadratic surface matrix Q in the camera coordinate system to the viewpoint coordinate system:

Is an expression showing a circle corresponding to the edge of the screen in the viewpoint coordinate system directly facing the screen.
Also, the circle defined by Ev and any point

The distance d of

There is a relationship.
here,

Expand as
e ₁₁ x ² _i + 2e ₁₂ x _i y _i
+ 2e ₁₃ x _i + 2e ₂₃ y _i
+ E ₂₂ y ² _i + e ₃₃
= D ² (28)
The point where the distance d is maximum is the center of the circle, and the distance at this time is the radius. Point that maximizes d

Satisfies the following relationship:

Solving the simultaneous equations shown above,

Since this coordinate point is the center of the circle, it is expressed as (x _c , y _c ). Using this center point (x _c , y _c ), the radius r is
r = √ (e ₁₁ x ² _c + 2e ₁₂ x _c y _c + 2e ₁₃ x _{c
^{+ 2e 23 y c + e 22}} y 2 c + e 33) (31)
It becomes.

これを用いて、図９（ａ）に示す原点をＯとする座標系から原点をＯ^ｊとする各視野の座標系への変換行列は、以下となる。

各視野は、各プロジェクタ１０２ｊ毎の映像が投影される領域を決める。 ([2-2-2] Field division)
As shown in FIG. 9A, if the overlap width of the projected images in the horizontal direction is W _over and the overlap width in the vertical direction is H _over , the divided visual field sizes Wrct and Hrct are as follows.

Using this, the transformation matrix from the coordinate system having the origin as O shown in FIG. 9A to the coordinate system of each visual field having the origin as O ^j is as follows.

Each field of view determines an area on which an image for each projector 102j is projected.

([2-2-3] Mapping function f ())
The following expression obtained by operating Hjrct to Expression (15) is a new mapping function f ^j () related to the projector 102j when the viewing angle is divided as described above.

([2-3] Brightness adjustment)
Since the overlapping portion of the projection area has high luminance, it is necessary to correct the luminance of the projected image. This luminance correction is performed by preliminarily holding a weighting coefficient at a pixel position of each projector overlapping on the screen as a table, and multiplying the pixel value and the corresponding weighting coefficient at the time of distortion correction. A method of determining the weight is shown in FIG. The program in FIG. 10 is written in a pseudo language.
Here, (p, q) is the pixel position in the camera coordinate system, i, j is the projector number of (1... 4), H is the number of pixels in the horizontal direction of the projector, W is the number of pixels in the vertical direction, and α ⁱ ( p ⁱ , q ⁱ ) represent weighting factors at the pixel position (p ⁱ , q ⁱ ) of the projector i, respectively. The overlapping part of the projector i and the projector j is determined, and the weight of the part is given in inverse proportion to the overlapping width.

([3] Reduction of black offset due to projector leakage light)
([3-1] Necessity of reducing black offset)
When a high-resolution video is generated using a plurality of projectors, an overlapping portion is generated in the projection area of each projector. The overlapping part is added with the brightness of each projector, and becomes brighter than the non-overlapping part. There is no problem as long as the projector can display completely black, but even if the projected image is black, the area projected by the leakage light of the projector is not completely black (see FIG. 11A). In addition, the overlapping area becomes brighter because the leakage light of the projectors overlaps. In other words, a black offset occurs due to light leaked from the projector, and the overlapped portion becomes conspicuous even when black is emitted.

([3-2] Reduction of black offset)
The black offset portion due to the leakage light of the projector depends on the projector and cannot be adjusted. Only the color of the projected image can be adjusted. If the color value becomes 1.0 or more with simple color raising (the color value is normalized to 0.0 to 1.0), the color is corrected to 1.0. It will be different from the color of the original video. Therefore, the contrast of the original image is lowered, and the color is not changed by raising the color while maintaining the color (see FIG. 11B). As an implementation method, by using the fragment shader function of the graphics board using the Cg (C for Graphics by nVidia) language and correcting the color of the video to be captured, the brightness of the overlapping portion is conspicuous (FIG. 11 (c ), (See FIG. 11D).
The color correction formula used in Cg is as shown in the following formula (35). By using this equation, it is possible to use the α blend function for adjusting the black level and blending overlapping portions (see FIG. 11E). In addition, it is possible to reduce luminance deviation due to black offset.
C_new
= K * C_org + C_offset (35)
However,
C_new: Corrected color (The corrected color)
K: the coefficient for reducing the contrast (the coefficient for reducing the contrast)
C_org: The original color of image (The original color of image)
C_offset: The brightness of color (The offset of color)

（図１２のステップＰ１２０３）
なお、式（３６）の分母のＷｉｍｇ，Ｈｉｍｇはそれぞれカメラでスクリーン１０１を撮像したときのスクリーン１０１の周囲にある矩形の幅、高さである（図９（ａ）参照）。式（３８）の分母の２５６は例えばカメラが持つ輝度値を８ｂｉｔで持つとした場合の２^８＝２５６である。

これらにより式（３５）を得て、補正した色を得る（図１２のステップＰ１２０４）。 ([3-3] K and C_offset calculation method)
Determined from the brightness of the camera image.
1. A state in which nothing is projected from the projectors 1021 to 1024 is captured by the camera 1041. (Used as Background Image.) Let the image be Img_bkgrnd (step P1201 in FIG. 12).
2. A state in which black is emitted from each of the projectors 1021 to 1024 is photographed by the camera 1041 (step P1202 in FIG. 12). Let the image be Img_blkofst_n. n is a projector number. When the luminance obtained from the image is I, the offset calculation unit 114j performs the following calculation.

(Step P1203 in FIG. 12)
Note that Wimg and Himg in the denominator of Expression (36) are the width and height of a rectangle around the screen 101 when the screen 101 is imaged by the camera (see FIG. 9A). The denominator 256 of the equation (38) is 2 ⁸ = 256 when the luminance value of the camera is assumed to be 8 bits, for example.

Thus, equation (35) is obtained, and a corrected color is obtained (step P1204 in FIG. 12).

とする）を利用し、プロジェクタ画像座標のある位置がテクスチャ座標のどの位置に対応するかを求めることができ、そのテクスチャ座標の位置の色をサンプリングして色情報として保持する（図１４参照）。
サンプリング結果を元に歪み補正を施した映像を作成し、プロジェクタから投影する。投影された映像はスクリーン上で正しく映像として表示される。
図２の各プロジェクタ１０３ｊ毎に備えられたコンピュータ１０３ｊの画像生成手段１１１ｊにより生成された画像を、ピクセルバッファ１１２ｊａにレンダリングし、この画像を当該プロジェクタ１０３ｊについて求めたマッピングの逆関数ｆ^−１（）により、歪み補正した映像をピクセルバッファ１１２ｊｂに生成する（図１（ｂ）のステップＳ１０１）。この歪み補正した映像を各プロジェクタ１０３ｊから二次曲面スクリーン１０１に投影する（図１（ｂ）のステップＳ１０２）。 ([3] Video generation)
Conventionally, when video rendering is performed, the video is generally written in a frame buffer (memory area) of the graphics board, and the buffer is displayed via a display or a projector.
In a projection cluster (the screen is covered by the entire projection area of a plurality of projectors), the video data stored in the buffer is sampled (video reference) when correcting the video distortion. The distortion is corrected by recreating the video based on the referenced data. The larger the buffer, the higher the resolution of the video. When this buffer is a frame buffer, the frame buffer depends on the resolution with the display or projector, and therefore the buffer cannot be made larger than the resolution of the display or projector. Therefore, it is difficult to increase the resolution beyond the function of the display or projector.
In the present invention, a pixel buffer is used instead of a frame buffer when storing video data. By storing directly in the pixel buffer and performing off-screen rendering, the resolution of the display or projector is not limited. Although the size of the pixel buffer is limited to a power of 2, it can be arbitrarily set. Actually, the size of the pixel buffer is limited because it depends on the function of the graphics card, but a buffer having a size larger than the frame buffer can be secured.
The size of the pixel buffer is arbitrarily determined, and video having a display resolution or higher is stored in the buffer. Here, the buffer size is 2048 × 2048.
When using a pixel buffer, it is not necessary to use a frame buffer because it can be rendered directly into the buffer. Further, by using an OpenGL extension instruction, data stored in the pixel buffer can be directly handled as a texture.
The pixel buffers 112a and 112b in FIG. 2 perform sampling of data using Quadric Transfer (QT). QT represents conversion from camera image coordinates to projector image coordinates, and inverse conversion (QT_inv) represents conversion from projector image coordinates to camera image coordinates. In the projection cluster, data sampling is performed using QT_inv.
As a sampling method, the coordinates from (0, 0) to (1024, 768) of the projector image coordinates are sequentially scanned, and the position corresponding to the position in the camera image coordinates is obtained. If the camera is considered as a viewpoint on OpenGL, an image stored in the texture memory can be regarded as an image viewed from the camera. This relationship (here transformation matrix

Can be used to determine which position in the texture coordinates the position of the projector image coordinates corresponds to, and the color at the position of the texture coordinates is sampled and held as color information (see FIG. 14). .
Based on the sampling result, an image with distortion correction is created and projected from the projector. The projected image is correctly displayed as an image on the screen.
The image generated by the image generation means 111j of the computer 103j provided for each projector 103j in FIG. 2 is rendered in the pixel buffer 112ja, and the inverse function f ⁻¹ () of the mapping obtained for this projector 103j. Thus, a distortion-corrected video is generated in the pixel buffer 112jb (step S101 in FIG. 1B). This distortion corrected image is projected from each projector 103j onto the quadric curved screen 101 (step S102 in FIG. 1B).

In the above ([2-3] luminance adjustment), the width of the overlapping portion of the projection area is set to be relatively smaller than the non-overlapping portion as shown in FIG. 9A. As described above, for example, the overlapping portions of the projection areas 1303 and 1304 by the two projectors 1301 and 1302 can be enlarged to generate a composite image, and the brightness of the projection area can be improved by superimposing. At this time, the screen may be a quadratic curved surface as described above, but may be a flat screen 1305.
Also in the case of using these two projectors, the two computers 1021 and 1022 in FIG. 2 generate images for the respective projectors 1021 and 1022, and the generated images by the inverse function of the mapping for each of the projectors 1021 and 1022 Is generated in the pixel buffer 112a (step S1301 in FIG. 13B), and the images corrected for distortion in the pixel buffer 112b are projected from the projectors 1021 and 1022 onto the quadric curved screen 101, and the common image is obtained. Thus, the brightness of the overlapping portion is increased (step S1302 in FIG. 13B).

In the above ([3] Reduction of black offset due to leakage light from projector), there is no problem if the projector can display perfect black, but even if the projected image is black, the area projected by the leakage light of the projector is I explained that it was not completely black. This is because when a dark scene image at night is projected, the human eye can be sensitive to even a slight difference in luminance, and thus tends to be more noticeable.
In this case, as a possible measure, by correcting the brightness of the original image to be projected softly, the brightness of the overlapping area caused by black offset can be reduced and the conspicuousness of the overlapping part can be reduced. It is difficult to prevent the overlap part from being completely recognized in the scene.
For this reason, it is possible to display an image in which the overlapping portion is not felt more by combining the polarizing plates in combination with the correction of the luminance of the original image and adjusting the dimming rate for the overlapping portion in a stepwise manner.
FIG. 15A shows a flow in which the light attenuation rate can be adjusted stepwise, and FIG. 15B shows the apparatus. In FIG. 15B, 1501 is a projector, 1502 is a polarizer, 1503 is an analyzer, 1504 is a rotating means including a motor, gears, other mechanical parts, and an electric circuit for rotating the analyzer 1503. Reference numeral 1506 denotes gear teeth provided around the analyzer so as to mesh with the gear of the rotating means 1504.
Polarizer 1502 covers a range corresponding to an overlapping area between the image area of projector 1501 in which it is located and the image area of another projector (not shown) (shown by hatching to cover it). . Depending on the rotation state of the analyzer 1503, the brightness of only the overlapping portion of the projection light from the projector 1501 onto the screen is adjusted.
According to Malu's law, I (θ) = I (0) cos2θ
However, I (θ): intensity of light when passing through the polarizer and analyzer I (0): intensity of light when passing through the polarizer, so the luminance of the overlapping portion is changed by changing the angle of the analyzer Can be adjusted.
Each computer 103j in FIG. 2 generates an image for each projector 102j in the pixel buffer 112a, and generates an image obtained by correcting the distortion of the generated image in the pixel buffer 112b by an inverse function of the mapping for each projector 102j ( P1501). Each projector 102j projects the image corrected for distortion in the pixel buffer 112b onto the quadric curved screen 101 (P1502). When representing a night scene, the operator operates the rotating means 1504 to rotate the analyzer 1503. In accordance with the rotation of the analyzer 1503, the light from the projector 102j is reduced (P1503).
If the luminance of the overlapping portion is reduced to the same level as the luminance of each projector 102j alone, the overlapping portion becomes inconspicuous.

(Application to stereo image display)
The above-described technique can also be realized in stereo video display. FIG. 16A is a flowchart for explaining an embodiment of a mapping function generation method related to a quadric curved screen, and FIG. 16B shows an embodiment of an image generation method related to a quadric curved screen. FIG. FIG. 17 is a functional block diagram for explaining an embodiment of a correction function generation apparatus and video generation apparatus relating to a quadratic curved screen. In FIG. 17, 1601 is a quadric curved screen, 1602j (j = 1, 2, 3,... N) is a projector provided in front of the screen, and 1603j is a plurality of computers provided corresponding to each projector 1602j. . Reference numerals 16041 and 16042 denote digital cameras, 1605 denotes a computer to which the digital camera 1604 is connected, 1606 denotes transformation matrix calculation means in the computer 1605, 1607 denotes quadric surface parameter (Qv) calculation means in the computer 105, and 1608 denotes in the computer 1605. 1609j is a mapping function and inverse function calculation means in each computer 1603j, 1610j is a storage device in each computer 103j, 1611 is an image generation means provided in each computer 1603j, and 1612a and 1612b are provided in each computer 1603j. Is a pixel buffer. Reference numeral 1613 denotes a network connecting the computers 103j and 105. Reference numeral 1614 denotes an offset calculation means of each computer 1603j.

([1] Mapping function related to quadratic surface for stereo display and its inverse function)
In the first embodiment, the stereo display is to generate a mapping function and an inverse mapping function for the viewpoints of both the left and right eyes instead of one viewpoint, and generate images for the left and right eyes.
That is, a mapping function related to a quadric curved screen for stereo display and its inverse function are obtained as follows.
With respect to a predetermined point of the test pattern projected from the projector onto the quadric surface screen by the transformation matrix (M) from the reference coordinate system to the viewpoint coordinate system and the transformation matrix (S) from the camera coordinates to the reference coordinate system, A transformation matrix (H) from the viewpoint coordinate system to the camera coordinate system is obtained by calculation of the transformation matrix calculation means 1606 (step P1601 in FIG. 16A). Using the transformation matrix (H), the viewpoint coordinate system of either the right eye or the left eye with the assumed viewpoint of either the right eye or the left eye as the origin from the quadric surface parameter (Q) obtained in the camera coordinate system Further, the quadric surface parameter (Qv) in the viewpoint coordinate system of either the right eye or the left eye with the other assumed viewpoint of the right eye or the left eye as the origin is obtained by calculation of the quadric surface parameter (Qv) calculation means 1607. (Step P1602 in FIG. 16A). These results are stored in the storage device 1608 as a table. The mapping function and inverse function calculating means 1609j uses the quadratic surface parameter (Qv) in the right-eye and left-eye viewpoint coordinate systems obtained as described above, and the points projected on the screen 1601 from the projector i coordinate system. Then, a mapping function indicating the correspondence with the points seen from the assumed viewpoint of any right eye and left eye is obtained (step P1603 in FIG. 16A). The inverse function is obtained from the mapping function (step P1604 in FIG. 16A). Steps P1601 to P1604 are repeated n times for the plurality of projectors, and the inverse functions of the right eye and left eye mapping functions are obtained for each projector and stored in the storage device 1601j (step P1605 in FIG. 16A).

([2] Stereo image generation)
Stereo display is performed as follows.
The image generation means 1611j of each computer 1603j generates the right-eye and left-eye images for each projector 1602j in the pixel buffer 1612ja, and the inverse function of the right-eye and left-eye mapping for each projector 1602j obtained as described above. Thus, an image obtained by correcting the distortion of the generated image is generated in the pixel buffer 1612jb (step S1601 in FIG. 16B). The right-eye and left-eye images corrected for distortion in the pixel buffer 1612b are alternately projected onto the quadric curved screen 1601 from each projector 1602j (step S1602 in FIG. 16B).

  The technique described as the second embodiment can also be applied to the stereo display. The technique described as the third embodiment can also be applied to stereo display.

FIG. 18A is a functional block diagram for explaining the fifth embodiment. Reference numeral 1801 denotes a video device, which uses a television device, a video playback device, a personal computer, or the like. 1802 is a distortion-corrected video generation unit, 1803 is a converter that converts a signal of the video device 1801 so that it is suitable for processing by the distortion-corrected video generation unit 1802, 1804 is a memory that temporarily stores an output signal from the converter 1803, 1805, 1806, 1807, and 1808 are distortion correction means for correcting distortion of an image by an inverse function of the mapping function as described in the first embodiment in accordance with the position of the projector and the screen, and 1809, 1810, 1811, and 1812 are distortion corrections. 1813, 1814, 1815, and 1816 are converters that convert the corrected video signals into signals suitable for projection onto the projector, 1817, 1818, 1819, and 1820 are projectors, and 1821 is the second memory. This is a second curved screen.
The mapping function and its inverse function are obtained by a computer (not shown) as described in the first embodiment.
The memory 1084 may be composed of a frame memory, but this need not be the case and may be a line memory. Further, the memory 1084 is not essential, and may be directly connected from the converter 1803 to the distortion correction means 1805 to 1808. The post-correction memories 1809, 1810, 1811 and 1812 may be composed of frame memories, but are not necessarily this, and may be line memories. Further, the post-correction memories 1809, 1810, 1811 and 1812 are not essential, and each of the distortion correction means 1805, 1806, 1807 and 1808 may be directly connected to the converters 1813, 1814, 1815 and 1816.
Arbitrary number n can be used as the number of projectors 1817, 1818, 1819, 1820. In this embodiment, four examples are shown.

  The video from the video device 1801 (FIG. 18B) is converted by the converter 1803 so as to be suitable for processing, and is temporarily stored in the memory 1804. The image in the memory 1804 is divided by ranges projected onto the projectors 1817, 1818, 1819, and 1820, and distortion is corrected by the distortion correction units 1805, 1806, 1807, and 1808 within the ranges (steps in FIG. 19). ST1901) and temporarily stored in the corrected memories 1809, 1810, 1811, 1812 (FIG. 18C). Video signals are processed by converters 1813, 1814, 1815, and 1816, and projected onto screen 1821 by projectors 1817, 1818, 1819, and 1820 (step ST1902 in FIG. 19).

  In the converters 1813, 1814, 1815, and 1816, the brightness of the overlapping projection areas of the projectors 1817, 1818, 1819, and 1820 is obtained by the brightness adjustment as described in ([2-3] brightness adjustment) of the first embodiment. Can be corrected.

  In FIG. 18, offset adjustment is performed by a computer (not shown) as described in the first embodiment ([3] Reduction of black offset due to leakage light of projector).

For example, the overlapping portion of the projection areas of two projectors can be enlarged, and the brightness of the overlapping portion can be increased as in the second embodiment.
The video from the video device 1801 is converted by the converter 1803 so as to be suitable for processing, and temporarily stored in the memory 1804. Images in the memory 1804 are divided by ranges projected onto the projectors 1817, 1818, 1819, and 1820, and distortions are corrected by the distortion correction units 1805, 1806, 1807, and 1808, respectively, within the ranges (steps of FIG. 20). ST2001) and temporarily stored in the corrected memories 1809, 1810, 1811, 1812. Video signals are processed by converters 1813, 1814, 1815, and 1816, and are projected onto a screen 1821 by projectors 1817, 1818, 1819, and 1820 to increase the brightness of the common overlapping portion (step ST2002 in FIG. 20). ).

  A polarizer and an analyzer as in the third embodiment are arranged in front of each projector 1817, 1818, 1819, 1820, and the light from the projector can be dimmed to make the leaked light from the projector inconspicuous.

The even number of projectors 1817, 1818, 1819, and 1820 are divided into those for the right eye and those for the left eye, and the mapping function and the inverse mapping function are obtained for the viewpoints of the left and right eyes by applying the fourth embodiment. Stereo images can be realized by using stereoscopic images as the images.
For example, projectors 1817 and 1819 are for the right eye, and projectors 1818 and 1820 are for the left eye.
Distortion correction is performed on the right-eye and left-eye images of the projectors 1817, 1818, 1819, and 1820 from the video equipment 1801 by the inverse functions of the right-eye and left-eye mappings of the projectors 1817, 1818, 1819, and 1820. (Step ST2101 in FIG. 21). The distortion-corrected right-eye and left-eye images are alternately projected on the quadric curved screen 1821 from the projectors 1817, 1818, 1819, 1820 (step ST2102 in FIG. 21).

It is a flowchart explaining one Example of the production | generation method of a mapping function, and a video generation method. It is a functional block diagram explaining one Example of the production | generation apparatus of a mapping function, and an image generation apparatus. It is a figure explaining the arrangement | positioning attachment of a projector. It is a schematic diagram of distortion correction by reverse distortion. It is a figure explaining the point on a screen. It is a figure explaining the point on a screen. It is a figure explaining the positional relationship in an image plane when the point on a screen is seen from a different viewpoint. It is a figure explaining the movement to a viewpoint position through a reference point from a camera installation position. It is a figure explaining division | segmentation and the viewing angle of the visual field of a screen. It is a program explaining the determination method of the weight of an overlap part. It is a figure explaining reduction of offset. It is a flowchart explaining the offset reduction method. It is a figure explaining the brightness improvement by the superimposition of a composite image. It is a figure explaining the sampling by a pixel buffer. It is a figure explaining the brightness | luminance adjustment of the duplication part at the time of the projection of a night scene. It is a flowchart explaining one Example of the production | generation method of a mapping function, and a video generation method. It is a functional block diagram explaining one Example of the production | generation apparatus of a mapping function, and an image generation apparatus. It is the functional block diagram explaining one Example of a video generator, and description of distortion correction. It is a flowchart explaining one Example of a video generation method. It is a flowchart explaining the brightness improvement by the superimposition of a composite image. It is a flowchart explaining one Example of a video generation method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Quadric surface screen, 102j (j = 1, 2, 3, ... n) ... Projector, 103j ... Computer, 1041, 1042 ... Digital camera, 105 ... Computer, 106 ... Conversion matrix calculation means, 107 ... Quadric surface Parameter (Qv) calculation means 108... Storage device 109 j. Mapping function and inverse function calculation means 110 j. Storage device 111. Image generation means 112 a and 112 b Pixel buffer 113 113 Network.

Claims (24)

A quadratic curved screen, a plurality of n projectors disposed in front of the screen and projected onto the screen, a camera disposed in front of the screen and photographed on the screen, and a computer; A storage device,
Computer processing,
Test pattern projected from the projector onto the quadric surface screen by performing matrix calculation on the transformation matrix (M) from the reference coordinate system to the viewpoint coordinate system and the transformation matrix (S) from the camera coordinates to the reference coordinate system. A first step of obtaining a transformation matrix (H) from the viewpoint coordinate system to the camera coordinate system for the predetermined point of
By using the transformation matrix (H) and the inverse matrix of the transformation matrix (H), a quadratic surface obtained in the camera coordinate system by performing matrix calculation on the quadric surface parameter (Q) obtained in the camera coordinate system. A second process of obtaining a quadric surface parameter (Qv) in a viewpoint coordinate system having an assumed viewpoint as an origin from the parameter (Q);
Using the quadric surface parameter (Qv) given by the matrix under the constraint that the test pattern is on the quadric surface, the points projected on the screen by the matrix calculation were viewed from the projector i coordinate system. A third process for obtaining a mapping function indicating a correspondence between the points and the points viewed from any hypothetical viewpoint as a matrix ;
Using the mapping function given by a matrix, and a fourth process of obtaining a more thereof inverse matrix calculation,
A fifth process in which a first process to a fourth process are repeated n times for the plurality of projectors, an inverse function of a mapping function is obtained for each projector, and stored in the storage device;
A mapping function generation method characterized by comprising:
A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a computer provided for each projector And a storage device,
An image for each area projected by each projector is generated in the pixel buffer by each computer, and an image obtained by correcting the distortion of the generated image by an inverse function of mapping for each projector previously obtained in claim 1 is used for the pixel buffer. A first process to generate;
A composite image generation method comprising a second step of projecting an image corrected for distortion in the pixel buffer from each projector onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors arranged so that each region projected on the screen in front of the screen overlaps with each other, and a screen located in front of the screen A camera arranged to take a picture, a computer, and a storage device,
Computer processing,
Test pattern projected from the projector onto the quadric surface screen by performing matrix calculation on the transformation matrix (M) from the reference coordinate system to the viewpoint coordinate system and the transformation matrix (S) from the camera coordinates to the reference coordinate system. A first step of obtaining a transformation matrix (H) from the viewpoint coordinate system to the camera coordinate system for the predetermined point of
By using the transformation matrix (H) and the inverse matrix of the transformation matrix (H), a quadratic surface obtained in the camera coordinate system by performing matrix calculation on the quadric surface parameter (Q) obtained in the camera coordinate system. From the parameter (Q), either the right eye or the left eye as the origin, and the right eye or the left eye as the origin, and the right eye or the left eye as the origin. A second process for obtaining a quadric surface parameter (Qv) in the other viewpoint coordinate system of:
The quadratic surface parameter (Qv) given by the matrix in the right-eye and left-eye viewpoint coordinate systems is used under the constraint that the test pattern is on the quadratic surface, and is projected onto the screen by matrix calculation. A third step of obtaining a mapping function indicating a correspondence between a point seen from the projector i coordinate system and a point seen from an arbitrary right eye and left eye assumed viewpoint as a matrix ;
Using the mapping function given by a matrix, and a fourth process of obtaining a more thereof inverse matrix calculation,
A fifth process in which a first process to a fourth process are repeated n times for the plurality of projectors, an inverse function of a mapping function for the right eye and the left eye is obtained for each projector, and stored in the storage device;
A mapping function generation method characterized by comprising:
A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a computer provided for each projector And a storage device,
The computer generates the right-eye and left-eye images for each projector in the pixel buffer and uses the inverse functions of the right-eye and left-eye mapping for each projector previously determined in claim 3 to generate the generated images. A first process of generating a distortion-corrected image in a pixel buffer;
A composite image generating method comprising: a second process of projecting right-eye and left-eye images corrected for distortion in the pixel buffer from each projector onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors disposed in front of the screen and arranged to have a common overlapping area projected onto the screen, a computer, and a storage device,
A first image for generating an image for each area projected by each projector by the computer and generating an image obtained by correcting the distortion of the generated image in a pixel buffer by an inverse function of mapping for each projector previously obtained in claim 1. And the process
A composite image generation method comprising a second step of projecting a distortion corrected image in each pixel buffer from each projector onto the quadric curved screen to increase the brightness of the common overlapping portion. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a video device,
The image generated from the video device is divided and generated so as to be projected onto the plurality of n projectors onto the screen, and the divided and generated image is subjected to distortion correction by using an inverse function of mapping for each projector previously obtained in claim 1. A first process of generating a video;
A composite image generation method comprising a second process of projecting the distortion-corrected image from each projector onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a video device,
The division generation is performed by dividing and generating the image from the video device so as to project the image onto the screen onto the plurality of n projectors, and using the inverse function of the right-eye and left-eye mapping for each projector previously obtained in claim 3. A first process of generating a distortion corrected image,
A composite image generation method comprising a second step of projecting the distortion-corrected right-eye and left-eye images from the respective projectors onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors disposed in front of the screen and arranged to have a common area projected onto the screen and overlapping, and a video device,
The image generated from the video device is divided and generated so as to be projected onto the plurality of n projectors onto the screen, and the divided and generated image is subjected to distortion correction by using an inverse function of mapping for each projector previously obtained in claim 1. A first process of generating a video;
A composite image generation method comprising: a second step of projecting the distortion-corrected image from each projector onto the quadric curved screen to increase the brightness of the common overlapping portion.   9. The composite video generation method according to claim 6, 7 or 8, wherein the video equipment is a television device, a video playback device, or a personal computer. In the composite image generation method according to claim 2, claim 4, claim 5, claim 6, claim 7, claim 8, or claim 9,
A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged to project onto the screen, a computer, and a camera located in front of the screen and arranged to photograph the screen; Have
A first step of photographing with a camera in a state where none of the projectors projects onto the screen;
A second process in which a black image is projected from the projectors onto the screen and photographed by a camera and repeated n times by a plurality of n projectors;
In the computer, the maximum value is obtained by the sum of the average value of the screen by each projector and the average value based on the difference between the brightness of the image for each projector in the second process and the brightness of the image obtained in the first process. A third step of obtaining an offset value by the difference between the maximum value and the average value;
A composite video generation method comprising: a fourth step of generating a video by lowering the contrast of the overlapping portion of the video of each projector by the offset value and increasing the luminance by the offset value in a computer. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and polarized light provided in front of each projector A board, a computer provided for each projector, and a storage device,
An image for each area projected by each projector is generated in the pixel buffer by each computer, and an image obtained by correcting the distortion of the generated image by an inverse function of mapping for each projector previously obtained in claim 1 is used for the pixel buffer. A first process to generate;
A second process of projecting, from each projector, a distortion corrected image in the pixel buffer onto the quadric curved screen;
A composite image generation method comprising a third step of dimming a distortion-corrected image projected from each projector using the polarizing plate when a night scene is represented using an image with corrected distortion .
A quadratic curved screen, a plurality of n projectors disposed in front of the screen and projected onto the screen, a camera disposed in front of the screen and photographed on the screen, and a computer; A storage device,
Test pattern projected from the projector onto the quadric surface screen by performing matrix calculation on the transformation matrix (M) from the reference coordinate system to the viewpoint coordinate system and the transformation matrix (S) from the camera coordinates to the reference coordinate system. Transformation matrix calculation means for obtaining a transformation matrix (H) from the viewpoint coordinate system to the camera coordinate system for the predetermined point of
By using the transformation matrix (H) and the inverse matrix of the transformation matrix (H), a quadratic surface obtained in the camera coordinate system by performing matrix calculation on the quadric surface parameter (Q) obtained in the camera coordinate system. A quadric surface parameter (Qv) calculating means for obtaining a quadric surface parameter (Qv) in a viewpoint coordinate system having an assumed viewpoint as an origin from the parameter (Q);
Using the quadric surface parameter (Qv) given by the matrix under the constraint that the test pattern is on the quadric surface, the points projected on the screen by the matrix calculation were viewed from the projector i coordinate system. Mapping function and inverse function calculating means for obtaining a mapping function indicating a correspondence between a point and a point viewed from an arbitrary hypothetical viewpoint as a matrix, and using the mapping function given by the matrix to obtain an inverse function by matrix calculation When,
A mapping function generation device comprising a storage device storing a mapping function and its inverse function for each of the plurality of projectors.
A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a computer provided for each projector And a storage device,
An image for each area projected by each projector is generated in the pixel buffer by each computer, and an image obtained by correcting the distortion of the generated image by an inverse function of the mapping for each projector previously obtained in claim 12 is stored in the pixel buffer. Video generation means for generating;
A composite image generating apparatus comprising: a plurality of n projectors for projecting each distortion corrected image from a pixel buffer of each computer onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors arranged so that each region projected on the screen in front of the screen overlaps with each other, and a screen located in front of the screen A camera arranged to take a picture, a computer, and a storage device,
Test pattern projected from the projector onto the quadric surface screen by performing matrix calculation on the transformation matrix (M) from the reference coordinate system to the viewpoint coordinate system and the transformation matrix (S) from the camera coordinates to the reference coordinate system. Transformation matrix calculation means for obtaining a transformation matrix (H) from the viewpoint coordinate system to the camera coordinate system for the predetermined point of
By using the transformation matrix (H) and the inverse matrix of the transformation matrix (H), a quadratic surface obtained in the camera coordinate system by performing matrix calculation on the quadric surface parameter (Q) obtained in the camera coordinate system. From the parameter (Q), either the right eye or the left eye as the origin, and the right eye or the left eye as the origin, and the right eye or the left eye as the origin. A quadric surface parameter (Qv) calculating means for obtaining a quadric surface parameter (Qv) in any one of the other viewpoint coordinate systems;
The quadratic surface parameter (Qv) given by the matrix in the right-eye and left-eye viewpoint coordinate systems is used under the constraint that the test pattern is on the quadratic surface, and is projected onto the screen by matrix calculation. A mapping function indicating a correspondence between a point seen from the projector i coordinate system and a point seen from an arbitrary right-eye and left-eye assumed viewpoint as a matrix, and using the mapping function given by the matrix, a mapping function and the inverse function calculating means for calculating the inverse function by calculation,
A mapping function generation device comprising a storage device for storing a mapping function for right eye and left eye and an inverse function thereof for each of the plurality of projectors.
A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a computer provided for each projector And a storage device,
The computers generate right-eye and left-eye images for each projector in a pixel buffer, and use the inverse functions of the right-eye and left-eye mapping for each projector previously determined in claim 14 to generate the generated images. Image generation means for generating a distortion corrected image in a pixel buffer;
A composite image generation apparatus comprising: a plurality of n projectors that project right-eye and left-eye images corrected for distortion in the pixel buffer onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors disposed in front of the screen and arranged to have a common overlapping area projected onto the screen, a computer, and a storage device,
An image for each area projected by each projector is generated in the pixel buffer by each computer, and an image obtained by correcting the distortion of the generated image by an inverse function of the mapping for each projector previously obtained in claim 12 is stored in the pixel buffer. Video generation means for generating;
A plurality of projectors arranged so as to project the distortion-corrected images from the pixel buffer of each computer onto the quadric curved screen to increase the brightness of the common overlapping portion. A composite video generator. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a video device,
The image generated from the video device is divided and generated so as to be projected onto the plurality of n projectors onto the screen, and the divided and generated image is subjected to distortion correction by the inverse function of the mapping for each projector previously obtained in claim 12. Video generation means for generating video;
A composite image generating apparatus comprising: a plurality of n projectors that project the distortion-corrected images onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged so that each region projected onto the screen overlaps with each other, and a video device,
The division generation is performed by dividing and generating the image from the video device so as to be projected onto the screen on the plurality of n projectors, and by the inverse function of the right-eye and left-eye mapping for each projector previously obtained in claim 14. Image generation means for generating a distortion corrected image,
A composite image generating apparatus comprising a plurality of n projectors that project the distortion-corrected right-eye and left-eye images onto the quadric curved screen. A quadratic curved screen, a plurality of n projectors disposed in front of the screen and arranged to have a common area projected onto the screen and overlapping, and a video device,
The image generated from the video device is divided and generated so as to be projected onto the plurality of n projectors onto the screen, and the divided and generated image is subjected to distortion correction by the inverse function of the mapping for each projector previously obtained in claim 12. Video generation means for generating video;
A composite video generation apparatus comprising a plurality of n projectors arranged to project the distortion-corrected video onto the quadric curved screen and increase the brightness of the common overlapping portion.   20. The composite video generation device according to claim 17, 18 or 19, wherein the video equipment is a television device, a video playback device, or a personal computer. In the composite video generation device according to claim 9 or claim 11 or claim 12 or claim 17 or claim 18 or claim 19 or claim 20,
A quadratic curved screen, a plurality of n projectors arranged in front of the screen and arranged to project on the screen, and a camera arranged in front of the screen and arranged to photograph the screen ,
A camera that shoots a screen in a first state that is not projected from any of the projectors and that shoots a second state of the screen that projects a black state from each projector;
The average value of the screen by each projector and the sum of the average values by the difference between the luminance of the video for each projector that has projected the second state photographed by the camera and the luminance of the video that has photographed the first state An offset calculation means for obtaining a maximum value by means of and obtaining an offset value by a difference between the maximum value and the average value;
A composite video generation apparatus comprising: video generation means for generating video by reducing the contrast of the overlapping portion of the video for each projector by the offset value and increasing the luminance by the offset value by the computers. A quadratic curved screen, a plurality of n projectors arranged in front of the screen and projecting onto the screen with overlapping portions, arranged so as to be continuous, a polarizing plate, and each projector Having a computer and a storage device,
An image for each area projected by each projector is generated in the pixel buffer by each computer, and an image obtained by correcting the distortion of the generated image by an inverse function of the mapping for each projector previously obtained in claim 12 is stored in the pixel buffer. Video generation means for generating;
A plurality of projectors for projecting each distortion-corrected image from the pixel buffer of each computer onto the quadric curved screen;
A composite image generating apparatus comprising: a polarizing plate provided in front of each projector for reducing the amount of light from each projector when representing a night scene.   15. The mapping function generation device according to claim 12, wherein each projector is attached to a base by a flexible arm having a degree of freedom. 23. The composite video generation apparatus according to claim 13, wherein each projector is attached to a base by a flexible arm having a degree of freedom.

Images