JP2006285482A - Device for correcting image geometry - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To produce a natural and correct three-dimensional CG image, by obtaining a continuous visual field without generating flexion or distortions, when an image, where a visual field is shifted after zooming operation, is projected on a screen with the use of a projection system, in a virtual imaging system by a plurality of virtual cameras. <P>SOLUTION: An image geometry correction device comprises a scene data producing means 1; a virtual imaging image data operation means 2; a projection image generating means 3 for generating a virtual projection image to be a three-dimensional CG or VR image for projection, based on the data from the producing means 1 and the operation means 2; a projection system arrangement adjusting means 4; a first correction parameter setting change means 6 for changing the setting of the correction parameters of the virtual imaging system; a second correction parameters setting change means 7 for changing the setting of the correction parameters of the projection system; a composition correction parameter setting means 8 for setting the composition correction parameter of the virtual imaging system and the projection system; and an image-deforming means 9 for performing deformations and corrections to obtain the image for projection, based on the composition correction parameter which is outputted from the setting means 8. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video geometry correction apparatus for correcting distortion or the like of a video image using a geometric technique when producing a 3D video image using 3D computer graphics.

  In general, when viewing an image by projecting it on a projection screen, it may be necessary to enlarge and display an object in the image to be viewed for the purpose of producing or explaining the image to the viewer.

  When magnifying and displaying an object in such an image, a zoom display method that zooms in on the object in the image field by a telescope and closes the object in the image field. There is an enlarged display method by a close-up observation operation for enlarged display.

  In particular, when the viewing environment of the video display is an immersive virtual reality (VR) environment oriented to the accuracy of the size as an apparent angle as a strict geometry (geometric) reproduction, the zoom operation In many cases, the image is observed by magnifying display by an approach observation operation for magnifying and displaying the object by approaching the video object.

  However, the magnified display method by this approach observation operation may cause the following inconvenience.

  For example, considering the case of celestial simulation during solar eclipse, the explanatory diagram used for the explanation of solar eclipse is the explanatory diagram (sun S, moon M, earth A, Observation point O, sun observation line of sight L1), and in the case of total solar eclipse, an explanatory diagram as shown in FIG. 18 (sun S, moon M, earth A, observation point O, moon observation line of sight L2), sun S, moon M The sizes of the celestial bodies arranged in the order of the earth A are larger in the order of the moon M, the sun S, and the earth A.

  However, in the image reproduction at the time of solar eclipse as viewed from the earth A, it is important that the small moon M at a short distance from the earth A and the large sun S at a long distance are apparently substantially the same size.

  In this case, if the viewpoint is moved closer to the direction of the phenomenon in order to observe the phenomenon in an enlarged manner, the apparent size of both is different from the substantially same size before the approach, and is viewed from the earth A. The purpose of image reproduction during solar eclipse cannot be achieved. Therefore, a zoom function and a zoom operation that can expand the field of view without moving the viewpoint are essential.

  Generally, as a video projection method, for example, a screen projection method using one flat projection screen and one projector, and a multi-screen projection method using three flat projection screens and three projectors. And a multi-projector projection method using a cylindrical curved projection screen and two or more projectors.

  Here, a configuration example of a typical virtual reality system (hereinafter referred to as VR) of the multi-projector projection method is shown below.

FIG. 19 shows three virtual cameras C1, C2, and C3 for virtual shooting directed in three directions within a VR scene F (VR virtual shooting field of view, dotted line) where a subject j is located from a certain virtual shooting point P1 (fixed point). FIG. 20 is a front view showing virtual shooting frames f1, f2, and f3, respectively, and FIG. 20 shows a cylindrical curved projection screen R and a projection fixed point P2 (at the same position as the VR observer in the cylindrical curve of the screen R). (Viewing point, ornamental point) and three projectors T1, T2, T3, which are set at the same angle of view and direction as the virtual cameras C1, C2, C3, and video projection frames t1, t2, t3 respectively. FIG.

  Normally, perspective projection is used for rendering in VR, but in this example, the geometric conditions (direction, angle of view) on the virtual photographing camera C1, C2, C3 side (rendering side) of the virtual photographing point P1, and Since the geometric conditions on the projector T1, T2, T3 side of the projection fixed point P2 are the same, a projection result without distortion can be obtained without correcting the image. If the depth of focus of the projection lens of the projector and the luminance unevenness are not taken into consideration, the cylindrical curved projection screen R does not need to be a cylinder centered on the projection fixed point P2 (observation point, viewing point).

  By the way, each of the virtual shooting frames f1, f2, and f3 by the virtual shooting cameras C1, C2, and C3 shown in FIG. 19 is zoomed in the reduction direction by simply zooming with the respective virtual cameras (the angle of view is changed). Accordingly, as shown in FIG. 21, the virtual photographing frames f1, f2, and f3 of the virtual cameras C1, C2, and C3 are naturally reduced, so that the photographing optical axes O1, O2,. Unless the direction of O3 is corrected, a continuous field of view in the VR scene F as shown in FIG. 19 cannot be obtained.

  Here, as shown in the plan view of FIG. 23, if there is a wall W of a finite height extending infinitely in the lateral direction in the VR scene F (see FIG. 19), each virtual camera C1, Of the virtual captured images of the walls W by the virtual captured frames f1, f2, and f3 of C2 and C3, the virtual captured images of the wall W by the virtual captured frames f1 and f3 from the left and right virtual cameras C1 and C3 are shown in FIG. As shown in the figure, it appears as an image that narrows toward the left end of the virtual photographing frame f1, and also appears as a perspective image that narrows toward the right end of the virtual photographing frame f3.

  Further, when attention is paid to the upper and lower edges of the virtual photographed image of the wall W in each of the virtual photographing frames f1, f2, and f3 shown in FIG. 24, there is a bending point at the screen boundary of each virtual photographing frame. Since the projection conditions have the same geometrical conditions as the virtual imaging system without any adjustment change of the imaging conditions (directions of the imaging optical axes O1, O2, O3, the sizes of the virtual imaging frames f1, f2, f3), the above-mentioned figure. At the time of projection by the projection system shown in FIG.

  Here, as shown in FIG. 25, when the same virtual photographing operation is performed in the state where zooming is performed from the state of FIG. 23, the elevation angles of the left and right virtual cameras C1 and C3 become small. In addition, as shown in FIG. 26, the virtual captured image of the wall W by the virtual captured frames f1 and f3 from the left and right virtual cameras C1 and C3 is seen as a video narrowed toward the left end of the virtual captured frame f1, Although it appears as a video that narrows toward the right end of the virtual shooting frame f3, its perspective is weaker than the virtual shooting images of the wall W by the virtual shooting frames f1 and f3 shown in FIG.

  By the way, when the virtual photographed image of the wall W shown in FIG. 26 is projected as an image by the projection system shown in FIG. 20, the adjustment change of the virtual photography condition of the zoom operation occurs between the photography system and the projection system. For this reason, the projection system has different geometric conditions from the virtual shooting system (the virtual shooting image of the virtual shooting system has a different bending angle from the video of the projection system). The upper and lower ends are not observed as one straight line, and the bending is not eliminated.

As a solution to the bending that occurs when such a projection system has different geometric conditions from the virtual shooting system (the virtual shooting image of the virtual shooting system has a different bending angle from the video of the projection system) As shown in FIG. 27, the virtual photographing system shown in FIG. 21 is obtained by projecting the virtual photographed image of the wall W by the virtual photographing frames f1 and f3 from the cameras C1 and C3 during the rendering operation in the computer graphic CG video production. While maintaining the same optical axis direction as the photographic optical axes O1 and O3, the fields of view (virtual photographic frames f1 and f3) are shifted in the direction perpendicular to the optical axes O1 and O3 (in the direction of the arrows), respectively. (Similar to tilting), a continuous field of view can be obtained, the bending is eliminated on the screen, and the upper and lower edges are observed as a single straight line from the viewing position. .

Known patent documents related to the present invention are described below.
JP 2004-206551 A JP 2004-258287 A

  However, for example, as shown in FIG. 28, virtual photographed images (that is, magnification factors in the projection system) in virtual photographed frames f1, f2, and f3 that are virtually photographed by zooming greatly in the reduction direction in the virtual photographing system. The angle formed by the boundary between the optical axis O1 of the virtual camera subject to field-of-view shift to be corrected for bending and distortion, for example, the virtual axis C1 of the virtual camera C2 and the virtual photographing frame f2 of the adjacent virtual camera C2. When the distance is considerably large (for example, 90 degrees or more), even if the visual field shift is performed, the bending and distortion cannot be corrected, and a continuous visual field cannot be obtained.

  Also, for example, as shown in FIG. 29, assuming an object j that is relatively close to the front surface of each of the virtual photographing frames f1, f2, and f3 of the respective virtual photographing cameras C1, C2, and C3, the line is aligned. Since the distance from the virtual shooting point P1 increases as the distance to the object j on the end side lined in the line increases, when each of the virtual shot images is projected on the screen R as a video in the projection system shown in FIG. From the viewing viewpoint P2, as shown in FIG. 30, the image looks as small as the object j on the end side.

  When the central portion F1 of the video shown in FIG. 30 is enlarged, as shown in FIG. 31, the objects j and j on both sides adjacent to the central object j are slightly different in apparent size. The only difference is that the object j on both sides should be observed slightly smaller than the object j at the center, and the image obtained by photographing the object j that is close and aligned is It is desirable that the feeling is weak.

  However, the virtual photographed images obtained by the virtual photographing frames f1, f2, and f3 of the virtual photographing cameras C1, C2, and C3 shown in FIG. 29 have a large reduction ratio in the virtual photographing system as shown in FIG. Is a virtual photographed image (that is, a video with a high magnification in the projection system) in the virtual photographing frames f1, f2, and f3 that are virtually photographed by zooming, and is a virtual subject of visual field shift to be corrected for bending and distortion. When the angle formed by the optical axis O1 of the camera, for example, the virtual camera C1 and the optical axis O2 of the adjacent virtual camera C2 is considerably distant (for example, 90 degrees or more), the visual field shift is performed to correct the difference. Even if an attempt is made to obtain a continuous field of view on the screen R with the projection system shown in FIG. 32, the objects j and j on both sides adjacent to the center object j are extremely Will be fence observed, very unnatural, becomes inaccurate images, can cope with field shift angle is the case within 90 °.

The present invention is to eliminate the above inconveniences, even when the angle formed by the virtual camera optical axis adjacent to the virtual camera subject to visual field shift and the adjacent virtual camera optical axis is considerably separated in the virtual photographing system. Even if the angle formed by the boundary between the virtual shooting frame of the virtual camera subject to the virtual shooting frame shift and the virtual shooting frame of the adjacent virtual camera is quite far away, the screen can be corrected in the projection system by performing field-of-view shift correction. In addition, it is to be able to obtain a natural and accurate three-dimensional CG image by obtaining a continuous field of view without occurrence of bending or distortion in the image corresponding to the boundary of each virtual photographing frame.

  The present invention relates to a virtual imaging system that creates scene data (scene data) such as an object (for example, an image of an object j) or a background image of a 3D CG scene F or VR scene F as virtual captured image data. Driven as a three-dimensional CG image or VR image based on the scene data creation means 1 corresponding to a plurality of virtual cameras (for example, C1, C2, C3,...) And the three-dimensional CG or VR virtual photographed image data. Based on the virtual photographed image data manipulation means 2 of the virtual photography system for creating manipulation data for manipulation, and the scene data and virtual photographed image manipulation data captured from the scene data creation means 1 and the virtual photographed image data manipulation means 2. Projection image generation means 3 for generating a virtual projection image of a virtual photographing system to be a three-dimensional CG or VR image for projection, and a projector (for example, T1) , T2, T3,...) And curve screen R, etc., based on the virtual projection image data from the projection system setting adjustment unit 4 for setting correction parameters related to physical installation conditions and the projection image generation unit 3. First correction parameter setting changing means 6 for changing the setting of the correction parameters of the virtual photographing system related to the rendering corresponding to the zoom operation and the adjustment of the viewing angle, and the correction parameters of the projection system set by the projection system setting adjusting means 4 The second correction parameter setting changing means 7 for changing the setting, and the composition for setting the composite correction parameter based on the virtual imaging system and projection system correction parameters output from the first and second correction parameter setting changing means 6, 7. Correction parameter setting means 8, composite correction parameters output from the composite correction parameter setting means 8, and the projection image generation means A geometric correction means 5 comprising a video deformation means 9 for obtaining a virtual projection image for curve screen projection without distortion by deformation correction based on the virtual projection image data outputted from the A video geometry correction apparatus comprising: a recording / reproducing unit 10 that records a curved screen projection image corrected in the above; or a projector unit T (for example, T1, T2, T3) that projects a projection image onto a curved screen. It is.

  The invention according to claim 2 of the present invention relates to rendering corresponding to a zoom operation and adjustment of a viewing angle based on virtual projection image data from the projection image generation means in the video geometry correction device according to claim 1. The first correction parameter setting changing means for changing the setting of the correction parameters of the virtual photographing system is composed of one or a plurality of correction parameter setting changing means, and the projection system correction parameters set by the projection system installation adjusting means are set. The second correction parameter setting changing means for changing the setting comprises one or a plurality of correction parameter setting changing means.

  The image geometry correction apparatus of the present invention can change and correct the correction parameters dynamically and smoothly at high speed (for example, 30 times or more per second) during projection. It can be changed and corrected following the zoom operation, and all correction factors (video correction related to the virtual shooting system and video correction related to the projection system) are held as composite correction parameters (one set of correction parameters). Therefore, when the zoom operation is performed, only the pixel portion of the video depending on the rendering viewing angle can be changed and corrected, and the correction parameters corresponding to the two types of factors of the virtual shooting system and the projection system are independent. Can be set to

Further, the video geometry correction device of the present invention uses the geometry correction means 5 to obtain a correction parameter related to the virtual photographing system obtained from the projection condition at the time of rendering and a correction parameter related to the projection system based on the physical projection condition, respectively. One or more can be stored and changed individually. In the correction by these, the final correction result is the correction by the combined correction parameter of the virtual photographing system and the projection system.

  For this reason, the number of image correction operations can be reduced by obtaining a composite correction parameter that combines multiple correction parameters before performing geometric correction on the video, and linked to the zoom operation. Smooth correction parameter change is possible.

  As described above, the image geometry correction apparatus of the present invention corrects image distortion caused by a zoom operation and image distortion caused by a projection system in order to realize a zoom function in a virtual photographing system by a curved screen multi-plane projection method. Both of the deformation correction and the correction can be performed at high speed.

  Embodiments of the image geometry correction apparatus according to the present invention will be described in detail below.

  First, as shown in the plan view of FIG. 1, images jf, jf, jf,... Of a plurality of objects of the same shape and size placed at equal distances so as to surround the viewing viewpoint P2 in the projection system. Consider a situation in which projection is performed along a circumferential direction on a cylindrical curved screen R.

  Considering the situation in which the images jf, jf, jf,... Of each object are enlarged on the curve screen R in the projection system, as shown in FIG. The images jf, jf, jf,... Of the objects that have been viewed can be viewed in the same size even after enlargement, and when enlarged or reduced in the projection system, they are photographed by the zooming technique in the virtual imaging system. There is no unnatural difference in size or distortion that occurs when enlarging or reducing an image.

  Here, as shown in the plan view of FIG. 3, in the virtual imaging system, a plurality of (two or more) virtual cameras for CG production, for example, three virtual cameras C1, C2, C3, etc. Among the objects j, j, j... Arranged at the distance, the center of each field of view (virtual photographing frames f1, f2, f3) of each virtual camera C1, C2, C3 (θo is the camera angle of view) As shown in FIG. 4, since CG rendering is performed by perspective projection when attention is paid to each object j (for example, bold circle J) separated by an angle θ from (imaging optical axes O1, O2, O3 of each virtual camera). These objects j (J) are the center lines O1, O1,... Of the respective visual fields in the virtual images j, j, j... Of the respective objects generated in the respective visual fields (virtual photographing frames f1, f2, f3). CG rendering is performed as an object J larger than the object j close to O2 and O3.

If the drawing size of the object j at the center of each camera field of view (f1, f2, f3) is Lo and the drawing size of the object J at the center of each camera field of view is La,
La = Lo / cos θ (1)
It becomes.

Also, if the angle of view of each virtual camera is θo and the drawing size of the object j at the left and right ends of each field of view (f1, f2, f3) is L,
L = Lo / cos (θo / 2). (2)
Then, the virtual images j, j, j,... Of each object obtained by CG drawing obtained by the virtual photographing system are apparently shown in FIG. 20 without adding a zoom operation. By projecting onto the screen R by a projection system set to a geometric condition for projecting without distortion, it is possible to view an image of each object without distortion.

Here, as shown in the plan view of FIG. 5, the zooming operation is performed by reducing the angle of view of each virtual camera from θo to θz. Here, in order to prevent the occurrence of the above-described problem, the shooting operations shown in FIGS. 5 and 6 are used instead of the shooting operations shown in FIGS. 27 and 28. As shown in FIG. The virtual photographing frames f1, f2, and f3 of the cameras C1, C2, and C3 are reduced from the virtual photographing frames f1, f2, and f3 shown in FIG. 3, and then the directions of the photographing optical axes O1, O2, and O3 are corrected. Thus, a continuous field of view in the CG scene F is obtained.

  As a result, a virtual captured image of each object j based on the CG rendering result as shown in FIG. 6 is obtained. At that time, since the ratio of the drawing size of the object j at the center and the left and right ends of the virtual camera field of view depends on the camera angle of view θz after the zoom operation as shown in the above equation (2), FIG. The value is different from the case of the camera angle of view θo as shown in FIG.

  In this way, the virtual shooting image that has been virtually shot with the zoom operation added in the virtual shooting system has a geometric condition for projecting the virtual shot image without adding the zoom operation without any apparent distortion. When projected onto the screen R by the set projection system shown in FIG. 20, the image is distorted.

Generally, the horizontal coordinate in one image (for example, in one unit image area such as each left (direction) image, intermediate image, and right (direction) image with parallax) is X (−1.0 <X <1.0). If the drawing size of the object j at the center of the visual field is Lo and the drawing size of the object at the coordinate X is Lx,
Lx = Lo / cos (atan (tan (θz / 2) × X) (3)
It becomes.

  Therefore, in order to project an image on the screen without distortion, it is necessary to correct the drawing size difference expressed by the above equation (3) between the non-zoom operation and the zoom operation for each X coordinate position. In the invention, the distortion correction of the video in the projection system when the zoom operation is added in the virtual photographing system is performed by the geometry correction.

  In many real-time three-dimensional computer graphics processing systems, the projection coordinate transformation at the time of rendering supports only the primary transformation, so the distortion correction calculation corresponding to the above equation (3) is performed as post-processing after image generation. Although there is a need for a geometry correction function, a multi-projector system using a curved screen often employs a geometry correction technique for correcting distortion by deforming an image, and is described with reference to FIGS. 19 and 20 described above. In the present invention, the virtual imaging system described with reference to FIGS. 19 and 20 is used to remove distortion caused by the difference between the ideal projection condition and the actual projection condition. By correcting the geometry of the zoomed virtual image, an image without distortion under the projection conditions in the ideal projection system on the screen R can be obtained. It is Utosuru thing.

  For example, as shown in the plan view of FIG. 7, a plurality of projectors (two or more projectors corresponding to the number of virtual cameras for CG production) arranged in the center of the cylindrical curved screen R, for example, each of the three projectors T1 Installation where the principal point on the optical axis of the projection lens of T2, T3 and the cylindrical center of the cylindrical curved screen R are completely matched is not necessary unless special measures such as bending the optical path with a pyramid mirror are taken. This is possible, and as shown in the plan view of FIG. 8, a simple method of shifting back and forth from the center is often adopted.

  In this case, since the distance from the principal point of the projection lens, which should be all equal, to the screen R is different in the screen, a barrel or pincushion distortion occurs.

Further, as shown in the side views of FIGS. 9 and 10, the vertical positions of the projection lenses of the projectors T1, T2, and T3 with respect to the screen R are not on the center line of the screen to prevent interference with the seats and the viewers. In this case, parallax occurs due to the depth of the curved screen R, and the image is distorted. In addition, the aberration of the optical system, the screen There are factors of image distortion, such as the shape error of the projector and the installation error of the projector.

  As described above, the present invention corrects image distortion in a projection system caused by a zoom operation in a virtual photographing system and performs projection in order to realize a zoom function in multi-plane projection by a curved screen R. Both correction processing for correcting image distortion caused by the system itself is realized.

  As shown in the block diagram of FIG. 11, the video geometry correction apparatus according to the present invention includes scene data (for example, an image of an object j, for example) or a background image of a three-dimensional CG scene F or VR scene F, such as a virtual image. Scene data creation means 1 (virtual camera) corresponding to a virtual camera (for example, C1, C2, C3,...) That creates scene data) as virtual photographed image data, and the three-dimensional CG or VR. And a virtual photographed image manipulation data creation means 2 of a virtual photography system that creates manipulation data for driving operation as a three-dimensional CG image or a VR image based on the virtual photographed image data.

  Further, based on the scene data and virtual photographed image manipulation data captured from the scene data creation means 1 and the virtual photographed image manipulation means 2, a virtual projection image of a virtual photographing system that becomes a three-dimensional CG or VR video for projection is generated. Projection image generation means 3 and projection system installation adjustment means 4 for setting correction parameters relating to physical installation conditions such as projection system projectors (for example, T1, T2, T3,...) And curve screen R. .

  The projection image signal (view angle information and the like) from the projection image generation unit 3 and the correction parameter signal related to the physical installation condition from the projection system installation adjustment unit 4 are input to the geometry correction unit 5.

  As shown in FIG. 11, the geometry correction unit 5 sets and changes the correction parameters of the virtual photographing system related to the rendering corresponding to the zoom operation and the adjustment of the viewing angle based on the virtual projection image data from the projection image generation unit 3. First correction parameter setting changing means 6 for performing the above, second correction parameter setting changing means 7 for changing the setting of the correction parameters of the projection system set by the projection system setting adjusting means 4, and the first and second correction parameter setting. Based on the composite correction parameter setting means 8 for setting the composite correction parameter based on the virtual imaging system and projection system correction parameters output from the changing means 6 and 7, and the composite correction parameter output from the composite correction parameter setting means 8. And an image deformation means 9 for deforming and correcting distortion as an image for projecting a curved screen.

  The scene data creation means 1 creates 3D scene data of a 3D CG scene F or VR scene F composed of the shape and position of an object.

  The virtual photographed image operation means 2 creates operation information data for driving operation as a three-dimensional CG image or VR image based on the three-dimensional CG or VR virtual photographed image data, and outputs the operation information data.

  The projection image generation means 3 drives the 3D CG or VR virtual captured image data as a 3D CG image or VR image based on the operation information data output from the virtual captured image operation means 2, and A video of a CG image or a VR image is generated.

  The projection system installation adjusting means 4 sets correction parameters relating to physical installation conditions such as a projection system projector (for example, T1, T2, T3,...) And a curve screen R.

  The first correction parameter setting changing unit 6 of the geometry correcting unit 5 is based on the virtual projection image data from the projection image generating unit 3 and is a virtual imaging system correction parameter related to the rendering corresponding to the zoom operation and the adjustment of the viewing angle. Change the setting.

  The second correction parameter setting changing unit 7 changes the setting of the projection system correction parameter set by the projection system installation adjusting unit 4.

  The composite correction parameter setting unit 8 combines the virtual imaging system correction parameter output from the first correction parameter setting change unit 6 and the projection system correction parameter output from the second correction parameter setting change unit 7. Set correction parameters.

  The video deformation means 9 is for correcting and correcting distortion as a video for curve screen projection based on the composite correction parameter output from the composite correction parameter setting means 8.

  The curved screen projection video data whose distortion has been corrected by the video transformation means 9 is recorded on a recording medium such as a magnetic disk, a magnetic tape, a magneto-optical disk (CD-ROM) by the recording / reproducing means 10, and Based on the image data for curve screen projection that has been reproduced and corrected for distortion by the image transformation means 9 in the projector T (for example, projectors T1, T2, T3), the image is projected in the projection system shown in FIG. Projecting onto the curve screen R.

  An example in which correction parameters are described by lattice deformation in the image correction operation by the image modification by the geometry correction means 5 will be described below.

  As shown in FIG. 12, as an example, a grid is defined by dividing the screen region into four equal parts in the horizontal (x-axis) direction and three equal parts in the vertical (y-axis) direction, and the coordinates of each grid point are defined as Pnm (xn , ym). For the sake of explanation, the number of divisions is reduced, but in practice it is subdivided.

  FIG. 13 shows the lattice shape (solid line portion) after the geometry correction based on the correction parameter. By defining the coordinate on the screen of the corrected lattice point as Pnm ′ (xnm, ynm), the deformation of the entire screen is shown. Can be expressed.

For example, in FIG. 12, when considering a rectangular region surrounded by four points of grid points P (00), P (01), P (10), and P (11), this deformation operation is described by affine transformation. be able to. If the coordinates of the point in the region before deformation are p (x, y) and the coordinates after deformation of this point based on the correction parameters are q (x ', y'),
q = Mp + d
It becomes. Where M; 2 × 2 transformation matrix d; amount of movement in the x-axis and y-axis directions (dx, dy)
By performing this affine transformation on all the pixels in all the regions in the image, the geometry correction is performed on the screen projection image.

  Further, as another example of the image correction operation by the image deformation by the other geometry correction means 5, in addition to the above method, in order to increase the degree of freedom of the affine conversion, the coordinates of the pixel before and after the conversion are set. There are a method using a coordinate conversion table associated one-to-one, a method using a higher-order function for coordinate conversion for smoothing, and the present invention apparatus is also applied in the same manner when using these methods. Can do.

Further, as the multi-stage geometry correction, a correction parameter P1 related to the virtual photographing system related to the rendering viewing angle shown in FIG. 14 and a correction parameter P2 related to the projection system shown in FIG. 15 are used.

  A correct projection image can be obtained by deforming the video based on the correction parameter P1 related to the virtual photographing system and further using the correction parameter P2 related to the projection system. The correction parameter setting means 8 generates a combined correction parameter P3 obtained by combining the P1 and P2, and the image transformation means 9 is used to perform data processing on the coordinate value of each pixel based on the combined correction parameter P3. By executing the calculation, the geometry correction can be performed on the screen projection image by one deformation process.

  Specifically, the composite correction parameter P3 shown in FIG. 16 is obtained by converting the respective grid point coordinates of the image of the virtual photographing system constituting the correction parameter P1 by the correction parameter P2. The generation of the composite correction parameter P3 is shown in FIG. Is a calculation by the synthesis correction parameter setting means 8 for a limited lattice point coordinate in the video, so that the synthesis correction parameter can be obtained at a very high speed compared to the image deformation processing for all the pixels in the video. Based on the composite correction parameter P3, the geometry correction for the screen projection image can be executed by the image deformation means 9 at a very high speed.

  This high-speed processing method using the composite correction parameter P3 is a method using a coordinate conversion table in which the coordinates of pixels before conversion (before deformation correction) and after conversion (after deformation correction) are associated one-to-one, coordinate conversion The apparatus of the present invention can be similarly applied to the case of using another deformation correction method such as a method using a higher order function.

  In addition, regarding the change of the correction parameter, in order to improve the change accuracy, the number of lattice points actually defined is about 1000 points, for example, when a 32-division lattice is used.

  This data amount is 4 Kbytes with each coordinate value being 4 bytes. If the parameter change is performed 30 times per second, the transmission data amount is 120 Kbytes / second. This data amount can be sufficiently transmitted by a relatively high-speed interface such as LAN or USB.

  In addition, the correction parameter that requires high-speed change processing is related to the CG rendering viewing angle. However, as shown in the above formulas (2) and (3), this correction parameter can be calculated from the viewing angle. Actually, only the viewing angle information is transmitted, and the correction parameters may be calculated on the geometry correction means 5.

  As a result, the amount of transmission data is much smaller. For example, if the viewing angle information is 4 bytes and the number of changes is 30 times per second, the transmission data amount is 120 bytes / second, which is a data amount that can be sufficiently transmitted even with a low-speed interface such as a serial interface.

  In addition to the structural form shown in the block diagram of FIG. 11, the apparatus of the present invention has the above-described geometry correction means 5 having a correction function, projection image generation means 3, projection system installation adjustment means 4, recording / reproduction means 10, and projector T. (T1, T2, T3, etc.) may be integrated, the projection image generation means 3 and the video deformation mechanism may be integrated, or a plurality of devices including the geometry correction means 5 may be provided. The correction parameters P1 and P2 may be connected in series and the deformation correction may be executed by different devices.

The top view of the image | video projection system explaining the image | video geometry correction apparatus of this invention. The top view explaining expansion of the picture in the picture projection system explaining the picture geometry correction device of the present invention. The top view of the virtual imaging | photography system explaining the image | video geometry correction apparatus of this invention. The top view of the picked-up image of the virtual imaging | photography system explaining the image | video geometry correction apparatus of this invention. The top view of zoom operation of the virtual imaging | photography system explaining the image | video geometry correction apparatus of this invention. The top view of the picked-up image by zoom operation of the virtual imaging | photography system explaining the image | video geometry correction apparatus of this invention. The top view of the image | video projection system explaining the image | video geometry correction apparatus of this invention. The top view of the image | video projection system explaining the image | video geometry correction apparatus of this invention. The side view of the image | video projection system explaining the image | video geometry correction apparatus of this invention. The side view of the image | video projection system explaining the image | video geometry correction apparatus of this invention. The apparatus block diagram explaining the image | video geometry correction apparatus of this invention. The top view of the image | video lattice and lattice point before the deformation | transformation correction | amendment of the image | video by the image | video geometry correction apparatus of this invention. The top view of the image | video lattice and lattice point after the deformation | transformation correction | amendment of the image | video by the image | video geometry correction apparatus of this invention. The top view of the image | video grid and grid point after the deformation | transformation correction | amendment of the image | video of the virtual imaging | photography system by the image | video geometry correction apparatus of this invention. The top view of the image | video grid and grid point after the deformation | transformation correction | amendment of the image | video of a projection system by the image | video geometry correction apparatus of this invention. The top view of the image | video grating | lattice and lattice point after the synthetic | combination correction | amendment by the image | video geometry correction apparatus of this invention. The schematic diagram explaining a three-dimensional CG image and a VR image. The schematic diagram explaining a three-dimensional CG image and a VR image. The top view of a general virtual imaging | photography system. The top view of a general projection system. The top view explaining zoom operation in a general virtual photography system. The top view explaining zoom operation in a general virtual photography system. The top view of a general virtual imaging | photography system. The top view of the picked-up image of a general virtual imaging | photography system. The top view explaining zoom operation in a general virtual photography system. The top view of the picked-up image by zoom operation of a general virtual imaging | photography system. FIG. 6 is a plan view illustrating a tilt shooting operation in a general virtual shooting system. FIG. 6 is a plan view illustrating a tilt shooting operation in a general virtual shooting system. The top view of a general virtual imaging | photography system. The top view of the picked-up image of a general virtual imaging | photography system. The top view of the picked-up image of a general virtual imaging | photography system. The top view of the picked-up image of a general virtual imaging | photography system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Scene data preparation means 2 ... Virtual picked-up image data operation means 3 ... Projection image generation means 4 ... Projection system installation adjustment means 5 ... Geometry correction means 6 ... 1st correction parameter setting change means 7 ... 2nd correction parameter setting change means 8: Composite correction parameter setting means 9 ... Video deformation means 10 ... Recording / reproducing means F ... 3D CG scene or VR scene C1, C2, C3 ... Virtual camera f1, f2, f3 ... Virtual shooting frame j ... Object jf ... Image P1 ... Virtual shooting point P2 ... Projection fixed point R ... Curve screen T ... Projector means T1, T2, T3 ... Projector t1, t2, t3 ... Image projection frame

Claims (2)

  A plurality of virtual imaging systems that create scene data (scene data) such as an object (for example, an image of the object j) or a background image of the 3D CG scene F or VR scene F as virtual captured image data. Virtual shooting image data of a virtual shooting system that generates scene data generation means corresponding to a camera and operation data for driving and driving as a three-dimensional CG image or VR image based on the virtual shooting image data of the three-dimensional CG or VR A virtual projection image of a virtual shooting system that becomes a three-dimensional CG or VR image for projection is generated based on the operation means and the scene data and virtual shooting image operation data captured from the scene data preparation means and the virtual shooting image data operation means. Projection image generation means to be used, and correction parameters related to physical installation conditions such as projection projectors and curve screens. And a first correction for setting and changing the correction parameter of the virtual photographing system related to the rendering corresponding to the zoom operation and the adjustment of the viewing angle based on the virtual projection image data from the projection image generation unit. Parameter setting changing means, second correction parameter setting changing means for changing the setting of the projection system correction parameters set by the projection system setting adjusting means, and virtual output from the first and second correction parameter setting changing means. Composite correction parameter setting means for setting a composite correction parameter based on correction parameters of the photographing system and the projection system, a composite correction parameter output from the composite correction parameter setting means, and a virtual projection image output from the projection image generation means Deformation correction based on data to obtain a virtual projection image for curved screen projection without distortion A geometric correction unit including a video deformation unit, and a recording / reproducing unit that records a video for projection of a curved screen corrected by the geometric correction unit, or a projector unit that projects the video for projection onto a curved screen. A video geometry correction device characterized by this.   2. The video geometry correction device according to claim 1, wherein setting of a correction parameter of a virtual photographing system related to rendering corresponding to a zoom operation and adjustment of a viewing angle is changed based on virtual projection image data from the projection image generation unit. One correction parameter setting changing means is composed of one or a plurality of correction parameter setting changing means, and the second correction parameter setting changing means for changing the setting of the correction parameter of the projection system set by the projection system installation adjusting means is An image geometry correction apparatus comprising one or a plurality of correction parameter setting changing means.

Images