JP3992045B2 - Video signal processing apparatus and method, and virtual reality generation apparatus - Google Patents

Description

  The present invention performs distortion correction on a curved surface in advance so that a normal input video signal for a flat display surface can be projected without distortion even when projected from a projector onto a curved surface having an arbitrary curvature such as a wall surface. The present invention relates to a video signal processing apparatus and method, and a virtual reality generation apparatus configured using the same.

  The above-described projection technology to a curved surface has already been put into practical use in a virtual reality generation device. For example, Patent Document 1 and the like can be cited as the prior art. This virtual reality generation device projects an image that exceeds the viewing angle of the observer by projecting the image onto a hemispherical screen with a concave surface facing the observer, which is greater than the effective viewing angle, and generates virtual reality. It is like that. The input video signal input to the virtual reality generation apparatus is a CG (computer graphic) signal generated for the virtual reality generation apparatus.

The effective viewing angle is a viewing angle at which a person can recognize an object in detail, and is about 140 degrees in the horizontal direction and about 85 degrees in the vertical direction.
Japanese Patent No. 3387487

  By the way, since the input video signal input to the virtual reality generation device according to Patent Document 1 is a signal generated for the virtual reality generation device, the input video signal is limited.

  The present invention has been made in view of the above circumstances, and is a video signal processing apparatus and video signal processing capable of performing distortion correction on a curved surface in advance, even if it is not a signal generated for the virtual reality generation apparatus It aims to provide a method. And it aims at providing the virtual reality production | generation apparatus using this video signal processing apparatus.

  In order to achieve the above object, in projecting an image from a projection unit to an arbitrary curvature projection surface according to an aspect of the present invention, distortion correction for the projection surface is performed in advance on a video signal for a flat display surface. The video signal processing device applied to the projection means stores a DV terminal for inputting a DV video signal for a flat display surface, the video signal inputted from the DV terminal in a frame memory, and the video Input video processing means for outputting the start memory address and the used memory size in the frame memory of the video signal stored in the frame memory by inputting the screen size of the signal, and a function related to a plurality of types of prescribed projection surface shapes Preliminary formula, select any one of the specified projection plane shapes or a combination of them, and input the variable parameters A model creation unit that creates a mesh model of the projection plane based on a functional formula, and a correspondence map between the display surface of the plane and the mesh model of the projection plane based on the positional relationship of the observer and the projection unit with respect to the projection plane And generating a video again by applying texture mapping coordinates using the corresponding map to the plane input video based on the top memory address and memory size output from the input video processing unit, Distortion correction processing means for realizing distortion correction is provided.

  The video signal processing apparatus described above further includes an output terminal that outputs a video signal whose distortion has been corrected by the distortion correction processing unit.

  In addition, the above-described video signal processing devices further include video signal storage means for storing the video signal whose distortion has been corrected by the distortion correction processing unit.

  Further, in these video signal processing devices described above, an input terminal for inputting a video signal of a system different from the DV system, and a video signal input from the input terminal is converted into a video signal of the DV system and the input Video signal converting means for outputting to the video signal processing unit is further provided.

  In the above-described video signal processing apparatus, a screen size storage unit that stores a screen size of the video signal corresponding to the input terminal method, and a screen size that corresponds to the input terminal method from the screen size storage unit. The image processing apparatus further includes a screen size selection unit that extracts and outputs the extracted screen size to the input video processing unit.

  Further, according to another aspect of the present invention, a wide viewing angle screen capable of displaying an image more than the effective viewing angle of the observer having a projection surface of an arbitrary curvature with a concave surface facing the observer, A video signal processing device that outputs an input video signal to a display surface after performing distortion correction on the projection surface in advance, and a projection unit that projects an image based on the video signal output from the video signal processing device onto the screen. In the virtual reality generation device provided, the video signal processing device is any one of the video signal processing devices described above.

  Furthermore, according to another aspect of the present invention, a spherical, specularly reflective screen with a concave surface facing the observer, and distortion correction for the projection surface in advance for an input video signal for a flat display surface Video signal processing devices provided for each of the left and right channels to be output, and synchronizing means for controlling the video signal processing devices to establish synchronization between the video signals of the left and right channels, and the video signal processing The image signal from the apparatus is projected onto the screen as polarized images that are orthogonal to each other, and the observer wears and transmits the polarized images corresponding to the left and right eyes, respectively, and combines them with the observer. In a virtual reality generation device comprising a video selection means for displaying a stereoscopic video image by recognizing the video signal processing device, any one of the video signal processing devices described above is provided. And characterized in that.

  Then, when projecting an image from the projection unit to a projection surface having an arbitrary curvature according to another aspect of the present invention, the image signal for the flat display surface is subjected to distortion correction for the projection surface in advance and then applied to the projection unit. A video signal processing method to be applied includes: a storage step of storing a DV format video signal for an input flat display surface in a frame memory; and the video signal stored in the frame memory by inputting a screen size of the video signal. An output step for outputting the start memory address and the used memory size in the frame memory, and a function expression relating to a prescribed projection surface shape over a plurality of types, and selecting any one of the prescribed projection surface shapes or a combination thereof The model creation step for creating a mesh model of the projection plane based on the function formula by inputting the variable parameters. And a correspondence map between the display surface of the plane and the mesh model of the projection plane based on the positional relationship between the observer and the projection means with respect to the projection plane, and the top memory address output in the output step and A distortion correction processing step that realizes the distortion correction by performing image generation again by applying texture mapping coordinates using the correspondence map to a plane input image based on a memory size. .

  In the video signal processing apparatus, method and virtual reality generation apparatus configured as described above, the input video processing means distorts the leading memory address and the used memory size when the input DV video signal is stored in the frame memory. Since the correction processing means is notified, the memory address storing the first pixel in the plane input video is known, so that the video generation can be immediately performed again by applying the texture mapping coordinates using the correspondence map. Therefore, it is possible to generate an output video signal that has been subjected to distortion correction in substantially real time from the input video signal.

Embodiments according to the present invention will be described below with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted.
(First embodiment)
FIG. 1 is a diagram illustrating an overall configuration of a virtual reality generation device including a video signal processing device according to the first embodiment. The virtual reality generation device M1 includes a video signal processing device 1, a screen 2 formed by combining an arbitrary curved surface or a plurality of planes, a projector 3 that projects video on the screen 2, and a television (TV) 4 respectively. −1, video signal source 4 such as a video playback device 4-2 that plays back a recording medium such as a DVD or a video tape, a video shooting device 4-3 that takes a video such as a video camera, and a game machine 4-4 The signal processing apparatus 1 includes a setting unit 6 that gives setting data such as the curved surface shape of the screen 2, the projection position of the projector 3, and the screen size. Then, the video signal processing apparatus 1 corrects the video signal with respect to the flat display surface input from the video signal source 4 based on the setting data set by the setting means 6 constituted by a personal computer or the like to correct the projector 3. Is output to the screen 2 in substantially real time. Here, the screen 2 may be a wide viewing angle screen capable of displaying an image that is larger than the effective viewing angle of the observer from the viewpoint of more effectively giving the observer virtual reality.

  The video signal processing apparatus 1 includes a hardware unit 11 and a software unit 12 that operates the hardware unit 11. The hardware unit 11 includes an input terminal 13 for inputting a video signal, a signal system conversion circuit 14 for converting a video signal of a signal system other than the DV system into a DV video signal, and a video signal for processing the video signal. A processing circuit 15, a memory 17 for storing operation data, a memory 18 for storing programs and input / output video signals, an output terminal 19 for outputting video signals subjected to distortion correction, and a central processing unit (CPU) ) And a digital signal processor (DSP) 16 that controls each part.

  The video signal is a known composite-type composite signal, a separate-type separate signal, a component-type component signal, a DV (Digital Video) -type DV signal, or the like. Each device of the video signal source 4 outputs a video signal according to one or a plurality of signal methods among a composite signal, a separate signal, a component signal, and a DV signal. The input terminal 13 includes a DV terminal for inputting a DV signal and a terminal for inputting a video signal having a signal system different from the DV system. The terminal for inputting a video signal of a signal system different from the DV system is, for example, a composite terminal for inputting a composite signal, an S terminal for inputting a separate signal, and a component terminal for inputting a component signal It is. The signal system conversion circuit 14 is provided according to the type of terminal for inputting a video signal of a signal system different from the DV system. For example, a converter circuit that converts a known composite signal into a DV signal, or a known separate It is a converter circuit that converts a signal into a DV signal, or a converter circuit that converts a known component signal into a DV signal. Setting data from the setting means 6 is input via a USB cable or the like.

  The software unit 12, as will be described in detail later, a screen model creation unit 21 that creates a model of the screen 2, and a projector arrangement / setup for setting the arrangement position of the projector 3 with respect to the screen 2 and setting the angle of view of the projector 3. And a user position setting unit 23 for setting the position of the observer with respect to the screen 2, a distortion correction unit 25, and an input video processing unit 26. The input video processing unit 26 first secures a memory area of a frame memory that is a video drawing memory in the memory 18. When a DV video signal is input from the DV terminal of the input terminal 13, the input video processing unit 26 expands the compressed DV video signal by a procedure reverse to the compression procedure. The expanded video signal data for one frame is stored in this frame memory so that the center of the video signal data matches the center of the frame memory. Alternatively, when input from a terminal other than the DV terminal of the input terminal 13, the input video processing unit 26 converts the DV video signal into a DV video signal, and then the input video processing unit 26 compresses the DV video signal. Is decompressed by a procedure reverse to the compression procedure, and the decompressed video signal data is stored in the frame memory so that the center of the video signal data coincides with the center of the frame memory. A DV video signal is usually subjected to inverse variable length coding such as inverse Huffman coding, inverse linear quantization, and inverse orthogonal transform such as inverse DCT (Discrete Cosine Transform) to be expanded. . Then, the input video processing unit 26 notifies the distortion correction unit 25 of the leading memory address and the used memory size in the memory area used by the video signal data stored in the frame memory. The start memory address and the used memory size are determined based on the start memory address of the frame memory when the memory area of the frame memory is secured in the memory 18 and the screen size (resolution) of the input video signal. It can be calculated from the fact that it is stored so that it matches the center of the frame memory. The screen size is input to the video signal processing apparatus 1 as one of setting data from the setting unit 6, for example.

  The distortion correction unit 25 converts the input image of the plane into pixels by the input image processing unit 26, and sets the start memory address and the used memory size of the input image signal stored in the frame memory set in the memory 18. Based on this, distortion correction corresponding to the processing results in the units 21 to 23, specifically, texture mapping is performed, and a distortion processed video is created. The following processing by the software unit 12 is performed using a video signal processing circuit 15 controlled by a digital signal processor (DSP) 16.

  FIG. 2 is a functional block diagram of the screen model creation unit, which is a model creation means. The screen model creation unit 21 creates a mesh model of the screen 2 as described above. First, the screen model creation unit 21 stores three basic models. That is, a spherical model 31, a cylindrical model 32, and a mixed model 33. The mixed model 33 includes a spherical mixed model 34 in which a sphere is mainly mixed with other planes or curved surfaces, a cylindrical mixed model 35 in which a cylinder is mainly mixed with other planes or curved surfaces, and a plane mainly mixed with other planes or curved surfaces. It is composed of a planar mixed model 36. The contents of each model 31, 32, 34 to 36 can be called to the setting means 6, one of which is set by the setting means 6, and the input parameters 31a, 32a, 34a to 36a are further set from the setting means 6. , Mesh models 31b, 32b, 34b to 36b adapted to the actual screen 2 can be created.

  The basic model and input parameters are as follows.

  As shown in FIG. 3, in the case of a spherical surface (a part of a true sphere), input parameters are a radius R of the sphere and a distance A from the center of the sphere to the cut surface.

Here, the sphere is expressed as Equation 1.
x ² + y ² + z ² = R ² (Formula 1)
Here, A ≦ x ≦ R, −y1 ≦ y ≦ y1, and −z1 ≦ z ≦ z1. y1 is obtained by substituting x = A and z = 0 in Equation 1, and z1 is obtained by substituting x = A and y = 0 in Equation 1, and y1 = z1. Therefore, in the case of a spherical surface as described above, the radius R of the sphere and the distance A from the center of the sphere to the cut surface may be input as input parameters.

  As shown in FIG. 4, when a part of the spherical surface is cut out at the position B from the origin (x axis) in the y direction, the screen surface is as shown in FIG. 5A, and B is negative. In the case of the position (down (−y direction) as shown in FIG. 4), A ≦ x ≦ R, −B ≦ y ≦ y1, −z1 ≦ z ≦ z1, and y1 in Equation 1 is x = A, z is obtained by substituting z = 0, and z1 is obtained by substituting x = A and y = 0 in Equation 1, and x1 is obtained by substituting y = −B and z = 0 in Equation 1. It is required by doing. Therefore, as the input parameters, the radius R of the sphere, the distance A from the center of the sphere to the cut surface, and the distance B from the origin to the notch position may be input.

  Similarly, when the top, right, and left are cut, the screens are as shown in FIGS. 5B, 5C, and 5D, respectively, and Equation 1 is as follows. As shown in FIG. 5 (b), when cutting at the height of B, A ≦ x ≦ R, −y1 ≦ y ≦ B, and −z1 ≦ z ≦ z1. As shown in FIG. 5C, when cutting to the right at a distance of B, A ≦ x ≦ R, −y1 ≦ y ≦ y1, and −z1 ≦ z ≦ B. As shown in FIG. 5D, when cut at a distance B to the left, A ≦ x ≦ R, −y1 ≦ y ≦ y1, and −B ≦ z ≦ z1. If the input parameters are increased, it is possible to combine two or more cutting positions.

  On the other hand, FIG. 6 shows an arch surface obtained by cutting a cylinder. In this case, the input parameters are the radius R of the arch, the distance A from the center of the circle to the cut surface, and the height H of the arch.

Here, the circle is expressed as Equation 2.
x ² + z ² = R ² (Formula 2)
Here, by adding restrictions of A ≦ x ≦ R, 0 <y ≦ H, and −z1 ≦ z ≦ z1, the equation of the surface of the arch is obtained. z1 is obtained by substituting x = A in Equation 2.

Further, FIG. 7A shows a case where a plurality of planes (a plurality of rectangular surfaces exist), and in this case, as an input parameter, a plane viewed from the top as shown in FIG. 7B. And the height H of the surface. In the case of FIG.
z = −x + 1 (0 ≦ x <1) (Formula 3-1)
z = 0 (1 ≦ x <3) (Formula 3-2)
z = x−3 (3 ≦ x ≦ 4) (Formula 3-3)
0 <y ≦ H (Formula 3-4)
It is.

  Further, in the case of a screen in which a plurality of spherical surfaces of the same size are arranged side by side, as input parameters, as in the example shown in FIG. 3 or FIGS. 4 and 5, the radius R, the distance A to the cut surface, What is necessary is just to input the distance B to the notch position, and the number. That is, for example, when arranging two spherical screens on the top and bottom, it can be realized by combining FIG. 5A and FIG. 5B, and when arranging two spherical screens on the left and right in the same way, FIG. 5 (c) and FIG. 5 (d) can be realized. As described above, the input parameters include the radius R, the distance A to the cut surface, and the distance to the notch position. It will be understood that B and the number may be entered.

  In the case of a plurality of screens having a cylindrical surface, the height H of the surface, the radius R and the distance A of each screen, and the number of the screens may be input in the layout viewed from above. In this way, the function expression of the screen surface is obtained. By substituting appropriate values for x, y, and z, a certain number or more of the points on the screen are recorded as sampling points. Can be specified. The function formula of each screen surface is stored in advance in each model 31, 32, 34 to 36 in the screen model creation unit 21, and can be called by the setting means 6. And mesh model 31b, 32b, 34b-36b which adapted to the actual screen 2 can be produced only by inputting said parameter as input parameter 31a, 32a, 34a-36a.

  The sampling points on the screen surface as described above can be rotated around the x axis, the y axis, and the z axis (an inclined screen can be defined).

  That is, the rotation of α degrees around the x axis can be expressed by Equation 4.

  Further, the rotation of β degrees around the y-axis can be expressed by Equation 5.

  Furthermore, the rotation of γ degrees around the z-axis can be expressed by Equation 6.

  Distortion is achieved by mapping each sampling point of the plane input video from the input video processing unit 26 to the sampling points on the screen surface obtained in this way using a mesh file in the distortion correction unit 25 as distortion correction means. Correction can be performed, and by assigning each pixel data of the input video signal to a corresponding pixel on the mesh file, a distortion-corrected video signal can be created. In order to generate the output video signal from the input video signal in substantially real time, the input video signal is stored in any of the memories 18 when each pixel data of the input video signal is allocated to the corresponding pixel on the mesh file. However, as described above, the input video processing unit 26 notifies the distortion correction unit 25 of the leading memory address and the used memory size of the input video signal. The mesh file is determined by the screen 2 to be used, inputs each parameter, and once created, there is no change unless the parameter is changed, and the mesh file is stored in the memory 17 of the hardware unit 11 and is being played back. Is used only to replace the corresponding pixels of the input video signal and the output video signal based on the leading memory address and the used memory size of the input video signal stored in the frame memory set in the memory 18, It will not be a big burden.

  The above description with reference to FIGS. 3 to 7 describes a correction method for the shape of the screen 2 as shown in FIG. That is, when a video signal for a flat display device is projected onto the spherical screen 2 as it is, a distorted image is obtained as shown in FIG. Further, as shown in FIG. 8B, the video is distorted in advance. However, further correction is required due to the difference in the arrangement position of the projector 3 with respect to the screen 2 and the difference in the position of the observer with respect to the screen 2. Therefore, the mesh file obtained as described above is further subjected to such correction and used for the replacement of the actual video signal. Hereinafter, a correction method for the difference in the positions of the projector and the viewer with respect to the spherical screen will be described in detail. This process is performed by the distortion correction unit 25 in response to inputs from the projector arrangement / setup unit 22 and the user position setting unit 23.

  FIG. 9 is a diagram for explaining a distortion correction method including the position correction of the projector 3 and the observer in addition to the spherical correction as described above. These corrections are performed by the distortion correction unit 25 in response to inputs from the projector arrangement / setup unit 22 and the position setting unit 23.

First, in the correction, as shown in FIG. 9, a view frustum is defined from the viewpoint position of the observer 51 and a video projection frustum from the projection position of the projector 3. The view frustum is expressed by a quadrangular pyramid with the vertex as the viewpoint position P ₀ and the bottom face P _0,0 , P _{m, 0} , P _{m, n} , P _{0, n,} and the projection frustum is the projector back focus The vertex as a position is expressed as Q ₀ , and the bottom is expressed as Q _0,0 , Q _{m, 0} , Q _{m, n} , Q _{0, n} . Here, m and n represent video resolutions. When the video signal is, for example, SXGA, m = 1279 and n = 11023. The bottom surface is also called a virtual screen surface.

Here, for simple expression, an image viewed from a yz two-dimensional section at m = i is shown in FIG. First, assuming a point P _{i, j} in the virtual screen surface 2a, an intersection R _{i, j} between the vector P ₀ P _{i, j} and the dome screen 2 (mesh models 31b, 32b, 34b to 36b) is obtained. Next, an intersection point Q _{i, j} between the vector Q ₀ R _{i, j} and the virtual screen surface 2b of the projection frustum is obtained. If i, j is changed as 0 ≦ i ≦ m and 0 ≦ j ≦ n, a P _{i, j} → Q _{i, j} correspondence map can be created, and this correspondence map is the reverse of image distortion. It becomes correction.

That is, first, normal video generation is performed based on the view frustum, then this video data is taken out, and the image is applied again by applying texture mapping coordinates using the P _{i, j} → Q _{i, j} correspondence map to this image. By performing the generation, the video distortion correction is realized. An image obtained by applying the P _{i, j} → Q _{i, j} correspondence map to the grid video is shown in FIG. 11 (that is, reverse to FIG. 8B). This distortion correction method does not limit the shape of the screen 2. Therefore, the present invention can be applied not only to the spherical mesh model 31b obtained as described above but also to other mesh models 32b, 34b to 36b, and the like.

  By performing such correction in the distortion correction unit 25 of the video signal processing apparatus 1, it is possible to sequentially distort the DV input video signal for the flat display surface and apply it to the projector 3. An output video signal is generated from the input video signal of the system, and an image without distortion can be displayed on the screen 2 having an arbitrary curvature.

  In the above-described embodiment, the screen size of the input video signal is input from the setting unit 6 to the video signal processing device 1. However, the screen size is stored in advance in the memory 18 in association with each terminal of the input terminal 13. The input video processing unit 26 also operates as a screen size selection unit, and the input video processing unit 26 detects the terminal to which the input video signal is input and searches the memory 18 for the corresponding screen size. You may comprise so that it may be used for calculation of a use memory size.

  In the above-described embodiment, the video signal input from each terminal of the input terminal 13 is switched sequentially or randomly at predetermined time intervals, and the video signal processing apparatus 1 performs distortion correction processing and the like, and the projector 3 You may comprise so that it may project to 2. By configuring in this way, images of various scenes can be projected on the screen 2.

Next, another embodiment will be described.
(Second Embodiment)
FIG. 12 is a block diagram illustrating a configuration of a virtual reality generation device including a video signal processing device according to the second embodiment. This virtual reality generation device M2 is arranged in a horizontal direction by an interval corresponding to human binocular parallax, and left and right general-purpose photographing devices (for example, video cameras) 111 whose time is adjusted to the same time, As described above, distortion correction is performed on the screen surface according to the present invention from the left and right channel DV video signals created using the 121, and the right and left channel projectors 102 and 103 are orthogonal to each other in synchronization. A virtual reality image is generated by projecting the image of the polarization plane. The observer looks at the screen with dedicated glasses that transmit only the images corresponding to the left and right polarization planes, so that the left and right eyes recognize the images of the left and right channels, respectively. The screen is a specular reflection screen such as a silver screen that can reflect polarized light as it is. In addition, this screen may be a wide viewing angle screen capable of displaying an image that is larger than the effective viewing angle of the observer from the viewpoint of more effectively giving the observer a virtual reality.

  The DV video signals obtained by the photographing apparatuses 111 and 121 are input to the input video processing units 113 and 123, which are configured in substantially the same manner as the video signal processing apparatus 1 via the DV terminals 112 and 122, respectively. The same processing is performed and output to the distortion correction units 114 and 124. In the distortion correction units 114 and 124 configured substantially in the same manner as the video signal processing apparatus 1 described above, distortion correction for distortion caused by the spherical shape of the screen and offset (initial deviation) from the center axis of the sphere of the projectors 102 and 103. The installation position is corrected for the distortion caused by.

  The video signals output from the distortion correction units 114 and 124 are written to one of two frame memories 117 and 118; 127 and 128, respectively, via the changeover switches 116 and 126, and the frame memories 117, A video signal is read from the other of 118; 127, 128 via the changeover switches 119, 129 and the output terminals 120, 130, and is supplied to the projectors 102, 103 for projection.

  The input video processing units 113 and 123 in the second embodiment further notify the distortion correction units 114 and 124 of the time code (time stamp) of the video signal that has been decompressed and stored in the frame memory, and in the second embodiment. Further, the distortion correction units 114 and 124 notify the synchronization management units 115 and 125 of the time codes of the frames waiting to be drawn that have been subjected to distortion correction and written in the frame memories 117 and 118; 127 and 128. As is well known, the time code is included in a DV video signal.

  The video creation means 107 and 108 are composed of, for example, a personal computer, and the input video processing units 113 and 123, the distortion correction units 114 and 124, and the synchronization management units 115 and 125 execute software stored in the personal computer and the software. Configured by hardware. That is, as video signal processing software, the input video processing units 113 and 123 have an input video processing process, and the distortion correction units 114 and 124 have a video distortion correction process. Also, the synchronization management units 115 and 125 that manage the synchronization of the input video processing units 113 and 123 and the changeover switches 116 and 126; 119 and 129 have a synchronization management process.

  Here, the video creation means 107 and 108 are slaves, and a synchronization means 109 is provided as a master. The synchronization means 109 is also composed of a personal computer, and the video creation means 107 and 108 are mutually connected by a network. The synchronization unit 109 uses the time code notified from the synchronization management units 115 and 125 of the video creation units 107 and 108 to change the switches 116 and 126; 119 and 129 as will be described later via the synchronization management units 115 and 125. A synchronization management unit 136 that establishes synchronization between the video signals from the video creation means 107 and 108 by controlling the switching of the video. The synchronization management unit 136 has a synchronization management process. The synchronization management process, the input video processing process and the video correction process executed by the video creation means 107 and 108, and the synchronization management process executed by the synchronization means 109 will be described in detail below.

  Input terminals 112 and 122, input video processing units 113 and 123, distortion correction units 114 and 124, synchronization management units 115 and 125, changeover switches 116 and 119; 126 and 129, and frame memories 117 and 118. And 127 and 128, the output terminals 120 and 130, and the synchronization management unit 136 in the synchronization means 109 can be realized in the form of a board or the like mounted in a PCI slot of a personal computer, for example. Alternatively, the video creation means 107 and 108 are configured as dedicated units for video distortion correction that realize the functions described above, and are inserted between the imaging devices 111 and 121 and the projectors 102 and 103, and further to them. In addition, a synchronization unit serving as the synchronization unit 109 may be provided.

  FIG. 13 is a flowchart for explaining the operation of establishing synchronization between the left and right video signals in the virtual reality generating apparatus configured as described above. FIG. 13A is a flowchart showing the operation of the synchronization management process of the video creation means 107, and FIG. 13B is a flowchart showing the operation of the synchronization management process of the synchronization means 109. FIG. These are flowcharts showing the operation of the synchronization management process of the video creation means 108. The main software that realizes the virtual reality generation apparatus M2 includes a synchronization management process, an input video processing process, and a video distortion correction process. The synchronization management process and the input video processing process operate in cooperation with each other in a series of processes in the video creation means 107 and 108. These video creation means 107 and 108, which are slave personal computers, are not limited to two, and can be further expanded to support multiple screens. Each process communicates using a network. In FIG. 12, the operation | movement by a synchronous management process is shown with a broken line.

  The synchronization unit 109 executes a synchronization process and waits for reception of a time code of a drawing wait frame (S101).

  The video creation means 107 and 108 execute an input video processing process, a video distortion correction process, and a synchronization management process. When instructed by the synchronization management process of the video creation means 107, the input video processing process of the video creation means 107 receives a video signal for one frame from the photographing devices 111 and 121, and expands the received video signal while Is stored in the frame memory, for example, the frame memory 117, and the start memory address, the used memory size, the time code, and the like are notified to the video distortion correction process. Upon receiving this notification, the video distortion correction process of the video creation means 107 performs the above-described distortion correction on the video signal in the frame memory 117 and stores it in the frame memory 117, and synchronously manages the time code of this drawing waiting frame. Notify the process. Upon receiving this notification, the synchronization management process of the video creation unit 107 notifies the time code of the drawing waiting frame to the synchronization management process of the synchronization unit 109 (S201), and waits for the reception of the swap synchronization message (S202).

  Upon receiving this notification, if the synchronization management process of the synchronization means 109 has received the time code of the drawing waiting frame from the synchronization management process of the video creation means 108, the synchronization of the received video creation means 108 is received. The time code of the drawing wait frame received from the management process is compared with the time code of the drawing wait frame received from the synchronization management process of the video creation means 107, and it is determined whether or not they match (S102). As a result of the determination, if they do not match (No), the synchronization management process of the synchronization means 109 returns the process to step S101.

  On the other hand, similarly, when instructed by the synchronization management process of the video creation means 108, the input video processing process of the video creation means 108 receives a video signal for one frame from the photographing device 121 and decompresses the received video signal. Then, the data is stored in one frame memory, for example, the frame memory 127, and the head memory address, the used memory size, the time code, and the like are notified to the video distortion correction process. Upon receiving this notification, the video distortion correction process of the video creation means 108 performs the above-described distortion correction on the video signal in the frame memory 127 and stores it in the frame memory 127, and synchronously manages the time code of this drawing waiting frame. Notify the process. Upon receiving this notification, the synchronization management process of the video creation unit 108 notifies the time code of the drawing waiting frame to the synchronization management process of the synchronization unit 109 (S301), and waits to receive a swap synchronization message (S302).

  Upon receiving this notification, when the synchronization management process of the synchronization means 109 receives the time code of the drawing waiting frame from the synchronization management process of the video creation means 107, the synchronization of the received video creation means 107 is received. The time code of the drawing wait frame received from the management process is compared with the time code of the drawing wait frame received from the synchronization management process of the video creation means 108, and it is determined whether or not they match (S102). If the result of the determination is that they match (Yes), the synchronization management process of the synchronization means 109 is performed by a changeover command of the changeover switches 116, 126; 119, 129 for switching between the two frame memories 117, 118; A swap synchronization message is notified to the synchronization management process of the video creation means 107 and 108, and both received time codes are discarded (S103).

  Upon receiving this notification, the synchronization management process of the video creation means 107 and 108 swaps the frame memories 117, 118; 127, 128 by switching the changeover switches 116, 126; 119, 129 (S203, S303). That is, the synchronization management process of the video creation means 107 switches the changeover switches 116 and 119. For example, when the distortion correction unit 114 switches the changeover switch 116 so as to correct the distortion of the video signal in the frame memory 117, and the projector 102 changes the changeover switch 119 so as to read the video signal in the frame memory 118. The changeover switch 116 is switched so that the distortion correction unit 114 performs distortion correction on the video signal in the frame memory 118, and the changeover switch 119 is switched so that the projector 102 reads the video signal in the frame memory 117. When the changeover switch 126 is switched so that the distortion correction unit 124 performs distortion correction on the video signal in the frame memory 127, and the changeover switch 129 is switched so that the projector 103 reads the video signal in the frame memory 128. The changeover switch 126 is switched so that the distortion correction unit 124 performs distortion correction on the video signal in the frame memory 128, and the changeover switch 119 is switched so that the projector 103 reads the video signal in the frame memory 117.

  As a result, the left and right projectors 102 and 103 project the moving image synchronized with each other onto the screen.

  Then, the synchronization management process of the video creation means 107 and 108 reads the video signal for the next one frame from the photographing apparatuses 111 and 121, and performs the above-described processing to the input video processing process of the video creation means 107 and 108. Notification is made (S204, S304), and the process returns to the process S201 and the process S301.

  With this configuration, a video signal such as a live-action video can be transmitted in substantially real time (for example, a frame rate of 24 frames / second) on a personal computer on which a plurality of video creation units 107 and 108 are mounted without using special hardware. And 30 frames / second), and can be projected onto the screen synchronously while performing distortion correction.

  Here, in the first and second embodiments described above, the output terminals 19, 120, and 130 may further include a video signal storage device that reads and writes video signals from and on a recording medium. By storing the distortion-corrected video signal in the recording medium, the distortion-corrected video signal can be distributed, and the distortion correction can be performed immediately by reading from the recording medium with the video signal storage device. The applied video signal can be obtained. Such a video signal storage device is, for example, a hard disk drive that reads data from and / or writes data to a recording medium such as a hard disk, CD-R (Compact Disc Recordable), and DVD-R (Digital Versatile Disc Recordable). , A CD-R drive, a DVD-R drive, and the like.

It is a figure which shows the whole virtual reality production | generation apparatus containing the video signal processing apparatus which concerns on 1st Embodiment. It is a functional block diagram of the screen model creation part which is a model creation means. It is a figure which shows the input parameter of a spherical model. It is a figure which shows the input parameter of the model which notched a part of spherical surface. It is a figure which shows the front shape which notched some spherical screens. It is a figure which shows the input parameter of a cylindrical model. It is a figure which shows the input parameter of a several plane. It is a figure for demonstrating the distortion correction process with respect to the spherical shape of a screen. It is a figure for demonstrating the method of distortion correction. FIG. 10 is a two-dimensional cross-sectional view of FIG. 9 taken at the center of the hemispherical screen. It is an image figure of distortion correction of a grid picture. It is a block diagram which shows the structure of the virtual reality production | generation apparatus containing the video signal processing apparatus which concerns on 2nd Embodiment. It is a flowchart for demonstrating the synchronization establishment operation | movement between the right-and-left video signals in the virtual reality production | generation apparatus comprised as mentioned above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video signal processing apparatus 2 Screen 3, 102, 103 Projector 4 Video signal source 4-1 Television 4-2 Video reproduction apparatus 4-3, 111, 121 Shooting apparatus 4-4 Game machine 6 Setting means 11 Hardware part 12 Software unit 13, 112, 122 Input terminal 14 Signal system conversion circuit 15 Video signal processing circuit 16 Digital signal processor 17, 18 Memory 19, 120, 130 Output terminal 21 Screen model creation unit 22 Projector arrangement / setup unit 23 User position setting unit 25, 114, 125 Distortion correction unit 26, 113, 123 Input video processing unit 107, 108 Video creation unit 109 Synchronization unit 116, 119, 126, 129 Changeover switch 117, 118, 127, 128 Frame memory 115, 125, 136 Synchronization Management Department

Claims (8)

In projecting an image from a projection unit to a projection surface having an arbitrary curvature, a video signal processing apparatus that applies distortion correction to the projection surface in advance to a video signal for a flat display surface and applies the distortion correction to the projection unit.
A DV terminal for inputting a DV video signal to a flat display surface;
The video signal input from the DV terminal is stored in the frame memory, and the start memory address and the used memory size in the frame memory of the video signal stored in the frame memory by inputting the screen size of the video signal Input video processing means for outputting,
A function formula relating to a predetermined projection surface shape over a plurality of types is previously stored, a mesh of the projection surface is selected based on the function formula by selecting any one or a combination of the predetermined projection surface shapes and inputting a variable parameter. A model creation means for creating a model;
Based on the positional relationship between the observer and the projection unit with respect to the projection plane, a correspondence map between the planar display plane and the mesh model of the projection plane is created, and the leading memory address and memory output from the input video processing unit And a distortion correction processing unit that realizes the distortion correction by generating a video again by applying texture mapping coordinates using the correspondence map to a plane input video based on the size. Signal processing device. The video signal processing apparatus according to claim 1, further comprising an output terminal that outputs a video signal subjected to distortion correction by the distortion correction processing unit. The video signal processing apparatus according to claim 1, further comprising a video signal storage unit that stores the video signal whose distortion is corrected by the distortion correction processing unit. An input terminal for inputting a video signal of a system different from the DV system;
The video signal conversion means for converting the video signal input from the input terminal into a DV video signal and outputting the converted video signal to the input video signal processing unit. The video signal processing device according to claim 1. Screen size storage means for storing the screen size of the video signal corresponding to the method of the input terminal;
5. The display apparatus according to claim 4, further comprising: a screen size selection unit that extracts a screen size corresponding to the method of the input terminal from the screen size storage unit and outputs the extracted screen size to the input video processing unit. The video signal processing apparatus described. A screen having a projection surface of an arbitrary curvature with a concave surface facing the viewer, a wide viewing angle screen capable of displaying an image larger than the effective viewing angle of the viewer, and a distortion to the projection surface in advance for an input video signal to a flat display surface In a virtual reality generation device comprising: a video signal processing device that performs correction and outputs; and a projection unit that projects video based on the video signal output from the video signal processing device onto the screen.
6. The virtual reality generation device according to claim 1, wherein the video signal processing device is the video signal processing device according to any one of claims 1 to 5. A spherical, specularly reflective screen with a concave surface facing the viewer, and a left and right channel that outputs the input video signal for the flat display surface after correcting distortion in advance for the projection surface. A video signal processing device, a synchronizing means for controlling the video signal processing devices to establish synchronization between the video signals of the left and right channels, and a video of polarized light orthogonal to each other from the video signals from the video signal processing devices Projection means for projecting onto the screen, and image selection means for displaying a stereoscopic video image by being attached to the observer and transmitting a polarized image corresponding to each of the left and right eyes and causing the observer to synthesize and recognize the image. In a virtual reality generation device comprising:
6. The virtual reality generation device according to claim 1, wherein the video signal processing device is the video signal processing device according to any one of claims 1 to 5. In projecting an image from a projection unit onto a projection surface having an arbitrary curvature, a video signal processing method for performing distortion correction on the projection surface in advance on a video signal for a flat display surface and providing the projection unit with the distortion correction,
A storage step of storing in the frame memory a DV video signal for the input flat display surface;
An output step of outputting a head memory address and a used memory size of the video signal stored in the frame memory by inputting the screen size of the video signal;
A function formula relating to a predetermined projection surface shape over a plurality of types is previously stored, a mesh of the projection surface is selected based on the function formula by selecting any one or a combination of the predetermined projection surface shapes and inputting a variable parameter. A model creation step to create a model;
Based on the positional relationship between the observer and the projection unit with respect to the projection plane, a correspondence map between the display surface of the plane and the mesh model of the projection plane is created, and the first memory address and memory size output in the output step are set. And a distortion correction processing step for realizing the distortion correction by applying the texture mapping coordinates using the correspondence map to the plane input image and generating the image again. Method.

Images