JP2006094458A - Video signal processor, virtual reality creating apparatus, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To project a video image with no distortion from a projector onto a projection screen of an arbitrary curvature. <P>SOLUTION: A video signal processor 1 is stored with a mesh model of a projection screen (screen 2) in a specific form, such as a part of a spherical surface, a part of a cylindrical surface or combination of a plurality of planes, creates a mesh model of an actual projection screen by inputting, from a setting means 6, variable parameters of a diameter, a distance from the center of a cross section and the like, creates a corresponding map between a planar display screen and a projection screen mesh model on the basis of a positional relationship input from the setting means, between an observer and the projection screen of a projector 3. The video signal processor 1 produces a video image again for an input video image of a plane input from a playback apparatus of a recording medium 4 or a computer graphics creating apparatus 5 by applying texture mapping coordinates utilizing the corresponding map, thereby correcting the distortion of the projection screen. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention corrects distortion of the curved surface in advance in the input video signal so that a normal input video signal with respect to a flat display surface can be projected without distortion even if it is projected from a projector onto a curved surface of an arbitrary curvature such as a wall surface. The present invention relates to a video signal processing apparatus that performs the above-described processing, a virtual reality generation apparatus configured using the video signal processing apparatus, and a recording medium that records a program for realizing the virtual reality generation apparatus.

  The projection technique to the curved surface has already been put into practical use in a virtual reality generation device. For example, Patent Document 1 is cited as the prior art. The virtual reality generation device projects an image that exceeds the viewing angle of the observer by projecting an image on a hemispherical screen to generate the virtual reality.

  However, the dome-shaped screen is expensive and difficult to install due to its large size, and it can be used only for permanent use.

On the other hand, Patent Document 2 discloses a projection display device that is compact and can be installed on a desk. FIG. 1 of the publication shows a projection on a substantially hemispherical screen, and FIG. 3 shows a projection on a screen with a folding screen.
Japanese Patent No. 3387487 US Pat. No. 5,050,481

  In the above-described conventional technologies, projection onto a predetermined screen is assumed. Therefore, using a predetermined distortion correction algorithm, distortion correction is applied to the input video signal so that a normal input video signal for a flat display surface can be projected without distortion even if it is projected onto a curved surface. It is like that. Accordingly, an image without distortion cannot be projected on a projection surface having an arbitrary curvature such as a wall surface in an advertisement or a presentation.

  An object of the present invention is to provide a video signal processing device, a virtual reality generation device, and a recording medium that can project a video without distortion on a projection surface having an arbitrary curvature.

  In the video signal processing apparatus according to the first means of the present invention, when projecting an image from a projection unit onto a projection surface having an arbitrary curvature, the input video signal for a flat display surface is subjected to distortion correction for the projection surface in advance. A video signal processing apparatus to be applied to a projection means, which has in advance a functional expression related to a plurality of types of prescribed projection surface shapes, selects one of the prescribed projection surface shapes or a combination thereof, and inputs variable parameters Based on the positional relationship of the observer and the projection unit with respect to the projection surface, based on the positional relationship of the observer and the projection unit with respect to the projection surface, the plane display surface and the projection surface mesh model, The map is generated again by applying texture mapping coordinates using the corresponding map to the input image of the plane, and generating the image again. Characterized in that it comprises a distortion correcting means for implementing.

  In the video signal processing apparatus according to the second means of the present invention, the prescribed projection surface shape is any one of a part of a spherical surface, a part of a cylindrical surface, or a combination of a plurality of planes. And

  Still further, the virtual reality generating apparatus according to the third means of the present invention is provided on each of the left and right channels respectively provided with a spherical and specularly reflecting screen having a concave surface facing the observer and each of the left and right channels. A video creation means having at least a decoding means for sequentially decoding the compressed video video file of each channel and converting it to a video signal of the video, and controlling the video creation means, and synchronizing the video signals of the left and right channels Synchronization means for establishing the image signal, the video signal processing device that sequentially performs distortion correction on the screen on the video signals of the left and right channels, and the video signals from the video signal processing devices having polarizations orthogonal to each other. Projection means for projecting each image as a screen, and the observer wearing the projector and transmitting the polarized images corresponding to the left and right eyes respectively. Characterized in that it comprises a video selection means for displaying a stereoscopic video image by causing recognized synthesized and.

  In the virtual reality generation apparatus according to the fourth means of the present invention, the synchronization means includes a synchronization management unit that executes a time management loop process and a synchronization management process, and each of the video creation means includes the projection Having two frame memories which can be used by switching to output to the means and writing the decoded video signal in sequence, the decoding means executes a synchronization management process for executing the synchronization management process and a movie file decompression process A time management loop process and a synchronization management process of the synchronization means, comprising: a decompression unit that performs a playback display unit that executes a video display process; To establish synchronization between the video signals of the left and right channels.

  Furthermore, the recording medium according to the fifth means of the present invention provides the computer that constitutes the synchronizing means in the fourth means, the synchronization management process for acquiring the current time, The time management loop process for delivering the current time to the decoding means is executed, and the computer constituting the video creating means in the fourth means receives the current time and receives a receipt indicating that it has been received. The video display process for returning the reference time to the synchronization means and updating the reference time, and delivering the reference time to the movie file decompression process, the reference time is acquired, and the time at which the currently acquired frame is to be displayed If the delay is within 2 frames, the frame data is updated and the frame buffer is updated. In addition to notifying the video display process of the address and notifying the video display process that the frame data has been updated, and receiving the frame data update notification, the frame data is received from the address in the already notified frame buffer. As a video signal, notifies the movie file decompression process of the completion of the frame data update, and draws the video signal in the two frame memories provided in the video creation means that are not selected on the output side When the video display process and the frame data update completion notification are received, the movie file decompression process for acquiring new frame data is executed, and the computer constituting the synchronization means in the fourth means further includes the receipt. Receive a swap sync message The synchronization management process is transmitted to the video display process, and the computer constituting the video creation means in the fourth means executes swap of the two frame memories in response to the swap synchronization message. A receipt is returned to the synchronization management process of the synchronization means, and a program for executing a video display process waiting for reception of the current time is recorded again.

  The video signal processing apparatus having such a configuration creates a mesh model of the projection plane by selecting one of a plurality of types of predetermined projection plane shapes stored in advance or a combination thereof and inputting a variable parameter. Then, distortion correction is realized by generating an image by texture mapping the input image based on the positional relationship between the observer and the projection unit with respect to the projection plane.

  Therefore, a distortion-free image can be projected on a projection surface having an arbitrary curvature such as a wall surface.

  The virtual reality generating device having such a configuration is a display device that generates so-called virtual reality by projecting the moving image of each of the left and right channels on a spherical screen to display a stereoscopic moving image. Each of the compressed moving image files is sequentially decoded and converted into the moving image signal, and the decoding process applied to the projection unit takes time and the required time varies depending on the image. Therefore, synchronization means is provided to control each of the video creation means to establish synchronization between the signals of the moving image.

  Therefore, it is possible to reproduce a stereoscopic image by accurately synthesizing a moving image such as a live-action image without deviation.

  The recording medium having such a configuration can cause a computer to realize the synchronization unit and the video creation unit that can accurately synthesize a moving image such as a live-action video and reproduce a stereoscopic video.

[Embodiment 1]
FIG. 1 is a diagram showing an overall configuration of a video reproduction apparatus including a video signal processing apparatus 1 according to an embodiment of the present invention. This video reproduction apparatus includes a video signal processing apparatus 1 having a screen 2 formed by combining an arbitrary curved surface or a plurality of planes, a projector 3 for projecting an image on the screen 2, a reproduction apparatus for a recording medium 4, and a computer graphic. A video signal source such as a video generator 5 and setting means 6 for giving the video signal processing device 1 setting data such as the curved surface shape of the screen 2 and the projection position of the projector 3. Then, the video signal processing device 1 corrects the video signal for the flat display surface input from the video signal source based on the data set by the setting means 6 and outputs the corrected video signal to the projector 3. Display images without distortion.

  The video signal processing apparatus 1 includes a hardware unit 11 and a software unit 12 that operates the hardware unit 11. The hardware unit 11 includes a video signal processing circuit 13 that processes video signals, a memory 15 that stores arithmetic data, a memory 16 that stores programs and input / output video signals, and a central processing unit (CPU). And a digital signal processor (DSP) 14 that controls each unit. The video signal is a digital signal such as VGA or XGA based on a JPEG file or an MPEG file, an analog composite signal, or the like. Setting data from the setting means 6 is input via a USB cable or the like.

  The software unit 12, as will be described in detail later, a screen model creation unit 21 that creates a model of the screen 2, and a projector arrangement / setup for setting the arrangement position of the projector 3 with respect to the screen 2 and setting the angle of view of the projector 3. A unit 22, a user position setting unit 23 for setting the position of the observer with respect to the screen 2, a distortion correction unit 25, and an input video processing unit 26. The distortion correction unit 25 performs distortion correction corresponding to the processing results of the units 21 to 23, specifically texture mapping, on the data obtained by decomposing the plane input video into pixels by the input video processing unit 26, Create processed video. The following processing by the software unit 12 is performed using a video signal processing circuit 13 controlled by a digital signal processor (DSP) 14.

  FIG. 2 is a functional block diagram of the screen model creation unit 21 which is a model creation unit. The screen model creation unit 21 creates a mesh model of the screen 2 as described above. First, the screen model creation unit 21 stores three basic models. That is, a spherical model 31, a cylindrical model 32, and a mixed model 33. The mixed model 33 includes a spherical mixed model 34 in which a sphere is mainly mixed with other planes or curved surfaces, a cylindrical mixed model 35 in which a cylinder is mainly mixed with other planes or curved surfaces, and a plane mainly mixed with other planes or curved surfaces. It is composed of a planar mixed model 36. The contents of each model 31, 32, 34 to 36 can be called to the setting means 6, one of which is set by the setting means 6, and further the input parameters 31a, 32a, 34a to 36a from the setting means 6. , Mesh models 31b, 32b, 34b to 36b adapted to the actual screen 2 can be created.

  The basic model and input parameters are as follows.

  As shown in FIG. 3, in the case of a spherical surface (a part of a true sphere), input parameters are a radius R of the sphere and a distance A from the center of the sphere to the cut surface.

Where the sphere equation is
x ² + y ² + z ² = R ² (1)
And A ≦ x ≦ R, −y1 ≦ y ≦ y1, and −z1 ≦ z ≦ z1. y1 is obtained by substituting x = A and z = 0 in Equation 1, and z1 is obtained by substituting x = A and y = 0 in Equation 1, resulting in y1 = z1. Therefore, in the case of a spherical surface as described above, the radius R of the sphere and the distance A from the center of the sphere to the cut surface may be input as input parameters.

  In addition, as shown in FIG. 4, when a part of the spherical surface is cut out at the position B from the origin (x axis) in the y direction, the screen surface is as shown in FIG. 5A, and B is a negative position. In the case of (lower (−y direction) as in FIG. 4), A ≦ x ≦ R, −B ≦ y ≦ y1, −z1 ≦ z ≦ z1, and y1 is represented by the following equation (1): x = A, z Z1 is obtained by substituting x = A and y = 0 in Equation 1, and x1 is obtained by substituting y = −B and z = 0 in Equation 1. It is done. Therefore, as the input parameters, the radius R of the sphere, the distance A from the center of the sphere to the cut surface, and the distance B from the origin to the notch position may be input.

  Similarly, when the top, right, and left are cut, the screens are as shown in FIGS. 5B, 5C, and 5D, respectively, and the equations are as follows. As shown in FIG. 5 (b), when cutting at the height of B, A ≦ x ≦ R, −y1 ≦ y ≦ B, and −z1 ≦ z ≦ z1. As shown in FIG. 5C, when cutting to the right at a distance of B, A ≦ x ≦ R, −y1 ≦ y ≦ y1, and −z1 ≦ z ≦ B. As shown in FIG. 5D, when cut at a distance B to the left, A ≦ x ≦ R, −y1 ≦ y ≦ y1, and −B ≦ z ≦ z1. If the input parameters are increased, it is possible to combine two or more cutting positions.

  On the other hand, FIG. 6 shows an arch surface obtained by cutting a cylinder. In this case, the input parameters are the radius R of the arch, the distance A from the center of the circle to the cut surface, and the height H of the arch.

Where the circle formula is
x ² + z ² = R ² (2)
By adding the restrictions of A ≦ x ≦ R, 0 <y ≦ H, and −z1 ≦ z ≦ z1, the equation of the surface of the arch is obtained. z1 is obtained by substituting x = A in Equation 1.

Further, FIG. 7A shows a case where a plurality of planes (a plurality of rectangular surfaces exist), and in this case, as an input parameter, a plane viewed from the top as shown in FIG. 7B. And the height H of the surface. In the case of FIG.
z = −x + 1 (0 ≦ x <1) (3)
z = 0 (1 ≦ x <3) (4)
z = x−3 (3 ≦ x ≦ 4) (5)
0 <y ≦ H (6)
It is.

  Further, in the case of a screen in which a plurality of spherical surfaces of the same size are arranged side by side, as input parameters, as in the example shown in FIG. 3 or FIGS. 4 and 5, the radius R, the distance A to the cut surface, What is necessary is just to input the distance B to the notch position, and the number. That is, for example, when arranging two spherical screens on the top and bottom, it can be realized by combining FIG. 5A and FIG. 5B. Similarly, when arranging two spherical screens on the left and right, FIG. This can be realized by combining (c) and FIG. 5 (d). As input parameters, as described above, the radius R, the distance A to the cut surface, and the distance B to the notch position are as follows. It is understood that the number and the number may be input.

  In the case of a plurality of cylindrical screens, the surface height H, the radius R and the distance A of each screen, and the number of the screens may be input in the layout viewed from above. In this way, the screen surface function is obtained. By substituting appropriate values for x, y, and z, a certain number or more of the points on the screen are recorded as sampling points. Can be specified. The function expression of each screen surface is stored in advance in each model 31, 32, 34 to 36 in the screen model creation unit 21, and can be called by the setting means 6. And mesh model 31b, 32b, 34b-36b which adapted to the actual screen 2 can be produced only by inputting said parameter as input parameter 31a, 32a, 34a-36a.

  Note that the sampling points on the screen surface as described above can be rotated around the x axis, the y axis, and the z axis (a tilted screen can be defined).

  That is, the rotation of α degrees around the x axis can be expressed by the following equation.

  Further, the rotation of β degrees around the y axis can be expressed by the following equation.

  Furthermore, the rotation of γ degrees around the z axis can be expressed by the following equation.

  By mapping each sampling point of the planar input video from the input video processing unit 26 to the sampling points on the screen surface obtained in this way using the mesh file in the distortion correction unit 25 which is a distortion correction unit. Distortion correction can be performed, and a distortion-corrected video signal can be created by assigning each pixel data of the input video signal to a corresponding pixel on the mesh file. The mesh file is determined by the screen 2 to be used, inputs each parameter, and once created, there is no change unless the parameter is changed. The mesh file is stored in the memory 15 of the hardware unit 11 and is being played back. Is only used for replacing corresponding pixels of the input video signal and the output video signal, and does not impose a heavy burden.

  The above description with reference to FIGS. 3 to 7 describes a correction method for the shape of the screen 2 as shown in FIG. That is, when a video signal for a flat display device is projected onto the spherical screen 2 as it is, a distorted image is obtained as shown in FIG. 8A, so that the distortion is eliminated when displayed on the spherical screen 2. Further, as shown in FIG. 8B, the video is distorted in advance. However, further correction is necessary due to the difference in the arrangement position of the projector 3 with respect to the screen 2 and the difference in the position of the observer with respect to the screen 2. Therefore, such a correction is further applied to the mesh file obtained as described above, and it is used to replace the actual video signal. Hereinafter, a correction method for the difference in the positions of the projector and the observer with respect to the spherical screen will be described in detail. This process is performed by the distortion correction unit 25 in response to inputs from the projector arrangement / setup unit 22 and the user position setting unit 23.

  FIG. 9 is a diagram for explaining a distortion correction method including the position correction of the projector 3 and the observer in addition to the spherical correction as described above. These corrections are performed by the distortion correction unit 25 in response to inputs from the projector arrangement / setup unit 22 and the position setting unit 23.

First, for correction, as shown in FIG. 9, a view frustum is defined from the viewpoint position of the observer 51 and a video projection frustum from the projection position of the projector 3. The view frustum is expressed by a quadrangular pyramid with the vertex as the viewpoint position P ₀ and the bottom P _0,0 , P _{m, 0} , P _{m, n} , P _{0, n,} and the projection frustum is the projector back focus The vertex as a position is expressed as Q ₀ , and the bottom is expressed as Q _0,0 , Q _{m, 0} , Q _{m, n} , Q _{0, n} . Here, m and n represent video resolutions. When the video signal is, for example, SXGA, m = 1279 and n = 11023. The bottom surface is also called a virtual screen surface.

Here, for simple expression, an image viewed from a yz two-dimensional section at m = i is shown in FIG. First, assuming a point P _{i, j} in the virtual screen surface 2a, an intersection R _{i, j} between the vector P ₀ P _{i, j} and the dome screen 2 (mesh models 31b, 32b, 34b to 36b) is obtained. Next, an intersection point Q _{i, j} between the vector Q ₀ R _{i, j} and the virtual screen surface 2b of the projection frustum is obtained. When i and j are changed as 0 ≦ i ≦ m and 0 ≦ j ≦ n, a P _{i, j} → Q _{i, j} correspondence map can be created, and this correspondence map is the reverse of image distortion. It becomes correction.

That is, first, normal video generation is performed based on the view frustum, then this video data is taken out, and the image is again applied by applying texture mapping coordinates using the P _{i, j} → Q _{i, j} correspondence map to this image. By performing the generation, the video distortion correction is realized. An image obtained by applying the P _{i, j} → Q _{i, j} correspondence map to the grid video is shown in FIG. 11 (that is, reverse to FIG. 8B). This distortion correction method does not limit the shape of the screen 2. Therefore, the present invention can be applied not only to the spherical mesh model 31b obtained as described above but also to other mesh models 32b, 34b to 36b, and the like.

  By performing such correction in the distortion correction unit 25 of the video signal processing apparatus 1, the input video signal with respect to the flat display surface can be sequentially distorted and given to the projector 3. An image without distortion can be displayed.

[Embodiment 2]
FIG. 12 is a block diagram showing an electrical configuration of a virtual reality generation apparatus according to another embodiment of the present invention. This virtual reality generation device decompresses the compressed video image files of both the left and right channels created using general-purpose shooting means such as a video movie, and performs distortion correction on the screen surface according to the present invention as described above. In other words, a virtual reality image is generated by projecting images of polarization planes orthogonal to each other from the projectors 102 and 103 of the left and right channels in synchronization. The observer looks at the screen with dedicated glasses that transmit only the images corresponding to the left and right polarization planes, so that the left and right eyes recognize the images of the left and right channels, respectively. Further, a specular reflection screen such as a silver screen that can reflect polarized light as it is is used for the screen.

  The compressed moving image video file is compressed and recorded on a medium such as a DVD (digital video disc). In FIG. 12, the medium is shown by the left and right DVDs 111 and 121, but may be another medium on which a moving image is compressed and recorded. In addition, video images shot by the left and right shooting units arranged horizontally apart by an interval corresponding to human binocular parallax are compressed and recorded on individual DVDs 111 and 121 by those shooting units. However, the left and right photographing means may be integrated and recorded on the same medium.

  Recorded videos of DVDs 111 and 121 compressed by MPEG2 and MPEG4 are taken into hard disks 112 and 122 by video creation means 107 and 108, respectively. The recorded video captured in the hard disks 112 and 122 is decompressed (expanded) in the decompression units 113 and 123, and further converted into video signals in the reproduction display units 114 and 124. The decompression units 113 and 123 and the reproduction display units 114 and 124 constitute decoding means. The thus obtained video signal is subjected to distortion correction for distortion caused by the spherical shape of the screen 2 and the central axes of the spheres of the projectors 102 and 103 in the distortion correction units 115 and 125 configured similarly to the video signal processing apparatus 1 described above. Installation position correction is performed for distortion caused by offset from above.

  The video signals output from the distortion correction units 115 and 125 are written into one of the two frame memories 118, 119; 128, 129 provided via the changeover switches 117, 127, respectively. From the other of 119; 128, 129, a video signal is read out via the change-over switches 120, 130, and is supplied to the projectors 102, 103 for projection. In the video creation means 107 and 108 configured as described above, the video signal created by CG does not need to be decompressed in the decompression units 113 and 123 and converted into video signals in the reproduction display units 114 and 124. Directly input from the distortion correction units 115 and 125.

  The video creation means 107 and 108 are composed of a personal computer. The decompression units 113 and 123, the reproduction display units 114 and 124, the distortion correction units 115 and 125, and the synchronization management units 116 and 126 are software and software stored in the personal computer. Configured by the hardware that executes it. That is, as video signal processing software, the decompression units 113 and 123 have a movie file decompression process, the reproduction display units 114 and 124 have a video display process, and the distortion correction units 115 and 125 have a video distortion process. Has a correction process. The synchronization managers 116 and 126 that manage the synchronization of the decompression units 113 and 123, the reproduction display units 114 and 124, and the changeover switches 117 and 127; 120 and 130 have a synchronization management process.

  Here, the video creation means 107 and 108 are slaves, and a synchronization means 109 is provided as a master. The synchronization means 109 is also composed of a personal computer, and the video creation means 107 and 108 are mutually connected by a network. Also in the synchronization means 109, the recorded video from the DVD 111 or 112 (111 in FIG. 12) is taken into the hard disk 132, decompressed by the decompression unit 133, and further converted into a video signal by the reproduction display unit 134. Using the time code in the video signal, the synchronization management unit 136 controls the switching of the changeover switches 117, 127; 120, 130 as will be described later via the synchronization management units 116, 126, and the video creation means. Synchronization between the video signals from 107 and 108 is established. The decompression unit 133 has a movie file decompression process, the reproduction display unit 134 has a video display process, and the synchronization management unit 136 has a synchronization management process and a time management loop process. The synchronization management process, movie file decompression process, video display process and video correction process executed by the video creation means 107 and 108 will be described in detail below. The time management loop process, synchronization management process, movie file decompression process and video display process executed by the synchronization means 109 will also be described in detail below.

  The decompression unit 113, the reproduction display unit 114, the distortion correction unit 115, the synchronization management unit 116, the changeover switches 117 and 120, and the frame memories 118 and 119 in the video creation unit 107 are, for example, boards installed in a PCI slot of a personal computer. It can be realized in the form. Similarly, the decompression unit 123, the playback display unit 124, the distortion correction unit 125, the synchronization management unit 126, the changeover switches 127 and 130 and the frame memories 128 and 129 in the video creation unit 108, and the decompression unit 133 in the synchronization unit 109, the playback display. The unit 134 and the synchronization management unit 136 can also be realized in the form of a board mounted in the PCI slot. Alternatively, the video creation means 107 and 108 are configured as a dedicated unit for video distortion correction that realizes the functions as described above, and a deck or personal computer that plays a compressed video file such as the DVD 111 or 121, and the projector 102. , 103, and a synchronizing unit serving as the synchronizing means 109 may be provided in accordance with them.

  FIG. 13 is a flowchart for explaining the operation of establishing synchronization between the left and right video signals in the virtual reality generating apparatus configured as described above. The main software that realizes the virtual reality generation apparatus includes the time management loop process, the synchronization management process, the movie file decompression process, the video display process, and the video distortion correction process as described above. The synchronization management process, the movie file decompression process, and the video display process operate in the video creation means 107 and 108 in cooperation with each other through a series of processes. These video creation means 107 and 108 which are slave personal computers are not limited to two, and can be further expanded to support multi-screens. The time management loop process is the only process and needs only to be executed at one place in a plurality of personal computers. However, in this embodiment, on the synchronization means 109 which is an independent master personal computer as described above. Execute. Each process communicates using a network. In FIG. 12, the operation | movement by a synchronous management process is shown with a broken line.

  The master personal computer executes a time management loop process and a synchronization management process. Thus, first, the current time is acquired by the time management loop process, and when the specified time has elapsed, the current time is distributed from the synchronization management process to the slave personal computer (S101, S102). That is, the synchronization management process performs swap synchronization, which is a switching command for the changeover switches 117, 127; 120, 130 for switching between the time and the two frame memories 118, 119; A message will be sent.

  On the other hand, the slave personal computer executes a movie file decompression process, a video display process, and a video distortion correction process. As a result, first, the video display process receives the current time, and sends back to the master personal computer as a receipt that it has been received (S201, S202). Further, the video display process updates the reference time, further distributes the reference time to the movie file decompression process (S203), and waits for a frame data update notification (S204).

  In response to this, the movie file decompression process obtains a reference time (S301) and compares it with the time at which the currently obtained frame is to be displayed (S302). If the delay is within 2 frames as a result of the comparison, the frame data is updated, that is, decompressed, and the address in the frame buffer is notified to the video display process (S303). Further, the movie file decompression process notifies the video display process that the frame data has been updated (S304).

  Upon receiving the frame data update notification (S204, S205), the video display process acquires frame data as video from the address in the frame buffer already notified in S303 (S206), and completes the frame data update process to the movie file decompression process. (S207), and the image is drawn in the back buffer (the two frame memories 118, 119; 128, 129 which are not selected on the output side (frame memories 118, 128 in FIG. 12)), that is, the image The signal is reproduced (S208).

  On the other hand, in the master personal computer, when the receipt is received in S103, the synchronization management process transmits a swap synchronization message to the video display process of the slave personal computer (S104). In response to this, the video display process calls the video distortion correction process to execute the processing (S209), and then executes buffer swap, and receives a receipt indicating that it has been received by the synchronization management process of the master personal computer. Reply (S210, S211, S212). Thereafter, the video display process waits for reception of the current time in S201. As shown in FIG. 10, the video distortion correction can be realized with a delay of a predetermined time by rereading the correspondence map, and thereby the left and right projectors 102 and 103 project moving image images synchronized with each other. It will be.

  In the movie file decompression process, when a frame data update completion notification is received (S305), one new frame is acquired from the compressed movie file (S306). Even if the delay exceeds 2 frames as a result of the comparison in S302, one frame is newly acquired in S306.

  In the synchronization management process of the master personal computer, the process is stopped until receipt of the swap synchronization message is received from the video display process, and when the receipt is received, the process starts with acquisition of the current time (S105).

  By configuring in this way, multiple interactive movie files can be transferred to multiple computers while performing real-time processing of decompression, playback, and distortion correction of movie files such as live-action video without using special hardware. It is possible to play back synchronously. In particular, the movie file formats such as MPEG2 and MPEG4 do not have the concept of each frame, and the file capacity is compressed by managing only the difference between video signals, so the process of reconstructing the frame from the difference information ( In the present embodiment, real-time processing (30 frames / second) can be realized by simultaneously executing playback, synchronization, and distortion correction while executing this heavy processing process. .

1 is a diagram illustrating an overall configuration of a video reproduction device including a video signal processing device according to an embodiment of the present invention. It is a functional block diagram of the screen model creation part in the software part of the video signal processing apparatus shown in FIG. It is a figure which shows the input parameter of a spherical model. It is a figure which shows the input parameter of the model which notched a part of spherical surface. It is a figure which shows the front shape which notched some spherical screens. It is a figure which shows the input parameter of a cylindrical model. It is a figure which shows the input parameter of a several plane. It is a figure for demonstrating the distortion correction process with respect to the spherical shape of a screen. It is a figure for demonstrating the method of distortion correction. FIG. 10 is a two-dimensional cross-sectional view of FIG. 9 taken at the center of the hemispherical screen. It is an image figure of distortion correction of a grid picture. It is a block diagram which shows the electric constitution of the virtual reality production | generation apparatus which concerns on the other embodiment of this invention. 13 is a flowchart for explaining a synchronization establishment operation in the virtual reality generation apparatus shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video signal processing apparatus 2 Screen 3; 102,103 Projector 4 Recording medium 5 Computer graphic creation apparatus 6 Setting means 11 Hardware part 12 Software part 13 Video signal processing circuit 14 Digital signal processor 15 Memory 16 Memory 21 Screen model creation Unit 22 Projector arrangement / setup unit 23 User position setting unit 25 Distortion correction unit 26 Input video processing unit 31 Spherical model 32 Cylindrical model 33 Mixed model 34 Spherical mixed model 35 Cylindrical mixed model 36 Planar mixed models 31a, 32a, 34a to 36a Input Parameters 31b, 32b, 34b to 36b Mesh model 51 Viewer 107, 108 Video creation means 109 Synchronization means 111, 112 DVD
112, 122, 132 Hard disk 113, 123, 133 Decompression unit 114, 124, 134 Playback display unit 115, 125 Distortion correction unit 116, 126, 136 Synchronization management unit 117, 127; 120, 130 changeover switch 118, 119; 128, 129 frame memory

Claims (5)

When projecting an image from a projection unit onto a projection surface having an arbitrary curvature, a video signal processing apparatus that applies distortion correction to the projection surface in advance to an input video signal for a flat display surface and applies the distortion correction to the projection unit,
A function formula related to a predetermined projection surface shape over a plurality of types is previously stored, and one of the specified projection surface shapes or a combination thereof is selected, and a variable parameter is input to input a projection parameter based on the function formula. A model creation means for creating a mesh model;
Based on the positional relationship between the observer and the projection means with respect to the projection plane, a correspondence map between the plane display plane and the mesh model of the projection plane is created, and the texture mapping coordinates using the plane map are used as the plane input image. A video signal processing apparatus comprising: distortion correction means that realizes the distortion correction by applying and generating video again.   2. The video signal processing apparatus according to claim 1, wherein the prescribed projection surface shape is one of a part of a spherical surface, a part of a cylindrical surface, or a combination of a plurality of planes. A screen with a wide viewing angle that can be mirror-reflected in a spherical shape with a concave surface facing the viewer,
Video creation means provided at each of the left and right channels respectively, and having at least decoding means for sequentially decoding the compressed video video files of the left and right channels and converting them into video signals of the video,
Synchronization means for controlling the video creation means and establishing synchronization between the video signals of the left and right channels;
The video signal processing apparatus according to claim 1 or 2, wherein distortion correction for the screen is sequentially performed on the video signals of the left and right channels,
Projecting means for projecting the video signals from the respective video signal processing devices onto the screen as polarized images orthogonal to each other,
A virtual reality comprising: a video selection means that is worn by the observer and displays a stereoscopic video image by transmitting a polarized image corresponding to each of the left and right eyes and causing the observer to synthesize and recognize the image. Feeling generation device. The synchronization means includes a synchronization management unit that executes a time management loop process and a synchronization management process,
Each of the video creation means has two frame memories that can be used by sequentially switching between output to the projection means and writing of the decoded video signal,
The decoding means includes a synchronization management unit that executes a synchronization management process, a decompression unit that executes a movie file decompression process, and a playback display unit that executes a video display process, and sequentially decodes the video signals of the left and right channels. 4. The virtual signal according to claim 3, wherein the video signal is converted into a video signal of the moving image and controlled by a time management loop process and a synchronization management process of the synchronization means to establish synchronization between the video signals of the left and right channels. Reality generator. The computer constituting the synchronization means in claim 4 executes the synchronization management process for acquiring the current time and the time management loop process for distributing the current time to each decoding means when a predetermined time has elapsed. Let
5. The computer constituting the video creation means in claim 4 receives the current time, returns a receipt indicating the reception to the synchronization means, updates the reference time, and sends the reference time to the movie file decompression process. The video display process to be distributed and the reference time are acquired and compared with the time at which the currently acquired frame is to be displayed. As a result, if the delay is within 2 frames, the frame data is updated and the frame buffer is updated. The movie file decompression process for notifying the video display process of the address of the video file and the video display process for notifying the video display process that the frame data has been updated. Acquire data as a video signal The video display process for notifying the movie file decompression process of completion of the frame data update, and drawing the video signal on the one not selected on the output side of the two frame memories provided in the video creation means; Upon receipt of the data update completion notification, the movie file decompression process for acquiring new frame data is executed,
Furthermore, when the receipt of the receipt, the computer constituting the synchronization means in claim 4 executes the synchronization management process for transmitting a swap synchronization message to the video display process,
In response to the swap synchronization message, the two frame memories are swapped in response to the swap synchronization message, a receipt is returned to the synchronization management process of the synchronization unit, and the current time again A computer-readable recording medium storing a program for executing a video display process waiting for reception.

Images