JP5214267B2 - Video production system and video production method - Google Patents

Description

  The present invention relates to a video effect system and a video effect method that allow an interactive video to be displayed on a display screen and that the viewer change the interactive video by performing an interactive operation on the video. Is.

  Conventionally, by providing a display screen and a touch panel that covers the display screen, it corresponds to the display position of the operation object when the operation object such as a character is displayed on the display screen. There is a game device that can change the position and posture of an operation target by touching a point on a touch panel with a pen tip or a fingertip and moving or rotating the operation target in real time (for example, , See Patent Document 1).

  Since the game apparatus of this patent document 1 is adapted to perform operations by bringing a pen tip or a fingertip into contact with a point on the touch panel corresponding to the display position of the operation target in the display screen, It was possible to obtain a feeling of conquest as if the object to be operated on the display screen was controlled as desired.

  Here, the video output by the game device of Patent Document 1 changes in response to an interactive operation between the person who operates this video and the game device. It is sometimes called interactive video because it focuses on the fact that it is.

Conventionally, similar to the game device of Patent Document 1, as a video output device configured to output an interactive video by providing a touch panel on a display screen, for example, a ceremony venue such as a ceremonial occasion There were some that were installed and used at some event venues.
JP 2006-122241 A

  However, the game device as described in Patent Document 1 or the above-described conventional video output device installed at a ceremony venue or event venue such as a ceremonial occasion, the operator is on the display screen. There is a problem that the operation target cannot be moved or changed unless it is operated by directly touching the touch panel.

  Therefore, the game apparatus as described in Patent Document 1, or the above-described conventional video output apparatus installed in a ceremony hall or an event hall does not touch the touch panel directly, but the operation target is on the spot. Because it does not move or change freely according to the atmosphere, there was a problem that it was not interesting.

  Also, at ceremonial venues, event venues, and other meeting places, such as ceremonial occasions, video production is desired so that viewers can enjoy interactive video naturally and intuitively according to the atmosphere of the venue. There was a background of being.

  In addition, the game device as described in Patent Document 1, or the above-described conventional video output device installed in the ceremony hall, event hall, or the like, is provided so that the touch panel covers the display screen. Therefore, if the touch panel becomes dirty due to the operation, there is a problem that it is difficult to see the image displayed on the display screen.

  Therefore, in view of the above problems, the present invention provides a video effect system and a video effect that allow a viewer to change an interactive video displayed on a display screen by performing an interactive operation on the video system. It is an object of the present invention to provide a video presentation system and a video presentation method that enable a viewer to enjoy an interactive video that changes by himself / herself by acting naturally or consciously without using a touch panel. .

In order to solve the above-mentioned problem, the video production system according to the present invention is:
Storage means for storing image data;
A video output means having a display screen, and displaying a video based on the image data on the display screen;
A camera sensor that captures an image of a predetermined area on the front side of the display screen;
Based on the image taken by the camera sensor, the fluctuation of the real world object that has entered the fixed area is extracted, and the virtual object in the video projected by the video output means corresponding to the fluctuation of the real world object is extracted. Control means for controlling to change the form,
The control means assumes that there is a virtual fluid that propagates the wind around the real world object around the virtual object in the video, and the fluctuation of the real world object is in the virtual fluid. By controlling the propagation, by controlling the form of the virtual object in the image projected by the image output means,
It is characterized in that the viewer can enjoy a video that changes by himself / herself by acting naturally or consciously.

In addition, the video production system according to the present invention includes:
The control means controls to cause a change in the form of a virtual object in the video to occur continuously in real time in response to a change in the real world object .

In addition, the video production system according to the present invention includes:
The control means extracts the position, size, shape, and / or speed of an object in the real world that has entered the certain area as a variation of the object in the real world, and extracts these as virtual changes in the video. It is characterized in that it is reflected in the change of the form of the object .

In addition, the video production system according to the present invention includes:
The control means exaggerates the simulation of the propagation of the real-world object in the virtual space of the virtual fluid and simulates the virtual fluid .

In addition, in order to solve the above-mentioned problem, the video presentation method according to the present invention is
Storage means for storing image data;
A video output means having a display screen, and displaying a video based on the image data on the display screen;
A camera sensor that captures an image of a predetermined area on the front side of the display screen;
Using the control means for controlling the video projected by the video output means,
The control means extracts fluctuations of a real-world object that has entered the fixed area based on an image captured by the camera sensor , and winds around the virtual object in the video are around the virtual object. Assuming that there is a virtual fluid propagating through the virtual fluid, the virtual object in the image projected by the image output means is simulated by simulating the propagation of changes in the real world object in the virtual fluid. Control to change,
It is characterized in that the viewer can enjoy a video that changes by himself / herself by acting naturally or consciously.

According to such a video production system of the present invention,
Storage means for storing image data;
A video output means having a display screen, and displaying a video based on the image data on the display screen;
A camera sensor that captures an image of a predetermined area on the front side of the display screen;
Based on the image captured by the camera sensor, the variation of the object that has entered the fixed area is extracted, and control is performed so as to change the form of the virtual object in the image projected by the image output unit in response to the variation. Control means,
By enabling viewers to enjoy images that change by themselves by acting naturally or consciously,
In a video production system in which an interactive video displayed on a display screen is changed by an interactive operation performed on the video system by the viewer, the viewer can naturally or without a touch panel. You can enjoy interactive images that change by yourself by acting consciously.

Moreover, according to the video production system of the present invention,
The control means assumes that there is a virtual fluid around a virtual object in the video, and simulates the propagation of the fluctuation in the virtual fluid, so that the video output means Because it controls to change the shape of the virtual object of
The viewer can enjoy interactive video that changes by himself or herself by acting naturally or consciously.

Moreover, according to the video production system of the present invention,
Since the control means controls the change of the form of the virtual object in the video to occur in real time and continuously corresponding to the change,
The viewer can enjoy interactive video that changes by himself or herself by acting naturally or consciously.

Moreover, according to the video production system of the present invention,
Since the control means extracts the position, size, shape, and / or speed of the object that has entered the fixed area as the fluctuation,
The viewer can enjoy interactive video that changes by himself or herself by acting naturally or consciously.

Moreover, according to the video production system of the present invention,
As we have usage fee calculation means to calculate usage fee based on usage time or usage count,
When renting the system to others, the usage fee system can be clarified.

Moreover, according to the video production method of the present invention,
Storage means for storing image data;
A video output means having a display screen, and displaying a video based on the image data on the display screen;
A camera sensor that captures an image of a predetermined area on the front side of the display screen;
Using the control means for controlling the video projected by the video output means,
The control means extracts the fluctuation of the object that has entered the fixed area based on the video imaged by the camera sensor, and changes the form of the virtual object in the video imaged by the video output means in response to the fluctuation. Control to let
By enabling viewers to enjoy images that change by themselves by acting naturally or consciously,
In a video presentation method in which an interactive video displayed on a display screen is changed by an interactive operation performed on the video system by the viewer, the viewer can naturally perform without using a touch panel, or You can enjoy interactive images that change by yourself by acting consciously.

Moreover, according to the video production method of the present invention,
Assuming that a virtual fluid exists around a virtual object in the video, the control means simulates the propagation of the fluctuation in the virtual fluid, thereby causing the video output means Because it controls to change the shape of the virtual object of
The viewer can enjoy interactive video that changes by himself or herself by acting naturally or consciously.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of a video effect system and a video effect method according to the present invention will be specifically described based on the drawings.
FIG. 1 to FIG. 15 are diagrams which are referred to for explaining the video effect system 2 and the video effect method according to the first embodiment of the present invention.

  FIG. 1 is a hardware configuration diagram of a video presentation system 2 according to the present embodiment. The video presentation system 2 according to the present embodiment is configured to include a display device 4 (video output means), a camera sensor 6, and a computer 8. Each of the display device 4 and the camera sensor 6 is wired to the computer 8.

  The computer 8 includes a storage device 10 (storage means), a central processing unit 12, and an input device 14 (input means) such as a keyboard and a mouse. The storage device 10 of the computer 8 stores a material file 30 and a video effect program 32 shown in FIG.

  Further, the central processing unit 12 of the computer 8 shown in FIG. 1 operates in accordance with a video effect program 32 to be described later read into the storage device 10 and operates as a control means for controlling the video displayed on the display device 4. It is like that.

  As shown in FIGS. 2 and 3, the display device 4 has a display screen 18. The display device 4 includes a speaker (not shown). As shown in FIG. 3, the display device 4 has a display screen 18 facing a viewer 16 who operates to change the video while watching the video displayed on the display screen 18. It is installed on the gantry 20 in a posture. As the gantry 20, for example, an easel can be used.

  Such a display device 4 and the gantry 20 can be installed as a welcome board near the entrance of a wedding reception hall, for example. As the display device 4, for example, a liquid crystal display device, a plasma display device, an organic EL display device, or the like can be used.

  In addition, the camera sensor 6 of the video effect system 2 is attached to the upper end portion of the display device 4. The camera sensor 6 takes an image of a predetermined area surrounded by a broken line in FIG. 3 on the front side of the display screen 18 of the display device 4.

  FIG. 4 is a configuration diagram of software 28 for operating the video effect system 2 according to the present embodiment. The software 28 for operating the video presentation system 2 includes a material file 30 and a video presentation program 32. The material file 30 and the video effect program 32 are stored in the storage means 10 of the computer 8 shown in FIG.

  The material file 30 includes moving image data 34, still image data 36a, 36b, 36c, 36d, 36e, 36f,..., 3D model data 38 (three-dimensional model data), audio data 40, and text data 42. Yes.

  Regarding the specific contents of the moving image data 34, still image data 36a, 36b, 36c, 36d, 36e, 36f,..., 3D model data 38, audio data 40, and text data 42 in the material file 30, The video presentation program 32 described later and the video 80 shown in FIG. 5 will be described together.

  Here, the moving image data 34, the still image data 36a, 36b, 36c, 36d, 36e, 36f,..., And the 3D model data 38 of the material file 30 may be referred to as image data for convenience. To do. In addition, each of the still image data 36a, 36b, 36c, 36d, 36e, 36f,..., Or all of them may be referred to as still image data 36 without adding an alphabet to the reference numeral. And

  Next, the video effect program 32 includes a sensor module 46 and a video effect module 48. Here, a module refers to a small program that is configured to perform some function.

  The sensor module 46 of the video effect program 32 acquires the video (real world video) captured by the camera sensor 6 shown in FIG. 3 in real time, and performs any movement shown in the video by the previous frame difference method. Information on the size, shape, moving direction, and speed of the object is extracted.

  That is, the sensor module 46 converts each image acquired from the camera sensor 6 to about 30 frames per second into a gray scale as needed, and extracts only the moving part by detecting the difference between the previous and next frames. Thus, the size and shape of any moving object shown in the video is extracted. Further, the sensor module 46 obtains the coordinates of the center position based on the size and shape of the moving object thus extracted, and extracts the moving direction and speed from the amount of change in the coordinates of the center position. It has become.

  As the moving object in the captured image that the sensor module 46 performs such an extraction operation, the viewer 16 or a part of the body that has entered the fixed area captured by the camera sensor 6, or the viewer An object that is carried or carried by the camera 16 or an object that is thrown into a certain area photographed by the camera sensor 6 or an object that has been rolled up is applicable.

  As shown in FIG. 4, the sensor module 46 extracts information on the size, shape, moving direction, and speed extracted from any moving object in the video captured by the camera sensor 6, and the video effect module 48. Is output continuously.

  Next, the video presentation module 48 in the video presentation program 32 includes a fluid simulation module 54, a video playback module 56, a video effect module 58, a CG synthesis module 60 (computer graphic synthesis module), an audio playback module 62, and a voice effect module. 64 and a speech synthesis module 66.

  The fluid simulation module 54 in the video effect module 48 sets a virtual space in which air exists and spreads inside the display screen 18 of the display device 4 shown in FIG. Flow) is simulated.

  That is, the fluid simulation module 54 causes the wind generated by a real-world object moving in a certain area on the front side of the display screen 18 of the display device 4 to penetrate the display screen 18 of the display device 4. Assuming that it propagates to the virtual space, the size, shape, moving direction, and speed of the moving object in the real image captured by the camera sensor 6 extracted by the sensor module 46 shown in FIG. Information is acquired continuously, and how the wind propagates in the virtual space is simulated based on the information.

  At this time, the fluid simulation module 54 performs simulation by exaggerating the strength of the wind propagating in the virtual space to some extent. Further, the fluid simulation module 54 continuously transmits such simulation information to each of the video reproduction module 56, the video effect module 58, the CG synthesis module 60, the audio reproduction module 62, the audio effect module 64, and the voice synthesis module 66. Output.

  The video reproduction module 56 of the video effect module 48 reads and reproduces the moving image data 34 and the still image data 36a, 36b, 36c, 36d, 36e, 36f,.

  The video reproduction module 56 can individually reproduce a plurality of arbitrary ones of the moving image data 34 and the still image data 36a, 36b, 36c, 36d, 36e, 36f,. It is like that.

  Each of the moving image data 34 and the still image data 36a, 36b, 36c, 36d, 36e, 36f,... Arbitrarily reproduced by the video reproduction module 56 is displayed in the display screen 18 of the display device 4 shown in FIG. How to be displayed in which range is determined by the mapping process of the CG synthesis module 60 described later.

  It should be noted that the video reproduction module 56 continuously reproduces the moving image data 34 and the still image data 36a, 36b, 36c, 36d, 36e, 36f, and so on, and at what timing. Whether the video is switched and played back will be described together when the video 80 in FIG.

  As shown in FIG. 4, the video data 34 and still image data 36a, 36b, 36c, 36d, 36e, 36f,... Reproduced by the video playback module 56 are sent to the video effect module 58. It has become.

  Each time the video effect module 58 receives simulation information indicating that wind has occurred in the virtual space from the fluid simulation module 54, the video effect module 58 alternates the red hue with respect to the video of the video data 34 reproduced by the video reproduction module 56. Image processing that mixes or prevents mixing is performed.

  The video effect module 58 adds such a hue effect (video effect) to the video of the moving image data 34 reproduced by the video reproduction module 56, and then sends it to the CG synthesis module 60. Yes.

  The CG synthesis module 60 generates CG (computer graphics) based on the 3D model data 38 and the text data 42 in the material file 30 or generates CG independently.

  The CG synthesis module 60 arranges the CG generated in this way, the video received from the video effect module 58, and the like at a predetermined position in the virtual space. In such an arrangement work of the CG synthesis module 60, the video received from the video effect module 58 is textured on the surface of the CG generated by the CG synthesis module 60 or on an arbitrary area in the virtual space. As described above, an operation such as mapping is included. The CG synthesis module 60 also synthesizes videos.

  Further, the CG synthesis module 60 is configured to deform the CG in real time or generate a new CG based on the simulation information sent from the fluid simulation module 54. Further, based on the simulation information, texture replacement and the like are performed in real time. Further, the CG synthesis module 60 outputs the completed video to the display device 4. Specific operation contents of the CG synthesis module 60 will be described together with a video 80 and the like in FIG.

  As shown in FIG. 4, the sound reproduction module 62 reads and reproduces the sound data 40 of the material file 30 based on the simulation information sent from the fluid simulation module 54, and sends it to the sound effect module 64. ing.

  The sound effect module 64 changes the pitch (frequency) of the sound data 40 reproduced by the sound reproduction module 62 based on the simulation information sent from the fluid simulation module 54, and sends it to the sound synthesis module 66. Yes.

  The voice synthesis module 66 generates electronic sound data, and outputs the generated electronic sound and the voice received from the voice effect module 64 to the built-in speaker of the display device 4.

  Specific operation contents of the audio reproduction module 62, the audio effect module 64, and the audio synthesis module 66 will be described together with the description of the video 80 and the like in FIG.

  FIG. 5 is a diagram showing a colored video 80 displayed on the display screen 18 of the display device 4 by the video presentation program 32 of the video presentation system 2. The video 80 is arranged in an area near the lower end of the video 80, and the desk 102 in which only a partial area is displayed without being fit within the frame of the video 80 is placed on the desk 102. The album 104, the translucent lace curtain 106, the window 108 seen through the curtain 106, and the sky 110 showing the cloud flowing on the outside of the window 108 are shown. .

  The video 80 includes a plurality of particles 112 that are generated near the upper end of the video 80, slowly fall on the album 104 while shining, and disappear. A designed character string 114 displayed so as to stick to the display screen 18 is displayed.

  This video 80 is displayed by combining the videos 82, 84, 86, 88, 90, and 92 shown in FIGS. 6 to 11 so as to overlap each other by using respective alpha channels of these videos. Is.

  Each of the images 82, 84, 86, 88, 90, and 92 is created as described below, and the composition of these images 82, 84, 86, 88, 90, and 92 is a video effect program shown in FIG. 4. The CG synthesis module 60 of the video effect module 48 in FIG.

  As shown in FIG. 6, a sky 110 in which a cloud is flowing is projected on the video 82. As shown in FIG. 4, the video 82 is read by the video playback module 56 of the video rendering module 48 in the video rendering program 32 and continuously loop-played the moving image data 34 of the material file 30 that is the basis thereof. The video effect module 58 performs image processing on the video, and the CG synthesis module 60 maps the video as a texture to the entire area within the frame of the video 82.

  Each time the video effect module 58 receives simulation information indicating that wind has occurred in the virtual space from the fluid simulation module 54, the video effect module 58 alternates the red hue with respect to the video of the video data 34 reproduced by the video reproduction module 56. 6 is performed so that the sky 110 projected in the video 82 shown in FIG. 6 alternately becomes a sunset sky or a blue sky. .

  As shown in FIG. 7, a desk 102 and a window 108 are projected on the video 84. As shown in FIG. 4, the video 84 is read by the video playback module 56 of the video rendering module 48 in the video rendering program 32 and played back by the still image data 36a of the material file 30 serving as the basis thereof. 60 is created by mapping the reproduced video as a texture to the entire area within the frame of the video 84.

  As shown in FIG. 8, the opened album 104 is displayed on the video 86. In addition, in this video 86, each of the images of two different photographs 120c and 120d showing a bride and groom (not shown) are pasted on both left and right pages of the album 104. It is reflected in.

  As shown in FIG. 4, the video of the album 104 portion excluding the video portions of the photos 120c and 120d in the video 86 is obtained by using the 3D model data 38 of the material file 30 as a basis for the video production program. 32, the CG synthesis module 60 of the video effect module 48 reads and generates a three-dimensional CG video, and this CG synthesis module 60 maps to a predetermined position in the display frame of the video 86 as shown in FIG. It was created by doing.

  In addition, as shown in FIG. 4, the video reproduction module 56 reads the still image data 36c and 36d of the material file 30 as the basis of each of the images 120c and 120d in the video 86, respectively. The CG synthesis module 60 three-dimensionally maps the reproduced video as a texture on each of both pages of the album 104 that are spread.

  Further, the album 104 shown in the video 86 shown in FIG. 8 is within a certain area that the fluid simulation module 54 of the video effect module 48 in the video effect program 32 shown in FIG. 4 takes by the camera sensor 6 shown in FIG. When a simulation that a certain wind has occurred in the virtual space is performed based on the operation of the viewer 16 and the like, the page is turned as shown in FIG.

  For example, the uppermost page on the right side of the left and right pages of the album 104 in FIG. 8 is turned up by the wind generated in the virtual space so as to overlap the upper side of the left page. In this case, the video effect program 32 shown in FIG. 4 operates as follows.

  First, basic data required when the CG synthesis module 60 generates an animation image in which the page of the album 104 is turned over is prepared as part of the 3D model data 38 of the material file 30. Yes.

  When the CG synthesis module 60 receives simulation information from the fluid simulation module 54 that the wind from right to left passes through the vicinity of the position where the album 104 is arranged in the virtual space, the 3D model data 38 of the material file 30 is received. Although not shown in the figure, on the right page of the album 104 (see FIG. 8), it is a page having the same shape as that of the right page. Fade in the first thing to make it appear.

  Also, the video playback module 56 shown in FIG. 4 is based on the simulation information sent from the fluid simulation module 54, and the stillness of the material file 30 that is the basis of the video 120c, 120d (see FIG. 8) of the album 104 is displayed. While continuing to reproduce the image data 36c, 36d, the reproduction of still image data 36e, 36f, which is the basis of the images of later-described photographs 120e, 120f, which are different from these images, is newly started.

  Although not shown, the CG compositing module 60 continues to display the image of the photo 120c on the left page of the album 104 (see FIG. 8), and appears to fade in on the right page as described above. At the same time as the fade-in operation, the image of the photo 120d based on the still image data 36d is mapped and displayed on the front and back surfaces of the page (not shown). . The photograph 120d mapped to the page appearing by fading in is the same as the photograph 120d mapped from the beginning to the lower page of this page.

  In addition, the CG composition module 60 maps a video of a new photograph 120e based on the still image data 36e on a surface on the back side of a page (not shown) that appears by fading in. This photo 120e cannot actually be seen before the page that appears after fading in is turned over, but it can be seen when the page has been turned and moved to the left. (See FIG. 13).

  In addition, before the page (not shown) that appears after fading in, the lower page of the page that appears after fading in is a still image instead of the image of the original photo 120d. An image of a new photograph 120f based on the data 36f is mapped (see FIG. 12).

  Based on the 3D model data 38 of the material file 30, the CG compositing module 60 projects a series of animations of the page turning and moving to the left side while mapping the images of the photos 120d and 120e. It has come to go. The CG synthesis module 60 projects a series of images of this animation based on the wind strength information in the simulation information sent from the fluid simulation module 54 so that the page turns slowly when the wind is weak, When the wind is strong, the pages are projected so that they can turn quickly.

  When the animation image is finished and the turned page has been moved to the left side as shown in FIG. 13, the CG synthesis module 60 is placed on a page (not shown) on the lower side of the turned left page. Instead of the original photo 120c (see FIG. 12), after the photo 120e is mapped, the video of the page that has been turned over is faded out. Also, the video playback module 56 shown in FIG. 4 ends the playback of the still image data 36c and 36d that was the basis of the video of the original photographs 120c and 120d shown in FIG.

  The video effect program 32 shown in FIG. 4 shows a state in which the page of the album 104 in the video 86 is turned up whenever the fluid simulation module 54 performs a simulation that a wind is generated in the virtual space. It is like that. In addition, when the fluid simulation module 54 performs a simulation to the effect that a wind in the opposite direction has occurred in the virtual space, the state in which the pages of the album 104 turn in the opposite direction is displayed.

  Such changes in the image 86 shown in FIGS. 8, 12, and 13 are completed by superimposing the image 86 and other images 82, 84, 88, 90, and 92 other than the image 86 in FIG. 5. The video 80 is also reflected.

  Further, in accordance with the state in which the page of the album 104 displayed in the video 86 is turned, the audio reproduction module 62 of the video effect module 48 in the video effect program 32 shown in FIG. 4 is sent from the fluid simulation module 54. Based on the coming simulation information, the audio data 40 which is the basis of the sound when the page is turned is read and reproduced. Thereby, the audio reproduction module 62 outputs a sound when the page is turned in accordance with the animation of the video 86.

  Also, the sound effect module changes the frequency of the sound when the page reproduced by the sound reproduction module 62 turns based on the wind strength information in the simulation information sent from the fluid simulation module 54. This makes a difference between when pages turn quickly and when they turn slowly.

  Next, as shown in FIG. 9, a curtain 106 is projected on the video 88. The image of the portion of the curtain 106 in the image 88 is generated by the CG synthesis module 60 shown in FIG. 4 as a free-form CG image, which is generated in a predetermined frame within the display frame of the image 88. 4, the video reproduction module 56 shown in FIG. 4 reads and reproduces the still image data 36b that is the basis of the translucent lace pattern of the curtain 106, and the CG synthesis module 60 uses the curtain 106 as a texture. It is created by mapping on the free-form surface.

  In the curtain 106 displayed in the video 88 of FIG. 9, the fluid simulation module 54 of the video rendering module 48 in the video rendering program 32 shown in FIG. 4 performs a simulation that a certain wind has occurred in the virtual space. In this case, as shown in FIG. 14, the skirt rises and moves toward the near side (the lower side of the display frame of the video 88). Such a deformation of the shape of the free curved surface of the curtain 106 is performed based on simulation information received from the fluid simulation module 54 by the CG synthesis module 60.

  Such a change in the image 88 is also reflected in the image 80 of FIG. 5 which is completed by superimposing the image 88 and other images 82, 84, 86, 90, 92 other than the image 88. It is like that.

  Next, as shown in FIG. 10, a plurality of particles 112 are displayed on the video 90. Each of the particles 112 in the image 90 is generated at a position near the upper end of the display frame of the image 90, and moves downward while slowly spreading in the horizontal direction, and then disappears at a position near the lower end of the display frame. It is projected to do. Such generation, movement and disappearance of the particles 112 are independently performed by the CG synthesis module 60 shown in FIG. Further, the CG synthesis module 60 also performs the work of mapping such particles 112 to predetermined positions in the display frame of the video 90.

  In addition, when the fluid simulation module 54 performs a simulation indicating that a certain amount of wind is generated in the virtual space, the particles 112 in the video 90 are stored in the virtual space as shown in FIG. It descends while being swept laterally by the wind. The movement direction of the particles 112 is changed based on simulation information received by the CG synthesis module 60 from the fluid simulation module 54.

  Such a change in the image 90 is also reflected in the image 80 in FIG. 5 which is completed by superimposing the image 90 and other images 82, 84, 86, 88, 92 other than the image 90. It is like that.

  Also, the speech synthesis module 66 shown in FIG. 4 generates an electronic sound that gives an impression that the particle 112 is shining in accordance with the generation timing of the particle 112.

  Next, as shown in FIG. 11, the designed character string 114 is displayed on the video 92. As shown in FIG. 4, the character string 114 is generated as a CG image based on the text data 42 after the CG synthesis module 60 reads the text data 42 of the material file 30.

  According to the video presentation system 2 and the video presentation method according to the present embodiment as described above, the touch panel is not used, so that the display is performed on the display screen 18 of the display device 4 as shown in FIG. When the viewer 16 performs an operation on the interactive video 80 itself, a plurality of other viewers (not shown) other than the viewer 16 can grasp changes in the interactive video 80 at the same time. it can.

  In addition, according to the video presentation system 2 and the video presentation method according to the present embodiment, an interactive video displayed on the display screen 18 of the display device 4 by the central processing unit 12 (control means) of the computer 8. Control is made so that changes in the shape of the virtual object in 80 are generated in real time and continuously in response to changes in the object such as the viewer 16, so that the viewer behaves naturally or consciously. You can enjoy interactive video that changes by itself.

  In addition, according to the video presentation system 2 and the video presentation method according to the present embodiment, the size and shape of the object such as the viewer 16 that has entered the certain area in front of the display device 4, and these move. The moving direction and the speed of the movement are extracted as the fluctuation of the object such as the viewer 16, and the wind generated by such fluctuation is reflected in the interactive video 80 displayed on the display screen 18 of the display device 4. Therefore, it is still possible to enjoy interactive video that changes by the viewer's behavior naturally or consciously.

  Next, FIG.1 and FIG.16 is a figure referred in order to demonstrate the video presentation system 150 which concerns on the 2nd Embodiment of this invention, and its video presentation method. The same parts as those in the video effect system 2 according to the first embodiment will be described using the same reference numerals, and redundant description of the same configuration and usage will be omitted.

  As shown in FIG. 1, the hardware configuration of the video effect system 150 according to the present embodiment is configured in exactly the same way as the hardware configuration of the video effect system 2 according to the first embodiment.

  The video presentation system 150 according to the present embodiment is shown in FIG. 16 in the storage device 10 of the computer 8 instead of the software 28 (see FIG. 4) of the video presentation system 2 according to the first embodiment. Software 152 is stored.

  The software 152 includes the material file 30 that is exactly the same as the material file 30 of the software 28 in the video presentation system 2 according to the first embodiment. In addition, the software 152 has a data structure substantially the same as that of the material file 30. It has material files 154 and 156 that have different image materials from the image 80 (see FIG. 5).

  Although not shown, each of the material files 154 and 156 includes data corresponding to each of the moving image data 34, the still image data 36, the 3D model data 38, the audio data 40, and the text data 42 of the material file 30. As many as necessary for projecting a unique video different from 80 (see FIG. 5).

  Here, in the material files 154 and 156, the contents data corresponding to the moving image data 34, the still image data 36, the 3D model data 38, the audio data 40, and the text data 42 of the material file 30 are described for convenience. The same reference numerals as those assigned to the data of the material file 30 will be used.

  The software 152 includes a video effect program 158 instead of the video effect program 32 of the software 28 in the video effect system 2 according to the first embodiment.

  This video effect program 158 includes the same video effect module 48 as the video effect module 48 of the video effect program 32 of the software 28 in the video effect system 2 according to the first embodiment. In addition, video effect modules 160 and 162 for performing video effects different from the video effect module 48 are provided.

  Although not shown, each of the video effect modules 160 and 162 includes a fluid simulation module 54, a video playback module 56, a video effect module 58, a CG synthesis module 60, an audio playback module 62, an audio effect module 64, and the like. A module corresponding to each of the speech synthesis modules 66 is provided, and thereby, a video effect different from that of the video effect module 48 is performed.

  Here, in each of the video effect modules 160 and 162, the fluid simulation module 54 of the video effect module 48, the video reproduction module 56, the video effect module 58, the CG synthesis module 60, the audio reproduction module 62, the audio effect module 64, and For convenience, the modules corresponding to each of the speech synthesis modules 66 are referred to with the same reference numerals as those assigned to the modules of the video effect module 48.

  Particularly, in each of the fluid simulation modules 54 of the video effect modules 160 and 162, the fluid simulation module 54 in the video effect system 2 according to the first embodiment simulates the wind (flow in the air) in the virtual space. The flow of water under the assumption that the virtual space is filled with water, and the assumption that the virtual space is filled with a transparent colloidal solution The flow of this colloidal solution is simulated.

  The video effect program 158 includes an effect control module 44, an effect configuration module 50, and an effect setting file 52.

  In the effect setting file 52 in the video effect program 158, setting data on which of the plurality of material files 30, 154, 156 is selected and used, and a sensor module in the video effect program 158 46, setting data necessary for the operation, setting data for selecting which of the plurality of video effect modules 48, 160, 162 to operate, and a video effect module 48 to select and operate. , 160, and 162, each of the fluid simulation module 54, the video reproduction module 56, the video effect module 58, the CG synthesis module 60, the audio reproduction module 62, the audio effect module 64, and the voice synthesis module 66. Setting data on how to operate Data, etc., are collectively registered.

  Various setting data in the effect setting file 52 are input by the maintenance worker from the input device 14 of the computer 8 shown in FIG. 1 in the maintenance mode before the start of the image effect or after the stop of the image effect. In such a case, the effect composition module 50 shown in FIG. 16 is changed based on such an operation input.

  The effect control module 44 is a plurality of material files 30, 154, 156 based on the setting data of the effect setting file 52 after the start operation for starting the image effect and before actually starting the image effect. The user selects and controls which one of the plurality of video effect modules 48, 160, and 162 is to be operated.

  Among the video effect modules 48, 160 and 162 in the video effect program 158, the fluid simulation module 54, the video reproduction module 56, the video effect module 58, and the CG synthesis module among those selected by the effect control module 44. 60, the sound reproduction module 62, the sound effect module 64, and the sound synthesis module 66 are under the control of the effect control module 44 in addition to the same operations as those described in the first embodiment. The following operations are performed as necessary.

  That is, when there are a plurality of moving image data 34 and / or still image data 36 among the material files 30, 154, 156 selected by the effect control module 44, the video reproduction module 56 simultaneously and It is determined under the control of the effect control module 44 whether to reproduce separately or sequentially.

  The video playback module 56 starts or stops video playback based on information sent from the sensor module 46 or simulation information sent from the fluid simulation module 54 under the control of the effect control module 44. The timing is changed, the reproduction speed is changed, and the moving image data 34 and / or the still image data 36 to be reproduced are switched.

  Further, the video effect module 58 receives from the video reproduction module 56 based on the information sent from the sensor module 46 and the simulation information sent from the fluid simulation module 54 under the control of the effect control module 44. For example, an effect that gives deformation such as distorting the image into a ripple shape, an effect that rotates or enlarges or reduces the image, or an enlargement range after enlarging only a part of the image. The effect of moving the image or the effect of changing the color or brightness of the video is added in real time.

  Further, the CG synthesis module 60 reads the 3D model data 38 in the material files 30, 154, 156 selected by the presentation control module 44 under the control of the presentation control module 44, and this 3D model data. 38 is generated, or a virtual object object is generated independently.

  Further, the CG composition module 60 reads the text data 42 in the material files 30, 154, 156 selected by the effect control module 44, and generates the designed special character as a two-dimensional CG. It has become.

  In addition, the CG synthesis module 60 arranges the generated CG and the video received from the video playback module 56 and the video effect module 58 at predetermined positions in the virtual space under the control of the effect control module 44. It has become. In such an arrangement work of the CG synthesis module 60, the video reproduction module 56 is placed on the surface of the object of the virtual object generated by the CG synthesis module 60, or on an arbitrary area in the two-dimensional or three-dimensional virtual space. And mapping the video received from the video effect module 58 as a texture.

  In addition, the CG synthesis module 60 arbitrarily controls the virtual space based on the information sent from the sensor module 46 or the simulation information sent from the fluid simulation module 54 under the control of the effect control module 44. Generation of new CG, synthesis of CGs, synthesis of CG and video, deformation of CG, and / or replacement of texture, etc. are performed in real time.

  The sound reproduction module 62 also controls the material files 30, 154, based on the information sent from the sensor module 46 and the simulation information sent from the fluid simulation module 54 under the control of the effect control module 44. The audio data 40 selected by the effect control module 44 of 156 is read and reproduced.

  The audio playback module 62 controls the assignment (assignment) of audio data 40 corresponding to the video created by the video playback module 56, the video effect module 58, and the CG synthesis module 60, the playback timing and speed, and the like. ing.

  The sound effect module 64 is reproduced by the sound reproduction module 62 based on the information sent from the sensor module 46 and the simulation information sent from the fluid simulation module 54 under the control of the effect control module 44. The effect (effect) such as reverb or delay is added to the audio data 40 in real time, and the pitch (frequency) is changed. The sound effect module 64 also performs filter processing for removing noise.

  In addition, the voice synthesis module 66 uses various sound synthesis algorithms under the control of the effect control module 44 to convert information sent from the sensor module 46 or simulation information sent from the fluid simulation module 54. Based on this, electronic sound data is generated. Since the speech synthesis module 66 has a plurality of sound oscillators (oscillators), it is possible to generate and synthesize (synthesize) electronic sounds by combining various waveforms.

  The voice synthesis module 66 synthesizes the electronic sound data generated in this way with the voice data received from the voice reproduction module 62 or the voice effect module 64. The voice synthesis module 66 also adds an effect to the synthesized voice data.

  According to the video presentation system 150 and the video presentation method according to the present embodiment, as shown in FIG. 3, the touch panel is not used, as in the first embodiment. When the viewer 16 performs an operation on the interactive video 80 displayed on the display screen 18 of the display device 4 or other video itself, other plurals (not shown) other than the viewer 16 are shown. The viewer can grasp the change of the interactive video 80 or other video at the same time.

  Further, according to the video presentation system 150 and the video presentation method according to the present embodiment, the central processing unit 12 (control means) of the computer 8 is connected to the display device 4 as in the first embodiment. A change in the form of a virtual object or the like in the interactive video 80 or other video displayed on the display screen 18 is generated in real time and continuously corresponding to the change in the object such as the viewer 16. Therefore, it is possible to enjoy interactive video that changes by the viewer's behavior naturally or consciously.

  In addition, according to the video presentation system 150 and the video presentation method according to the present embodiment, the viewer 16 or the like who has entered the fixed area in front of the display device 4 can be obtained as in the first embodiment. The size and shape of the objects, and the moving direction and speed when they move are extracted as the fluctuations of the object such as the viewer 16, and the wind generated in the virtual space due to such fluctuations, or water or a colloidal solution Since the flow inside is reflected in the interactive video 80 displayed on the display screen 18 of the display device 4 or other video, it is changed by the viewer's behavior naturally or consciously. You can enjoy interactive video.

  Note that in the video effect system 150 according to the present embodiment, only the three material files 30, 154, 156 are stored in the storage means 10 of the computer 8, but these are separate new files. Even if a storage medium such as a flexible disk, an optical disk, or a memory card storing various material files is read by a storage medium reading device (not shown) provided in the computer 8, it can be added at any time. Good.

  Further, by downloading via a network (not shown) connected to the computer 8, new material files may be added at any time. Further, when any one of the material files 30, 154 and 156 becomes unnecessary, it may be deleted from the storage means 10 of the computer 8.

  In the video presentation system 150 according to the present embodiment, the video presentation program 158 includes only the three video presentation modules 48, 160, and 162. The effect module may be added at any time via a storage medium or a network in the same manner as the material file. Further, when any one of the video effect modules 48, 160, 162 becomes unnecessary, it may be deleted from the storage means 10 of the computer 8.

  In the video presentation systems 2 and 150 according to the first and second embodiments, the sensor module 46 of the video presentation programs 32 and 158 is based on the actual video shot by the camera sensor 6 shown in FIG. The image processing of the gray scale conversion and the previous frame difference method is performed, but the image processing such as the background difference method may be performed.

  In the video presentation systems 2 and 150 according to the first and second embodiments, the sensor module 46 of the video presentation programs 32 and 158 is based on the actual video shot by the camera sensor 6 shown in FIG. The information on the size, shape, moving direction, and speed of the object in the video was extracted, but any of the information on the size, shape, moving direction, and speed You may come to extract only. Information such as color and position may be extracted.

  Further, in the video effect system 150 according to the second embodiment, whether the sensor module 46 performs image processing of the previous frame difference method or image processing such as the background difference method, or FIG. In the video effect programs 32 and 158, information about the size, shape, moving direction, speed, color, position, etc. of the object or the like in the video imaged by the camera sensor 6 shown in FIG. The settings may be stored in the effect setting file 52, and the effect control module 44 may control the sensor module 46 based on the settings of the effect setting file 52.

  In addition, the video effect systems 2 and 150 according to the first and second embodiments are configured to use an easel as the gantry 20, but may use a gantry other than the easel. Good.

  Further, as shown in FIG. 3, in the video effect systems 2 and 150 according to the first and second embodiments, the camera sensor 6 is attached to the upper end portion of the display device 4. The sensor 6 may be attached anywhere as long as it can capture the inside of a certain area on the front side of the display screen 18 of the display device 4.

  For example, by providing eaves at the upper end of the display device 4 and attaching the camera sensor 6 to the front end of the eaves, the camera sensor 6 looks down at the lower direction substantially vertically, and the display screen of the display device 4 The inside of the fixed area | region of 18 front may be image | photographed.

  In the video presentation systems 2 and 150 according to the first and second embodiments, the fluid simulation module 54 of the video presentation module 48 of the video presentation programs 32 and 158 always performs a strict simulation. It does not have to be, and a simple simulation may be performed.

  For example, when the sensor module 46 extracts information on the size, shape, moving direction, and speed of any moving object in the image actually captured by the camera sensor 6, the fluid simulation module 54 extracts these information. Based on the information of the above, assuming a plurality of virtual circles centered on each of the plurality of arbitrary positions at a plurality of arbitrary positions on the outline of the moving object in the video, each of the plurality of virtual circles Is expanded at the same speed as the moving object in the image, the movement of the envelope in contact with all of these virtual circles is determined by the moving object in the image actually captured by the camera sensor 6. You may be made to treat it as the wind which arose.

  The video presentation systems 2 and 150 according to the first and second embodiments are used in the video presentation programs 32 and 158 based on the usage time or the number of times of use of the video presentation systems 2 and 150. A usage fee calculation module (a usage fee calculation means) for calculating a fee may be provided.

  In addition, in the video presentation systems 2 and 150 according to the first and second embodiments, such a usage fee calculation module limits the number of times and the usage conditions of the video presentation systems 2 and 150. It may be. In addition, such a usage fee calculation module takes a record (log) of the usage time and system status of the video production system 2,150, and transfers these records to an arbitrary server via a network. May be.

It is a figure which shows the hardware constitutions of the video production systems 2 and 150 which concern on the 1st and 2nd embodiment of this invention. It is a front view which shows the display apparatus 4, the camera sensor 6, and the mount frame 20 of the image | video production system 2,150 which concern on the 1st and 2nd embodiment of this invention. It is a side view which shows the display apparatus 4, the camera sensor 6, the mount frame 20, and the viewer 16 of the image | video production system 2,150 which concern on the 1st and 2nd embodiment of this invention. It is a conceptual diagram which shows the structure of the software 28 of the video presentation system 2 which concerns on the 1st Embodiment of this invention. It is a figure which shows the image | video 80 displayed on the display screen 18 of the display apparatus 4 in FIG. It is a figure which shows the image | video 82 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 84 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 86 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 88 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 90 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 92 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 86 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 86 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 88 which comprises a part of image | video 80 in FIG. It is a figure which shows the image | video 90 which comprises a part of image | video 80 in FIG. It is a conceptual diagram which shows the structure of the software 152 of the video presentation system 150 which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

2 video production system 4 display device 6 camera sensor 8 computer 10 storage device 12 central processing unit
DESCRIPTION OF SYMBOLS 14 Input device 16 Viewer 18 Display screen 20 Stand 28 Software 30 Material file 32 Image production program 34 Movie data 36, 36a, 36b, 36c, 36d, 36e, 36f Still image data 38 3D model data 40 Audio data 42 Text data 44 Production control module 46 Sensor module 48 Video production module 50 Production composition module 52 Production setting file 54 Fluid simulation module
56 Video playback module
58 Video Effect Module
60 CG synthesis module
62 voice playback module 64 voice effect module 66 voice synthesis module 80, 82, 84, 86, 88, 90, 92 video 102 desk 104 album 106 curtain 108 window 110 sky 112 particle 114 character string 120c, 120d, 120e, 120f photo 150 Video production system 152 Software 154, 156 Material file 158 Video production program 160, 162 Video production module

Claims (5)

Storage means for storing image data;
A video output means having a display screen, and displaying a video based on the image data on the display screen;
A camera sensor that captures an image of a predetermined area on the front side of the display screen;
Based on the image taken by the camera sensor, the fluctuation of the real world object that has entered the fixed area is extracted, and the virtual object in the video projected by the video output means corresponding to the fluctuation of the real world object is extracted. Control means for controlling to change the form,
The control means assumes that there is a virtual fluid that propagates the wind around the real world object around the virtual object in the video, and the fluctuation of the real world object is in the virtual fluid. By controlling the propagation, by controlling the form of the virtual object in the image projected by the image output means,
A video production system characterized by allowing viewers to enjoy a video that changes by themselves by acting naturally or consciously. Said control means changes the form of the virtual object in the image, the real-time in response to variation of the object in the real world, and, to claim 1, wherein the controller controls so that makes continuously occur The video production system described. The control means extracts the position, size, shape, and / or speed of an object in the real world that has entered the certain area as a variation of the object in the real world, and extracts these as virtual changes in the video. The video effect system according to claim 1 , wherein the video effect system is reflected in a change in the shape of the object . The video effect according to any one of claims 1 to 3, wherein the control means exaggerates and simulates the propagation of the fluctuation of the real world object in the virtual space of the virtual fluid. system. Storage means for storing image data;
A video output means having a display screen, and displaying a video based on the image data on the display screen;
A camera sensor that captures an image of a predetermined area on the front side of the display screen;
Using the control means for controlling the video projected by the video output means,
The control means extracts fluctuations of a real-world object that has entered the fixed area based on an image captured by the camera sensor , and winds around the virtual object in the video are around the virtual object. Assuming that there is a virtual fluid propagating through the virtual fluid, the virtual object in the image projected by the image output means is simulated by simulating the propagation of changes in the real world object in the virtual fluid. Control to change,
A video production method characterized in that a viewer can enjoy a video that changes by himself or herself by acting naturally or consciously.

Images