WO2013179992A1

WO2013179992A1 - Stage device and stage facility

Info

Publication number: WO2013179992A1
Application number: PCT/JP2013/064293
Authority: WO
Inventors: ラクロワポール; 健典星
Original assignee: 株式会社セガ
Priority date: 2012-05-31
Filing date: 2013-05-22
Publication date: 2013-12-05
Also published as: JP2013250773A

Abstract

In order to increase the realism of an image object (230) in a stage device that displays the image object (230) on a display unit (211) installed on a stage (210), this stage device (10) is equipped with: a stage (210); lights (213L, 213R) capable of shining light on the stage (210); a lighting control means (410) that controls the lights (213L, 213R); a display unit (211) that is installed on the stage (210) and displays an image object (230); and image data generation means (450, 500) that generate image data for the purpose of displaying the image object (230) on the display unit (211). In addition, the color of the image object (230) is adjusted in synchronization with the control of the lights (213L, 213R) by the lighting control means (410).

Description

Stage device and stage facility

The present invention relates to a stage apparatus and a stage facility used for performance, presentation, and the like, and more particularly to a stage apparatus and a stage facility in which a display unit for displaying an image on a stage is installed.

Patent Document 1 describes that a display unit (screen) for displaying a video on a stage where performance such as a play is performed is displayed on the display unit so as to display a video having performer performance and interactiveness. .

In recent years, real performers (dancers, performers, etc.) perform on stage and display animations (video objects) of characters such as animations generated by rendering virtual objects on a display unit installed on the stage. There are also known stage apparatuses or stage facilities that can perform an effect as if a video object is performing with a real performer.

Special table 2003-533235 gazette

When displaying a video object on a display unit installed on the stage, especially when performing an effect as if the video object is performing with a performer, whether the video object actually exists in the space on the stage It is desirable to have such a sense of reality, and for that purpose, various improvements have been made in terms of improving the image quality and facilitating the movement of the character. Improvement is demanded.

The present invention has been made in the above situation, and in a stage apparatus and stage equipment for displaying a video object on a display unit installed on a stage, the realism of the video object and / or video quality is improved. Objective.

The present invention includes a stage, illumination capable of irradiating the stage with light, illumination control means for controlling the illumination, a display unit for displaying a video object installed on the stage, and the video object Is a stage apparatus or stage facility provided with video data generation means for generating video data for displaying on the display unit, wherein the video data generation means is synchronized with the illumination control by the illumination control means, A stage apparatus or stage facility for adjusting the color of the video object.

“Stage” in this specification means a place or space where performances and presentations by real people and things are performed. The performance, presentation type, and content are arbitrary. Examples of performances and presentations include human dances, performances, singing, plays, etc. and display of goods.

In this specification, “illumination” means a member, equipment, device or the like that can irradiate the stage with light. For “illumination”, a halogen lamp, a xenon lamp, an LED illumination, or the like may be used. The number and arrangement of “lights” are arbitrary. “Illumination” includes spotlights, ceiling lights, footlights, front lights, moving lights, and the like depending on their roles, functions, and positional relationship with the stage.

“Illumination control means” in this specification is means for controlling illumination (irradiation of light to the stage). Examples of “control” of lighting include turning on / off (lighting / extinguishing) lighting, changing light intensity, changing lighting color, lighting position and light irradiation direction (directivity) Is included. “Control” may be performed manually or automatically. In the case of manual control, the illumination control means can be configured by a console (operation panel) operated by an operator. The console may have operation members such as buttons, switches, and volumes for the operator to input operations. When the computer automatically controls lighting according to a preset program, the computer can constitute lighting control means.

“Display section” in this specification means a portion, member, device, system, or the like on which an image is displayed. The display unit may be a CRT, LCD, plasma display, LED display or screen thereof, a screen for projecting a projector image, or the like.

“Virtual object” in this specification means shape data in a virtual space. A “virtual object” can be defined by a plurality of polygons. The shape of the “virtual object” can be three-dimensional. A “virtual object” can have the shape of a person, animal, fictional character, or other object.

“Video object” in this specification refers to an object such as a person or an object displayed as a video. A “video object” may be generated by rendering a “virtual object”.

“Color” in this specification includes brightness.

In this specification, the “stage facility” may be an indoor facility having an outer wall, a ceiling, or the like, or an outdoor facility installed in a space such as a stadium.

In the present invention, since the color of the video object is adjusted in synchronization with the lighting control by the lighting control means, the effect corresponding to the lighting effect such as a shadow generated on a real person or object on the stage by the lighting is displayed. Thus, it is possible to increase the realism of the video object and / or to improve the quality as an image.

The adjustment of the color of the video object in the present invention is as follows.
(1) The right side of the video object is brightened in synchronism with the control of irradiating the stage with illumination light from the right direction (this produces an effect such that the right side of the person or object on the stage is bright and the left side is dark). The left side is darkened (2) The color (brightness) of the video object is brightened in synchronization with the control that irradiates the stage with strong illumination light (this produces an effect such that a person or object on the stage looks bright). (3) To irradiate the stage with colored light such as red or blue (this causes the person or object on the stage to have a red or blue effect on the part where the illumination light hits) In synchronization, the corresponding part of the video object can be changed to a color corresponding to the illumination light.

In the present invention, the video data generation unit includes a virtual space setting unit that arranges a virtual object and a virtual light source in a virtual space, and a color adjustment unit that adjusts the color of the virtual object by reflecting the influence of the virtual light source. It is preferable to have a rendering means for generating the video data by rendering the virtual object, and this makes it easy to adjust the color of the video object in real time in synchronization with the lighting control by the lighting control means. It becomes.

In the present invention, the virtual space setting means arranges the virtual object and the virtual light source in the virtual space with a positional relationship corresponding to the positional relationship between the display unit and the illumination, and / or the virtual light source. The number, intensity, color and / or direction preferably corresponds to the number, intensity, color and / or direction of the illumination, so that the color of the video object can be faithfully adjusted by controlling the illumination by the illumination control means. Can be done.

In the present invention, there is further provided main motion data generating means for generating main motion data by detecting a measurement subject's motion, wherein the video data generating means is configured to detect the video object based on the main motion data. It is preferable to generate video data that operates simultaneously with the aforementioned operation. In this invention, it becomes possible to operate the video object in an appropriate manner according to the situation at the time (for example, the progress of the stage or the situation of the audience seat).

In the present invention, it further includes a controller that can be operated by the person being measured while performing the operation, and a sub-motion data generating unit that generates sub-motion data in accordance with an operation on the controller. It is preferable to change the shape and / or pattern of all or part of the video object based on motion data. In this invention, the person to be measured inputs sub-motion data for changing the shape and / or pattern of all or part of the video object by operating the controller while performing the operation for generating the main motion data. It is possible. Therefore, for example, the video object has a shape of a person or the like, and the shape of the video object's hand (hand gesture, etc.) or the shape / pattern of the head (expression, hair shape, etc.) is changed by sub-motion data. In this case, it becomes easy to change the gesture or expression of the hand of the video object at an appropriate timing according to the motion of the video object.

In the present invention, preset motion data storage means for storing preset motion data defining the operation of the video object, and switching means for switching the video data generation means between the first state and the second state are further provided. The video data generating means generates video data for operating the video object based on the main motion data in the first state, and based on the preset motion data in the second state. It is preferable to generate video data on which the video object operates.

In this invention, a state (first state) in which video data for operating the video object is generated based on the main motion data, and a state (first state) in which video data for operating the video object is generated based on the preset motion data. Switching between two states). Therefore, even if the main motion data cannot be generated properly due to, for example, a device failure or the movement of the measurement subject to a place where the motion cannot be detected, the first state to the second state By switching to, it is possible to prevent inconvenience such as an unnatural or unintentional motion of the video object, or by switching to the second state according to the scene, It is possible to cause the video object to perform an abrupt motion or the like that cannot be realized by the main motion data based on the motion.

Explanatory drawing which shows schematic arrangement | positioning of the stage facility 100 which has the stage apparatus 10 which concerns on one Embodiment of this invention. Explanatory drawing which shows the principal part of the stage apparatus 10 from the audience area 240 side. FIG. 3 is an explanatory diagram showing a configuration of an exemplary motion capture system 310. 3 is an explanatory diagram illustrating a functional configuration of the video processing device 450. FIG. FIG. 5 is an explanatory diagram illustrating an exemplary virtual space 600 set by a virtual space setting unit 530. FIG. 4 is an explanatory diagram showing a state in which a video object 230 is displayed on a display unit 211 installed on a stage 210. FIG. 5 is an explanatory diagram showing a video object 230 displayed on the display unit 211 when switching is performed by a data switching unit 570.

Hereinafter, the stage apparatus 10 and the stage facility 100 according to a preferred embodiment will be described based on the drawings.

FIG. 1 is an explanatory diagram showing a schematic arrangement of a stage facility 100 having a stage apparatus 10 according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing the main part of the stage apparatus 10 from the audience area 240 side.

As illustrated, the stage facility 100 includes a venue 200, a motion capture room 300, and an operation room 400.

The venue 200 has a stage 210 and a spectator area 240.

Stage 210 is a place where performers 220 can perform performances and present things. The shape of the stage 210 is arbitrary, and may have a flat surface, a curved surface, or a three-dimensional shape.

The display unit 211 is installed on the stage 210. The display unit 211 is an arbitrary part, member, apparatus, or system capable of displaying a video such as the video object 230. In this embodiment, a screen that displays a video from the projector 212 is used as the display unit 211. Is done. The shape of the display unit 211 is arbitrary, and may have a flat surface, a curved surface, or a three-dimensional shape.

The projector 212 is disposed at an arbitrary position where an image can be projected on the display unit 211. As shown in the figure, when the projector 212 is arranged on the back side of the display unit 211, it is possible to prevent the image of the projector 212 from being blocked by a performer 220 performing on the front side of the display unit 211. . In this case, a rear projection screen made of a transparent sheet-like member or the like can be used as the display unit 211.

In the figure, a single display unit 211 is installed at the approximate center of the stage 210, but the arrangement and number of the display units 211 on the stage 210 are arbitrary. For example, a display unit made of a three-dimensional model that displays an image by projection mapping on both sides of the display unit 211 shown in the figure, and / or a display unit made of LED vision or the like behind the display unit 211 By arranging a plurality of display units so as to surround the stage 210 such as arranging them, it is possible to perform an effect in which the performance on the stage 210 and the presentation and the video on the display unit are integrated.

The stage apparatus 10 to the stage facility 100 have

illuminations

213L and 213R for irradiating the stage 210 with light. In the figure, one

illumination

213L and 213R is shown near the left and right ceilings of the stage 210, respectively, but the illumination can be arranged at any other position where the stage 210 can be irradiated with light. The number of lights may be one or three or more.

In the preferred embodiment, the stage apparatus 10 to the stage facility 100 further include stage equipment such as one or a plurality of speakers 214 that output sound such as music, a smoke machine 215 for performing a SFX-like stage, and a laser device 216. Can do.

The audience area 240 is a place for the audience to see the performance performed at the stage 210. The audience area 240 may have one or more seats 241 on which the audience can sit.

The stage apparatus 10 to the stage facility 100 may further include a photographing unit 217 for photographing the audience area 240. The photographing means 217 is arranged at an appropriate position such as near the ceiling at the center of the stage 210 so that the entire audience area 240 or a specific part can be appropriately photographed.

In the motion capture room 300, a motion capture (MC) system 310 for generating main motion data by detecting (capturing) an action such as a dance performed by a measurement subject (motion capture actor) 320 is arranged. The operation of the measurement subject 320 can be detected by an arbitrary method such as an optical method, a mechanical method, or a magnetic method.

FIG. 3 is an explanatory diagram showing a configuration of an exemplary motion capture system 310.

As shown in the figure, the motion capture system 310 includes a plurality of markers 311 attached to appropriate positions such as a limb, a head, and a waist of the measured person 320, and a predetermined motion area (an area where the measured person 320 operates). One or a plurality of sensors 313 capable of detecting markers 311 arranged around 312 (for example, up, down, front, back, left, and right) are provided.

The sensor 313 is connected to the processing device 314 wirelessly or by wire, and the processing device 314 calculates the three-dimensional position of each marker 311 based on the data from the sensor 313, thereby generating main motion data. .

The main motion data is data for determining the position and posture of the virtual object 610 (see FIG. 5) to be arranged in the virtual space 600 in accordance with the operation of the person 320 to be measured. Through the processing, the position of the reference point of the virtual object (for example, the origin of the object coordinate system fixed to the virtual object 610) and the parts (head 611, hand 612, foot of the virtual object 610 with respect to the reference point) Data of the relative position of the unit 613) is generated.

The processing device 314 can be composed of a computer having a CPU and a storage device, and the sensor 313 can be composed of photographing means such as a camera. The number and arrangement of the sensors 313 are appropriately set in consideration of the accuracy of position detection of the marker 311 and the like.

The motion capture system 310 can have

controllers

315L and 315R operated by the person 320 to be measured as means for generating sub motion data. The specifications, number, etc. of the

controllers

315L, 315R are arbitrary, but in this embodiment, the left and right

handy controllers

315L, 315R that can be gripped / operated with one hand are used, and the person 320 to be measured can It is possible to input sub motion data from the

handy controllers

315L and 315R while performing the operation. The

handy controllers

315L and 315R have operation members such as buttons and joysticks. The

handy controllers

315L and 315R are preferably connected to the processing device 314 wirelessly.

The sub motion data is data for determining the shape and / or pattern of the whole or a part of the virtual object 610. In the present embodiment, the data input by operating the buttons of the

handy controllers

315L and 315R is It is used as hand gesture data, and the data input by operating the joystick is used as facial expression and hair shape data.

In a preferred embodiment, a display device 330 that displays live video of the spectator area 240 photographed by the photographing means 217 is disposed in front of the motion area 312. In this case, the person 320 to be measured can operate while viewing the video on the display device 330 or can operate the

handy controllers

315L and 315R. As a result, the person 320 to be measured can input the main motion data and / or the sub-motion data according to the real-time situation of the spectator area 240. For example, the video object 230 interacts with a specific spectator. It becomes possible to generate video data of the video object 230 that operates according to the situation of the audience area 240.

In the following, the main motion data and sub-motion data generated by the motion capture system 310 are collectively referred to as “real-time motion data”.

In the operation room 400, there are devices for the operator 460 to control various devices installed in the stage facility 100 such as a lighting console 410, a video console 420, a sound console 430, an SFX console 440, and a video processing device 450. Be placed.

The lighting console 410 is for controlling the

lighting

213L and 213R. When the operator 460 performs a predetermined operation on the lighting console 410, the

lighting

213L and 213R are individually turned on / off, and the intensity ( It is possible to perform control such as changing brightness) and / or color. In the present embodiment, a control signal S1 based on the operation of the operator 460 is transmitted from the lighting console 410 to the

lights

213L and 213R, and the

lights

213L and 213R are controlled based on the control signal S1. When the lighting consoles 213L and 213R are controlled in the lighting console 410, a notification signal S2 for notifying the control or the contents of the control is transmitted to the video generation device 450. The notification signal S2 may be the same signal as the control signal S1 or a separate signal, and a signal obtained by branching the control signal S1 can be used as the notification signal S2.

The video console 420 is for performing control related to video data generated by the video generation device 450. In the present embodiment, the operator 460 performs a predetermined operation on the video console 420, thereby realizing real-time motion data. It is possible to switch between a first state in which video data for operating the video object 230 is generated based on the video and a second state in which video data for operating the video object 230 is generated based on the preset motion data. It is. When the above-described control is performed in the video console 420, a notification signal S3 notifying that the control has been performed or the contents of the control is transmitted to the video generation device 450. The preset motion data will be described later.

The sound console 430 controls the sound output from the speaker 214, and the sound source data stored in an appropriate medium is reproduced from the speaker 214 at the timing, volume, etc. designated by the sound console 430.

SFX-related production equipment such as the smoke machine 215 and the laser device 216 is controlled by operating the SFX console 440.

The video processing device 450 is connected to the lighting console 410, the video console 420, and the motion capture system 310, and generates video data to be displayed on the display unit 211 based on the notification signals S2, S3 and data received from these. The processing is performed. The video processing device 450 can be configured by a CPU and a computer including a storage device that stores various programs and data necessary for generating video data. The storage device can be configured by a known storage device such as a ROM, a RAM, a hard disk, or a flash memory, alone or in combination.

The stage apparatus 10 of this embodiment includes a stage 210,

illuminations

213L and 213R, an illumination console 410, a video processing apparatus 450, and the like.

FIG. 4 is an explanatory diagram showing a functional configuration of the video processing device 450.

As illustrated, the video processing device 450 includes an object storage unit 510, a texture storage unit 520, a virtual space setting unit 530, a color adjustment unit 540, a rendering unit 550, a preset motion data storage unit 560, a data switching unit 570, and the like. A video data generation unit 500 is included.

The object storage unit 510 stores object data that defines one or a plurality of virtual objects 610 to be arranged in the virtual space. In the present embodiment, object data about a virtual object 610 having a shape such as a person or a character is stored. It is remembered. The format for defining the virtual object 610 is arbitrary. For example, the virtual object 610 can be defined by the position coordinates of polygons constituting the virtual object 610 in the local coordinate system fixed to the virtual object 610. The posture of the virtual object 610 can be changed by changing the arrangement of the parts (the head 611, the hand 612, the foot 613, etc.) constituting the virtual object 610 in the local coordinate system. In order to make it possible to change the gesture, facial expression, hair shape, and the like of the virtual object 610, the object storage unit 510 stores a plurality of types of shapes for the head 611 and the hand 612.

The texture storage unit 520 stores a texture for mapping to the virtual object 610. The texture is image data drawn in the form of a bitmap or the like for defining the pattern, color, transparency, etc. of the virtual object 610. By mapping the texture on the surface of the virtual object 610, a facial expression is displayed on the virtual object 610. It is possible to give details such as clothes and clothes. In order to make it possible to change the facial expression or the like of the virtual object 610, the texture storage unit 520 stores a plurality of types of textures for the head 611 and the like.

The virtual space setting unit 530 executes processing for arranging the virtual object 610, the virtual

light sources

620L and 620R, and the virtual viewpoint in the virtual space 600 in the manner illustrated in FIG.

Here, the virtual object 610 is arranged in the virtual space at the position and the posture determined by the main motion data supplied from the motion capture system 310 by streaming (in real time). That is, the position coordinates of the reference point of the virtual object 610 in the virtual space 600 and the arrangement of parts such as the head 611, the hand 612, and the foot 613 with respect to the reference point are determined by the real-time main motion data. As a result, the motion of the measured person 320 is reflected in real time on the motion of the virtual object 610, and the virtual object 610 performs the same or corresponding action as the measured person 320 simultaneously with the motion of the measured person 320. Will do.

Further, object data and texture data having a shape / pattern corresponding to the sub motion data input from the

handy controllers

315L and 315R are applied to the head 611 and the hand 612 of the virtual object 610. Thereby, the expression of the head 611 and / or the shape of the hair can be changed variously as shown in the frame 621, and the shape (hand gesture) of the hand portion 612 can be changed variously as shown in the frame 622. It becomes.

The virtual viewpoint is set at an appropriate position and direction in the virtual space 600. The position and direction of the virtual viewpoint may be fixed or movable, but here, a description will be given assuming that the virtual viewpoint oriented in the direction of the paper surface is set at a predetermined position on the front side of the paper surface in FIG.

The arrangement and / or number of the virtual

light sources

620L and 620R can be determined according to the control of the

lights

213L and 213R performed in the lighting console 410. For example, when the lighting console 410 is controlled to turn on the right side illumination 213R of the stage 210 as viewed from the audience area 240 and turn off the left side illumination 213L, the right side of the virtual object 610 viewed from the virtual viewpoint. If the control is performed so that both the left and

right illuminations

213L and 213R are turned on, the virtual

light sources

620L and 620R can be arranged on both the left and right sides of the virtual object 610.

When the control of changing the color, intensity, directivity, position, etc. of the

lights

213L, 213R is possible by the operation on the lighting console 410, the colors of the virtual

light sources

620L, 620R corresponding to these controls, Parameters such as intensity, directivity, and position can be controlled.

The placement, movement, parameter control, and the like of the virtual light sources 620 and 620R as described above are performed in synchronization with the control of the

lights

213L and 213R in the lighting console 410. Synchronization is automatically performed based on the notification signal S2 output from the lighting console 410.

The color adjustment unit 540 adjusts the color of the virtual object 610 by reflecting the influence of the virtual

light sources

620L and 620R (virtual light rays from the virtual

light sources

620L and 620R). Specifically, a process such as causing a shadow 614 by the virtual

light sources

620L and 620R in the virtual object 610 or changing the color of the virtual object 610 corresponding to the colors of the virtual

light sources

620L and 620R is performed. In such color adjustment, a known method such as von shading or glow shading can be used.

The rendering unit 550 renders an image of the video object 230 displayed on the display unit 211 by rendering an image when the virtual object 610 whose color has been adjusted by the color adjustment unit 540 is viewed from a virtual viewpoint at every predetermined frame time. Video data is generated. Rendering can be performed using a known method such as projection conversion, rasterization, and hidden surface removal. The video data generated by the rendering unit 550 is output to the projector 212, whereby the video object 230 as a moving image is displayed on the display unit 211.

FIG. 6 is an explanatory diagram showing a state where the video object 230 rendered by the rendering unit 550 is displayed on the display unit 211 installed on the stage 210.

As shown in the example, when the lighting console 410 is operated to turn off the left side lighting 213L and turn on only the right side lighting 213R, the performer 220 on the stage 210 has the lighting 213R. As a result of adjusting the color of the virtual object 610 by reflecting the influence of the virtual light source 620R arranged on the right side of the virtual object 610 in synchronization with the above control, the effect of the shadow 221 and the like by the light from An effect such as a shadow 231 corresponding to the video object 230 is also produced. In this way, the effect is as if the video object 230 is illuminated by the

actual illumination

213L, 213R (the effect as if the performer 220 and the video object 230 are illuminated by the

common illumination

213L, 213R). By causing the object 230 to occur, the realism of the video object 230 can be enhanced.

Referring back to FIG. 4, the preset motion data storage unit 560 stores one or more types of preset motion data. The preset motion data defines the motion of the virtual object 610 over a predetermined time. In this embodiment, the position and posture of the virtual object 610 in the virtual space 600, and the shapes / patterns of the head 611 and the hand 612 ( Data that defines the hand gesture, facial expression, hair shape, etc.) in time series is used.

In response to an operation performed on the video console 420, the data switching unit 570 includes a first state in which the video generation unit 500 generates video data of the video object 230 based on real-time motion data, and preset motion data. Based on this, the second state of generating video data of the video object 230 is switched. In the first state, the data switching unit 570 receives the real-time motion data from the motion capture system 310 in the virtual space setting unit 550. In the second state, the preset motion data stored in the preset motion data storage unit 560 is supplied to the virtual space setting unit 550. The processes executed by the virtual space setting unit 530, the color adjustment unit 540, and the rendering unit 550 in the first state are as described above. In the second state, the placement of the virtual object 610 in the virtual space 600 by the virtual space setting unit 530 is performed based on the preset motion data in the preset motion data storage unit 560, except for this point, the virtual space setting unit The color adjustment unit 540, the color adjustment unit 540, and the rendering unit 550 execute processing such as the placement of the virtual

light sources

620L and 620R, color adjustment, and rendering in the same manner as in the first state, and thereby based on the preset motion data. The video data of the video object 230 that operates in this manner is output to the projector 212.

FIG. 7 is an explanatory diagram showing a video object 230 that is rendered when switching is performed by the data switching unit 570, and arrows at the bottom of the figure indicate the passage of time.

In this example, the period from the time point T0 to T1 is the first state, and the video objects 2301 and 2302 that operate based on the real-time motion data are rendered, and the data switching unit 570 changes to the second state at the time point T1. Thereafter, the video objects 2303 to 2305 that operate based on the preset motion data are rendered until the time T3, and the data switching unit 570 switches to the first state at the time T3. Thereafter,

video objects

2306 and 2307 operating based on real-time motion data are rendered.

Before and after the switching in order to seamlessly switch between the first state and the second state, that is, to switch between the first state and the second state without causing discontinuity in the operation of the video object 230. It is possible to perform data interpolation between real-time motion data and preset motion data. For example, when switching from the first state to the second state at the time T1, the real-time motion data used for rendering the video object 2302 at the switching time T1 and the rendering of the video object 2303 immediately after the switching are used. Data interpolation is performed between the preset motion data, and a video object that operates based on the motion data generated by this data interpolation is a period from time T1 to time T2 when the first video object 2303 based on the preset motion data is rendered. 230 can be rendered.

As described above, the present invention has been described based on the exemplary embodiment. However, the present invention is executed in the stage apparatus and the stage facility or the members, apparatuses, systems, functional configurations, parameters, the stage apparatus and the stage facility in the above embodiment. The contents of the processing are described only as examples, and these can be arbitrarily changed within the scope of the claims.

For example, in the above embodiment, the case where the control of the

lighting

213L and 213R and the switching between the first state and the second state is performed based on the operation to the lighting console 410 and the video console 420 has been described. It is also possible to automatically control the

lighting

213L, 213R and / or switch between the first state and the second state according to a pre-programmed sequence using the means configured.

Moreover, although the said embodiment demonstrated the case where control of

illumination

213L, 213R was performed via control signal S1 output from the illumination control means (illumination console 410), a control signal is used for control of

illumination

213L, 213R. Whether or not to use is arbitrary. For example, the lighting control means controls the

lighting

213L and 213R without using a control signal such as turning on / off the power supply to the

lighting

213L and 213R or adjusting the power supply amount by switch operation or volume adjustment. In this case as well, the control of the

illuminations

213L and 213R is performed, or the content of the control is notified to the video generation means (video processing device 450, video processing data generation unit 500). Thus, the color of the video object can be adjusted in synchronization with the illumination control in the same manner as in the above embodiment.

DESCRIPTION OF SYMBOLS 10 ... Stage apparatus 100 ... Stage facility 210 ... Stage 211 ... Display unit 212 ...

Projector

213L, 213R ... Illumination 214 ... Speaker 217 ... Imaging means 230 ... Video object 310 ... motion capture system 400 ... operation room 410 ... lighting console 420 ... video console 450 ... video processing device 500 ... video data generation unit 510 ... object storage unit 520 ... Texture storage unit 530 ... Virtual space setting unit 540 ... Color adjustment unit 550 ... Rendering unit 550 ... Virtual space setting unit 560 ... Preset motion data storage unit 570 ... Data switching Part 600 ... virtual space 610 ... virtual object 620L, 20R ··· virtual light source

Claims

Stage,
Illumination capable of irradiating the stage with light;
Illumination control means for controlling the illumination;
A display unit for displaying a video object installed on the stage;
A stage apparatus comprising video data generating means for generating video data for displaying the video object on the display unit,
The stage apparatus characterized in that the video data generation means adjusts the color of the video object in synchronization with the illumination control by the illumination control means.
The video data generation means is
Virtual space setting means for arranging a virtual object and a virtual light source in the virtual space;
Color adjustment means for adjusting the color of the virtual object by reflecting the influence of the virtual light source;
The stage apparatus according to claim 1, further comprising: a rendering unit that generates the video data by rendering the virtual object.
3. The stage apparatus according to claim 2, wherein the virtual space setting unit arranges the virtual object and the virtual light source in the virtual space with a positional relationship corresponding to a positional relationship between the display unit and the illumination.
4. The stage apparatus according to claim 2, wherein the number, intensity, color, and / or direction of the virtual light source corresponds to the number, intensity, color, and / or direction of the illumination.
Further comprising main motion data generating means for generating main motion data by detecting the measurement subject's movement;
5. The video data generating means generates video data in which the video object operates simultaneously with the movement of the measurement subject based on the main motion data. The stage apparatus according to item.
A controller operable by the person being measured while performing the operation;
Sub-motion data generation means for generating sub-motion data according to the operation to the controller,
6. The stage apparatus according to claim 5, wherein the video data generating unit changes the shape and / or pattern of all or part of the video object based on the sub-motion data.
Preset motion data storage means for storing preset motion data defining the operation of the video object;
A switching means for switching the video data generation means between the first state and the second state;
The video data generation means includes
In the first state, video data for operating the video object is generated based on the main motion data, and in the second state, video data for operating the video object is generated based on the preset motion data. The stage apparatus according to claim 5 or 6, wherein
Stage,
Illumination capable of irradiating the stage with light;
Illumination control means for controlling the illumination;
A display unit for displaying a video object installed on the stage;
A stage facility comprising video data generating means for generating video data for displaying the video object on the display unit,
The stage facility characterized in that the video data generation means adjusts the color of the video object in synchronization with the control of the lighting by the lighting control means.