KR20030031956A - Interactive virtual reality performance theater entertainment system - Google Patents

Abstract

An interactive virtual reality (VR) execution theater and environment system 100 is provided that enjoys a plurality of participants and / or educational experiences. Through VR displays, such as head mounted devices (HMDs), participants watch performances and see both immersive graphical environments and live and / or pre-stored actors. The participant may have limited control over the content and results of the action by handheld input device 16 with a plurality of buttons 80 and / or voice communication while viewing the actors mixed within the immersive VR environment. Interconnected computer 102, video and audio processing devices 106 and 108 are used. The network 104 connects computers, participants and live / prestored actors to each other to transmit video, audio and graphics.

Description

Theater-type entertainment system running interactive virtual reality {INTERACTIVE VIRTUAL REALITY PERFORMANCE THEATER ENTERTAINMENT SYSTEM}

Virtual reality entertainment systems utilizing participants' interactive display and immersion devices or methods while viewing and interacting with the virtual reality environment created by the software have been used for training and entertainment purposes for many years. Immersive virtual reality entertainment, as used and described herein, is a computer in which a participant is “immersive” within the environment to provide the participant with a sense of physical location within the graphical environment, although the participant exists with other objects in the environment. Refers to the generated graphics environment. Thus, immersive virtual reality creates illusions in the participant in an artificially formed environment using three-dimensional (3D) graphics and computer software that mimic the relationship between the participant and the surrounding environment.

Typically, participants are represented in a software environment by projecting shapes that appear in a work such as an avatar. Participants control their avatar by combining input devices, including but not limited to data generated from electronic hand held input devices and electronic and electromagnetic tracking devices that move around the monitor. Objects in virtual reality that are not controlled by the participant's movement are controlled by a computer software program and respond to the participant's input.

In the virtual reality environment of the related art, shapes or objects that are not controlled by the participant move automatically only in accordance with the software program and respond to each participant's input device (Carmein, U.S. Patent No. 5,490,784, Latipov) Latypov, US Pat. No. 5,846,134. Passive and active objects such as plants and animals move in some way within the virtual reality environment. In addition, participants can interact with such objects in the virtual reality environment, but the objects are limited to computer-generated images.

In general, an object or focus in a virtual reality environment typically interacts with an object that is created solely by a computer graphics program that does not represent or substantially control human participants in the simulation. In addition, the participant group may be a graphically enhanced live or pre-stored actor on stage or on television, film or other non-immersive media, without an immersive device or method, and further, without a voice communication from the participant to the actor and the communication between the participants. (Trumbull et al., US Pat. No. 5,795,228).

Other known entertainment systems, such as US Pat. No. 5,682,196 to Freeman et al., Are capable of two-way communication between computer controlled hosts and participants. However, such a system is not capable of voice communication between participants or between participants and a live host. Trumbull's system can make one-way voice communication from live actor to participant but does not provide voice communication from participant to live actor. Trumbull's system is also capable of voice communication between participants and is not further enhanced by the immersive virtual reality environment.

Some conventional virtual reality systems have developed rudimentary sensory interfaces. For example, US Pat. No. 5,737,505 to shaw et al. Describes an interface similar to upper armor that detects limb and torso position. The device may provide force feedback or share movement to a participant in conversation with the virtual reality environment. However, virtual reality systems such as shows are not only able to provide communication between participants, nor do they include live and / or pre-stored actors.

Thus, known entertainment systems lack some advantageous features. In conventional systems, the participant can use some input device to talk to the host and the environment. However, the host is rarely live and instead is pre-stored or computer controlled and thus lacks spontaneity in the participant's conversation. Trumbull's host is a live actor, but the system itself is not in a virtual reality environment, so the immersion effect shared with participants for a more interesting experience. In addition, there is no voice communication from participant to actor and no voice communication between participants further limiting the entertainment or educational experience.

Thus, there is a need for virtual reality based entertainment and / or education that provides 1) participant 2) immersive environment and 3) immersive interactive communication between live and / or pre-stored actors. There is also a need for a system that includes facilities for individual and / or combined video, audio and graphical input and output to enhance the overall entertainment and / or educational experience. In addition, in response to scripted and / or improvised communication from actors and / or virtual reality environments, there is a need for output of content and experiences controlled at least in part by input from participants. . Finally, there is a need for a robust and modular system that is designed to accommodate rapidly changing and changing computer technologies for multiple participants, actors, and virtual reality environments input and output. The synergistic combination of these factors can provide participants with what they have never felt about entertainment and / or experience excitement and quality.

Cross Reference of Related Application

This application is based on US patent application Ser. No. 09 / 617,388, entitled "Interactive Virtual Reality Performance Theater Entertainment System," filed July 17, 2000, claiming the benefit of priority.

The present invention relates to the field of electronic imaging and live entertainment. In particular, the present invention relates to an entertainment and educational system in which a participant interacts with an electronic or computerized environment in addition to a live and / or pre-recorded performer. In addition, the present invention utilizes immersive interactions with the virtual reality environment and live actors using various immersive and input devices such as head mounted displays and hand-held keypads for participants. (immersive) virtual reality computer system.

The foregoing and other features and objects of the present invention and methods for achieving the same will become more apparent and will be better understood by reference to the following embodiments of the present invention in conjunction with the accompanying drawings.

1 is a block diagram illustrating communication between a participant, an actor, and an immersive virtual reality environment.

Fig. 2 is a perspective view of a computer display showing the definition of "immersion" as defined in the present invention.

3 is a perspective view of a virtual reality performance theater according to the present invention.

4 is a perspective view of a presenter enclosure according to the present invention.

5 is a front view of the same video display as the participant sees in accordance with the present invention.

6 is a plan view of a keypad that is a handheld input device according to the present invention.

7 is a block diagram of system components in accordance with the present invention.

8 is a software flow diagram in accordance with the present invention.

9 is a network diagram in accordance with the present invention.

10 is a plan view of another embodiment of a virtual reality execution theater representing a theater with sixteen chairs.

11 is a top view of another embodiment of a virtual reality execution theater representing a theater with eight chairs.

Corresponding reference characters indicate corresponding parts in some drawings. Although the drawings show embodiments of the present invention, the drawings are enlarged or reduced in order to better illustrate the present invention.

According to the virtual reality performance theater (VRPT) of the present invention, the inventors have the participants, live and / or pre-stored actors and immersions from the interaction between individual participants and computer-generated graphic objects in the virtual reality environment. Transferred the perspective of software programs and entertainment or educational performance to the virtual reality environment. Inclusion of three-way interactive communication between participants, live and / or pre-stored actors, and immersive virtual reality environments results in synergistic effects that create unparalleled entertainment and / or educational environments.

Accordingly, it is a primary object of the present invention to provide an interactive interactive virtual reality performance theater where three-way communication between participants, live and / or preset actors, and the virtual reality environment exists. Three-way communication includes, but is not limited to, audio, video, and graphics input and output data. The interaction between the participant and the actors and the virtual reality environment determines to some extent the end result of the content and action.

Accordingly, another object of the present invention is to provide an interactive virtual reality environment in which a plurality of participants immerse themselves in a software generated simulation with live and / or pre-stored actors.

Another object of the present invention is to allow the participant to control the outcome of the action.

Another object of the present invention is to provide a new immersive performance in which each participant and a plurality of actors are interconnected by voice communication with a voice conversation system. The system should be built to allow participants to have a constant conversation, which enhances interactivity by allowing them to share options and work together in entertainment games or educational settings.

It is another object of the present invention to provide a virtual reality environment that can easily adapt to the needs of a given application and can be easily upgraded to keep pace with rapidly changing computer technologies.

In summary, the present invention provides a virtual reality performance platform and environment that can interact with each other, between participants and live and / or pre-stored actors, and between the participants and the virtual reality environment for a more enjoyable entertainment and / or educational experience. will be. It is important for the participant to have limited control over the content and the results of the actors through voice communication with the input device.

The system further includes a program that normally uses input and output devices for each of the participants and actors, and which performs a predetermined processing task with a computer having system data. The system further includes various audio and visual elements that mix and switch audio and visual data for presentation to the participant. Through innovative mixing and switching techniques, the system can show each participant a graphical virtual reality environment in which the live actor's image is added into the graphical environment. Accordingly, participants see and communicate new display devices relating to both graphical data and live image data for interesting entertainment and / or educational environments.

With the aid of image mixing and replacement, the virtual reality presentation consists of chromakeys and other electronically generated costings, with the image and / or audio input from the actor (s) being presented by the participant, for example. For example, it is mixed with an immersive graphic environment using a head mounted display device (HMD). Thus, the participant experiences an immersive graphical environment in which the actor can be seen and heard.

Participants can view images or electronic replicas of actors / actors, software simulations, and graphical output of other visual, audio, and virtual reality environments, using handheld input devices such as keypads or cyberloves to interact with content. Control the outcome of the actor. The actor asks the audience for a response based on a specific question or opinion on, for example, a scripted and / or improvised act. The participant's response may indicate additional content and continue to indicate the outcome of the actor, for example, declaring a "winner" based on the scores earned in response to actor and / or system queries. .

The system further includes audio and motion devices, including, but not limited to, speakers and motion-equipped seats for each participant to enhance the immersive experience. The participant's immersion experience is supplemented by the auditory and sixth sense of a chair (hereinafter referred to as a "rumble seat") where each participant sits and moves throughout the performance. The chair includes a speaker for a background sound effect, the movement effect being generated by the software in response to the participant's voice communication, decision and physical movement.

In addition, a network is included to connect computers for data transmission and to facilitate remote interconnection and participation. With the implementation of the network, participants and actors can be represented physically from geographical locations while experiencing the same virtual reality environment and actors.

Thus, the present invention adds immersive virtual reality technology and electronic interaction to live and / or action by actors. By placing the participant in the virtual reality environment, the participant and the actor can interact more directly with each other while feeling physically close and immersive virtual reality environment.

As described in more detail below, an immersive environment is defined in which more than 25 degree diagonal fields of view are provided within 10 feet of the viewer. To achieve an immersive environment, VRPT can use an HMD device that creates an immersive feeling in a 3D graphics environment. Also, bringing the display closer to a headless display device, such as an eye-level monitor, can create a similar immersive environment. Still other immersion devices and methods are described in detail below.

As a result, the entire system can be classified as a fusion or combination of virtual reality, computer graphics and remote video conferencing. The actor need not be located in some area or place where the participant is located. Rather, the actor may be located as far as the communication channel allows and may transmit the act to multiple locations from different places via the Internet, satellite transmissions or other channels.

One advantage of the system according to the invention is that human participants interact more quickly with other participants and actors than with previous forms of communication. Participants have a much more "personal and unique" experience through the use of immersive devices and interactive communication technologies.

Another advantage of the present invention is that VRPT is used as a medium for visual projection between individuals, as well as entertainment, such as stand-up comedy, drama presentations, teaching and teaching, and other actions related to the communication of ideas and concepts. And any immersive simulation medium, including the business sector.

Moreover, the ability to include multiple actors simultaneously adds a new dimension to the actors. The actors added are added with audio only, voice only or with audio and voice. Although effective, a single actor is only involved in a monologue, and participants in such a system are only allowed to talk. Multiple actors in the electronic simulation media allow these media to be used for live and stored conversations between actors and participants to show actual drama.

Other objects and features are apparent to those skilled in the art and may be particularly pointed out below.

To highlight the novelty of the present invention, FIG. 1 illustrates three-way communication between participants, actors, and immersive virtual reality environments. The VRPT system provides voice and video communication between participants, audio and video communication between participants and actors, voice communication between actors, and graphical, audio and video communication between participants and actors and immersive virtual reality environments. The combination of audio / voice, video and graphical input and output of each participant, actor and immersive environment results in a synergistic effect that results in an unparalleled entertainment and / or educational experience as described in more detail below.

Throughout the following description, a single or multiple participants and actors may be used in a virtual reality performance theater to speak a single or multiple contents unless otherwise indicated.

With respect to the following description of the preferred and other embodiments of the present invention, the present invention defines a "immersive" display device having a viewing angle of 25 degrees or more diagonally within a distance of 10 feet from the viewer. A viewing angle of 25 degrees diagonally at a distance of 10 feet is shown in FIG. 2, and for a 14 inch computer terminal device, it can be called "immersive" when the viewer is 31.6 inches from the display device.

Referring to FIG. 3, a preferred embodiment of the present invention is shown and represented as a virtual reality performance theater (VRPT) 10. The VRPT 10 accommodates a plurality of participants and includes a display device (HMD) 12 mounted on the head of each participant, a microphone 14 and headphones 15 mounted on the HMD 12, and a handheld input device 16. ). The HMD 12 provides each participant with a visual indication of an immersive environment through one or more cathode ray tubes (CRTs), one or more liquid crystal displays (LCDs), or other visual devices. Participants can view the virtual reality environment through the HMD 12 and enter the system through the handheld input device 16 and the microphone 14. In addition, the participant may hear voice and / or audio communications from other participants, actors, and the VRPT system with headphones 15.

The audio output from the VRPT 10 to the participant is provided through a chair 18 equipped with a suitable speaker system or other suitable voice transponder so that it can be called a "rumble chair." The rumble chair 18 further includes a vibration panel or other voice or motion transducer or device that complements the immersive experience in the virtual reality environment and provides output to the participant.

The VRPT 10 further includes a control center 30 for entering the participant's name into the registration computer 32 or allowing the live host to interact with the participant, for example, before the participant sits in the rumble chair 18. . Each participant registers before entering the VRPT 10 through the control center.

Before or during the performance, the participant's physical behavior and audio input is captured by video camera 20 and theater microphone 22. The video images and audio signals captured by the video camera 22 and the theater microphone 22 are communicated to live actors or actors in the presenter enclosure 24 during entertainment and / or educational performances. Output from video camera 20 and theater microphone 22 is further processed by VRPT 10 as described in more detail below.

The interior of the indicator enclosure 24 is shown more clearly in FIG. 4, where the live actor is shown viewing the participant through the video monitor 44 and listening to what the participant is speaking through the speaker 46. The output of video camera 20 and theater microphone 22 are sent to video monitor 44 and speaker 46 located within indicator enclosure 24, respectively. The indicator enclosure 24 is specifically designed to conceal the presence of a live actor so that the participant does not recognize the entity. The indicator enclosure 24 also creates a live image of the actor and a backdrop 48 to facilitate mixing of the actor's live image with the graphic data in the VRPT 10, as described in more detail below. And a stereoscopic image camera 42 for capturing.

Key features of the present invention include mixing video data or images in a virtual reality environment with video input from a live actor, and this combination can be seen by the participant through the HMD 12. As shown in FIG. 4, the stereoscopic video camera 42 captures a live video image of the actor and then transmits a corresponding video signal to a system for processing and mixing graphic data in the virtual reality environment.

The result is a novel virtual reality environment shown and generally represented by an immersive display device 60 as shown in FIG. Participants can view graphical data in the virtual reality environment, as well as view live images of actors added or mixed with the graphical data in the environment. Thus, since the participant communicates with and interacts with the graphical environment 62, the video image 64 of the live actor is periodically added to the environment to allow the participant and the actor to communicate with each other using a communication device as described above. Can be.

All of the video images 64 of the actors who have lived with the graphics environment 62 in the virtual reality environment may be controlled by the computer software or the live actors of the VRPT 10. The immersive display 60 may further include a scoreboard 66 listing each participant's name and scores during the performance indicating success levels in response to questions imposed by the live actor and / or virtual reality environment. . It is clear that the challenge of the participant related to skill, input or luck may also be imposed. The immersive display 60 described herein is for illustrative purposes only and is not limited to the contents of as many graphic data and image data as possible.

As shown in more detail in FIG. 6, hand held input device 16 is used in the preferred embodiment of the present invention to facilitate the user's response. The handheld input device 16 includes a plurality of buttons 46, illustrated with a keypad, that include a label corresponding to a response or input to a question imposed by a live actor and / or a virtual reality environment. For example, the system may impose on the participant a plurality of optional questions with four possible responses, A through D. The participant simply presses the button corresponding to the correct answer and the system tabulates the results entered by the participant. Also, for example, based on the content of the immersive environment, the participant can ask a simple question of "yes" or "no" and the participant can answer the corresponding "yes" or "no" in the handheld input device 16. Press the button. As another example, the participant may press a button to reveal more complex and varied meanings or concepts, such as pressing "TRUST" to indicate that the actor has "trusted" and agreeing with the action suggested by the actor. In another example, the participant may press one or more buttons as quickly as possible as a reflex or memory test to achieve a certain goal or overcome a challenge. The foregoing is only an example of how the participant uses the input device to communicate with the actor and the virtual reality space, and does not limit the type of conversation possible for VRPT.

Processing and communication of input and output of all systems are made clear in the system diagram shown in FIG. The computerized system of the preferred embodiment of the present invention is shown as a VRPT system 100 and is generally represented. Control computer 102 includes audio processor 106, image processor 108, audio switch 110, image switch 112, external audience video 114, external audience audio 115, live The type actor video monitor 116 and the VR display 118 are connected via a network 104. The external audience image 114, the live actor image monitor 116, and the VR display 118 each project a visual display in perspective of the performance of the VRPT system 100 and a virtual environment controlled by software. The VR display 118 further includes a headphone 121 for delivering audio data to the participant. Control computer 102, audio processor 106, image processor 108, audio switch 110, and image switch 112 may be a variety of pre-programmed data in software in addition to input from participants, actors, and the immersive environment of VRPT. To process

Inputs to the VRPT system 100 include participant microphone 119, participant input device 120 (such as the handheld input device shown in FIG. 3), live actor camera 122, live actor microphone 124. , A live actor keypad 126, an auxiliary video camera 128, and an auxiliary video, graphics, and audio source 130. Additional system components of the VRPT system 100 include a chair 132, a power grid 134, and an engineer device 134. Chair 132 represents a plurality of rumble chairs 18 as described above in connection with FIG. 3.

The power grid 134 supplies the amount of power required for each device requiring power within the VRPT system 100. The engineer device 134 is monitored by an "engineer" or "system operator" independent of the actor or host (who enters participant information into the registration computer, receives money, places the participant's seat, or introduces the participant's seat). And electronic components to adjust. Engineer device 134 controls the size of the volume, special effects or other elements of the performance.

The audio processor 106 and the audio switch 110 are connected to the network 104, the live actor microphone 124 and the participant microphone 119, respectively. The audio processor 106 and the audio switch 110 are controlled by the control computer 102 and thus distribute the audio signal throughout the system.

The microphone 119 may perform voice communication with a small microphone embedded in the HMD. The voice signals of the microphones are mixed together using a voice mixing component as described in detail below. The voice signal is then mixed with the voices of other actors, music, special audio effects and pre-stored voice signals generated by other live and software.

The system uses a small condenser built into the HMD and every participant has a unique microphone that detects his or her voice. High speed internet audio transmission may be used in the Internet embodiment of the VRPT system, with voice mixed together on the host side, for example, using a software program.

In another embodiment of the present invention, a player's microphone is switched on and off or the volume is automatically adjusted to enhance team play or to filter out participants who interfere with inappropriate language or live actors. Can be. The switching of the microphones can be computer controlled or manually.

The headphones 102 may be replaced or reinforced with other devices that do not attach to the HMD. Additional voice devices may include desktop speakers, wall mounted speakers, or other types of audio generating devices.

Image processor 108 and image switch 112 are used in VRPT to deliver more graphics to the participants. Various functions of the mixer can be used to provide graphic effects such as chromaking of several sources. Live actors or actors are located, for example, in front of lit blue screens and are chromatographed on graphics from primary or secondary computers or other sources, including other actors or other camera input sources. . In the system of the present invention, a group of participants can interact with an actor in a virtual reality environment and enjoy "immersion" in the environment in which it is formed.

The player input device 120 includes a hand held input device as described above. Participants press a button on the keypad in response to various games such as answering a question, voting, solving a problem, or participating in a challenge.

Another embodiment of the invention may include another player input device 120 in communication with the virtual reality environment. Other embodiments of the player input device 120 may be a keypad, joystick, steering wheel, cyber glove, flying mouse, desktop mouse, trackball or pad. Other types of player input devices that communicate with a computer by various hardware and software means.

In a preferred embodiment of the present invention, the network 104 uses a Windows operating system with a TCP / IP communication protocol and is a 100 base-T network with a multiport hub and a 3Com 905BTX network card. network). Other embodiments use active / passive, or switching / de-switching hubs, as well as other network cards (Linksys or others), other communication protocols (such as NetBEUI or IPX), and other operating systems (Windows 95, 98, 2000, NT, OS / 2, or Macintosh). In a preferred embodiment, Microsoft's network software is used, which is client-to-client or client-server based. Other embodiments may include FTP, Banyan, Sunsoft or Novell networking software.

The preferred embodiment of the remote system uses four network machines to control the computer. Four network machines include a main server, secondary and tertiary computers that perform audio, video and other functions, and a registration computer (shown in FIG. 3) used to enter names or other information. The control computer 102 communicates using the TCP / IP protocol with file and drive sharing. Other embodiments of the present invention may use some network computer, including one computer dedicated to each participant, to provide individual unique graphics to the participant.

The device used for the VR display device 118, for example, has a viewing angle of about 30 degrees and a resolution of 320 × 240, transmits a stereoscopic view, uses two LCDs, and is manufactured by the iO Display System. glasses "model LC, HMD. Other forms of HMD can be used to achieve immersive effects for other embodiments of the present invention, such as LCDs, CRTs, projection display devices, and the like. The only graphical perspective transmitted to each of the two eyes is that the stereoscopic graphics defined here are not necessary. Monoscopic graphics, in which the same image is delivered to both eyes, can achieve almost the same effect for the purposes of the present invention.

Additional VR displays 118 other than HMD may be used in other embodiments, such as CRTs or LCD monitors. The monitor is a large projector positioned in front of the participant with a viewing angle of at least 25 degrees or within 10 feet of the participant with a viewing angle of at least 25 degrees. A typical 14 inch desktop computer monitor must be viewed at a normal distance of 31.6 inches before it can be used as an immersive device.

A handheld input device for a preferred embodiment of the present invention is a numeric keypad of 17 or 18 keys, such as a model 623 or 633 manufactured by Generation Systems. However, there are several numeric keypads of different manufacturers that can be used in the VRTP of the present invention. The keypad, or other participant's input device, communicates with the control computer 102 via a serial RS-232 communication protocol and is attached to a multi-port RS-232 device mounted on an ISA or PCI system bus. Multi-port devices are manufactured by Willis Computer Software and control COM ports in a Windows environment. Custom device driver software reads the port and passes player information to the game software.

Other embodiments of the invention may include other player input devices that communicate with a serial keypad or input device, or operate on entirely different protocols such as USB, parallel, wireless or wired.

The microphone 119 is powered and sends a signal to an amplifier / mixer consisting of components such as Advantage One, Advantage EX, 801, 601e and 301e, which are models of Biamp Systems'. Other embodiments may use other brands of amplifier / mixer devices or use desktop microphones or other individual microphones. The signals can be mixed with several hardware or software solutions or a combination of the two.

A preferred embodiment of the present invention uses a Panasonic MX-50 video mixer. The system may use other chroma key and switching functions, controlled by the main or secondary computer via RS-232 communication. VRTP uses a burst video switcher to direct various signals to the Panasonic mixer. Other embodiments of the invention may use different mixing or switching techniques, implemented in hardware or software.

The software for the VRPT system 100 is shown in a software flow diagram 200 as shown in FIG. 8 and is generally represented. The VRPT software system is a combination of low level and high level software programming code and is a multimedia script. The command represented by the top level is formed using Macromedia Authorware and includes an entire array of orderware commands and function icons. At the top level, the show program in the control computer (described above with respect to FIG. 7) begins by entering the participant's name from the registration computer, which includes the reading of the names and as a variable to be used during the show. The software not only lets the actors speak information about the participants, but also waits while each participant gets acquainted with the performance of the equipment and microphone. When using a team in a show, the player may have a team name set.

With the actor signing the start, the program is a series of animated clips (while the actor overlaps and disappears), branches, challenges, improvisations, scripted parts, score checks, decision points, player or team actions, etc. Is processed. Software initiated by the actor's startup signature plays video, audio, and show graphics, reads and calculates player input devices, controls hardware devices, communicates with other computers on the network, and performs other functions. Simultaneously running on different network computers (any of a plurality of secondary computers including an intensor chair computer) provides additional functionality and making the final show smoother and more seamless.

Certain functional programs operate in conjunction with an orderware application and are written in the C ++ language. The functional program communicates with the main order software program software and monitors the status of hardware components such as keypad control and key press communication to the software, control of audio and signals delivered to the participants' rumble chairs, audio / video mixers and video switches. And perform a function or other function. The control computer and all other computers communicate via TCP / IP network protocols and share hard drives with text files and other types of files.

The software system according to the invention can be easily created using several types of multimedia authoring packages other than Orderway available from the manufacturer for PC, Macintosh or other forms of computer systems, or C ++ or It can be written in a low level programming language such as Java. Underlying control programs and device drivers can be written in many different forms of programming languages besides C ++ and bring the same performance and functionality. Network communication functions can be accomplished by various other protocols or by means of existing or future means.

The network 104 as shown in FIG. 7 is shown in more detail in FIG. 9 and is generally represented as a diagram 300 implementing the network. The network generally includes a local end node site 310, a broadcast station 320, the Internet 330, a private wide area network 340, a router 360, and a remote terminal node site 380. . Using the network, the VRPT can communicate with one or more geographically remote locations for the purpose of transmitting data related to the immersive environment.

The VRPT network consists of two main components: a broadcast station 320 that creates an immersive environment and a plurality of remote terminal node sites 380 that allow participants to recognize and communicate with the broadcaster 420 and a live actor. The broadcast station 320 and the remote terminal node site 380 are connected with additional network components including the Internet 330, private wide area network 340, and router 360.

The broadcast station 320 includes the hardware and software necessary to deliver computer graphics, audio information, tactile feedback and audio and visual content of live actors that form a VRPT experience. Each station may receive input from a participant or a plurality of participants, including but not limited to data from an input device, audio and video transmission from a microphone.

A plurality of sites can be combined to form one broadcasting station. For example, multiple server systems can be deployed at different locations to create redundancy or more diverse experiences, thus allowing more processing on a large number of participants and / or networks.

Each broadcaster may be located at at least one remote broadcast site 320 geographically separated from one or more actors, while supporting live performances on the local network. As shown by broadcast station 320B, an actor at broadcast site 2A may host an experience from broadcast station 320A or broadcast site 2B or both.

The remote terminal node site 380 includes network uplinks and equipment that recognizes and interacts with the immersive experience hosted by the broadcaster 320. From the flexibility of networking technology, the network topology can change from application to application. The topology can be set up as a host server (e.g. C), a remote kisko (e.g. E and G), or a television "set top box (e.g. set top box) "(e.g., D). Any such examples include at least a feedback system for relaying data from input devices and / or voice communications, such as head mounted displays, forced feedback output devices and / or the rumble chair described in connection with FIG. 3. It may include but is not limited to a device.

Three main categories of network connectivity are implemented to provide a minimum level of performance that allows real-time transmission of live audio and video information to the remote terminal node site 380 and feedback of data inputs and audio responses. The three categories are

One). Local Area Network (LAN): A LAN provides an economical approach to connecting a remote terminal node site 380 to a broadcast station 320 using a network technology such as Ethernet.

2). Private network (PN): Also implements a PN consisting of dedicated circuits from one or more carriers. PN links also use telco circuitry as the physical medium for communication, including but not limited to xDSL, T1, DS3 or OC3 level links. In addition, data utilizing this type of circuitry can be delivered via SONET, satellite, microwave, or other conventional communications infrastructure systems.

3). Public networks (Internet): The Internet is used as the most efficient connection in terms of price, especially for remote terminal node participants living at home. In addition, data transmission via the Internet can be used in place of the above-described PN. Common connections with the Internet include analog modems, cable modems, T1, ISDN, and xDSL connections.

A plurality of computer and processing components on the Internet allow video cameras to project images, interactive and scripted and improvised, onto a vision device worn by a plurality of audiences, thereby allowing an actor to exit the audience. Provides software-controlled, immersive virtual environment execution even when you are in remote locations. Similarly, the player's chair and headphones provide audio signals. Through vision, sound intercommunication, sound emission, vibration emission, and handheld input devices, the audience can communicate to some extent while watching the virtual reality environment with live and / or pre-stored actors. have.

Another embodiment of the VRPT is shown in the sixteen chair theater shown in FIG. 10 or the eight chair theater shown in FIG. VRPT may be tailored to the specific needs of the audience and the educational purpose of entertainment and / or performances. The number of chairs can range from just one to twelve or hundreds, depending on the application. The VRPT's system can be designed to accommodate the number of participants and associated input / output devices. As shown in FIG. 10, the VRPT generally includes a participant's chair, indicator enclosure, control station, and computer / electrical system as described above.

According to the configuration features of the system shown in more detail in FIG. 7, the system provides a method of projecting an actor into a virtual reality environment such that the entertainment programming delivered is improvised and suitable for the current behavior of the actor and the participant.

From the foregoing description, it can be seen that the system provides an innovative immersive virtual environment that delivers an interactive entertainment and / or educational experience to a group of participants. For example, a game or entertainment type system may provide a show that ranges from minutes to tens of minutes, depending on the number of participants from one or two players to hundreds.

The components of the system described here allow for virtual performance entertainment by performing live performances using interface technologies such as cameras, electronics, mechanicals, and magnetic devices that capture the attitude, voice and motion of the actors. . In the system of the present invention, a group of participants can communicate with actors in the virtual reality environment and enjoy a “immersion” feeling in the environment so formed.

Live and / or pre-stored actors and / or participants may be represented in the virtual reality environment as avatars, ie animated characters, by the system. Actors and participants can be integrated into graphics and immersive environments to experience interactive and more enjoyable virtual reality. The actor and the participant may do so in a place as far from the participant as the actor permits, as far as possible. Thus, the system forms actor avatars, participant avatars and other virtual reality objects with realistically programmed movement and special visual effects.

The player participates in the system with one or more voices, electronic voice intercommunication, voice recognition, handheld device, forced feedback device, motion device, body motion, keypad input, skin resistance , Joystick control and steering wheel.

The system receives feedback from participants through voting, surveys, questions, and the like, and can monitor the participants' views for current and later analysis of key areas of interest. Artificial intelligence can be added to the system to monitor and respond to feedback from participants.

The system may implement a large screen image display device while being enhanced with software. Because of the modular structure of the system, it can be used to deliver these performances to almost any audience, including audiences that are distributed in many places. The voice processor can be used to enforce the actor's proxy for the virtual character.

In operation, system hardware, software, and integration provide a technically innovative way to:

1. Multiple Actors and Participants in a Virtual Reality Environment

2. The actor's voice and visual information are broadcast simultaneously to multiple computers.

3. Interaction and integration of actors and computer graphics is achieved through image mixing and chroma key techniques.

4. Background The stage operator and show host send commands to the system's network.

5. The actor may interact with the participants by scripting, improvisation, ad-lib or voluntary action.

6. It is possible to achieve technically uniquely enhanced processing in accordance with certain scripts during the course of the show, while allowing some deviation from the script in accordance with the actions of participants and actors.

7. The system includes signals from multiple cameras, video sources such as VCRs, laserdisc players, computers, phase inputs, internet inputs, etc., and signals from microphones and / or live or pre-stored broadcasts, or signals in entertainment and multimedia formats. Can be mixed.

8. Innovative communication structures can increase indefinitely the number of participants, audiences and participants, and the number of categories separated from each other, distributed among multiple sites.

9. Three-dimensional visual graphics can be obtained.

10. The method of the present invention can be used in a system for generating special visual effects by pre-programmed motion paths, mesh transformations and computer animation.

11. Computer controlled microphone switching is possible.

12. Artificial intelligence can be used for visual cameras to show to the audience.

New systems to illustrate only some of the various approaches include, but are not limited to:

1. A special way of gathering people in a trade show booth.

2. Gathering people at special marketing events for entertainment and / or information.

3. A single medium for multinational companies

4. More effective and enhanced means to communicate ideas and concepts by group.

5. Means to entertain the audience in places of public entertainment.

6. An effective means for examining people's new ideas or products and / or testing reactions through various feedback mechanisms.

7. An alternative method of delivering stand or improvised comedy to the audience through technology.

8. An alternative way of delivering the performing arts to the audience.

9. The only way to communicate historical information to the audience.

10. New ways to deliver teaching and training materials with increased memory recall and learning efficiency.

11. The only way to gather people on special days or holidays.

12. Substitution or enhancement for theater use for entertainment or pre-show promotion.

13. Performing dialogue and drama using immersive virtual reality simulation media.

14. Interesting means for games such as bingo or poker.

In view of the foregoing and feasible embodiments of the present invention, several objects and other advantages of the present invention are achieved. The examples and examples are intended to best explain the principles of the invention, and those skilled in the art may consider them in practical use in order to best apply various embodiments and variations of the invention.

Modifications may be made without departing from the principles of the present invention in the structures and methods described and illustrated herein, and all of the details contained in the above description or shown in the accompanying drawings are merely illustrative and not restrictive.

The scope of the present invention is not limited by the above-described exemplary embodiments, but should be defined according to the following claims.

Claims (7)

A system that interacts with participants and performers. Immersive virtual reality environment, At least one actor input device in electronic communication with the immersive virtual reality environment, At least one participant input device in electronic communication with the immersive virtual reality environment, At least one actor output device in electronic communication with the immersive virtual reality environment, and At least one participant output device in electronic communication with the immersive virtual reality environment Including; At least one live performer interacts with at least one participant and the immersive virtual reality environment, The immersive virtual reality environment includes a live video image of the actor, At least one participant communicates with the live actor and the immersive virtual reality environment to experience an experience partially controlled by the participant and the participant input device. Interactive conversation system. In claim 1, The immersive virtual reality environment At least one processing device, System data, Output data, and network Include more, The network connects the processing device, the actor input device and the output device, and the participant input device and the output device by electronic communication to transmit the system data and the output data to the actor and the participant. Interactive conversation system. In claim 2, The participant output device At least one chair that provides participants with sound or movement, or both sound and movement, At least one virtual reality display, and At least one participant audio device, More, The participant input device At least one hand-held keypad, and At least one participant microphone More, The actor output device At least one live video image of the participant, and At least one speaker for conveying the participant microphone input to the actor More, The actor input device At least one live video image of the actor, and At least one actor microphone Including; The participant views the immersive virtual reality environment through the virtual reality display device and the output data from the live video image of the actor, and feeds back to the immersive virtual reality environment through the handheld keypad and the participant microphone. (feedback), The output data further includes a graphical image and audio data. Interactive conversation system. In claim 3, The processing device At least one control computer, At least one audio processor, At least one audio switcher At least one image processor, and At least one video switch Including, The control computer processes and distributes data from the actor input device and output device and the participant input device and output device, The audio processor processes the participant microphone input, the actor microphone input, and the audio data according to the system data of the control computer, The audio switch switches the participant microphone input, the actor microphone input, and the audio data according to the system data of the control computer, The image processor processes a live video image of the participant, a live video image of the actor, and the graphic image according to the system data of the control computer; The image switch switches the live video image of the actor to the graphic data according to the system data of the control computer, Selectively providing the graphic image, the audio data and the live video image of the actor to the virtual reality display device, and selectively providing the participant microphone input, the actor microphone input and the audio data to the participant audio output means. doing Interactive conversation system. In claim 4, The immersive virtual reality environment further includes a control station, The control computer further includes a registration computer, The system data further includes participant information; The participant registers participant information with the registration computer for use for mutual conversation in the immersive virtual reality environment. Interactive conversation system. A system for interacting with participants and actors, Immersive Virtual Reality Environment Including, The immersive virtual reality environment At least one processing unit, System data, Output data, network, At least one actor input device in electronic communication with the immersive virtual reality environment, At least one participant input device in electronic communication with the immersive virtual reality environment, At least one actor output device in electronic communication with the immersive virtual reality environment, and At least one participant output device in electronic communication with the immersive virtual reality environment Including; At least one live actor interacts with at least one participant and the immersive virtual reality environment, The immersive virtual reality environment includes a live video image of the actor, At least one participant interacts with the live actor and the immersive virtual reality environment to have a partially controlled experience by the participant and the participant input device, The network connects the processing device, the actor input and output device, and the participant input and output device by electronic communication to transmit the system data and the output data to the actor and the participant, The network allows the immersive virtual reality environment and the participant to connect input and output devices over the Internet. Interactive conversation system. A method of providing interactive communication between a participant and an actor, (a) providing an immersive virtual reality environment, (b) having at least one actor input device in electronic communication with the immersive virtual reality environment, (c) having at least one participant input device in electronic communication with the immersive virtual reality environment, (d) having at least one actor output device in electronic communication with the immersive virtual reality environment, (e) providing at least one participant output device in electronic communication with the immersive virtual reality environment, (f) including a live video image of the actor with the immersive virtual reality environment, such that at least one actor interacts with at least one participant and the immersive virtual reality environment, and (g) causing at least one participant to interact with at least one live actor and the immersive virtual reality environment for a partially controlled experience by the participant and the participant input device; Interactive communication providing method comprising a.