JP2011215701A - Event participation support system and event participating support server - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an event participation support system for enabling a participant to obtain such a feeling that the participant has actually participated in the event although he or she has not participated in the actual event.SOLUTION: In this system in which a support server 2 and a participant side device 1 are connected via a communication network, the support server 2 transmits at least the image information of the event to the participant side device 1. The image information of the support image for supporting the participant's prescribed virtual participation in the event is transmitted to the participant side device 1 instead of the image information of the event or as the image information of the support image superimposed on the image information of the event. The participant side device 1 receives the input information input by the participant in response to the support image displayed by a display means 11, and transmits it to the support server 2. The support server 2 receives the input information input by the participant, and executes processing corresponding to the input information.

Description

  The present invention relates to an event participation support system that supports virtual participation in events such as ceremonies, ceremonies, conferences, and concerts. The present invention also relates to an event participation support server used in this event participation support system.

  If an event such as a ceremonial occasion, a ceremony, a meeting, or a concert is held far away from the prospective participant's residence, the prospective participant may hesitate to participate in the event. In addition, there are cases where it is not possible to participate directly in the event to be held due to circumstances of the prospective participant such as having been ill.

  In spite of such circumstances that event participation is impossible, if the event participant is, for example, a grandmother of a wedding bride, the event participant should watch the event in real time. There is a request to try.

  For example, like a videophone, it is possible to have an image sent from a remote place and view the image. For example, as described in Patent Document 1 (Japanese Patent Laid-Open No. 2009-272717), a remote conference system is also known.

  Therefore, by using such image transmission technology, event participants who are no longer able to participate in the event receive the event in real time and monitor it on the display screen so that the event can be viewed in real time. It is possible.

JP 2009-272717 A

  However, simply watching a real-time image of the event does not give you the feeling that you are participating in the event, even if it is virtually.

  In addition, when an event is a ceremonial event such as a ceremonial ceremonial occasion, there is an aspect that participation in the event fulfills its obligation as a member of a social organization that maintains human relations. One of these aspects is, for events such as ceremonial occasions, the act of registering the address and name of the participant at the reception, the act of delivering a celebration bag or a non-celebration bag, and the act of burning in the funeral. Through these actions, participants can get a sense of fulfilling their obligation to participate in the event.

  In view of the above points, an object of the present invention is to enable a participant to obtain a sense of actually participating in an event even if the participant does not participate in the actual event.

In order to solve the above problems, the present invention provides:
An event participation support system comprising an event participation support server and a participant side device that is provided on the side of a participant who virtually participates in an event and is connected to the event participation support server through a communication network,
The event participation support server is:
Event image transmission means for transmitting at least image information of the event to the participant side device through the communication network;
Support image information generating means for generating image information of a support image for supporting predetermined virtual participation of the participant in the event;
Instead of the image information of the event, or superimposed on the image information of the event, support image transmission means for transmitting the image information of the support image to the participant side device through the communication network;
Corresponding processing means for receiving input information input by the participant corresponding to the support image from the participant side device, and executing processing according to the input information;
With
The participant side device is:
Display means for displaying the image information of the event received through the communication network and the image of the support image information for the participant to view;
Accepting means for accepting the input information inputted by the participant corresponding to the support image displayed on the display means;
Input information transmitting means for transmitting the input information received by the receiving means to the event participation support server through the communication network;
An event participation support system is provided.

  According to the present invention configured as described above, at least image information of the event to be held is transmitted from the event participation support server to the participant side device through the communication network, and the event image is displayed on the display unit of the participant side device. Is displayed. Thus, the participant can view the event image, for example, in real time.

  Then, the event participation support server participates by replacing the image information of the support image for supporting the predetermined virtual participation of the participants in the event, instead of the image information of the event or by superimposing the image information on the event. To the party side device through the communication network.

  In the participant side device, the receiving means receives input information input by the participant in correspondence with the support image displayed on the display means. And a participant's side apparatus transmits the input information received by the reception means to an event participation support server through a communication network.

  The event participation support server receives the input information from the participant side device and executes a process according to the input information.

  For example, taking as an example a case where the image of reception at the ceremonial occasion is sent from the event participation support server to the participant side device as the support image, the input information and event participation received by the participant reception means The response processing by the response processing means of the support server can be as follows.

  Participants record their name and address at the reception (input act) while looking at the received support image, and for example, a virtual operation of handing a celebration bag or a non-celebration bag with a specified amount of money to the reception. Do. The participant side device sends the name and address information input by the participant to the event participation support server as part of the input information. In addition, the participant-side device sends information such as an image selection operation in which a celebration bag or a non-celebration bag is handed over to the reception as a part of input information to the event participation support server.

  The event participation support server stores the name and address of the participant as input information from the participant side device, and participates according to the operation information virtually handed over to the reception of a celebration bag or a non-festive bag. Process for billing.

  In this way, the interactive operation of providing the support image from the event participation support server to the participant side device and transmitting the input information from the participant side device to the event participation support server allows the participant to You can expect to feel like you actually participated in the event.

  According to the present invention, even if the participant does not participate in an actual event, the participant can obtain a feeling as if he / she actually participated in the event.

It is a figure which shows the structural example of 1st Embodiment of the event participation assistance system by this invention. It is a block diagram of the structural example of the participant side apparatus in 1st Embodiment of the event participation assistance system by this invention. It is a block diagram of an example of composition of a meeting place side device in a 1st embodiment of an event participation support system by this invention. It is a block diagram of the example of composition of the event participation support server in a 1st embodiment of the event participation support system by this invention. It is a figure used in order to demonstrate the principal part of the processing operation in 1st Embodiment of the event participation assistance system by this invention. It is a figure which shows a part of flowchart for demonstrating the processing operation example in the participant side apparatus of FIG. It is a figure which shows a part of flowchart for demonstrating the processing operation example in the participant side apparatus of FIG. It is a figure which shows a part of flowchart for demonstrating the example of a processing operation in the event participation assistance server of FIG. It is a figure which shows a part of flowchart for demonstrating the example of a processing operation in the event participation assistance server of FIG. It is a figure which shows a part of flowchart for demonstrating the example of a processing operation in the event participation assistance server of FIG. It is a figure which shows the flowchart for demonstrating the example of a processing operation in the venue side apparatus of FIG. It is a figure which shows the structural example of 2nd Embodiment of the event participation assistance system by this invention. It is a block diagram of the structural example of the event participation assistance server in 2nd Embodiment of the event participation assistance system by this invention. It is a figure used in order to demonstrate the principal part of the processing operation in 2nd Embodiment of the event participation assistance system by this invention. It is a figure which shows a part of flowchart for demonstrating the example of a processing operation in the event participation assistance server of FIG. It is a figure which shows a part of flowchart for demonstrating the example of a processing operation in the event participation assistance server of FIG. It is a figure which shows a part of flowchart for demonstrating the example of a processing operation in the event participation assistance server of FIG.

  Hereinafter, embodiments of an event participation support system and an event participation support server according to the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram for explaining the outline of the first embodiment of the event participation support system. In the event participation support system according to the first embodiment, for example, an event to be held is a wedding, and a participant who could not participate due to various circumstances (hereinafter referred to as a virtual participant) lively marries. You can watch the ceremony.

  And in this 1st Embodiment, it is comprised so that it may project as a hologram in this example in the actual event site. In addition, the virtual participant is configured to feel as if he / she went to the event venue, made a book at the reception, and actually handed the celebration bag.

  As shown in FIG. 1, the event participation support system of the first embodiment is a participant-side device 1 provided in a location 5 of a virtual participant 7, an event participation support server 2, and a venue provided in an event venue 6. Side device 3. Participant side device 1, event participation support server 2 and venue side device 3 are connected to communication network 4. The communication network 4 includes the Internet, an optical cable network, a public telephone network, and the like.

  In this example, the participant apparatus 1 and the support server 2 are connected through the communication network 4 by accessing the event participation support server (hereinafter abbreviated as support server) 2 from the participant apparatus 1. Further, the support server 2 and the venue-side device 3 are connected through the communication network 4 by accessing one of the support server 2 and the venue-side device 3 to the other.

  The participant side device 1 includes a participant side processing device 10, a display device 11, an operation unit 12, and a microphone 13. The participant processing apparatus 10 is connected to the communication network 4 in order to communicate with the support server 2. The display device 11 is connected to the participant side processing device 10 and displays a display image based on image information sent from the support server 2 on its display screen. Note that the display device 11 has a built-in speaker for sound reproduction, and in this example, multi-surround sound, for example, 5.1 channel surround sound, can be acoustically reproduced.

  The operation unit 12 is connected to the participant-side processing device 10, receives an operation input from the virtual participant 7, and supplies the operation input information to the participant-side processing device 10. In this example, the operation unit 12 includes a touch panel, and has a function of accepting a character input operation using a touch pen by the virtual participant 7 and supplying the operation input information to the participant-side processing device 10. In this example, the operation unit 12 includes a gyro, and the virtual participant 7 detects a movement while holding the operation unit 12 (an action of the virtual participant 7), and the detected information is processed by the participant. A function of supplying to the device 10 is also provided.

  The microphone 13 is connected to the participant-side processing device 10, picks up the voice uttered by the virtual participant 7, converts it into a voice signal, and supplies the voice signal to the participant-side processing device 10. In this example, the microphone 13 includes a switch for turning on and off the sound collecting operation, and performs sound collecting operation when the virtual participant 7 turns on the switch.

  The hall-side device 3 includes a hall-side processing device 30, an imaging device 31, a microphone 32, a hologram projection device 33, and a speaker 34. The venue side processing device 30 is connected to the communication network 4 in order to communicate with the support server 2. In this example, the imaging device 31 includes a 3D camera (three-dimensional stereoscopic image camera), images a wedding event at the event venue 6, and supplies the 3D captured image to the venue-side processing device 30. The microphone 32 picks up the voice of the wedding event at the event hall 6 and converts it into a voice signal, and supplies the voice signal to the hall-side processing device 30. In FIG. 1, only one microphone 32 is shown. However, since the sound of the event venue 6 is realistic, for example, 5.1-channel surround sound is collected, the microphone 32 has six microphones. Consists of.

  The hologram projecting device 33 is a device that projects the hologram 35 of the virtual participant 7 based on the hologram projecting image information sent from the support server 2. The speaker 34 constitutes an acoustic reproduction means for acoustically reproducing the audio signal of the virtual participant 7 sent from the support server 2. The speaker 34 is disposed in the vicinity of the hologram 35 of the virtual participant 7.

  Next, detailed configuration examples of the participant side device 1, the support server 2, and the venue side device 3 will be described.

<Configuration example of participant side device 1>
FIG. 2 is a block diagram for explaining a specific configuration example of the participant processing apparatus 10 of the participant apparatus 1. The participant-side processing apparatus 10 in this example is configured by, for example, a personal computer, and a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102 for storing programs and the like, A RAM (Random Access Memory) 103 serving as an area is connected.

  Further, a communication interface 104, a reception processing unit 105, a transmission processing unit 106, an image / audio processing unit 107, an operation unit interface 108, and an audio input interface 109 are further connected to the system bus 100.

  The CPU 101 controls the overall operation of the participant side processing apparatus 10. The ROM 102 stores in advance a program for supporting virtual participation (virtual participation) in an event by a virtual participant as will be described later in the participant processing apparatus 10. For example, in the ROM 2, a program for displaying characters input through the touch panel from the operation unit 12 on the screen of the display device 11, or the input characters are converted into text data and transmitted to the support server 2. Programs are stored. The CPU 101 uses the RAM 103 as a work area memory and executes control processing according to the program of the ROM 102.

  The communication interface 104 is connected to the communication network 4 and communicates with the support server 2. The reception processing unit 105 has a function of extracting image information and audio information from information received from the support server 2 through the communication interface 104 and supplying the extracted image information and audio information to the image / audio processing unit 107. The transmission processing unit 6 has a function of generating information to be transmitted to the support server 2 and transmitting the information through the communication interface 104. Note that the reception processing unit 105 and the transmission processing unit 106 may be software function processing units in which the CPU 101 executes processing based on a program in the ROM 102.

  The image / sound processing unit 107 sends the image information and sound information received from the reception processing unit 105 to the display device 11. In this example, as described above, the image information from the venue side device 3 is 3D image information, and the virtual participant 7 wears so-called 3D glasses 8 and looks at the display screen of the display device 11. Thus, the event venue image can be viewed as a stereoscopic image. Further, since the audio information is multi-channel surround sound such as 5.1 channel surround sound, the virtual participant 7 can listen to the sound of the event venue with a sense of reality.

  The operation unit interface 108 is connected to the operation unit 12, and operation input information (including information on movement of the operation unit 12) by the virtual participant 7 is transmitted to the system bus 100 through the operation unit interface 108. The CPU 101 monitors operation input information input through the operation unit interface 108 and detects character input through the touch panel, movement of the operation unit 12 according to the movement of the virtual participant detected by the gyro, and the like.

  A microphone 13 is connected to the audio input interface 109. As described above, the microphone 13 of this example picks up the sound uttered by the virtual participant 7 only when the sound pickup operation on / off switch is turned on, and outputs the collected sound signal. , Input to the system bus 100 through the voice input interface 109. The input audio signal is converted into a transmission signal by the transmission processing unit 106 and transmitted to the support server 2 through the communication interface 104.

<Configuration example of the venue side device 3>
FIG. 3 is a block diagram for explaining a specific configuration example of the venue-side processing device 30 of the venue-side device 3. The venue-side processing device 30 in this example is configured by a personal computer, for example, and is connected to a system bus 300 with a CPU 301, a ROM 302 for storing programs and the like, and a RAM 303 serving as a work area.

  Further, the system bus 300 further includes a communication interface 304, a reception processing unit 305, a transmission processing unit 306, an imaging device interface 307, a voice input interface 108, a voice output interface 109, a hologram image data storage unit 310, a hologram projection. A device interface 311 is connected.

  The CPU 301 controls the overall operation of the venue side processing apparatus 30. The ROM 302 displays a real-time image and sound (live image / sound) of the event venue as will be described later in the venue-side processing device 30, and the hologram of the virtual participant received from the support server 2. Programs such as processing to be performed and processing to reproduce sound of the virtual participant received from the support server 2 through the speaker 34 are stored in advance. The CPU 301 uses the RAM 303 as a work area memory and executes control processing according to the program stored in the ROM 302.

  The communication interface 304 is connected to the communication network 4 and communicates with the support server 2. The reception processing unit 305 has a function of supplying information received from the support server 2 through the communication interface 304 to the imaging device interface 307, the hologram image data storage unit 310, and the audio output interface 309. The transmission processing unit 6 converts captured image information (live image information) from the imaging device 31 and audio information (live audio information) from the microphone 32 into a form to be transmitted to the support server 2, and sends it through the communication interface 104. It has a function to do. Note that the reception processing unit 305 and the transmission processing unit 306 may be software function processing units in which the CPU 301 executes processing based on a program stored in the ROM 302.

  The imaging device interface 307 is connected to the imaging device 31, and transfers 3D captured image information obtained by imaging by the imaging device 31 to the transmission processing unit 306 under the control of the CPU 301. The imaging device interface 307 also supplies the imaging device 31 with the imaging direction or zoom change control signal received by the reception processing unit 305 in accordance with the control of the CPU 301.

  The audio input interface 308 is connected to the microphone 32, and transfers audio information (multi-surround audio information) obtained by collecting sound with the microphone 32 to the transmission processing unit 306 under the control of the CPU 301.

  The hologram image data storage unit 310 stores hologram participant image data sent from the support server 2. Then, the CPU 301 controls to read out the hologram image data of the virtual participant from the hologram image data storage unit 310 and supply it to the hologram projector interface 311 while the event is being executed.

  A hologram projection device 33 is connected to the hologram projection device interface 311, and the image data for hologram of the virtual participant is sent to the hologram projection device 33. The hologram projecting device 33 receives this image data and projects the virtual participant's hologram at a preset position in the event venue 6.

<Configuration example of support server 2>
FIG. 4 is a block diagram for explaining a specific configuration example of the support server 2. The support server 2 in this example is configured by a personal computer, for example, and is connected to a system bus 200 with a CPU 201, a ROM 202 for storing programs and the like, and a RAM 203 serving as a work area.

  The system bus 200 further includes a communication interface 204, a virtual participant information storage unit 205, a reception information distribution unit 206, a reception image / audio information processing unit 207, a monitor interface 208, a support image / audio information generation unit 209, A participant transmission information generation unit 210, a venue transmission information generation unit 211, a virtual participant operation analysis unit 212, a virtual participant settlement processing unit 213, and an operation unit interface 214 are connected.

  The CPU 201 controls the overall operation of the support server 2. The ROM 202 stores in advance a program for supporting virtual participation (virtual participation) in an event by a virtual participant as will be described later in the support server 2. For example, the ROM 2 generates a program that transmits live images and audio of the event at the event venue 6 sent from the venue-side device 3 to the participant-side device 1 and support image / audio information to generate the participant-side device. The program for transmitting to 1 and the program for transmitting information from the participant side device 1 to the venue side device 3 are stored. The CPU 201 uses the RAM 203 as a work area memory and executes control processing according to the program stored in the ROM 202.

  The communication interface 204 is connected to the communication network 4 to communicate with the participant side device 1 and the venue side device 3.

  The virtual participant information storage unit 205 stores information about participants who participate in virtual events in advance. In this example, the virtual participant information includes correspondence information between the event identifier and the virtual participant identifier, image data for hologram of the virtual participant 7, and information for settlement processing for the money of the virtual participant 7. Including.

  The event identifier and the virtual participant identifier are notified in advance to the virtual participant 7 from the operating service provider of the support server 2 when an application for virtual participation by the virtual participant 7 is made to the support server 2. The These identifiers are information for specifying which virtual participant participates in which event, and are necessary when the virtual participant 7 accesses the support server 2 to virtually participate in the event. Become.

  The hologram image data of the virtual participant 7 is created by the operation service provider of the support server 2. That is, when an application for virtual participation by the virtual participant 7 is made to the support server 2, the operation service provider of the support server 2 photographs the virtual participant 7, creates hologram image data in advance, This virtual participant information storage unit 205 stores and holds it. At the start of the event, the support server 2 transmits the hologram image data stored and held in the virtual participant information storage unit 205 to the hall processing device 30 through the communication network 4. The hall-side processing device 30 stores the received hologram image data in the hologram image data storage unit 310, and reads out and displays it as a hologram during an event.

  The information for the settlement process for the money of the virtual participant 7 is a payment method and information accompanying the payment method. The credit number is included at the time of credit payment, and the destination information of the invoice is included at the time of transfer payment. It is.

  The reception information distribution unit 206 sorts information received from the participant side processing apparatus 10 or the venue side processing apparatus 30 through the communication interface 104 and distributes the information to the corresponding units. That is, the reception information distribution unit 206 supplies the captured image information and audio information received from the venue side processing apparatus 30 to the reception image / audio information processing unit 207. Further, the reception information distribution unit 206 supplies the operation input information of the virtual participant 7 received from the participant processing apparatus 10 to the virtual participant operation analysis unit 212. Further, the reception information distribution unit 206 supplies the audio signal of the virtual participant 7 received from the participant side processing apparatus 10 to the venue transmission information generation unit 211.

  The received image / audio information processing unit 207 supplies live image information and live audio information from the venue side processing device 30 to the transmission information generation unit 210 for participants, and information in a format that can be reproduced and monitored by the monitor device 215. And supplied to the monitor interface 208.

  A monitor device 215 is connected to the monitor interface 208. The monitor interface 208 supplies the live image information and live audio information from the received image / audio information processing unit 207 to the monitor device 215. The monitor device 215 monitors and reproduces the image and sound based on the received live image information and live audio information. Thereby, the operator of the support server 2 can check the state of the event venue in real time.

  The support image / sound information generation unit 209 is a support image / sound for enabling the virtual participant 7 to have a feeling that the virtual participant 7 is participating in a pseudo-event at the event venue. Generate information. The support image / sound information generation unit 209 supplies the generated support image / sound information to the participant transmission information generation unit 210 based on an instruction from the operator through the operation unit 216.

  In this example, the support image / sound information generation unit 209 generates support image / sound information that supports the virtual event participation operation of the virtual participant 7 at the reception of the wedding hall as the event venue. The reception support image / sound information is realistic when it is generated as including an image obtained by photographing an actual reception at the event venue 6, but it may of course be generated as a virtual image. . In this example, the support image / speech information generation unit 209 causes a person character image (hereinafter referred to as an avatar) to be a substitute for the virtual participant 7 to appear in the image based on the support image information. Support image information is generated so that an operation corresponding to the operation using the operation unit 12 is performed.

  A character image used as an avatar can be generated as a two-dimensional image or a three-dimensional animation image. In this case, if a captured image of the virtual participant 7 is provided to the support server 2, the support server 2 generates an avatar that resembles the seven virtual participants by reflecting the captured image on the avatar. You can also

  In this example, the virtual participant 7 is configured to be able to give a celebratory speech. The speech sound of the virtual participant 7 is collected by the microphone 13, supplied to the venue processing device 30 through the support server 2, and reproduced as sound by a speaker. Also during the speech of the virtual participant 7, the support image / speech information generation unit 209 generates an avatar to be displayed superimposed on the image information of the event venue.

  The participant transmission information generation unit 210 follows the instruction of the operator through the operation unit 216, and the live image information and live audio information of the event venue from the received image / audio information processing unit 207, and the support image / audio information generation unit 209. Participant transmission information to be sent to the participant processing apparatus 10 is generated from the support image / sound information from

  That is, when the operator's instruction from the operation unit 216 is “live image / sound transmission”, the transmission information generation unit 210 for participants sends the live image information of the event venue from the received image / sound information processing unit 207 and Only live audio information is generated as transmission information for participants.

  When the operator's instruction from the operation unit 216 is “acceptance support”, the transmission information generation unit 210 for participants uses the live image information and live audio information during the “acceptance support” instruction period. Instead, the support image / sound information received from the support image / sound information generation unit 209 is generated as transmission information for the participant. Furthermore, when the operator's instruction from the operation unit 216 is “speech support”, the transmission information generation unit 210 for participants transmits support image / audio to live image information during the “speech support” instruction period. The image information on which the avatar image information from the information generation unit 209 is superimposed and the live audio signal are generated as the transmission information for the participant. During the speech of the virtual participant 7, the virtual participant 7 in the participant side device 1 listens to live sound through headphones in order to prevent sound howling.

  The transmission information generation unit 210 for participants transmits the transmission information for participants generated as described above to the communication network 4 through the communication interface 204 with the participant processing apparatus 10 as a transmission destination.

  The venue-oriented transmission information generation unit 211 generates the audio information of the virtual participant 7 from the reception information distribution unit 206 as venue information. The venue direction transmission information generation unit 211 also generates, as venue direction information, the direction of the imaging device 31 (imaging direction such as pan and tilt) and the zoom control signal from the virtual participant operation analysis unit 212.

  The venue-specific transmission information generation unit 211 transmits the venue-specific transmission information generated as described above to the communication network 4 through the communication interface 204 with the venue-side processing device 30 as a transmission destination.

  The virtual participant operation analysis unit 212 analyzes the operation input information of the operation unit 12 of the virtual participant 7 from the participant-side processing device 10, and information on the bookkeeping operation at the reception by the virtual participant 7 is a support image / It is sent to the voice information generation unit 209. Receiving this, the support image / speech information generation unit 209 generates a support image for reception so that the avatar performs a bookkeeping operation. In addition, when the operation input information of the operation unit 12 of the virtual participant 7 is information specifying the amount of money to be put in the celebration bag and indicates an operation of delivering the celebration bag, the virtual participant operation analysis unit 212 In addition to supplying the money amount information to the virtual participant settlement processing unit 213, information indicating an operation for delivering the celebration bag is sent to the support image / audio information generation unit 209.

  The virtual participant settlement processing unit 213 performs settlement processing on the amount indicated by the amount information received from the virtual participant operation analysis unit 212 based on the information for settlement processing acquired from the virtual participant information storage unit 205. .

  Further, the support image / sound information generation unit 209 supports the reception so that the avatar performs the operation of delivering the celebration bag according to the information of the delivery operation of the celebration bag received from the virtual participant operation analysis unit 212. Generate an image.

  The operation unit interface 214 is connected to the operation unit 216, and transmits the operation input through the operation unit 216 of the operator of the support server 2 described above to the CPU 201 through the system bus 200.

  In FIG. 4, the received information distribution unit 206, the received image / audio information processing unit 207, the support image / audio information generation unit 209, the transmission information generation unit 210 for participants, the transmission information generation unit 211 for venues, and the virtual participants Each unit such as the operation analysis unit 212 and the virtual participant settlement processing unit 213 may be a software function processing unit in which the CPU 101 executes processing based on the program in the ROM 102.

<Overview of operations in the event participation support system>
The virtual participant 7 applies to the operator of the support server 2 for virtual participation, acquires the event identifier and the virtual participant identifier, and has his / her own hologram image data created. The support server 2 holds the event identifier, the virtual participant identifier, and the hologram image data in the virtual participant information storage unit 205.

  The operating service provider of the support server 2 installs the venue side device 3 in the event venue 6 and notifies the support server 2 of the preparation OK when the live information of the event can be transmitted.

  In such an environment, when the virtual participant 7 accesses the support server 2 through the participant processing device 10 in accordance with the event holding time, the support server 2 confirms the virtual participant identifier and the event identifier, Event support processing for the participant 7 is started.

  First, the operator of the support server 2 inputs an instruction for “acceptance support” through the operation unit 216. In response to this instruction input, the support server 2 generates a support image / speech information for reception including, for example, an image obtained by capturing the reception of the event venue 6 in advance using the support image / sound information generation unit 209, It transmits to the apparatus 10 through the communication network 4. The participant-side processing device 10 supplies the received support image / sound information to the display device 11 and reproduces the support image and sound.

  An example of the support image for this reception is shown in FIG. In this example, the support server 2 uses the support image / sound information generation unit 209 to generate the reception image and sound information acquired by the imaging device 31 and the microphone 32 of the hall side device 3 as support image / sound information ( (See FIG. 5A).

  Next, the support image / sound information generation unit 209 generates the entry screen of the participants at the reception, and the sound information of “Thank you for attending today. 5 (B)). The transmission information generation unit 210 for participants sends the support image / audio information to the participant side processing apparatus 10 through the network 4.

  In response to this, when the virtual participant 7 performs a bookkeeping operation through the operation unit 12, the participant processing apparatus 10 sends information on the bookkeeping operation to the support server 2. When the support image / sound information generation unit 209 of the support server 2 receives the operation input information through the virtual participant operation analysis unit 212, the avatar performs an operation as if it is writing according to the operation input. A support image (see FIG. 5C) is generated. The participant transmission information generation unit 210 sends the generated support image / audio information to the participant processing apparatus 10 through the network 4. Note that the registered character information actually input to the virtual participant 7 is determined by the participant-side processing device 10, converted into text information, and sent to the support server 2. The support server 2 stores and holds this text information in the virtual participant information storage unit 205.

  Next, the support image / sound information generation unit 209 displays a celebration bag and generates an image for the virtual participant 7 to input the amount of money (see FIG. 5D). In addition to this image, a message “Please enter the amount” is displayed. In response to this, when the virtual participant 7 performs an operation for inputting an amount of money through the operation unit 12, the operation input information is sent to the support server 2.

  In the support server 2, the operation input of this amount is determined by the virtual participant operation analysis unit 212, and the amount information is sent to the virtual participant settlement processing unit 213.

  Next, when the virtual participant 7 performs an operation of submitting a celebration bag at the reception using the operation unit 12, operation input information of the operation is sent from the participant-side processing device 10 to the support server 2.

  The support image / speech information generation unit 209 of the support server 2 receives this operation input information through the virtual participant operation analysis unit 212 and reflects it on the movement of the avatar of the support image (see FIG. 5E). Then, the virtual participant operation analysis unit 212 of the support server 2 determines that the operation of submitting the celebration bag to the reception is complete, and notifies the virtual participant settlement processing unit 213 of the determination result.

  The virtual participant settlement processing unit 213 of the support server 2 refers to the settlement method of the virtual participant 7 in the virtual participant information storage unit 205 and executes settlement according to the settlement method.

  Next, the support image / speech information generation unit 209 generates an image of a response to the virtual participant who has completed the transfer of the bookkeeping and the celebration bag (see FIG. 5F), and further guides the image into the event venue. (See FIG. 5G). These are transmitted from the transmission information generation unit 210 for participants to the participant processing apparatus 10 through the communication interface 204 and the communication network 4.

  Then, the transmission information generation unit 210 for the participant of the support server 2 displays the live image information (FIG. 5) from the received image / audio information processing unit 207 after the support image to be guided to the event venue in FIG. (See (H)) and the live audio information are supplied to the participant processing apparatus 10 through the communication network 4.

  With the support image / sound of the reception as described above, the virtual participant 7 can feel as if the reception has been completed at the event venue.

  When the reception support image is completed, the operator inputs “live image / audio transmission” through the operation unit 216. In response to this instruction input, the support server 2 thereafter transmits the live image information and live audio information of the event venue 6 to the participant processing device 10 until the speech order of the virtual participant 7 is reached.

  Then, when the order of the speech of the virtual participant 7 is reached, the host of the event venue reports that there is a speech of the virtual participant 7, and the operator inputs “speech support” through the operation unit 216. In response to this instruction input, the support server 2 causes the support image / sound information generation unit 209 to execute a speech to the virtual participant 7 such as “Please turn on the microphone and make a speech”. The message image and sound to be urged are generated, and the message image and sound are superimposed on the live image and live sound and transmitted to the participant processing apparatus 10.

  In response to this, the participant-side processing device 10 displays the message on the display device 11 and reproduces the sound. When the virtual participant 7 makes a speech through the microphone 13 in response to this message, the participant-side processing device 10 sends the voice information to the support server 2.

  In the support server 2, the audio information is received, and the transmission information generation unit 211 for the venue transmits the audio signal of the virtual participant 7 to the hall processing device 30 through the communication interface 204.

  The hall-side processing device 30 plays back the received audio information through the speaker 34. Since the speaker 34 is disposed in the vicinity of the hologram of the virtual participant 7, the virtual participant 7 feels as if speaking.

  At this time, the support image / speech information generation unit 209 generates a support image that behaves as if the avatar is making a speech, and supplies the support image to the participant transmission information generation unit 210. The transmission information generation unit 210 for participants generates an image to be displayed by superimposing the avatar generated by the support image / speech information generation unit 209 on the live image, and sends the generated image to the participant processing apparatus 10.

  On the display 11 of the participant processing apparatus 10, an image behaving like an avatar is speaking is displayed. Thereby, the virtual participant 7 can obtain a feeling as if he / she was speaking at the event venue.

  When the event ends, the transmission of live image information and live audio information from the support server 2 to the participant processing apparatus 10 stops, and the event participation support service by the support server 2 ends.

<Examples of processing operations of each part of the event participation support system>
[Example of processing operation in the participant processing apparatus 10]
FIG. 6 and FIG. 7 show a flowchart of an example of processing operation in the participant processing apparatus 10 performed to realize the system operation as described above. The processing of each step in FIGS. 6 and 7 is executed by the CPU 101 according to the program in the ROM 102.

  The CPU 101 waits for the virtual participant 7 to give an instruction to access the support server 2 through the operation unit 12 (step S101). When it is determined that the access instruction has been given, the event sent from the support server 2 An input screen for an identifier and a virtual participant identifier is received (step S102).

  Then, the CPU 101 waits for the completion of the input of the event identifier and the virtual participant identifier through the operation unit 12 by the virtual participant 7 (step S103), and when determining that the input of the identifier is completed, the CPU 101 The participant identifier is sent to the support server 2 (step S104).

  If the event identifier and the virtual participant identifier are correct, the support server 2 sends either event start preparation OK or NG. Therefore, the CPU 101 determines whether or not the event start preparation OK has been received (step S105). When the CPU 101 determines that the event start preparation OK has not been received, it displays “Please wait for a while” on the display screen. The process returns to step S105.

  If it is determined in step S105 that the event start preparation OK has been received, the CPU 101 receives support image / audio information of a procedure for accepting the event (step S107). Then, the CPU 101 determines whether or not operation input information by the virtual participant 7 has been received through the operation unit 12 (step S108).

  When it is determined in step S108 that the operation input information by the virtual participant 7 is received through the operation unit 12, the CPU 101 transmits the operation input information of the virtual participant 7 to the support server 2 through the communication network 4 (step S109). . Next, the CPU 101 determines whether or not the support image / sound information has ended based on the notification information from the support server 2 (step S110). When determining that the support image / sound information has not ended, the CPU 101 returns to step S107. The processing after step S107 is repeated. If it is determined in step S108 that the operation input information by the virtual participant 7 is not accepted through the operation unit 12, the process proceeds to step S110, bypassing step S109, and the processes after step S110 are repeated.

  When it is determined in step S109 that the support image / sound information has ended, the CPU 101 receives live image information and live sound information from the support server 2, and receives a live image (3D image) and sound (event image) of the event venue 6. Multi-sound sound) monitor playback is started (step S121 in FIG. 7).

  Then, the CPU 101 determines whether or not an instruction to change the orientation or zoom of the imaging device 31 through the operation unit 12 has been received from the virtual participant 7 (step S122). The direction of the apparatus 31 and zoom change instruction information are transmitted (step S123).

  When it is determined in step S122 that the virtual participant 7 has not received an instruction to change the orientation or zoom of the imaging device 31 through the operation unit 12, and after step S123, the CPU 101 causes the microphone 13 to It is turned on and it is determined whether or not an audio signal is input (step S124).

  In other words, in the live image and live audio, when the virtual participant 7 makes a speech, as described above, in response to the operator's “speech support” instruction input, the virtual participant 7 is given a speech. A message prompting you to do so is sent superimposed. In response to this, the virtual participant 7 turns on the microphone 13 and executes speech input. When the input of the audio signal from the microphone 13 is detected, the CPU 101 transmits the audio signal of the virtual participant 7 collected by the microphone 13 to the support server 2 (step S125).

  Next, the CPU 101 determines whether or not the event has ended based on whether or not an end notification has been received from the support server 2 (step S126). If it is determined in step S126 that the event has not ended, the CPU 101 returns to step S122 and repeats the processing from step S122 onward.

  If it is determined in step S126 that the event has ended, the CPU 101 stops receiving information from the support server 2, and notifies the virtual participant 7 of the end of the event by display display or audio notification (step S127). Then, the CPU 101 disconnects the communication path with the support server 2 (step S128), and ends this processing routine.

[Example of processing operation in support server 2]
FIG. 8 to FIG. 10 show flowcharts of processing operation examples in the support server 2 performed to realize the system operation as described above. The processing of each step in FIGS. 8 to 10 is executed by the CPU 201 controlling each unit in accordance with the program in the ROM 202.

  The CPU 201 of the support server 2 waits for reception of access from the participant processing device 10 (step S201), and when determining that it has received access from the participant processing device 10, the event identifier and the virtual participant identifier The input screen is generated by the participant transmission information generation unit 210 and transmitted to the participant processing apparatus 10 (step S202).

  Then, the CPU 201 receives the event identifier and the virtual participant identifier sent from the participant processing apparatus 10 (step S203), determines the corresponding event, and checks whether the event is ready to start. (Step S204). Then, as a result of the check, the CPU 201 determines whether or not the event start preparation is OK (step S205). If the CPU 201 determines that the event start preparation is not OK, the CPU 201 notifies the participant processing apparatus 10 of the event start preparation NG. Is transmitted (step S206), and the process returns to step S204.

  When it is determined in step S205 that the event start preparation is OK, the CPU 201 causes the support image / audio information generation unit 209 and the transmission information generation unit 210 for the participant to be activated in response to an instruction input of “reception support” by the operator. Control is performed to generate support image / sound information for reception at the event venue, and the generated support image / sound information is transmitted to the participant processing apparatus 10 (step S207).

  Next, the CPU 201 monitors reception from the participant-side processing device 10, and when there is reception, controls the reception information distribution unit 206 to determine whether or not the reception information is operation input information of the virtual participant 7. (Step S208).

  When it is determined in step S208 that the received information is the operation input information of the virtual participant 7, the CPU 201 causes the virtual participant operation analysis unit 212 to analyze the received operation input information. The virtual participant operation analysis unit 212 supplies the analysis result to the support image / audio information generation unit 209 and the virtual participant information storage unit 205. The support image / audio information generation unit 209 reflects the operation input information of the virtual participant 7 in the support image, such as controlling the movement of the avatar according to the operation input information of the virtual participant 7. Further, the virtual participant information storage unit 205 stores the book entry information input by the virtual participant 7 (step S209).

  When it is determined in step S208 that the received information is not the operation input information of the virtual participant 7, and after step S209, the CPU 201 executes the support image / audio information generation process in the support image / audio information generation unit 209. It is determined whether or not the generation process has been completed (step S210). If it is determined that the generation process has not been completed, the process returns to step S207, and the processes after step S207 are repeated.

  In step S210, when it is determined that the generation processing of the support image / sound information has been completed in response to an instruction input of “live image / live sound” through the operation unit 216 of the operator, the CPU 201 analyzes the virtual participant operation. It is determined from the analysis result of the unit 212 whether or not there has been a virtual money delivery action by the virtual participant 7 during the processing based on the support image / audio information (step S221). In step S221, when it is determined that there has been a virtual money delivery action by the virtual participant 7, the CPU 201 performs a settlement process for the delivery of money for the virtual participant 7 to the virtual participant settlement processing unit 213. A control instruction is given as follows (step S222).

Next, the CPU 201 accesses the venue-side processing device 30 through the communication network 4 and generates a communication path (step S223). Then, the CPU 201 starts the transmission of the live captured image information and the live audio information sent from the venue side processing device 30 to the participant side processing device 10, the received information distribution unit 206, the received image / audio information. The processing unit 207 and the transmission information processing unit 210 for participants are controlled (step S224).
Then, the CPU 201 reads the hologram image data of the virtual participant 7 from the virtual participant information storage unit 205 and controls the venue-oriented transmission information generation unit 211 and the communication interface 204 to transmit the image data to the venue-side processing device 30. (Step S225).

  Then, the CPU 201 checks whether or not the reception information distribution unit 206 has received information from the participant processing apparatus 10. When the information is received, the received information is captured from the analysis result of the virtual participant operation analysis unit 212. It is determined whether or not it is change instruction information such as the orientation and zoom of the device 31 (step S226). When it is determined in step S226 that the received information is change instruction information such as the orientation and zoom of the imaging device 31, the CPU 201 controls the venue-oriented transmission information generation unit 211 to determine the orientation of the imaging device 31. Change instruction information such as zoom is transmitted to the venue-side processing device 30 (step S227).

  When it is determined in step S226 that the received information is not change instruction information such as the orientation or zoom of the imaging device 31, and after step S227, the CPU 201 receives an audio signal from the participant processing device 10. It is determined whether or not it has been received (step S228).

  When it is determined in step S228 that an audio signal has been received from the participant side processing device 10, the CPU 201 transmits to the hall side processing device 30 so as to transmit the audio signal from the participant side processing device 10 to the hall. The information generator 211 and the communication interface 204 are controlled (step S229).

  When it is determined in step S228 that an audio signal has not been received from the participant processing apparatus 10, and after the processing in step S229, the CPU 201 determines whether or not the event has ended (FIG. 10). Step S231). This event end determination may be performed automatically by determining the event end time, or may be a process of determining an event end input operation by an operator who monitors the event with the monitor device 215. Furthermore, the process which discriminate | determines the completion | finish instruction | indication from the venue side apparatus 3 of the event venue 6 may be sufficient.

  If it is determined in step S231 that the event has not ended, the CPU 201 returns to step S226 and repeats the processing from step S226 onward. If it is determined in step S231 that the event has ended, the CPU 201 stops receiving the imaging information and audio information from the hall-side processing device 30, and notifies the participant-side processing device 10 of the end of the event (step S232). ).

  Then, the CPU 201 waits for the communication path between the participant side processing apparatus 10 and the venue side processing apparatus 30 to be disconnected (step S233), and ends this processing routine.

[Example of processing operation at the venue-side processing apparatus 30]
FIG. 11 shows a flowchart of an example of processing operation in the venue-side processing apparatus 30 performed to realize the system operation as described above. The processing of each step in FIG. 11 is executed by the CPU 301 controlling each unit according to the program stored in the ROM 302.

  First, the CPU 301 prepares for transmission of an image captured by the image capturing device 31 and collected sound by the microphone 32 (step S301). Then, the CPU 301 waits for connection to the support server 2 through the communication network 4 (step S302). When it is determined that the support server 2 is connected, the CPU 301 captures an image picked up by the image pickup device 31 and sound collected by the microphone 32. Each unit is controlled to start transmission to the support server 2 (step S303).

  Next, the CPU 301 receives the hologram image data of the virtual participant 7 sent from the support server 2, stores the hologram image data in the hologram image data storage unit 310, and stores the hologram image data in the hologram projector 33. And reproduction of the hologram is started (step S303).

  Next, the CPU 301 monitors information sent from the support server 2 and determines whether or not the direction of the imaging device 31 and zoom change instruction information have been received (step S305). When it is determined in step S305 that the orientation change information or the zoom change instruction information of the imaging device 31 has been received, the CPU 301 performs change control in accordance with the change instruction information received for the orientation of the imaging device 31 or the zoom (step S306). ).

  When it is determined in step S305 that the orientation change information or zoom change instruction information of the imaging device 31 has not been received, and after step S306, the CPU 301 has received audio information from the support server 2 or not. A determination is made (step S307). If it is determined in step S307 that audio information has been received from the support server 2, the CPU 301 reproduces the received audio signal through the speaker 34 (step S308).

  When it is determined in step S307 that no audio information has been received from the support server 2, and after step S308, the CPU 301 determines whether the event has ended (step S309). If it is determined in step S309 that the event has not ended, the CPU 301 returns to step S305 and repeats the processes in and after step S305. If it is determined in step S309 that the event has ended, the CPU 301 performs event end processing such as disconnecting the connection with the support server 2 (step S310), and ends this processing routine.

  As described above, in this embodiment, the virtual participant 7 can view the scene of the event being held at the event venue 6 as a 3D stereoscopic image on the display device 11 and also can output the event sound to the multi-surround sound. Can be heard as. Therefore, the virtual participant 7 can view the scene of the event with a sense of realism, and can have a sense of virtually participating in the event.

  In addition, in this embodiment, not only viewing the image and sound of the event with the support image / sound information, it is as if actually going to the event venue and accepting it, and actually participating in the event such as a speech. You can get a feeling like that.

  Further, since the virtual participant is displayed as a hologram in the venue side device of the event venue 6, the event participant of the event venue 6 can surely recognize the virtual participant's virtual participation in the event. .

  In the first embodiment described above, the hologram of the virtual participant 7 is a hologram of a still image of the virtual participant 7 prepared in advance. However, an imaging device is provided in the participant's device 1 so that image information captured by the imaging device is always supplied to the support server 2 during the event holding time, and the support server 2 Hologram image data may be generated in real time and transmitted to the venue side apparatus 1 in real time.

  In the above-described embodiment, the support image / sound information for reception is supplied to the participant processing unit 10 instead of the real-time image / sound information of the event venue based on an instruction from the operator. It can also be configured to automatically switch according to the time according to the event holding schedule.

  In the above-described embodiment, the event is a wedding, but the present invention can also be applied to funerals, concerts, meetings, and the like. For example, in the case of a funeral, the support server can provide support image / speech information about incense in addition to the support image / speech information of the reception to the participant processing device. In that case, in the support image of the burning incense action, the support server generates the support image so that the avatar moves according to the burning action of the virtual participant.

  If the event venue is a concert venue, the support server generates support image / audio information to support sales of CDs and DVDs after breaks and concerts, as well as support image / audio information for admission. Can also be provided to virtual participants. In that case, in the sale of CDs and DVDs, the settlement process for the virtual participant can be performed in the same manner as in the above-described embodiment.

[Second Embodiment]
The first embodiment is a system that supports virtual participation of a virtual participant in the event when the event is actually performed in an event venue set up in a real space. In a case where a virtual event is performed at the created event venue, a system that supports virtual participation of virtual participants can be provided. The second embodiment is an example in that case.

  An example of the configuration of the event participation support system of the second embodiment is shown in FIG. That is, in the second embodiment, a plurality of participant devices 1 are connected to the event participation support server 20 through the communication network 4. Since the event is a virtual event, the venue-side device 3 is not provided in the second embodiment.

  In the second embodiment, the participant apparatus 1 is configured in exactly the same manner as the first embodiment described above, and the participant processing apparatus 10, the display device 11, the operation unit 12, A microphone 13 is provided.

  Unlike the first embodiment, the support server 20 does not require information communication with the venue side device 3 through the communication network 4. FIG. 13 shows a block diagram of a configuration example of the support server 20 in the second embodiment. In FIG. 13, the same parts as those of the support server 2 of the first embodiment shown in FIG.

  That is, in the support server 20, the CPU 201, the ROM 202, the RAM 203, the communication interface 204, the virtual participant information storage unit 205, the support image / audio information generation unit are the same as the first embodiment with respect to the system bus 200. 209, a participant transmission information generation unit, a virtual participant operation analysis unit 212, and a virtual participant settlement processing unit 213 are connected.

  The reception information distribution unit 206, the reception image / audio information processing unit 207, the monitor interface 208, the transmission information generation unit 211 for the venue, the operation unit interface 214, the monitor device 215, and the operation unit 216 of the first embodiment are the second This embodiment is not provided. Instead, the support server 20 of the second embodiment is provided with a virtual event information generation holding unit 221, a clock unit 222, and a virtual event record information generation unit 223 connected to the system bus 200.

  The virtual event information generation / holding unit 221 generates virtual event information created in response to a request from a virtual event host, and stores and holds the generated virtual event information. Here, the generated virtual event information includes information on event participation points of virtual participants. The virtual event participation points include a switching point at which virtual event information is switched to support image / sound information and a superimposition point at which support image / sound information is superimposed on virtual event information. In the second embodiment, the transmission information generation unit 210 for participants switches virtual event information and support image / audio information at a switching point and controls virtual image information at a superimposition point according to control of the CPU 201. Audio information is superimposed.

  The clock unit 222 is for providing time information of year, month, day, hour, minute, and second. The virtual event record information generation unit 223 records information on the date and time of participation of the virtual participant in the virtual event, information input by the virtual participant in the virtual event, and the like in association with the virtual event identifier and the virtual participant identifier. . Then, the virtual event record information generation unit 223 generates record information including each virtual event information and virtual participant input information after the period of each virtual event ends. The recording information generated by the virtual event recording information generation unit 223 is provided to the virtual event organizer.

<Virtual Event Generation and Provision Procedure in Support Server 20>
In order to facilitate understanding, the procedure from the creation of the virtual event to the end of provision in the second embodiment will be described with reference to FIG.

  A virtual event host (hereinafter simply referred to as a requester) applies to the operation service provider of the support server 20 for a virtual event creation request (procedure T1). Next, the client tells the operation service provider of the support server 20 what kind of virtual event he / she wishes to hold, such as wedding, funeral, legal affairs, 60th birthday celebration (procedure T2).

  Next, the operating service provider of the support server 20 listens to the requester's wish regarding the virtual event holding conditions and determines the holding conditions (procedure T3). In this example, if the client is, for example, a wedding ceremony, it will be held in the form of gods, buddhas, publics, christian churches, or “in Hawaii”, “at the top of Mt. You can request selection of the ground. Also, the client can select how many event participants are to be sent and how many of the bots are to be prepared. The operation service provider of the support server 20 accepts selection of these conditions by the client, and further requests for detailed conditions such as whether the avatar should reflect the person's image or how to wear the costume. The conditions for the event will be determined according to the intentions of the participants.

  As described above, when the holding conditions for generating the virtual event are determined, the operation service provider of the support server 20 generates virtual event information in the support server 20, for example. Of course, the virtual event information may be created by a dedicated device other than the support server 20. Then, the operating service provider of the support server 20 stores and holds the generated virtual event information in the virtual event information generation holding unit 221 of the support server 20 in association with the identifier (procedure T4). Each virtual event information stored in the virtual event information generation / holding unit 221 includes information on the holding period of each virtual event designated by the requester.

  Next, the management service provider of the support server 20 notifies the event attendees of a virtual event holding notice by e-mail or mail based on the event attendee list from the client ( Procedure T5). This virtual event holding notification includes a virtual event identifier and a virtual participant identifier of each event participant. When an event planner accesses the support server 20 from the participant-side device 1, by sending these identifiers to the support server 20, virtual participation in the corresponding virtual event becomes possible.

  Next, the support server 20 monitors information on each virtual event holding period in the virtual event information generation holding unit 221 to determine whether or not the holding period has come (procedure T6). When it is determined that the holding period has come, the support server 20 determines whether or not access from the notified virtual participant is accepted (procedure T7).

  When it is determined that the access from the virtual participant is accepted, the support server 20 provides the virtual event information whose identifier is specified by the virtual participant to the virtual participant, and the virtual participant The information input in the virtual event is recorded and held corresponding to the virtual participant identifier (step T8).

  When it is determined in step T7 that access from the virtual participant is not accepted, and after providing virtual event information to the virtual participant in step T8, the support server 20 ends the virtual event holding period. It is discriminated whether or not (step T9).

  When it is determined in step T9 that the virtual event holding period has not ended, the support server 20 returns to step T7 to monitor access by other virtual participants. When it is determined in step T9 that the virtual event holding period has ended, the support server 20 generates recording information including virtual event information and input information of all virtual participants who have participated in the virtual event. Provide to the event organizer (procedure T10).

  In addition, the virtual prospective person contacts the operation service provider of the support server 20 before the event period of the virtual event so that his / her avatar appearing when participating in the virtual event is as similar as possible to his / her own image. Can be requested. The support server 20 obtains an image of the virtual participant in response to the request from the virtual participant as described above for the avatar, and corresponds the virtual participant's avatar in advance to the virtual participant identifier. Add and save. When the support server 2 accesses the support server 20 for participation in the virtual event, the support server 2 causes the stored avatar to appear in the virtual event.

<Example of processing operation in support server 20>
15 to 17 are flowcharts of processing operation examples in the support server 20 of the second embodiment. Each step in FIGS. 15 to 17 is executed by the CPU 201 controlling each unit in accordance with the program stored in the ROM 202.

  The CPU 201 of the support server 20 waits for reception of access from the participant processing apparatus 10 (step S401), and when determining that the access from the participant processing apparatus 10 has been received, the virtual event identifier and virtual participant identifier Is generated by the participant transmission information generation unit 210 and transmitted to the participant processing apparatus 10 (step S402).

  Then, the CPU 201 receives the virtual event identifier and the virtual participant identifier sent from the participant processing device 10 (step S403), and determines the corresponding virtual event. In this example, the CPU 201 controls the support image / sound information generation unit 209 and the transmission information generation unit 210 for participants to generate and generate support image / sound information for reception at the event venue. The support image / sound information is transmitted to the participant processing apparatus 10 (step S404).

  Next, CPU201 discriminate | determines whether the operation input information of the virtual participant 7 was received from the participant side processing apparatus 10 (step S405).

  If it is determined in step S405 that the operation input information of the virtual participant 7 has been received, the CPU 201 causes the virtual participant operation analysis unit 212 to analyze the received operation input information. The virtual participant operation analysis unit 212 supplies the analysis result to the support image / audio information generation unit 209 and the virtual event record information generation unit 223. The support image / audio information generation unit 209 reflects the operation input information of the virtual participant 7 in the support image, such as controlling the movement of the avatar according to the operation input information of the virtual participant 7. In addition, the virtual event record information generation unit 223 stores the participation date / time information of the virtual participant 7, book entry information input by the virtual participant 7, and the like in association with the virtual participant identifier (step S 406).

  When it is determined in step S405 that the received information is not the operation input information of the virtual participant 7, and next to step S406, the CPU 201 executes the support image / sound information generation unit 209 for the support image / sound information. It is determined whether or not the generation process has been completed (step S407). If it is determined that the generation process has not been completed, the process returns to step S404, and the processes after step S404 are repeated.

  When it is determined in step S407 that the generation processing of the support image / speech information has been completed, the CPU 201 determines from the analysis result of the virtual participant operation analysis unit 212 that the virtual participant during the process using the support image / speech information. It is determined whether or not there has been a virtual money delivery action according to 7 (step S411 in FIG. 16). When it is determined in this step S411 that there has been a virtual money delivery action by the virtual participant 7, the CPU 201 performs a settlement process for the delivery of money for the virtual participant 7 to the virtual participant settlement processing unit 213. A control instruction is issued (step S412).

  Next, the CPU 201 starts the process of reading the virtual event information of the virtual event identifier instructed by the virtual participant from the virtual event information generation / holding unit 221 and transmitting the virtual event information to the participant processing apparatus 10 through the communication network 4. (Step S413).

  Then, the CPU 201 waits for an event participation point of the virtual participant in the virtual event information being transmitted (step S414).

  When it is determined in step S414 that the event participation point of the virtual participant has not been reached, the CPU 201 determines whether or not the virtual event has been executed and ended (step S424 in FIG. 17). When it is determined in step S424 that the virtual event has ended, the CPU 201 stops transmission of the virtual event information to the participant processing apparatus 10 (step S425), and ends this processing routine. If it is determined in step S424 that the virtual event has not ended, the CPU 201 returns to step S414.

  When it is determined in step S414 that the event has become an event participation point, the CPU 201 transmits support image / audio information to the participant-side processing device 10 instead of virtual event information at the switching point, and virtual event information at the superimposition point. The support image / audio information is superimposed on the image, and the support image / audio information is transmitted to the participant processing apparatus 10 (step S415).

  Next, CPU201 discriminate | determines whether the operation input information of the virtual participant 7 was received from the participant side processing apparatus 10 (step S416).

  If it is determined in step S416 that the operation input information of the virtual participant 7 has been received, the CPU 201 causes the virtual participant operation analysis unit 212 to analyze the received operation input information. The virtual participant operation analysis unit 212 supplies the analysis result to the support image / audio information generation unit 209 and the virtual event record information generation unit 223. The support image / audio information generation unit 209 reflects the operation input information of the virtual participant 7 in the support image, such as controlling the movement of the avatar according to the operation input information of the virtual participant 7. Further, the virtual event record information generation unit 223 stores the book entry information and the like input by the virtual participant 7 in association with the virtual participant identifier (step S417).

  When it is determined in step S416 that the received information is not the operation input information of the virtual participant 7, and after step S417, the CPU 201 executes the support image / audio information generation in the support image / audio information generation unit 209. It is determined whether or not the generation process has been completed (step S418). If it is determined that the generation process has not been completed, the process returns to step S415, and the processes after step S415 are repeated.

  When it is determined in step S418 that the generation processing of the support image / speech information has been completed, the CPU 201 determines that the virtual participant during the process using the support image / speech information from the analysis result of the virtual participant operation analysis unit 212. It is determined whether or not there has been a virtual money delivery action by No. 7 (step S421 in FIG. 17). In step S421, when it is determined that there has been a virtual money delivery action by the virtual participant 7, the CPU 201 performs a settlement process for the delivery of money for the virtual participant 7 to the virtual participant settlement processing unit 213. The control is instructed (step S422).

  For example, when the virtual event is a concert, as described above, in the support server, in addition to support image / sound information for admission, support images / sounds for supporting sales of CDs and DVDs after breaks and concerts. Generate information and provide it to virtual participants. In that case, when the virtual participant purchases a CD or DVD, a settlement process is performed for the virtual participant, and therefore the settlement process is performed in steps S421 and S422.

  Next, the CPU 201 stops reading support image / speech information from the support image / speech information generation unit 229. Then, the virtual event information from the virtual event information generation and holding unit 221 is transmitted to the participant processing apparatus 10 through the communication network 4 (step S423).

  Next, the CPU 201 determines whether or not the virtual event has been executed to the end (step S424), and when determining that the virtual event has not ended, the CPU 201 returns to step S414. If it is determined in step S424 that the virtual event has ended, the transmission of the virtual event information to the participant processing apparatus 10 is stopped (step S425), and this processing routine is ended.

<Example of Processing Operation of Participant Processing Device 10>
The processing operation of the participant processing apparatus 10 when receiving a virtual event from the support server 20 in the procedure T8 is different only in that the event information received from the support server 20 is virtual event information. This is similar to the first embodiment described with reference to FIG. Therefore, in the second embodiment, the description of the processing operation in the participant processing apparatus 10 is omitted.

  According to the second embodiment described above, the virtual participant can go to the virtual event venue, accept it, and obtain a feeling as if he / she actually participated in a virtual event such as speech.

[Other Embodiments or Modifications]
In the first and second embodiments described above, the participant side device 1 is configured using a personal computer, but a portable device such as a mobile phone terminal can also be used as the participant side device.

  Further, in the first embodiment, the imaging device provided in the venue side device 3 uses a 3D camera so that virtual participants can experience the presence of the event venue more. However, the imaging device is limited to this. Of course, a normal 2D camera may be used.

  Also, the event venue audio in the first embodiment is not limited to multi-surround audio, and may be 2-channel stereo audio or monaural audio.

  In the first embodiment, the event information in the real space is provided to the participant side device as live event information. However, the event information recorded in the real space is recorded in the participant side device. You may make it provide.

  In the above-described embodiment, the event information includes both image information and audio information. However, only the image information may be included.

  DESCRIPTION OF SYMBOLS 1 ... Participant side apparatus, 2,20 ... Event participation support server, 3 ... Venue side apparatus, 4 ... Communication network, 10 ... Participant side processing apparatus, 11 ... Display apparatus, 12 ... Microphone, 13 ... Operation part, 30 ... Venue side processing device, 31 ... Imaging device, 32 ... Microphone, 33 ... Hologram projection device, 34 ... Speaker, 35 ... Hologram

Claims (8)

An event participation support system comprising an event participation support server and a participant side device that is provided on the side of a participant who virtually participates in an event and is connected to the event participation support server through a communication network,
The event participation support server is:
Event image transmission means for transmitting at least image information of the event to the participant side device through the communication network;
Support image information generating means for generating image information of a support image for supporting predetermined virtual participation of the participant in the event;
Instead of the image information of the event, or superimposed on the image information of the event, support image transmission means for transmitting the image information of the support image to the participant side device through the communication network;
Corresponding processing means for receiving input information input by the participant corresponding to the support image from the participant side device, and executing processing according to the input information;
With
The participant side device is:
Display means for displaying the image information of the event received through the communication network and the image of the support image information for the participant to view;
Accepting means for accepting the input information inputted by the participant corresponding to the support image displayed on the display means;
Input information transmitting means for transmitting the input information received by the receiving means to the event participation support server through the communication network;
An event participation support system. In the event participation support system according to claim 1,
A venue-side device that is installed at the event venue and connected to the event participation support server through the communication network;
The venue-side device has an imaging means for imaging actually performed at the event venue,
Means for transmitting captured image information captured by the imaging means to the event participation support server through the communication network;
The event image transmission means of the event participation support server transmits the image information received from the venue side device to the participant side device through the communication network. In the event participation support system according to claim 1 or 2,
The event participation support server includes event image information generation means for virtual event image information,
The event image transmission unit of the event participation support server transmits the image information of the virtual event generated by the event image information generation unit to the participant side device through the communication network. In the event participation support system according to any one of claims 1 to 3,
The input information input by the participant includes information on virtual delivery of money in the event,
The event participation support system, wherein the response processing means of the event participation support server performs processing for charging money to the participant in the virtual delivery of money based on the input information. In the event participation support system according to any one of claims 1 to 4,
The input information input by the participant is operation input information representing the operation of the participant,
The response processing means of the event participation support server displays a person image as a part of the participant in the support image, and also according to the operation input information received from the participant side device. An event participation support system that moves a person image as a substitute. In the event participation support system according to any one of claims 2 to 5,
The accepting means of the participant side device comprises sound collecting means for acquiring a voice signal of the voice of the participant as the input information,
The response processing means of the event support server performs processing of transmitting the audio signal from the participant side device to the venue side device through the communication network,
The event site support system, wherein the venue side device includes means for acoustically reproducing the audio signal transmitted from the event support server. In the event participation support system in any one of Claims 1-6,
The accepting means of the participant side device comprises sound collecting means for acquiring a voice signal of the voice of the participant as the input information,
The event participation support system in which the response processing means of the event support server receives and stores the audio signal from the participant side device. An event participation support server connected through a communication network with a participant side device provided on the side of a participant who virtually participates in an event,
Event image transmission means for transmitting at least image information of the event to the participant side device through the communication network;
Support image information generating means for generating image information of a support image for supporting predetermined virtual participation of the participant in the event;
Instead of the image information of the event, or superimposed on the image information of the event, support image transmission means for transmitting the image information of the support image to the participant side device through the communication network;
Corresponding processing means for receiving input information input by the participant corresponding to the support image from the participant side device and executing processing according to the input information;
An event participation support server comprising:

Images