JP2003333572A - Virtual customer forming apparatus and method thereof, virtual customer forming reception apparatus and method thereof, and virtual customer forming program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a virtual customer forming apparatus in which a virtual conference configuration is formed as if many viewers were gathered in a studio to participate in a viewer participating program, so that the program is enriched. <P>SOLUTION: A video compositing apparatus 12 composites a viewer's video image as a virtual customer transmitted from the viewer and received by a receiver 11 with respect to a site video image picked up by a bird's-eye camera 13 and a studio camera 14, a mixer 18 puts together a site sound picked up by a studio microphone 17 with viewer's voice transmitted from the site and received by the receiver 11 to form as virtual customer's voice, and a transmitter 19 transmits the viewer's image and the viewer's voice to the viewer. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention allows a viewer to virtually participate in a shooting site as a virtual audience by synthesizing the viewer video and the viewer audio of the viewer with the site video and the site audio of the shooting site. The present invention relates to a virtual spectator formation device and method, a virtual spectator formation reception device and method, and a virtual spectator formation program.

[0002]

2. Description of the Related Art Conventionally, in a viewer participation type program in which a viewer participates, when the viewer goes to a shooting place,
There are cases where the performer talks to the viewer by telephone (including videophone) from the shooting location of the program in which the performer appears, when taking the form of participation by facsimile, or when taking the form of participation by data broadcasting. is there. Furthermore, in a viewer participation type program, as a large-scale format, a relay crew is dispatched to the home of the target viewer to connect with the studio so that the viewer can participate in the program. Things are also going on.

[0003]

However, in the conventional viewer participation type program, it has been desired to solve the following problems. (1) When the viewer actually goes to the shooting location,
When participating in the program, it took a lot of time and money to go to the shooting location, so it was not possible for anyone to participate casually.

(2) In the program form in which the viewer participates in the program by transmitting information by telephone conversation or facsimile, the expression and appearance of the other party cannot be recognized, so that there is a limit to the reproduction of the realistic sensation. . In addition, with the facsimile, the viewer could not obtain a real-time reaction.

(3) In the program form in which the viewer participates in the program by data broadcasting, since the program participates in the program by transmitting / receiving information such as characters from the viewer, the sense of presence in the studio is present. Cannot be reproduced, of course,
It was difficult to represent the individual identities of the participating viewers.

(4) The number of viewers who can participate in the program is limited in the live program in the program form in which the viewer participates because the performers visit the viewer's house with the relay crew. In addition, dispatching a relay crew is a big problem, and it takes a lot of money and time.

The present invention was devised in view of the above-mentioned problems, and the video and audio effects are enhanced by effectively utilizing the image processing technology, and a large number of viewers gather in a studio to participate. PROBLEM TO BE SOLVED: To provide a virtual audience formation apparatus and method, a virtual audience formation receiving apparatus and method, and a virtual audience formation program for forming a virtual audience like this and enriching viewer participation type programs. .

[0008]

In order to solve the above problems, a virtual spectator forming apparatus according to the present invention includes a station side image pickup means for picking up a spot image of a shooting spot, and a spot image from the station side image pickup means. An image synthesizing device for synthesizing and outputting the viewer video of the viewer transmitted from the viewer side, a station-side sound collecting means for collecting the on-site sound of the shooting site, and a sound collecting means from the station-side sound collecting means. A voice synthesizer for synthesizing and outputting the on-site voice and the viewer voice of the viewer transmitted from the viewer side is configured.

With such a configuration, the virtual spectator forming apparatus captures the site image of the shooting site such as a studio by the station side image pickup means, and collects the site voice by the station side sound collection means. The viewer video and the viewer audio transmitted from the viewer are synthesized by the video synthesizer and the audio synthesizer, respectively, to generate a virtual spectator video from the viewer video and to generate a virtual spectator sound from the viewer voice. ing. Then, in the virtual spectator formation device, as the output of the combined video and audio, for example, it may be transmitted to the viewer side as a station side signal. Of course, in this virtual spectator formation device, video and audio may be output to the transmission device, and the transmission device may multiplex and transmit as a station side signal.

Further, in the virtual spectator forming device according to the present invention, the image synthesizing device synthesizes the viewer image with a scene image from the station side image pickup means as a virtual spectator image in which a virtual spectator is arranged. The voice synthesizer is configured to synthesize the viewer voice as a virtual spectator voice of the virtual spectator with respect to the on-site voice from the station side sound collecting means. With such a configuration, the virtual audience formation device can combine the viewer video with the scene video to provide a state in which the viewer is participating as an audience at the shooting site.

Further, in the virtual spectator forming device according to the present invention, the voice synthesizing device provides a speech path for synthesizing the viewer voice with the on-site voice, and a mutual conversation voice between the virtual spectator voices. The communication path switching means for switching between the communication path and the communication path is used. With such a configuration, the virtual spectator forming device can switch to the call path in which the viewers can talk with each other by the call path switching means.

The virtual audience formation method according to the present invention is
The step of collecting the on-site video of the shooting site of the station side and collecting the on-site audio of the shooting site of the station side, and the viewer side consisting of the viewer video and the viewer audio of the viewer transmitted from the viewer side And a step of receiving a signal, synthesizing a viewer image included in the viewer-side signal as a virtual audience image of the scene image, and adding the viewer sound included in the viewer-side signal to the site audience sound. And a step of synthesizing as a virtual audience voice.

With such a configuration, in the virtual spectator formation method, first, an image pickup means such as a television camera is used to pick up an image of an on-site image of an on-site image taken by a station, for example, in a studio, and the on-site voice is collected. The sound is collected by a sound collecting means such as a microphone, and the viewer image and the viewer sound sent from the viewer side are received. It should be noted that the viewer video and the viewer audio are related to a predetermined number of viewers selected in advance. Then, the viewer video is combined with the scene video so as to be a virtual spectator video, and the viewer sound is combined with the scene sound so as to be a virtual spectator sound of the virtual spectator.

Further, the virtual audience formation receiving apparatus according to the present invention comprises a viewer side image pickup means for picking up a viewer video of a viewer, and a viewer side sound pickup means for collecting a viewer's voice. Transmitting means for transmitting the viewer image from the viewer-side image pickup means and the viewer sound from the viewer-side sound collecting means to the station side as a viewer-side signal; A viewer side, which is received as a station side signal generated by synthesizing a viewer image and a viewer sound included in the viewer side signal as a virtual audience image and a virtual audience sound with respect to the scene image and the site sound, respectively. And a receiving means.

With this configuration, the virtual spectator formation receiver receives the viewer image and the viewer image obtained by the viewer-side image pickup means and the viewer-side sound collection means via the transmission means such as the Internet. The viewer voice is transmitted to the station side as a viewer side signal. A station-side signal generated by synthesizing a viewer-side image and a viewer-sound included in the viewer-side signal as a virtual spectator image and a virtual spectator sound with respect to a scene-side image and a site-sound of a shooting side on the station side. By receiving, the viewer, who has been selected as a virtual audience by the station side in advance, can recognize his / her own video on the display screen of the television or the like.

Further, in the virtual spectator formation receiving apparatus according to the present invention, a speech path for synthesizing the viewer's voice with the on-site speech to become a virtual audience sound, and a speech path for an interactive speech between the virtual audience sounds. The switching operation means for operating the communication path switching means on the station side for switching between and is used.

With such a configuration, in the virtual spectator formation receiving device, by operating the speech path switching means, the speech path switching means for switching the speech path set on the station side is operated. It is possible to have a conversation with a specific viewer who can be recognized as a virtual audience.

Further, the virtual audience formation receiving method according to the present invention captures the viewer video of the viewer and collects the viewer audio of the viewer to view the viewer video and the viewer audio. Transmitting to the station side as a person-side signal, a virtual audience image in which the viewer image is combined as a virtual audience with the scene image of the shooting site of the station side, and the viewer audio to the site sound of the shooting site. Receiving as a station-side signal including a virtual audience voice synthesized as the voice of the virtual audience; displaying video and audio by the received station-side signal on a display device on the viewer side; A switching operation means installed on the viewer side is provided with call path switching means for switching between a call path which is a virtual spectator voice by synthesizing a voice with the on-site voice and a call path which is a mutual conversation voice between the virtual spectator voices. It was to include a step of further switching operation.

With such a configuration, in the virtual spectator formation receiving method, the virtual spectator video synthesized from the scene video and the viewer video on the viewer side, the virtual spectator audio synthesized from the site sound and the viewer voice. Can be received and displayed, and if necessary, conversation can be held between adjacent virtual spectators.

In the virtual spectator formation program according to the present invention, a computer is provided to realize a virtual spectator by synthesizing the viewer video of the viewer with the scene video of the shooting site on the station side. An image synthesizing unit for synthesizing the viewer image of the viewer received by the station side as a virtual audience on the scene image, and the viewer received by the station side on the site sound of the shooting site corresponding to the image synthesizing unit. Voice synthesizing means for synthesizing the viewer's voice as a virtual spectator voice of a virtual spectator,
Configured to run as.

With this configuration, the computer is caused to form a virtual spectator video from the studio video and the viewer's video, and a virtual spectator voice is generated from the studio audio and the viewer's audio. It is possible to realize a virtual audience formation program that can be transmitted to the user.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION A virtual spectator formation device and a virtual spectator formation reception device according to the present invention will be described in detail below with reference to the accompanying drawings.

The overall system configuration of the present invention is shown in FIG. The overall system configuration of the present invention includes a virtual audience formation device 1 installed on the station side and virtual audience formation reception devices 2a, 2b ... 2n installed on the viewer side. As will be described later, in the virtual audience formation receiving devices 2a, 2b ... 2n, the viewer video and the viewer audio of each viewer are sent to the station side as viewer side signals. The virtual spectator formation device 1 on the station side synthesizes the viewer video and the viewer audio of these individual viewers to create a virtual spectator, and a performer image that is a scene video actually shot by the camera on the station side. And, when the sound is actually picked up by the microphone on the station side, it is synthesized with the performer voice which is the on-site voice to create a viewer participation program and broadcast it to the viewer as a station side signal.

An embodiment of the virtual audience formation apparatus and the virtual audience formation reception apparatus of the present invention is shown in FIGS. 2 and 3. Figure 2
FIG. 3 is a block diagram showing a configuration of a virtual audience forming device. FIG. 3 is a block diagram showing the configuration of the virtual audience formation receiving device. It should be noted that, in the case where each component is generally shown on the viewer side, the virtual audience formation receiving device 2, the television camera 21 as the viewer image pickup means, the microphone 22 as the viewer sound collection means, the transmission means 23, the viewer side. The receiving means 24, the display device 25, the switching operation means 26, etc., are shown individually,
An explanation will be given by adding alphabetic letters a, b ... N after each symbol.

As shown in FIG. 2, the virtual audience forming apparatus 1
Is a receiving device 11 that receives a viewer signal from the virtual audience formation receiving device 2 on the viewer side, and a virtual audience formation receiving device 2
An image synthesizing device 1 for synthesizing a plurality of viewer images from the above, an image of the bird's-eye view camera 13 overlooking the studio, or an image of the studio camera 14 capturing the image of the studio performer.
2, an image switcher 15 for selectively outputting the output image of the image synthesizing device 12 or the image of the studio camera 14, a switching device 16 for controlling the image switcher 15, and a studio microphone 1 for collecting studio sound.
7, a plurality of viewer sounds from the virtual spectator formation receiving device 2, a mixer 18 for synthesizing with the on-site sound of the studio microphone 17, an output of the image switcher 15 and an output of the mixer 18 are transmitted to the virtual spectator formation receiving device 2. Transmitting device 19
It is configured to include and. Here, the overhead camera 13 and the studio camera 14 correspond to the station-side image pickup means, the studio microphone 17 corresponds to the station-side sound collection means, the mixer 18 corresponds to the voice synthesizer, and the transmitter 19 corresponds to the output means.

The receiving device 11 of the virtual spectator forming device 1 is a device for receiving and decoding the coded video signal and audio signal sent from the viewer side as data communication which is the viewer side signal. An example is a personal computer or workstation connected to the viewer side via the Internet.

The image synthesizing device 12 is a device which can synthesize images and images in a natural manner and combine them into one image. For example, the image synthesizing device 12 has photo retouching software and has an image processing function and a computer graphic. It can be realized with a personal computer or workstation with functions. In addition, here, a virtual audience is generated by synthesizing a viewer image with a blue background installed in a studio.

The image switcher 15 is for switching a plurality of camera images, and the one conventionally used in a broadcasting station can be used as it is. Further, as the mixer 18, a conventional audio mixer capable of synthesizing a plurality of audio inputs while balancing the volume can be used as it is.

Although it has been described here that the transmitting device 19 can already multiplex and transmit the site image and the viewer image and the site sound and the viewer sound as the station side signal, the transmitting device 19 It may be for outputting to a device that can be transmitted as a station side signal.

As for the transmission device 19 of the virtual spectator formation device 1, for a general screen, for example, a broadcasting facility such as terrestrial broadcasting or satellite broadcasting is used as the transmission device 19 to send a station side signal to the viewer side. it can. When transmitting an individual signal from the individual recipient proximity information extraction device 41 (41a to 41n) (see FIG. 8) described below, the station side signal must be sent as data communication to an individual line such as the Internet for each viewer. Therefore, it is necessary to use a personal computer or a workstation as the transmitting device 19, and cooperation with broadcasting is required.

As shown in FIG. 3, the virtual audience formation receiving device 2 includes a television camera 21 for picking up an image on the viewer side,
The microphone 22 that collects the audio from the viewer side, the transmission unit 23 that transmits the video output from the television camera 21 and the audio output from the microphone 22 to the virtual spectator formation device 1, and the signal from the virtual spectator formation device 1 Viewer side receiving means 24 for receiving
And a display device 25 for displaying the output video of the viewer-side receiving means 24 and reproducing the output sound. The television camera 21 corresponds to the viewer side image pickup means, and the microphone 22 corresponds to the viewer side sound collection means.

The television camera 2 of the virtual audience formation receiving device 2
The 1 and the microphone 22 may each be a single device or a portable video camera device or the like. The transmission means 23 encodes and transmits the viewer video and the viewer audio, which are the viewer side signals, as a data signal, and a personal computer or the like connected to the station side via the Internet is one example.

Viewer side receiving means 24 for receiving the station side signal.
As the display device 25, a normal television receiver can be used when receiving a general screen. A personal computer or the like connected to the station side via the Internet is used for individual signal reception.

Next, the process of synthesizing the viewer side information will be described with reference to FIGS. 1 and 4. In the virtual audience formation receiver 2 on the viewer side, a TV camera 21 and a microphone 22 are set near the TV screen of each viewer, and when the viewer wants to participate in the program, he / she informs the station side that the viewer wants to participate. It is transmitted by communication means. Then, when the bureau side informs of the participation winning, it is possible to participate in a specific program as a virtual audience. Further, the viewers who have decided to participate take a picture of the viewer image and the viewer's voice of themselves or their family (in the case of one person, only himself) with the TV camera 21 and the microphone 22 in accordance with the number of participants notified from the station side. Sound is collected, coded through the transmission means 23, and transmitted to the station.

In the virtual spectator forming device 1 on the station side, the information sent from the transmitting means 23a to 23n of each viewer is decoded by the receiving device 11 to be separated into an image and a sound, and each viewer image is image-synthesized. Each viewer voice input to the device 12 is input to the mixer (voice synthesizer) 18.

The video synthesizing device 12 extracts a person's face or whole / half body image from the sent viewer image from the background and synthesizes it with the on-site video so as to be a spectator corresponding to the program in the studio, As shown in FIG. 5, a virtual spectator image, which is a virtual image as if the spectator seat is being imaged, is generated. Even if the video synthesizing device 12 has a configuration in which viewer images are arranged and arranged in a predetermined order,
It suffices that the viewer images are combined with the scene image so as to be a virtual audience, for example, by deciding only the arrangement place to be aligned and arranging the viewer image at the arrangement place. The virtual audience video from the video synthesizer 12 is output to the image switcher 15 (see FIG. 2) and the like.

As shown in FIG. 2, the image switcher 15
A switching device 16 for controlling the virtual audience seats is switched and shown between camera images capturing the program performers, and is broadcast to a studio or a hall as if the audience had actually seen it. be able to. This virtual spectator video can be distributed to virtual 1st floor and 2nd floor seats, like a hall, or by zooming some spectators and panning the cameras that capture images between spectators and the stage. It is also possible to show an image. Note that instead of the face image captured by the camera, it is possible to display a deformed illustration or a character selected or created by the viewer.

The mixer 18 has essentially the same operation as a conventional audio mixer, but the voices of the respective viewers are combined by adding a reverberation sound, etc., and synthesized, as if from a large audience from the audience seats. Can be processed like voices and sounds, or when only two viewers need to interact with each other, only the audio of any viewer can be picked up. Furthermore, the voice of the viewer can be transformed into a voice that only the user can understand and reproduced.

In the above-described embodiment, the station side can use the image of each viewer as a virtual audience, but the viewer side does not know when he / she will appear. To that extent, the sense of realism is impaired and the viewer is anxious, so it is possible to eliminate this by using the configuration shown below.

As shown in FIGS. 6 and 2, in the second embodiment of the present invention, the image of the bird's-eye view camera 13 for taking a bird's-eye view of the studio and the image of the studio camera used in the studio or the hall. (Real camera) 14 images,
The image of the virtual camera 31 created by computer graphics or the like for photographing the virtual audience is synthesized by the video synthesis device 12 and transmitted to the viewer side via the transmission device 19. FIG. 7 shows an example of a video image of the overhead camera 13 at this time.

As shown in FIG. 3, on the viewer side, the video sent in this way is demodulated by the viewer side receiving means 24 and displayed on the display device 25. In the method of transmitting and receiving the video shown in FIG. 7, the image signal may be transmitted as it is or may be compressed and multiplexed and transmitted. Of course, it is also possible to transmit using another line such as the Internet. At the home of each viewer, it is possible to know where they are in the virtual spectator from the image shown in FIG. 7, and to predict in advance the time when one will be reflected from the movement of the virtual camera 31 in the image. It is possible to prepare for shooting video.

Next, a third embodiment of the present invention will be described mainly using FIGS. 7 to 8. In the third embodiment of the present invention, the specific extraction area (neighbor information extraction area) of the virtual audience is displayed by the individual recipient vicinity information extraction device 41. This individual recipient neighborhood information extraction device 4
From the outputs of the video synthesizing device 12 and the mixer 18, reference numeral 1 denotes each individual viewer (temporary viewer (audience A1)) and a spectator (viewer (audience A2) and viewer (spectator AN) adjacent to the viewer. )), Such as the voice, the image / sound information in the vicinity of the viewer, for example, in the portion that is the vicinity information extraction area in FIG. 7, is extracted, and the transmitter 19 and the viewer-side receiver 24 are connected. After that, it is displayed on the display device 25 of each individual viewer's house.

The individual-receiver vicinity information extraction device 41 is installed corresponding to each viewer who is a virtual spectator in FIG. 8 in order to extract a video centering on the viewer, but to some extent. When extracting the range of and transmitting it to the viewers in the range, it is not necessary to install it for each viewer. In this way, each individual viewer can see the surrounding spectators and hear voices such as shouting and speaking, as if they were watching directly in the studio or hall. The size of the neighborhood information extraction area can be selected within a certain range.

Furthermore, in the fourth embodiment of the present invention, in addition to the third embodiment described above, each individual viewer (virtual audience forming receiver) has a microphone / speaker 44a-44n.
And a switching operation means 26 (for switching connection of the communication path by the switching device (communication path switching means) 42 on the station side via the relay devices 43a to 43n on the station side. (See FIGS. 1 and 9).

The exchange device 42 is installed on the station side, and has a communication path for synthesizing the viewer's voice sent from the viewer as a virtual spectator voice, and an interactive conversation voice with which the virtual spectators can talk. It is for switching the communication path. Further, as shown in FIGS. 8 and 9, the switching operation means 26 is for performing the switching operation of the exchange device 42 by the operation of the viewer, and here, the virtual adjacent to each other on the image depending on the rotation angle of the switch. It is designed to allow conversation between the audience. If you select a channel for mutual conversation voice between virtual spectators, you can exchange opinions as if you were actually speaking with the next spectator in the theater, with the same spectators sitting next to each other as virtual spectators or sitting near you. And can exchange conversations.

That is, the viewer has, for example, a switch of the switching operation means 26 having a panel surface as shown in FIG. 9, and the viewer A2 sends his voice to the studio.
It is possible to select whether to deliver to the viewer AN on the left side or the viewer A1 on the right side by switching the switch.

Switching of the switch of the switching operation means 26 is performed by A
When set to N, a conversation signal is sent to the viewer AN on the left side, for example, as a voice signal. When AN agrees with the conversation and pushes AN's switch to the right, the viewer A2 hears the voice of AN on the left side, and the AN hears the story of the viewer A1. Similarly, if the switch is set to A1, if A1's consent is obtained, A on the right side
I can hear the voice of Mr. 1 and have a conversation.

In this case, the switching device 42 has the relay device 43.
Voices are switched via a to 43n. At the same time, the exchange device 42 sends the information of the current switching state to the video synthesizing device 12. Therefore, viewer A2 and Mr. AN,
If A1's conversation is taking place, video synthesizer 1
2 is a state in which a video image of a virtual audience having a conversation is attached by synthesizing a peripheral monitor image of the viewer A2 or the like with a part of a main image in which a performer as shown in FIG. 10 appears. It can also work as it can be sent to the viewer. At this time, when a plurality of virtual spectators are having a conversation with each other, the image switcher 15 (see FIG. 2) switches the monitor images within a predetermined time period at a fixed time. The video synthesizer 12
If output means for outputting a monitor image to each viewer-side terminal device (not shown) is provided via the network, the monitor images of the virtual spectators are individually transmitted to the parties having conversation. It is also possible to do.

The virtual spectator forming device and the virtual spectator forming receiving device of the present invention have been described above. The virtual spectator forming method used in these devices and the virtual spectator forming for causing a computer to realize such a virtual spectator forming method. The program is also an object of the present invention.

[0050]

As described above, according to claim 1, claim 2, claim 4 and claim 8 of the present invention, the studio video imaged by the station side imaging means and the viewer transmitted from the viewer side. By synthesizing the video with a video synthesizing device and synthesizing the studio voice collected by the station side sound collecting means and the viewer's voice transmitted from the viewer side with the voice synthesizing device and transmitting, the image of the virtual audience The voice can be sent to the viewer to easily realize the viewer participation type program. Further, the virtual spectator forming device according to the third aspect has the communication channel switching means, and thus it is possible to switch the communication channels and expand the usage of the audio of the viewer.

According to claim 5, claim 6 and claim 7 of the present invention, the studio image including the studio sound generated by the station side by transmitting the viewer image and the viewer sound to the station side as a viewer side signal. By receiving the station-side signal that is a combination of the signal and the viewer-side signal, it is possible to obtain the video / audio of the virtual audience and easily participate in the viewer participation type program. Further, in claim 6 and claim 7, since the switching operation means is provided, when the viewer is a virtual audience who is a neighbor or a virtual audience sitting in the vicinity, as if he / she actually speaks with a neighbor in the theater. You can exchange opinions and exchange conversations as in.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing an overall configuration of a system according to the present invention.

FIG. 2 is a block diagram showing a configuration of a virtual spectator formation device according to the present invention.

FIG. 3 is a block diagram showing the configuration of a virtual audience formation receiving device according to the present invention.

FIG. 4 is a block diagram showing a device configuration for synthesizing viewer side information according to the present invention.

FIG. 5 is an explanatory diagram showing an example of a virtual image of a spectator seat synthesized by the image synthesizing apparatus according to the present invention.

FIG. 6 is a block diagram showing a second embodiment of a virtual spectator forming device according to the present invention.

FIG. 7 is an explanatory diagram showing an example of an image from the overhead camera according to the embodiment of FIG.

FIG. 8 is a block diagram showing third and fourth embodiments of the virtual audience formation device of the present invention.

FIG. 9 is a front view showing an example of a panel surface of a switching operation means provided to a viewer in the fourth embodiment of the virtual spectator forming device of the present invention.

FIG. 10 is a front view showing an example of a display screen of the virtual audience formation receiving device of the present invention.

[Explanation of symbols]

1 Virtual Spectator Formation Device 2 Virtual Spectator Formation Reception Device 11 Reception Device 12 Video Compositing Device 13 Overhead Camera (Station Side Imaging Means) 14 Studio Camera (Station Side Imaging Means) 15 Image Switcher 16 Switching Device (Call Path Switching Means) 17 Studio Microphone (station-side sound collecting means) 18 Mixer (speech synthesizer) 19 Transmitter (output means) 21, 21a to 21n Television camera (viewer-side image pickup means) 22, 22a to 22n Microphone (viewer-side sound collecting means) 23, 23a to 23n Transmitting means 24 Viewer side receiving means 25 Display device 26 Switching operation means 31 Virtual camera 41, 41a to 41n Individual receiver proximity information extracting device 42 Exchange device 43, 43a to 43n Relay device 44, 44a to 44n Microphone speaker

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) // H04N 5/262 H04N 5/262 (72) Inventor Toshihiro Shimizu 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Corporation Broadcasting Technology Research Institute (72) Inventor Akira Otani 1-10-11 Kinuta, Setagaya-ku, Tokyo F-Term inside Broadcasting Technology Research Institute of Japan Broadcasting Corporation (reference) 5C023 AA11 AA13 AA14 CA01 CA04 5C025 BA30 CA02 CA19 DA10 5C064 AA02 AB04 AC04 AC06 AC11 AC12 AC16 BA01 BB10 BC16 BC20 BC23 BD01 BD02 BD08

Claims (8)

[Claims] 1. A station-side image pickup means for picking up a scene image of a shooting site, and a picture output by synthesizing a scene image from the station-side image pickup means and a viewer image of a viewer transmitted from a viewer side. A synthesizer, a station-side sound collector that collects the on-site sound of the shooting site, and a site sound from this station-side sound collector and a viewer audio of the viewer transmitted from the viewer side are synthesized. A virtual audience forming apparatus, comprising: a voice synthesizing device for outputting. 2. The video synthesizing device synthesizes a scene image from the station side image pickup means as a virtual spectator image in which the viewer images are arranged as a virtual spectator, and the voice synthesizing device collects the station side sound. The virtual spectator formation device according to claim 1, wherein the viewer voice is synthesized as a virtual spectator voice of the virtual spectator with respect to the on-site voice from the means. 3. The speech synthesizing device switches a speech lane for synthesizing the viewer's voice with the on-site speech and a speech lane for forming a mutual conversation voice between the virtual audience voices. The virtual spectator formation device according to claim 2, further comprising: 4. A step of capturing an on-site video of the shooting site on the station side and collecting the on-site audio of the shooting site on the station side, and a viewer image and a viewer audio of the viewer transmitted from the viewer side. Receiving a viewer-side signal, the viewer image included in the viewer-side signal is synthesized as a virtual audience image of the scene image, and the viewer audio included in the viewer-side signal is included in the scene. Synthesizing the voice as a virtual spectator voice of the virtual spectator. 5. A viewer side image pickup means for picking up a viewer video image of a viewer, a viewer side sound pickup means for collecting a viewer voice of the viewer, and a viewer video image from the viewer side image pickup means. And a viewer sound from the viewer-side sound collecting means as a viewer-side signal to the station side, and the viewing with respect to the site image and site sound generated at the station-side shooting site. And a viewer-side receiving unit that receives as a station-side signal generated by combining a viewer video and a viewer audio included in the viewer-side signal as a virtual spectator video and a virtual spectator audio, respectively. Audience formation receiver. 6. A call path switching means on the station side for switching between a call path which is a virtual audience voice by synthesizing the viewer voice with the on-site voice and a call path which is a mutual conversation voice between the virtual audience voices. The virtual spectator formation receiving device according to claim 5, further comprising switching operation means for operating. 7. A viewer image of a viewer is captured, and
Collecting the viewer audio of the viewer and transmitting the viewer video and the viewer audio to the station side as a viewer side signal, and the viewer video in the scene video of the shooting site of the station side. Receiving as a station-side signal including a virtual spectator video synthesized as a virtual spectator, and a virtual spectator voice synthesized with the viewer's voice in the on-site voice of the shooting site as a virtual spectator's voice; A step of displaying video and audio by a station side signal on a display device on the viewer side, a speech path that synthesizes the viewer audio with the on-site audio to become a virtual audience sound, and an interactive speech between the virtual audience sounds A virtual audience formation receiving method, which comprises a step of performing a switching operation of a communication path switching means for switching a communication path to be performed by a switching operation means installed on the viewer side. 8. A computer for synthesizing a viewer image of a viewer with a site image of a shooting site on a station side to realize a virtual audience is received on the site image of a shooting site on the station side by the station side. Video synthesizing means for synthesizing the viewer video of the viewer as a virtual spectator, and corresponding to this video synthesizing means, the audio of the viewer received by the station side is added to the on-site audio of the shooting site. A virtual audience formation program characterized by being executed as a voice synthesizing means for synthesizing as a voice.