JP2001036881A - Voice transmission system and voice reproduction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reproduce a voice from an opposite party in a video conference at a prescribed sound image position. SOLUTION: Each of clients 12-18 connected to an internet 10 in this voice transmission system is provided with a voice input output means and an image input output means. A control server 20 controls connection.interruption or the like of a video conference by the clients 12-18. An image server 22 collects images from the clients 12-18 and distributes the images to the clients 12-18. A voice server 24 collects the voice from the clients 12-18 and distributes it to the clients 12-18. Furthermore, in the case of transmitting voice data to each of the clients 12-18, the voice server 24 adds voice image localization information to the voice data and transmits the resulting voice data to the clients 12-18. Each of the clients 12-18 reproduces a received voice so that it is reproduced at a voice image position according to the voice image localization information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio transmission system and an audio reproducing apparatus, and more specifically, to an audio transmission system and an audio reproducing apparatus for reproducing each transmitted audio so that its transmission source can be easily specified. About.

[0002]

2. Description of the Related Art In recent years, computer communication networks represented by the Internet have become widespread, and usage fees for public communication networks and private line networks have been reduced. The acceptance of image / voice communication using such inexpensive networks has increased, and videoconferencing devices or videoconferencing devices have exploded. In a video conference using the Internet or a public communication network, a personal computer is used as a main device. The increasing functionality of personal computers, in contrast to the declining price of personal computers, has spurred the spread of videoconferencing systems.

[0003] However, considering the current usage environment in Japan, housing is too small, and the space allocated to one person at work is also small. Therefore, the number of speakers is generally limited to two, and a large-screen video conference system is not common.

Conventionally, video conferencing systems used in the Internet, public networks and leased line networks are one-to-one or multipoint systems, both of which are widely used. The transmitting / receiving device used in the system is generally based on a personal computer. Voice is ITU-T
Recommendation G. 711, G.R. 723 and G.R. 729 and the like. On the audio transmitting side, a dedicated microphone is connected to a microphone input terminal of an audio board or the like provided in the computer to input an audio signal. The input audio signal is
According to an encoder consisting of software and / or hardware, Recommendation G. 711, G.R. 723 and G.R. 729 and the like, and are connected to a network connection device (modem, terminal adapter and digital service unit (D
SU)) to a communication network such as the Internet.

[0005] When the receiving apparatus is based on a personal computer, the received sound is monaurally output from two speakers arranged on the left and right of the monitor screen. That is, even in a video conference between multiple points, sounds from a plurality of parties are output from the same position.

[0006] Although sound image localization devices have become quite popular, most of them have been developed for audio and have not been applied to communication devices such as video conference systems. In current audio equipment, the sound image position is defined by digital signal processing of the original sound into a multi-channel signal. It is not preferable to output a multi-channel audio signal to a narrow band transmission path, and such a method cannot be adopted in a video conference system or the like.

[0007]

In a communication application in which an audio signal is transmitted after being encoded by high-efficiency compression coding, there has been no request for sound image localization in the first place. However, assuming a situation in which a plurality of people are involved in a multipoint video conference, it is difficult to determine which communication partner is the sound from the output sound from the same position. In addition, when a plurality of people talk at the same time, it becomes difficult to understand the content.

[0008] An object of the present invention is to provide an audio transmission system and an audio reproducing apparatus which have solved such inconveniences.

[0009]

A voice transmission system according to the present invention comprises: at least one transmitting device for inputting voice information and outputting the input voice information to a transmission path; A receiving device that receives the audio information and reproduces the audio indicated by the audio information in accordance with the sound image localization information added to the audio information.

[0010] The voice transmission system according to the present invention further includes one or more voice transmitting devices for inputting voice and outputting the input voice to a transmission path, and receiving each voice from the one or more voice transmitting devices. And a sound receiving device that reproduces each sound at a sound image position specified in advance for each sound transmitting device as a sound transmitting source.

The voice transmission system according to the present invention further includes a plurality of terminal devices for inputting voice information, outputting the input voice information to a transmission path, and receiving and reproducing the voice information from the transmission path. And one of the plurality of terminal devices
One is characterized in that audio information from another terminal device is reproduced at sound image positions different from each other.

An audio reproducing apparatus according to the present invention is an audio reproducing apparatus connected to a transmission path, which receives audio information from one or more transmitting apparatuses connected to the transmission path,
The sound indicated by each sound information is reproduced according to the sound image localization information added to the sound information.

[0013] An audio reproducing apparatus according to the present invention is an audio reproducing apparatus for reproducing audio information transmitted via a transmission path, comprising: receiving means for receiving information from the transmission path; And sound reproducing means for reproducing sound information included in the obtained information at a sound image position corresponding to the transmission source.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a schematic block diagram showing an embodiment of a video conference system to which the present invention is applied. Here, the Internet is used as a communication network.

Reference numeral 10 denotes the Internet, to which clients 12, 14, 1 which are terminal devices of a video conference.
6, 18; a control server 20 for controlling connection / disconnection of the video conference of this group; collecting images from clients 12 to 18 of this group;
8 and an audio server 24 that collects audio from the clients 12 to 18 of this group and distributes the audio to the clients 12 to 18.

FIG. 2 shows the state of sound image localization in the client 12. The same applies to the other clients 14 to 18. Reference numeral 30 denotes a monitor for displaying an image,
On the screen, images 32, 34, 36 of participants B, C, D (users of clients 14, 16, 18) of this video conference and an image of user A of client 12 (so-called self-portrait) are displayed. 38 is displayed. Monitor 30
The camera 40 is arranged on the camera 12 and photographs the face of the user A of the client 12. Speakers 42 and 44 are arranged on the left and right of the monitor 30.

In this embodiment, different sound image localization positions 48, 50, and 52 are set for the other participants B, C, and D in the sound image localization area 46 including the user A.

FIG. 3 shows a schematic configuration diagram of the clients 12 to 18. Monitor 30, camera 40, and speaker 4
2, 44, a keyboard 60, a terminal adapter (TA) 62 for connecting to an ISDN, and a microphone 64.
Is connected to the PC body 66.

FIG. 4 is a schematic functional block diagram of each of the clients 12 to 18. 70 is a motherboard, 72 is an audio input / output board, 74 is an audio processing board, 76 is a digital signal processing circuit (DSP), and 78 is a video capture board.

A video conference operation according to the present embodiment will be described.

For example, one of the clients (for example, the client 12)
Other clients (e.g., clients 14, 16,
18) and a request to hold a video conference. Specifically, the dedicated software is started on the client 12, and the request signal is transmitted to the I / O of the control server 20.
The P address is specified and transmitted to the control server 20. This signal reaches the control server 20 via the Internet 10.

In response to the request from the client 12, the control server 20 transmits a request for participation in the video conference to the clients 14, 16, and 18 requested to participate in the video conference. This request for participation also reaches each of the clients 14, 16, and 18 via the Internet 10. Each of the clients 14, 16, 18 notifies the control server 20 of acceptance or rejection of the participation request. Only the client 14, 16 or 18 that has given the consent will join the video conference with the client 12. Here, it is assumed that four clients 12 to 18 participate in the video conference.

An image distribution operation during a video conference will be described. The control server 20 notifies the image server 22 of the start of the multipoint video conference by specifying the participating clients 12 to 18 of the video conference. Each of the clients 12 to 18 captures the video of its own camera 40 with the video capture board 78, compresses and encodes the video, and transmits it to the image server 22.
The image server 22 transmits images from the clients 12 to 18 to clients 12 to 18 other than the transmission source via the Internet 10. For example, the image server 22
Transmits an image from the client 12 to the client 14,
16 and 18.

On the screen of the monitor 30 of each of the clients 12 to 18, a self-portrait and an image from another client are displayed as shown in FIG. The dedicated software manages which image from which client is displayed at which position.

Next, the transmission process of the audio signal will be described. In each of the clients 12 to 18, the user's voice is captured by the microphone 64, A / D converted by the voice input / output board 72, and ITU-
T Recommendation G. 729, and is encoded at 8 kbit / s. The encoded audio signal is sent to the motherboard 70, TA6
2. It is transmitted to the voice server 24 via the ISDN and the Internet 10.

The voice server 24 is provided for each of the clients 12 to
The voice from the server 18 is transmitted to the clients 12 to 18 other than the transmission source via the Internet 10. The transmission destination is the same as the image. The audio server 24 transmits a sound image localization signal to each of the clients 12 to 18 in addition to audio information from other clients. For example, client 12
When each voice from the clients 14 to 18 is transmitted to the client 12, the voice server 24 also transmits to the client 12 a signal specifying where in the sound image localization area of the user A of the client 12 to localize. The same applies to the other clients 14, 16, and 18.

The meaning of the sound image localization information for the sound image localization area 46 of each of the clients 12 to 18 is described in the audio server 2.
If you unify between 4 and each client 12-18,
Sound image localization information can be simplified.

Referring to FIG. 2, a description will be given of the case where the client 12 localizes the sound signals from the other clients 14, 16, and 18 to a sound image.

The monitor screen of the client 12 is divided into four parts, an image 32 of the user of the client 14 in the upper left window, an image 34 of the user of the client 16 in the lower left window, and an image of the user of the client 18 in the upper right window. 36, an image 38 of the user of the client 12 is displayed at the lower right. Client 12
Image 38 of the user without passing through the image server 22
The output image of the camera 40 is displayed as it is.
The image 38 of the user of the client 12 is displayed
This is because the image of the camera 40 of the own station can be confirmed.

Upon receiving the sound image localization information from the audio server 24, the client 12 receives the sound image localization area 46 of its own station.
It is determined at which position the audio signal of which client is to be localized. Then, the voice of each of the clients 14 to 18 transmitted from the voice server 24 is decoded by the voice processing board 74, and the digital signal processing circuit 76 processes the received voice so as to be at the previously determined sound image localization position. . The processed audio signal is supplied to the speakers 42 and 44 via the audio input / output board 72. For example, FIG.
As shown in FIG. 2, each sound image is such that the voice of the user of the client 14 is heard from the left of the user of the client 12, the voice of the user of the client 16 is heard from behind, and the voice of the client 18 is heard from the right. Is localized.

The position at which the sound image from which client is localized may be different for each of the clients 12-18. It is easy for each client to arbitrarily change the position where the sound image is localized.

Here, the sound image localization information and the sound processing in each client will be described in detail using a simple example. First, the voice server 24 checks each client 1
2, 14, 16, or 18, it is determined which of the left, right, and center is to be the sound image localization position for the compression-encoded audio signal from a client other than the client to be transmitted. I do. Briefly, for the three clients except the destination client, the clients 12, 1
The order may be determined to be left, center and right in the order of 4, 16, 18.

The sound image localization positions determined to be left, center and right are '10', '11' and '01', respectively.
And the like, and add 2-bit sound image localization information, and transmit the information to each client. For example, when the voice is transmitted from the voice server 24 to the client 18, the voice information from the client 12 is “10” (left), the voice information from the client 14 is “11” (center), and the voice information from the client 16 is With '01' (right).

The monaural audio signal to which the sound image localization information is added is in a state of being compression-encoded, but is decoded by the audio processing board 74 of each client as described above. Although the decoded audio signal is still a signal of one channel here, this is a stereo signal on the board. That is, when the sound image localization information is “10”, the monaural signal is set to the left (L) signal, and when the sound image localization information is “01”, the monaural signal is set to the right (R).
When the sound image localization information is “11”, the monaural signal is divided into a left (L) signal and a right (R) signal by a half.

Then, the thus decoded audio information from the three clients is added for each of the left (L) signal and the right (R) signal, and the left (L) signal and the right (R) signal thus obtained are added. The signal is time-division multiplexed and the audio input / output port 72
Output to The audio input / output port 72 synchronizes the time-division multiplexed left (L) signal and right (R) signal, and after DA conversion, outputs the left (L) signal to the speaker 42 and the right (R) signal to the speaker. 44 respectively.

As described above, even when a network having a limited band, such as the Internet network, is used and audio signals need to be encoded at a high compression rate, the multipoint video conference system can be used to transmit signals from each participant. The sound image can be localized at a desired position, and even if a plurality of people speak at the same time, it becomes easy to determine and the conversation can be smoothly performed.

It is a matter of course that general window management software may be used so that the users of the clients 12 to 18 can arrange the images of the clients at desired positions. In that case, the position of the sound image of the voice from the client also depends on the position on the monitor screen. For example, when displaying an image on the left side of the monitor screen, sound is heard from the left side, and the sound is displayed on the right side of the monitor screen. When displaying an image, it is preferable to set so that sound can be heard from the right side. From this point of view, the voice server 24
Therefore, the voice from each of the other clients 12 to 18 may be transmitted to each of the clients 12 to 18 together with the information specifying the transmission source. Each of the clients 12 to 18 may set a different sound image localization position for each of the transmission clients 12 to 18.

[0039]

As can be easily understood from the above description, according to the present invention, even when a plurality of people speak at the same time, it is easy to identify the contents spoken by each person, and the conversation can be smoothly performed. To progress.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of an embodiment of the present invention.

FIG. 2 is a schematic diagram of an example of sound image localization in the present embodiment.

FIG. 3 is a schematic configuration diagram of each of clients 12 to 18.

FIG. 4 is a schematic configuration block diagram of each of clients 12 to 18.

[Explanation of symbols]

 10: Internet 12, 14, 16, 18: Client 20: Control Server 22: Image Server 24: Audio Server 30: Monitor 32: Image from Client 14 34: Image from Client 16 36: Image from Client 18 38: Image of client 12 40: Cameras 42, 44: Speaker 46: Sound image localization area 48, 50, 52: Sound image localization position 60: Keyboard 62: Terminal adapter 64: Microphone 70: Motherboard 72: Audio input / output board 74: Audio processing board 76: Digital signal processing circuit (DSP) 78: Video capture board

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C064 AA02 AB04 AC06 AC16 AD02 AD09 5D062 AA67 5K015 AA00 AA02 AB01 AB02 JA00 JA10 JA11 5K101 KK07 LL00 LL03 MM07 NN36 SS08 TT02

Claims (22)

[Claims] 1. One or more transmitting devices for inputting voice information and outputting the input voice information to a transmission path, receiving voice information from the one or more transmitting devices, and adding the voice information to the voice information. A sound transmission system comprising: a receiving device that reproduces sound indicated by each sound information in accordance with sound image localization information. 2. The sound information from each of the transmitting devices is one-channel sound information, and the sound image localization information defines a sound image position when the one-channel sound information is reproduced by a plurality of speakers. Voice transmission system. 3. The audio transmission system according to claim 2, wherein each of the transmission devices compresses and encodes the one-channel audio information and outputs the information to the transmission path. 4. The apparatus according to claim 1, wherein the transmitting device inputs image information together with the audio information, and the receiving device receives each image information and reproduces the image information on a single monitor. Voice transmission system. 5. The receiving apparatus according to claim 4, wherein the receiving apparatus inputs image information, and the receiving apparatus reproduces each image information from each transmitting apparatus and image information from the receiving apparatus on a single monitor. An audio transmission system as described. 6. The audio transmission according to claim 1, further comprising an intermediate device for mediating audio transmission connected to the transmission path, wherein the intermediate device generates the sound image localization information. system. 7. One or more voice transmitting devices for inputting voice and outputting the input voice to a transmission path, receiving each voice from the one or more voice transmitting devices, and receiving the voice from the one or more voice transmitting devices. An audio transmission system comprising: an audio reception device that reproduces each audio at a sound image position specified in advance for each audio transmission device. 8. The audio transmission system according to claim 7, wherein the audio transmitting device and the audio receiving device are video conference terminals. 9. A transmission mediating device for mediating voice transmission from the voice transmitting device to the voice receiving device,
The audio transmission system according to claim 7, wherein the sound image position is defined by the mediation device. 10. A plurality of terminal devices each for inputting audio information, outputting the input audio information to a transmission path, and receiving and reproducing the audio information from the transmission path. An audio transmission system in which one reproduces audio information from another terminal device at different sound image positions. 11. The audio information output from each of the terminal devices is one-channel audio information, and one of the plurality of terminal devices reproduces one-channel audio information from another terminal device through a plurality of speakers. The audio transmission system according to claim 10, wherein a sound image position is different for each terminal. 12. The audio transmission system according to claim 11, wherein each of the terminal devices compresses and encodes the one-channel audio information and outputs the information to the transmission path. 13. The terminal device according to claim 10, wherein the terminal device inputs image information together with audio information, and the terminal device receives the image information from each terminal and reproduces the image information on a single monitor.
3. The audio transmission system according to any one of 2. 14. The audio transmission system according to claim 10, wherein an intermediate device for mediating audio transmission is further connected to the transmission path, and the intermediate device defines the sound image position. . 15. An audio reproducing apparatus connected to a transmission path, receiving audio information from one or more transmission apparatuses connected to the transmission path, and sound image localization information added to the audio information. A sound reproducing device that reproduces a sound indicated by each sound information in accordance with the information. 16. The sound information according to claim 15, wherein the sound information from each of the transmitting devices is one-channel sound information, and the sound image localization information defines a sound image position when the one-channel sound information is reproduced by a plurality of speakers. Audio playback device. 17. The audio reproducing apparatus according to claim 16, wherein the one-channel audio information is compression-encoded. 18. The audio according to claim 15, wherein the audio information is received together with the image information from the one or more transmitting devices, and the image information is reproduced on a single monitor. Playback device. 19. The receiving apparatus according to claim 15, wherein the receiving apparatus inputs image information, and the receiving apparatus reproduces each image information from each transmitting apparatus and image information from the receiving apparatus on a single monitor. The audio reproduction device according to the above. 20. The audio reproducing device according to claim 15, wherein an intermediary device for mediating audio transmission connected to the transmission path generates the sound image localization information. 21. An audio reproducing apparatus for reproducing audio information transmitted via a transmission line, comprising: a receiving unit for receiving information from the transmission line; and a sound included in the information received by the receiving unit. A sound reproducing device comprising: sound reproducing means for reproducing information at a sound image position corresponding to a transmission source. 22. The information indicating the sound image position of each sound information,
22. The sound reproducing device according to claim 21, wherein the sound reproducing device transmits the sound information together with the sound information via a transmission path.