JPH10290225A - Digital voice mixing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce delay and absent of a reproduced sound when voices that speakers speak are generated one over another by superposing and reproducing the voices that the speakers speak at the same time. SOLUTION: When a detecting means 14 inspects a buffer means 13 and detects voice data being stored in two buffers, a control means 17 judges that voices of terminals are spoken at the same time and instructs an output means to send the voice data out of the terminals. In a next cycle, the buffer means 13 is inspected and when there is voice data from another terminal, the output means 14 is instructed to read and send the voice data out of the buffer means 13. Thus, two voice data are sent out alternately in every fixed time and then when the voice data are reproduced at a receiving terminal, two voices can be heard artificially one over the other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital audio mixing apparatus for superimposing and reproducing voices uttered simultaneously by a plurality of persons in a wireless digital audio communication system such as a cellular phone and a wired digital audio communication system such as a LAN. The present invention relates to a device for realizing multi-party communication. Further, the present invention relates to a technique which is effective when applied to a technique in which when two or more sounds are reproduced by a terminal which has received a plurality of sounds, two sounds can be heard in an overlapping manner.

[0002]

2. Description of the Related Art In a system for realizing multi-party communication by connecting to a server via a transmission path of a plurality of terminals, a plurality of persons are required to realize the same conversation as in a real environment. It is necessary to convey the multiplex sound of all the participants at the same time to all participants. For this purpose, it is necessary to treat a plurality of voices as one voice by overlapping them.
This is because it is not practical for a terminal to reproduce a plurality of sounds at the same time.

On the other hand, PCM data is rarely transmitted as it is from the viewpoint of transmission capacity, and is generally transmitted after information compression. As a method for efficiently compressing voice PCM data with high quality, PCM
There is a method of converting data into "audio data" processed based on the audio signal processing theory. This “audio data” is not waveform information but an audio information compression parameter calculated from an audio generation mechanism.

[0004] Specifically, it is composed of a parameter expressing the shape of the throat and a parameter expressing the sound source signal. At the time of voice reproduction, a voice signal is calculated by simulating a vocalization mechanism using these parameters. Since such audio data does not directly represent an audio signal waveform, when two audio signals are added, even if these audio data are added and synthesized, the added signal waveform is not obtained. In order to add two audio waveforms, it is necessary to encode the audio data once, return it to waveform information, add it, and then decode it again to return it to audio data.
Depending on the type of audio data, some audio data can be converted so that they can be added, and the addition formula can be converted and added at the audio data level, but a complicated operation is required. In order to avoid such a large amount of calculations for the mixing process, there is also a method in a multi-participatory communication system in which people who can simultaneously speak are restricted by the system, and only those who have the right to speak are allowed to speak. However, as a matter of course, since free conversation cannot be performed with this, the system becomes unsatisfactory from the viewpoint of providing a natural conversation environment.

Next, the effect of an increase in the amount of computation for mixing on the system will be described.

[0006] Audio data continuously transmitted from the transmission path is sequentially stored in a buffer in the server, and is periodically read out at the same time to perform a mixing process. If the timing of reading the audio data from the buffer is later than the timing of storing the data, the data accumulates in the buffer more and more and the audio data is discarded after the buffer becomes full. For this reason, there is a problem that the reproduced sound is lost for the discarded audio data.

In order to prevent this, it is necessary to shorten the time required for the mixing process to be shorter than the period in which the audio data is stored in the buffer. For this reason, measures are taken to prepare a buffer size that can store audio data longer than the mixing processing time. However, since data is stored in the buffer for a longer time, a problem arises in that the delay time of reproduced sound is increased. Different.

[0008]

As described above, in the conventional method, there is a problem that a reproduced sound is delayed or lost due to a large amount of calculation due to mixing processing in the server.

An object of the present invention is to provide a technique capable of reducing delays and omissions of reproduced sounds when a plurality of voices uttered at the same time are reproduced in a superimposed manner.

Another object of the present invention is to provide a technique capable of reducing the cost of a mixing device.

The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0012]

The outline of a typical invention among the inventions disclosed in the present application will be briefly described as follows.

[0013] Data input means for taking in digital voice data of a plurality of persons from a transmission path, a plurality of data storage means for temporarily storing the taken-in voice data for each speaker, and voice data respectively stored in the plurality of data storage means. Detection means for inspecting whether or not, data output means for reading out the temporarily stored audio data and transmitting it to a specific device connected to the transmission line, and control means for controlling each of the means, This is a digital audio mixing device that reproduces voices uttered simultaneously by a plurality of persons.

According to the above-described means, when reading and transmitting temporarily stored audio data, transmission control is performed in consideration of time and uttered sound as the audio data, thereby reducing delay and loss of reproduced sound. Can be. In addition, since this enables the reproduction of the voice uttered repeatedly,
Sounds uttered by a plurality of persons at the same time can be reproduced in a superimposed manner.

[0015]

Embodiments of the present invention will be described below in detail with reference to the drawings.

In all the drawings for describing the embodiments, components having the same function are denoted by the same reference numerals, and their repeated description will be omitted.

(Embodiment 1) FIG. 1 is a block diagram showing a schematic configuration of an audio mixing apparatus according to an embodiment (Embodiment 1) of the present invention. In FIG. 1, a LAN (Local Area Network) is connected to an input terminal (a plurality of transmitting terminals) 11.
k: a local communication network), and the input means 12 receives digital voice data transmitted from a plurality of terminals connected to the data transmission path. The digital voice data of the input means 12 is sent to a specific area assigned to the buffer means 13 for each source address (FIGS. 3 to 6). That is, the input unit 12 refers to the source address included in the audio data and sends it to the buffer to which the audio data is allocated.

The buffer means 13 comprises a plurality of buffers 21 to 2n, as shown in FIGS. 1 and 2, and transmits a signal from the input terminal 11 in accordance with an instruction from a control means (for example, a central processing unit: using a CPU) 17. Buffers 21 to
A predetermined area of 2n is allocated. For example, terminal A
To the buffer 21, the audio data from the terminal B to the buffer 22, and the audio data from the terminal C to the buffer 23.

The detecting means 16 checks whether or not audio data is stored in the buffers 21 to 2n at a specified cycle, and sends the result to the control means 17. Buffers 21 and 2
If the audio data is stored in a plurality of buffers in n, the control means 17 determines that the audio has been uttered simultaneously by the plurality of terminals, and starts the mixing process. The inspection unit and the determination unit use, for example, known ordinary software, and a description thereof will be omitted.

The specific operation of the audio mixing apparatus according to the present embodiment will be described with reference to FIGS. 3 (a), 3 (b), 3 (c), 4 (a), and 4 (a).
This will be described with reference to FIGS. 5B, 5A, 5B, 6A, and 6B.

In FIG. 3A, p1, p2, p3,
.. represent the time series of audio data from the terminal P, and q1, q2, q3, q4, q5,... represent the time series of audio data from the terminal Q. As shown in FIGS. 1 and 3B, the detecting means 14 checks the buffer means 13 and stores the audio data p in the two buffers 1 and 2.
When detecting that q1 and q1 are stored, the control means 17 determines that the voices of the terminal P and the terminal Q are uttered at the same time, and as shown in FIG. 1 and FIG. The output unit 14 is instructed to transmit the data p1.
In the next cycle, as shown in FIG. 4A, the detecting means 14 checks the buffer means 13 and outputs the audio data q from the terminal Q.
2 exists, as shown in FIG.
4, the audio data q2 is read from the buffer means 13 and transmitted. Similarly, in the next cycle, the terminal P
If the audio data p3 is present in the buffer 1 of the buffer means 13 allocated to store the audio data from the device (FIG. 5A), the audio data p3 is transmitted from the output means (FIG. 5B). If the audio data q4 exists in the buffer 2 allocated to the audio data of the terminal Q in the next cycle (FIG. 6A), the audio data q4 is transmitted from the output means (FIG. 6B). As described above, by alternately transmitting two audio data at regular intervals, in the above example,
When the terminal receiving the audio data of p1, p2, p3, p4, p5,...
.. and the voices q2, q4,.

The output means 14 shown in FIG.
The audio data is read out from the buffer designated by and output from the output terminal 15 to the data transmission path. The control means 17
The detection means 16 is periodically instructed to check whether audio data is stored in the buffer means 13 and is caused to execute the check. Further, the control means 17 instructs the output means 14 to read out the audio data from a specific buffer containing the audio data to be transmitted to the transmission path and output the audio data to the transmission path according to the inspection result of the detection means 16. Thus, the control means 1
Reference numeral 7 controls the timing of data processing of the entire processing. In the present apparatus, the mixing processing is realized only by the selection processing of the audio data, so that the amount of calculation required for the mixing is almost zero.

On the other hand, in the conventional method, the audio data is once converted into a form that can be added, the addition process is performed, and then the audio data is returned to the format of the audio data again. According to the method of the present invention, the amount of calculation can be significantly reduced.

(Embodiment 2) FIG. 7 is a block diagram showing a schematic configuration of a system using an audio mixing apparatus according to another embodiment of the present invention (Embodiment 2).

In FIG. 7, the user inputs voice to the terminals 41, 42 and 43, and the reproduced sound is output. Specifically, audio input from an audio input device such as a microphone
After the conversion, the encoded audio data is created by the audio analysis processing and transmitted to the transmission path 44. The audio data transmitted from the transmission path is decoded in reverse, DA converted, and output from an audio output device such as a speaker.

The audio data transmitted from the terminals 41, 42, 43 to the transmission line 44 are all audio mixing servers 45.
Collected in. The audio mixing server 45 uses the audio mixing device of the present invention. Each audio data is switched by the audio mixing server 45, sent to each of the terminals 41, 42 and 43, and reproduced by each of the terminals.
For example, when the users of the terminals 41 and 42 are having a conversation and the user of the terminal 43 listens to the joint message, the audio data input from the terminal 42 is transmitted from the audio mixing server 45 to the terminal 41. To the terminal 42, the audio data input from the terminal 41, and further to the terminal 43, the audio data input from the terminal 41 and the audio data input from the terminal 42 are alternately transmitted at regular intervals, Terminal 4
The users 1, 42, and 43 can obtain a reproduced sound as if three people were talking in the same place. As described above, the communication environment in which a plurality of people can participate and freely speak can be constructed by the system of the second embodiment.

Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above-described embodiments, and it is needless to say that various modifications can be made without departing from the gist of the present invention. It is.

[0028]

The effects obtained by typical aspects of the invention disclosed in the present application will be briefly described as follows.

(1) It is possible to repeatedly reproduce sounds uttered by a plurality of persons at the same time without deterioration of the reproduced sound such as delay or omission of the reproduced sound.

(2) The cost of the mixing device can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an audio mixing device according to an embodiment (Embodiment 1) of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a buffer in the audio mixing device according to the first embodiment.

FIG. 3 is a diagram for explaining an operation of the audio mixing device of the first embodiment.

FIG. 4 is a diagram for explaining the operation of the audio mixing device of the first embodiment.

FIG. 5 is a diagram for explaining an operation of the audio mixing device of the first embodiment.

FIG. 6 is a diagram for explaining the operation of the audio mixing device of the first embodiment.

FIG. 7 is a block diagram showing a schematic configuration of a system using an audio mixing device according to another embodiment of the present invention (Embodiment 2).

[Explanation of symbols]

11 input terminal, 12 input means, 13 buffer means, 14 output means, 15 output terminal, 16 detection means, 17 control means, 41-43 terminal, 44 transmission line, 45 audio mixing server.

Claims (1)

[Claims] 1. A data input means for receiving digital voice data of a plurality of persons from a transmission line, a plurality of data storage means for temporarily storing the captured voice data for each speaker, and the plurality of data storage means each storing the voice data. Detecting means for checking whether or not the data is stored; data output means for reading out the temporarily stored audio data and transmitting it to a specific device connected to a transmission line; and control means for controlling each of the means. A digital audio mixing device comprising: a plurality of voices simultaneously uttered by a plurality of persons;