WO2014076129A1

WO2014076129A1 - Method for operating a telephone conference system, and telephone conference system

Info

Publication number: WO2014076129A1
Application number: PCT/EP2013/073720
Authority: WO
Inventors: Christian Hoene; Michael Haun; Patrick SCHREINER
Original assignee: Symonics GmbH
Priority date: 2012-11-13
Filing date: 2013-11-13
Publication date: 2014-05-22
Also published as: WO2014076129A8; DE102012220688A1

Abstract

A method for operating a telephone conference system for at least two participants comprises the following steps: detection of verbal activity of a respective participant, and, in the event that verbal activity is detected for more than one participant, reproduction of only one verbal activity and time-delayed reproduction of the other verbal activities.

Description

TELEPHONE CONFERENCE WITH AVOIDANCE OF LANGUAGE COLLISIONS

The invention relates to a method for operating a telephone conference system and a telephone conference system.

In the course of a telephone conference, speech-based audio signals of the respective call participants of the telephone conference are mixed, so that a respective call participant can hear the other call participants. A system that interconnects several call participants in this way is called a telephone conference system.

For telephone conference systems which use single-channel (mono) transmission channels for the transmission of speech-based audio signals, it should be avoided that several call participants speak at the same time, because it is difficult to separate the call participants based on the superimposed audio signal.

In a traditional telephone conference based on single-channel audio signal transmission, a moderator typically decides who is allowed to speak. As a rule, all participants in the conversation try to avoid interrupting others. A technologically sophisticated alternative to deal with multiple simultaneous conversation participants or speakers is the use of surround sound, surround or stereo transmission technologies. For a listener can separate voices that come from different directions and focus on one. This is also called cocktail party effect. The invention has for its object to provide a method for operating a telephone conference system and a telephone conference system available, which allow the use of single-channel (mono) transmission channels while separability of the individual call shares without, for example, a moderator decides who may speak. The invention solves this problem by a method according to claim 1 and a telephone conference system according to claim 3.

The method is used to operate a telephone conference system for at least two (call) participants. The method comprises the following steps:

It is, in particular continuously, a linguistic activity of a respective conversation participant determined, i. For example, it is determined whether a respective conversation participant speaks or is silent. In the event that a linguistic activity is detected in more than one interviewee, i. if several call participants are speaking at the same time, only a single linguistic activity (i.e., the conversation contribution of a single conversation participant) is played back at a time, and the remaining linguistic activities or conversational messages are played back one after the other with a time delay. Here, the linguistic activities can be sequentially reproduced in the order of their temporal origin.

The determination of the linguistic activity may include a so-called Voice Activity Detection (VAD). Basically, in the course of the VAD, the presence or absence of human speech or conversation contribution is determined, i. It is determined whether a conversation participant is silent or speaks. Consequently, silence and speech activity can be detected or recognized by means of VAD. Switching between conversation participants or conversation contributions can be carried out, for example, at the beginning of a speech break. Incidentally, reference is also made to the relevant specialist literature.

Linguistic activity determination can be further performed based on whether important or unimportant speech segments are involved. In the case of an audio signal or speech segment classified as important, a linguistic activity is determined or recognized. In the case of a classified as not important audio signal or speech segment, no linguistic activity is detected or detected.

Audio signals from different participants in the conversation, in which several speech segments are present, which are classified as unimportant and thus not as linguistic activity, can be mixed with one another and simultaneously played back. Unimportant parts would be, for example, speech segments in which speech characteristics do not change, for example during an "aaah".

Merely with important speech segments, ie with determined linguistic activity, mixing and simultaneous reproduction is not recommended, since the different and mixed conversation parts are difficult to separate again by a listener, so that the important parts of the conversation are to be reproduced one after the other, although at least partly arise at the same time. Important parts are usually the onsets as soon as you start talking.

If a so-called codec is used which compresses an audio signal in the form of an audio stream at a variable bit rate, then a linguistic activity can be excluded with high probability if the codec generates a compressed data stream with a low bit rate.

In addition, it is possible to transform the audio signals into the frequency domain and to examine whether the audio signals are superimposed in one or more frequency bands. If there is an overlay, a simultaneous linguistic activity is concluded and no simultaneous playback occurs. If there is no overlay, it can be mixed and played simultaneously. For example, one high and one low voice would be played simultaneously, whereas two equally high voices would not be mixed and played simultaneously.

A combination of the above methods is possible. The telephone conference system is intended for at least two call participants and is preferably designed to carry out the above-mentioned method.

The telephone conference system comprises at least two audio signal sources, wherein a respective audio signal source is assigned to a conversation participant and designed to generate an audio signal in dependence on a speech activity of the conversation participant. The audio signal may be an analog audio signal. Typically, however, it is a digital audio signal.

Furthermore, at least one buffer or intermediate memory is provided, wherein the intermediate memory is designed to temporarily store at least a part of a respective audio signal. The buffer can be designed as a first-in-first-out memory. The intermediate memory can also be designed as a ring memory. Further, the latch may store the (digital) audio signal in response to a control signal generated by a controller upon storage need. The buffer may be an electrical memory such as RAM, magnetic memory, etc. It can be provided exactly one buffer or it can be provided per audio signal, a buffer. Furthermore, at least one voice activity recognition device is provided, which is designed to determine a linguistic activity in a respective audio signal. The voice activity recognition device can be designed as a microprocessor on which a suitable voice activity recognition software runs. Precisely one voice activity recognition device can be provided, or one voice activity recognition device can be provided per audio signal.

Further, there is provided a (digital) audio signal mixing device configured to mix input (digital) audio signals with each other and to output the mixed input audio signals as the output audio signal. The output audio signal can be output simultaneously to a plurality of outputs of the audio signal mixing device.

Furthermore, at least two loudspeakers are provided, wherein a loudspeaker is assigned to a conversation participant, receives the output audio signal of the audio signal mixing device and outputs it as a corresponding sound signal.

A control device of the telephone conference system is adapted, in the event that in more than one of the audio signals, a linguistic activity is detected, only a single one of those audio signals in which a linguistic activity is detected, output to the audio signal mixing device as an audio input signal and not audio signals output to the audio signal mixing device as audio input signals in which a linguistic activity is detected, to be buffered in the buffer, ie For example, to control the latch for caching the audio signals, and time-delayed output as an audio input signal to the audio signal mixing device, so that they are reproduced with a time delay. The audio signals can be cached before their reproduction exclusively during their determined linguistic activity.

In order to detect whether there is a linguistic activity in more than one of the audio signals, ie to detect a speaker collision, at least one speaker collision detection device can be provided. The speaker collision detection device is designed to detect or recognize simultaneous linguistic activities in a plurality of associated audio signals. The speaker collision detection device can be part of the voice activity recognition device or can be formed separately from the voice activity recognition device. If the voice activity recognition device and the speaker collision recognition device are implemented separately from one another, the speaker collision recognition device can be arranged, for example, between the voice activity recognition device and the control device, wherein the voice collision engine Identification device, for example, recognizes multiple simultaneous speakers and passes this information to the controller.

The control device may be designed to check, in the event that no linguistic activity is detected in any of the audio signals, whether cached audio signals are present, and if cached audio signals are present, the cached audio signals one by one or mixed so long to the audio signal Mixer output as audio input signals until no cached audio signals are available.

The control device can be configured to output all audio signals as audio input signals to the audio signal mixing device in the event that no linguistic activity is detected in any of the audio signals and no cached audio signals are present.

The control device may be configured to output only the audio signal in which a linguistic activity is detected as audio input signal to the audio signal mixing device in the event that a linguistic activity is detected in only one of the audio signals and no cached audio signals are present ,

The audio signals can be mono audio signals.

The audio signal sources may be microphones of headsets and / or telephone handsets, and the loudspeakers may be the speakers of the headsets or handset, i. be integrated into the headsets and / or the telephone handset.

The voice activity recognition device or the speaker collision detection device can be designed to transform the audio signals into the frequency domain. The voice activity recognition device or the speaker collision detection device can also be designed to examine whether the transformed audio signals are superposed in one or more frequency bands, ie to investigate whether linguistic activities can be recognized in a plurality of frequency bands. In the event that the transformed audio signals are superimposed in one or more frequency bands, ie if a linguistic activity can be detected in one or more frequency bands for a plurality of audio signals, the voice activity recognition device or the speaker collision detection device can be designed to indicate the thus determined speaker collision to transmit the control device. The invention will be described below with reference to the drawing. This shows schematically:

Fig. 1 is a telephone conference system for several participants in the conversation.

Fig. 1 shows a telephone conference system for several participants. The telephone conference system has, by way of example, two audio signal sources in the form of microphones 1_1 and 1_2 of two telephone handsets or headsets and two loudspeakers 5_1 and 5_2, which are part of the telephone handset or headsets.

The microphones 1_1 and 1_2 are assigned to a conversation participant and designed to generate a mono audio signal AS_1 or AS_2 depending on a linguistic activity of the respective conversation participant. The audio signals AS_1 and AS_2 are digital audio signals, wherein the digitization of the linguistic activity in the microphones or in a downstream, not shown A D converter can be done.

The telephone conference system further has two latches 2_1 and 2_2, wherein a respective latch 2_1 and 2_2 is configured to latch an associated audio signal AS_1 or AS_2 in response to a memory control signal generated by a control device 6a and 6b when memory is required.

The telephone conference system further has a voice activity recognition device 3, which is designed to detect or recognize a linguistic activity in a respective audio signal AS_1 and AS_2 and to transmit the result of the determination to the control device 6a and 6b.

Furthermore, a (digital) audio signal mixing device 4 is provided, which is designed to mix input audio signals with one another and to output the mixed input audio signals as output audio signal OS to two output terminals by way of example.

The control device has components 6a and 6b. The control device component 6a is in data connection with the voice activity recognition device 3 and receives information regarding the linguistic activity of the audio signals AS_1 and AS_2 from the voice activity recognition device 3. The control device Component 6a is further connected to the buffers 2_1 and 2_2 and controls these if necessary for buffering the associated audio signals AS_1 or AS_2.

The control device component 6b is connected on the input side to the microphones 1_1 and 1_2 and the latches 2_1 and 2_2 and on the output side to the audio signal mixing device 4. The control device component 6b has internal switching logic, not shown, whose switching positions are determined by the control device component 6a. By means of the switching logic is determined which input or which inputs are looped through to the audio signal mixing device 4.

The control device or its components 6a and 6b is / are designed for the case in which a linguistic activity is detected in both audio signals AS_1 and AS_2, i. Both participants in the conversation speak to output only one of the two audio signals AS_1 or AS_2 via the control device component 6b to the audio signal mixing device 4 as the audio input signal and to buffer the other audio signal AS_1 or AS_2 in the associated buffer 2_1 or 2_2 and then as an audio input signal to the audio signal mixing device 4 with a time delay issue.

The audio signal AS_1 or AS_2 that is not output can be delayed in time until no linguistic activity is detected in the other audio signal AS_1 or AS_2.

The decision as to which of the two audio signals AS_1 and AS_2 is first output to the audio signal mixing device 4 may be based on the fact in which the audio signal AS_1 or AS_2 has first been recognized as having a linguistic activity.

The control device 6a and 6b is further adapted, in the event that in any of the audio signals AS_1 and AS_2 a linguistic activity is detected to check whether cached audio signals are present, and if cached audio signals are present, the cached audio signals one by one long to the audio signal mixing device 4 output as audio input signals until no cached audio signals are available.

In the event that no linguistic activity is detected in any of the audio signals AS_1 and AS_2, for example because the audio signals AS_1 and AS_2 are classified as unimportant, and no cached audio signals (more) are present, all the audio signals AS_1 and AS_2 can be used as audio input signals are output to the audio signal mixing device 4. In the event that a linguistic activity is detected in only one of the audio signals AS_1 or AS_2 and no cached audio signals are present, only the audio signal AS_1 or AS_2 in which a linguistic activity is detected can be output to the audio signal mixing device 4 as an audio input signal , According to the invention, an audio signal mixing device 4 is provided which mixes two or more audio signals AS_1 and AS_2 in order to generate one or more identical output audio signals OS therefrom.

First, it is checked whether there are noises on one or more of the audio signals AS_1 and AS_2 which are considered to be relevant or important for the respective application of the system. For this purpose, a conventional voice activity detector, as used in the telephone system, can be used to distinguish phases of silence from phases of active speech.

The audio signals AS_1 and AS_2 can be delayed (reproduced). This means that they are not forwarded directly to the loudspeakers 5_1 and 5_2 or the audio signal mixing device 4, but can be buffered for any period of time in order to be forwarded later to the audio signal mixing device 4.

The latches 2_1 and 2_2 may be implemented as FIFO memories. For algorithmic efficiency reasons, it may be useful to store not only the audio signals AS_1 and AS_2 in the latches 2_1 and 2_2, but also the associated linguistic activity information.

The control device 6a and 6b monitors whether the voice activity recognition device 3 detects active or relevant signals and whether acoustic signals are stored in the latches 2_1 and 2_2.

If no active or relevant audio signal AS_1 and AS_2 is present and no audio signals are stored in the intermediate memories 2_1 and 2_2, all the audio signals AS_1 and AS_2 in the audio signal mixing device 4 can be mixed. This is the case, for example, when all conference or conversation participants are silent.

If only one relevant audio signal AS_1 or AS_2 is present, it is forwarded to the audio signal mixing device 4. This is the case, for example, when a conversation participant begins to talk. If further audio signals now become relevant or active, they are not forwarded without delay to the audio signal mixing device 4, but are stored in the latches 2_1 and 2_2. It avoids thus that a second participant in the conversation breaks the first participant in the conversation. As soon as the previously reproduced audio signal no longer contains any relevant information, ie is no longer active in speech, one of the stored audio signals is retrieved from the associated buffer memory 2_1 or 2_2 and routed to the audio signal mixing device 4 and thus to the loudspeakers 5_1 and 5_2. The second caller is now played delayed after the first is silent. This process continues until no latches 2_1 and 2_2 have stored more audio signals.

It is understood that the two audio signal sources 1_1 and 1_2, the two latches 2_1 and 2_2 and the two loudspeakers 5_1 and 5_2 represent only an exemplary number. Of course, any number of these components can be used.

In order to detect the case in which there is linguistic activity in both audio signals AS_1 and AS_2, the voice activity recognition device 3 can be designed to transform the audio signals AS_1 and AS_2 into the frequency domain, to examine whether the transformed audio signals are superposed in one or more frequency bands and, in the event that the transformed audio signals in one or more frequency bands overlap, determine that there is linguistic activity in more than a single one of the audio signals AS_1 and AS_2. The voice activity recognizer 3 may transmit the result of the determination to the controller 6a.

Claims

claims

Method for operating a telephone conference system for at least two call participants, with the steps:

Determine a linguistic activity of a respective conversation participant and in the event that a linguistic activity is detected in more than one participant, reproducing only a single linguistic activity and delayed playback of the remaining linguistic activities.

A method according to claim 1, characterized in that for determining the case that there is more than one conversation participants a linguistic activity, the following steps are performed:

Transforming an audio signal (AS_1, AS_2) assigned to a respective call partner into the frequency domain,

Examine whether the transformed audio signals are superimposed in one or more frequency bands, and

if the transformed audio signals overlap in one or more frequency bands, determining that there is more than one conversation participant having a linguistic activity.

Telephone conference system for at least two call participants, in particular for carrying out the method according to claim 1 or 2, comprising:

at least two audio signal sources (1_1, 1_2), wherein a respective audio signal source is assigned to a call participant and is designed to generate an audio signal (AS_1, AS_2) in dependence on a linguistic activity of the call participant,

at least one temporary store (2_1, 2_2), the temporary store being designed to temporarily store at least a part of a respective audio signal, at least one voice activity recognition device (3), which is designed to determine a linguistic activity in a respective audio signal,

audio signal mixing means (4) adapted to mix input audio signals with each other and to output the mixed input audio signals as output audio signal (OS);

at least two loudspeakers (5_1, 5_2), wherein a loudspeaker is assigned to a conversation participant, receives the output audio signal of the audio signal mixing device and outputs as a corresponding sound signal, and

a control device (6a, 6b) which is designed in the event that a linguistic activity is detected in more than a single one of the audio signals, outputting only one of those audio signals in which a linguistic activity is detected to the audio signal mixing device as the audio input signal and the audio signals not output to the audio signal mixing device as the audio input signal, which a linguistic activity is detected, in the buffer store and time-delayed output as an audio input signal to the audio signal mixing device.

Conference telephone system according to claim 3, characterized in that the control device is designed for this purpose

in the event that no linguistic activity is detected in any of the audio signals, check for cached audio signals, and

if cached audio signals are present, output the cached audio signals individually one after the other as long as the audio signal mixing device as audio input signals until no cached audio signals are more present.

Conference telephone system according to claim 3 or 4, characterized in that the control device is designed for this purpose

in the event that in any of the audio signals, a linguistic activity is detected and no cached audio signals are present to output all the audio signals as audio input signals to the audio signal mixing device.

Conference telephone system according to one of claims 3 to 5, characterized in that the control device is designed for this purpose

in the event that a linguistic activity is detected only in a single one of the audio signals and no cached audio signals are present, only the audio signal in which a linguistic activity is detected, output as an audio input signal to the audio signal mixing device.

Conference telephone system according to one of Claims 3 to 6, characterized in that the audio signals are mono audio signals.

Conference telephone system according to one of Claims 3 to 7, characterized in that the audio signal sources are microphones of headsets and / or telephone handsets and the loudspeakers are integrated in the headsets and / or the telephone receivers.

9. telephone conference system according to one of claims 3 to 8, characterized in that the voice activity recognition device (3) is adapted to

to transform the audio signals into the frequency domain,

to investigate whether the transformed audio signals are superimposed in one or more frequency bands, and

if the transformed audio signals overlap in one or more frequency bands, determining that there is linguistic activity in more than a single one of the audio signals.