FR2852438A1 - Voice messages translating system for use in multi-lingual audio-conference, has temporizing unit to temporize messages such that terminals except terminal which transmits current message, receive endings in speakers language - Google Patents

Abstract

The system has a translating unit to translate text messages into text messages translated in I-1 languages. A temporizing unit (4) temporizes synthesized I-1 messages and current voice message into temporized messages such that terminals other than a terminal which transmits the current message, receive simultaneously endings of the temporized messages in the languages of the speakers.

Description

1 2852438

  The present invention relates to a system for translating voice messages during telephone conversations between speakers of different languages during an audio conference. A telephone conversation is divided according to silence into voice messages respectively attributed to 10 of the speakers and identified by languages. A voice message is converted to a text message, the text message is translated into text messages translated from different languages, then each translated text message is synthesized.

  Currently, some systems translate audio signals in real time. According to European patent application EP 1093059, a translation system is composed of a voice recognition unit, a translation engine, and a voice synthesis unit. The voice recognition unit continuously analyzes an incoming audio signal in a first language, and gradually submits a text result to the translation engine. The translation engine 25 continuously generates a text translation into a second language from the result of the voice recognition. The translation is provided to the text-to-speech unit. Based on a comparison between the current translation and the previous 30 translations, the text-to-speech unit synthesizes the result of the translation into the second language.

  However, the current multilingual single-speaker translation systems do not take into account the differences between the languages of each speaker, 2,285,438 other than those relating to the vocabularies of these languages.

  The objective of the present invention is to automatically translate and synthesize voice messages of indefinite duration continuously during a multilingual audio conference between speakers, by overcoming in particular the temporal differences between a current voice message of a given language. and synthesized messages translated into other languages, so that speakers can intervene instantly in the audio conference despite these temporal differences.

  1 5 To achieve this objective, the system translates voice messages transmitted by K speaker terminals during a multilingual audio conference, K being an integer greater than 2, and includes a voice recognition means converting a current voice message transmitted from '' a speaker terminal in a common text message. It is characterized in that it comprises: a means for translating the current text message into current translated text messages in 25 I-1 languages other than the language of the speaker whose terminal has transmitted the current voice message, I being a number of different languages spoken by the speakers, ie I <K, - a means for synthesizing the 30 current text messages by voice, respectively into current synthesized messages, and - a means for timing the I-1 synthesized messages and the current voice message , for predetermined durations respectively in I current timed messages, so that the terminals other than the terminal having transmitted the current message, receive substantially simultaneously the ends of the current timed messages respectively in their languages.

  Preferably, the means for timing delays the following synthesized messages respectively the current synthesized messages, so that the following synthesized messages and the current timed messages do not overlap.

  In order to improve the quality of speech recognition and translation, the system may include means for segmenting the current text message into periodic time text segments, means for determining a context of a current text segment in according to the content and the language of the current text segment and according to averages of voice parameters of the voice message on the current text segment in order to determine contexts of the current text message and of the translated text messages.

  Other characteristics and advantages of the present invention will appear more clearly on reading the following description of several preferred embodiments of the invention with reference to the corresponding appended drawings in which: - Figure 1 is a schematic block diagram of a translation system for multilingual audio conference according to a first preferred embodiment of the invention in the environment of several speaker terminals and several audio conference servers.

  FIG. 2 is a schematic block diagram of a multilingual audioconference server for translation 4 2852438 according to a first preferred embodiment of the invention; FIG. 3 is a schematic block diagram of an audio conference server for multilingual translation according to a second preferred embodiment of the invention; and - Figure 4 is a schematic block diagram of an audio conference server for multilingual translation according to a third preferred embodiment of the invention.

  The detailed implementation of an audio conference system described below as well as its variants relate to an audio conference, known as a conference or telephone meeting organized between several telephone terminals T1 to TK, with KÄ2.

  The audio conference is accessible by reservation or by subscription or immediately via an SE telephone audio conference server.

  The translation system for multilingual audio conference comprises K speaker terminals T1, ..., Tk, ..., TK connected to access networks RA1 to RAK and at least one local audio conference server SE. The RAK access networks are linked to the switched telephone network. In this audio conference system, a TO terminal, confused for example with the T1 terminal, such as a personal computer or a specific audio conference console, is dedicated to an audio conference organizer for managing, that is to say, say organize, establish, monitor and animate the audio conference, in particular to welcome participants during the audio conference, filter channels and call participants during the audio conference, organize votes, etc. According to the illustrated embodiment, the terminal TO is in relation to the local audio conference server SE. In the case of a paid service, the audio conference organizer may take charge of the billing for this service.

  By way of example, four speaker terminals T1, T2, Tk and TK are illustrated in FIG. 1. The terminal T1 = T0 described above exchanges voice messages through an access network RA1 including the switched telephone network and the Internet, and therefore 10 voice over "IP" (Internet Protocol). The terminal T2 is a conventional fixed telephone terminal served by the switched telephone network (PSTN) or by a digital integrated services network (ISDN), constituting an access network RA2. 15 The terminal Tk, with 1 2 k 2 K, is a mobile radiotelephone terminal served by a digital cellular radio network RAk of the GSM or UMTS type. The terminal TK is a mobile radiotelephone terminal linked to another RAK digital cellular radiotelephone network of the GSM or UMTS type.

  The local audio conference server SE is connected to the switched telephone network by at least K two-wire analog or two-wire digital telephone lines LT1 to LTK via network interfaces IR1 to IRK.

  The main role of the network interface IRk, independently of the type of the line LTk, is to extract conventional telephone signaling and audio conference signaling originating from the terminal Tk in the signal received by the line LTk by separating from a participant voice signal.

  Conversely, the IR network interface inserts telephone signaling and audio signaling for audio conferencing intended in particular for the terminal Tk into the signal transmitted in the line LTk by mixing it with a voice signal.

  As shown in FIG. 1, a signaling bus BS exchanges the signals extracted 5 and to be inserted between the interfaces IR1 to IRK and a multilingual translation and audio conference management module MTG. A digital voice reception channel REk connects the network interface IRk to an input of the translation and management module, and respectively whatever is 1 <k <K. A digital voice emission voice EMk connects an output of the translation and management module at the network interface IRk and respectively whatever 1 <k <K. According to a first embodiment shown in FIG. 2, the multilingual translation and audio conference management module MTG comprises a detector activity 11, at least one voice recognition module 12, at least one translation module 2, at least one voice synthesizer 3 and a timer unit 4.

  The activity detector it periodically detects among the reception channels RE1 to REK 25 relating to participants in an audio conference whose number is less than or equal to K, that which is most active and which constitutes that conveying the current voice message. MVk of the current speaker during the audio conference. An identification address k is assigned to each set comprising a terminal Tk, a line LTk, and a network interface IRk. This address identifies a speaker by the line taken by his voice message. The detector 11 establishes the address of the most active line 35 LTk, that is to say the most active speaker or terminal. This address forms part of the speaker's identification. This identification may consist in searching for the largest of the average powers of voice segments 5 in the reception channels RE1 to REK provided that it is greater than a predetermined threshold. If the maximum power is less than a predetermined threshold, the activity detector 11 confirms a "silence" during the teleconference at the terminal TO 10 of the organizer of the audio conference.

  The organizer can intervene to invite speakers to speak louder. The incoming signal transported in the most active reception channel for a predetermined time is hereinafter assumed to be the voice signal SVk in the reception voice REk, hereinafter called current voice message MVk at the output of the activity detector 11.

  The MVk voice messages are temporarily stored temporarily in a buffer memory 20 contained in the activity detector 11. As with a conventional audio signal, the MVk voice messages include periodic time marks such as frame alignment words, words packet synchronization, frame or line synchronization signals, etc. These time marks are counted modulo a predetermined number and stored in the buffer memory contained in the detector 11. These time marks are refreshed periodically, in order to synchronize the processing of the voice messages in the components of the audio conference server MTG.

  It is assumed that the incoming signals received by the SE server are digital; otherwise, the incoming signals received are analog and converted by analog-to-digital converters included in the network interfaces IR1 to IRk. The activity detector 11 contains a filter separating a purely voice signal from a residual signal in an incoming signal, in order to improve the performance of voice recognition, translation and speech synthesis by applying a voice message to them. no residual signal. The residual signal comes in particular from noise due to the sound environment of a speaker around his terminal.

  In the embodiments described below, each speaker selects one of the different languages L1 to L1 in which the speakers participating in the audio conference can express themselves, with 1 <i I and I <K. In the first embodiment, the I different languages of the K speakers participating in the audio conference are considered to be known by the server MTG as well as the language Lk of the current voice message MVk. For example, prior to the audio conference, each speaker declared a language identifier L1 to LI to the MTG module, directly or through the organizer.

  A current voice message MVk of language Lk is translated into I-1 languages. For example in an audio conference with 5 speakers speaking 3 different languages, French, English and German, each voice message in French is translated into English and German and vice versa for a voice message in English or German. The translation system 30 according to the invention finds all its advantage when I is strictly greater than 2.

  The voice recognition module 12 converts the current voice message MVk applied by the activity detector 11 into a current text message 35 MTk. The translation module 2 translates the current text message 2852438 MTk of language Lk into I-1 textual messages MTTi translated into the other languages Li with 1 <i <I-1 and iÉk. Each textual voice message MTk is translated only once into each of the 5 different languages L1 to LI regardless of the number of speakers K. As a variant, the translation module 2 is based on exceptional lexicons. An exceptional lexicon contains rules to be applied for the translation of 10 predetermined words from a given language. For example a patronymic name is not translated.

  The speech synthesizer 3 vocally synthesizes the I-1 text messages translated MTT1 to MTTI respectively into I-1 current synthesized messages 15 MS1 to MSI, with iÉk. The speech synthesizer 3 notably comprises a buffer memory for storing the translated text messages MTTi to be synthesized. The speech synthesis in the synthesizer 3 is based on the language Lk of the current voice message MVk, on the other 20 languages L1 to LI with iÉk and on predetermined voice parameters PVk of the current voice message MVk considered in this first embodiment as known. The vocal parameters are in particular acoustic and particularly prosodic parameters like the vibration frequency, the intensity, the flow, the timbre and also other parameters like the relative age or the sex of the speaker. As a variant, the speech synthesis module 3 uses exceptional lexicons.

  The timer unit 4 includes a buffer memory and a processing unit. On the one hand, the unit 4 delays the I-1 synthesized messages MS1 to MSI, with iÉk, and the current voice message MVk, for first predetermined durations 35 respectively in I timed messages Ml to MI, so that the terminals T1 to TK, other than the speaker terminal of the current message MVk, receive substantially simultaneously the ends of the current timed messages Ml to MI respectively their 5 languages. On the other hand, the unit 4 delays the synthesized messages respectively following the current synthesized messages, so that the following synthesized messages do not at least partially overlap these with the current timed messages M1 preceding to MI respectively. The ends of the current synthesized messages are thus synchronized.

  The timer unit 4 directly receives the original current voice message MVk at the output of the activity detector 11 so as not to unnecessarily translate the current voice message MVk for speakers having the same language Lk as the current message MVk.

  For example, the timer unit 4 determines the first durations by comparing the durations of the current I-1 synthesized messages and the current vocal message MVk in order to determine the current synthesized message having the longest duration. Each of the synthesized messages MSi is timed for a first duration equal to the difference between the duration of the synthesized message having the longest duration and the duration of the synthesized message to be timed.

  In the case where common voice messages are relatively long, the first durations can be extremely different, which degrades the hearing quality of the audio conference. To overcome this drawback, the voice recognition module 12 progressively segments the current voice message MVk into segmented messages having a duration equal to or less than a predetermined duration, when the duration of the current voice message MVk is greater than said predetermined duration. Each of the segmented messages is processed successively in the voice recognition module 1, the translation module 2, 5 at the voice synthesizer 3 and the timing unit 4 as previously for the current voice message MVk.

  The predetermined duration is of the order of a few tens of seconds.

  The periodic segmentation improves the synchronization of the timed messages M1 to MI by the timing unit 4, so that the audio conference continues to offer all the speakers the same listening conditions during a long monologue by one of them. Without this additional segmentation, each speaker other than the intervening speaker must wait for the translation of the current voice message to finish before starting to receive the corresponding translated message, which would cause very long delays and manifest discomfort during the audio conferencing.

  As a variant, the timer unit 4 or the voice synthesis module 3 reduces or increases the duration of a synthesized message MS1 by a variable duration in order to equalize the durations of all the current synthesized messages MS1 to MSI by acting between another on the bit rate and frequency of each MSi synthesized message. This variable duration does not exceed 10% of the duration of the synthesized message in order to maintain the "audible" character of the synthesized voice message for the human ear.

  K multiplexers DM1 to DMK each have I inputs receiving the current timed messages M1 to MI and have outputs constituting the transmission channels EM1 to EMI connected to the K network interfaces. The I 35 timed messages are sent according to their duration of 12 2852438 timing at times different or identical to the K multiplexers. A selection input of each demultiplexer DMk receives the respective language identifier Lk to select the corresponding current timed message 5 from the messages M1 to MI so that the network interface IRk transmits to the terminal Tk only the timed message in the language Lk.

  In a second preferred embodiment according to the invention shown in FIG. 3, the MTG multilingual translation and audio conference management module also includes a language determination unit 5, a voice analyzer 6 and a voice database 7.

  The database 7 contains voice data characterizing the audio conference. The voice data are stored during a phase of initialization of the audio conference controlled by the organizer of the audio conference or by the speakers likely to participate in the audio conference, during a phase of determining the languages commented below. The characteristics of an audio conference include, for example, the total number of speakers participating in the audio conference, an appointment time for all the speakers, etc. The language determination unit 5 determines the K languages L1, ..., Lk, ..., LK of the voice signals SV1 30 to SVK transmitted respectively by the network interfaces IR1 to IRK associated with the terminals T1 to TK, at start of the audio conference during a language determination phase. The language determination unit 5 contains voice mail. The voicemail or the organizer invites each 13 2852438 speaker to pronounce a free sentence of the following genre, exceeding a predetermined minimum duration, "Hello, I am the speaker XXX". This sentence is analyzed by unit 5 in order to determine the language 5 of the respective speaker. The identifiers of the languages thus determined are stored in the voice database 7 in correspondence with the speaker and the respective terminal.

  The language determination unit 5 delays the voice signals SV1 to SVK in the reception channels RE1 to REK during the language determination phase to determine the language of these. The duration of this phase depends on the number K of speakers participating in the audio conference. At the end of this phase, if the number of languages determined is less than the number of participants in the audio conference, a default language, for example English, is assigned to the terminal of the speakers whose language remains unknown. Alternatively, the 20 speaker terminals whose language remains unknown are ousted from the audio conference. In another variant, as long as the determination unit 5 has not determined the languages of the K speakers, this delays the voice signals reaching it.

  The current voice message MVk, the language Lk of which is identified among the languages determined previously in association with the network interface IRk, is processed in parallel by the voice analyzer 6 and the voice recognition module 12. The voice analyzer 6 analyzes the current voice message MVk produced by the activity detector 11 in order to continuously determine the voice parameters PVk characterizing the current voice message MVk. The voice parameters as defined above are used in the reproduction of the voice of the speaker who spoke during the speech synthesis. The number of voice parameters is not fixed.

  As a variant, a second voice analyzer (not shown) in relation to the voice database 7 5 is arranged in parallel with the language determination unit 5. The second voice analyzer establishes a voice model or a voice print of the speaker's voice. each SVk voice signal from voice parameters. This voice print is stored in the voice database 7 when this is not similar to a voice print already contained in the voice database. A speaker is thus identified by his voice print and his language. As a variant, the voice fingerprint is used to identify each speaker during the audio conference and to improve the quality of his synthesized voice during the speech synthesis in the speech synthesizer 3.

  The translation module 2 retrieves from the voice database 20 the languages LI to LI into which it translates the current text message MTk, that is to say the I-1 different languages other than the language Lk of the voice message current MVk.

  Then the translated text signals MTT1 to 25 MTTI are synthesized respectively in I-1 synthesized signals MS1 to MSI. The speech synthesis in the synthesizer 3 is based on the speech parameters PVk of the current voice message MVk, such as the tone, determined in the speech analyzer 6 and supplied continuously so as, inter alia, to break the monotony of the synthesized messages. Speech synthesis also considers the evolution of a speaker's voice from one voice message to the next. In the case where a voiceprint for each speaker has been established, the voice synthesis module 3 updates 2852438 the voiceprints in the voice database 7 to associate with each speaker a voice very distinct from those of the other speakers, in order to break the monotony of the synthesized messages. The processing of the synthesized messages MS1 to MSI is then identical to the first preferred embodiment in the timing unit 4 and the multiplexers DM1 to DMK.

  In a third preferred embodiment according to the invention shown in FIG. 4, the MTG multilingual translation and audio conference management module comprises the same elements as the second embodiment, as well as a multilingual context determination unit 8 and a contextual database 9.

  The multilingual context determination unit 8 segments the current text message MTk produced by the voice recognition module 12 into periodic temporal textual segments Sn. The unit 8 determines a context CSn of the current text segment as a function of averages of the voice parameters of the current voice message, supplied continuously by the voice analyzer 6, on the current text segment 25 Sn, and as a function of the content of the text segment current Sn and the language Lk of the current text segment Sn. The contexts thus determined are stored in the contextual database 9. Likewise the context determination unit determines contexts for the messages translated MTT1 to MTTI in their respective languages. The multilingual contextual database thus contains contexts of the audio conference in the I languages used by the K speakers. The voice recognition module 35 12 and the translation module 2 being able to use only contexts in the language of the message they are processing and in the language in which they convert the message, the multilingual contextual database 6 directly sets 5 arrangement of these two modules for contexts in these latter languages.

  Various contexts in the form of key words and expressions, as defined above, deduced from segments preceding the current segment Sn and / or from the context study constitute contexts CSn pre-stored and managed in the contextual database 9 linked to the voice recognition module 12, to the translation module 2 and to the multilingual context determination unit 8.

  As a variant, the contexts in the contextual database 9 are also completed and refined by automatic consultation of contextual databases external to the audio conference server SE as a function of the contexts 20 recently detected. Context lists contained in external context databases are updated manually and / or automatically.

  Each context is characterized by additional information such as the tone of the context, for example serious or happy, the category of population concerned, for example children, managers, workers, etc. The contexts written in the base 9 as they are determined in the unit 8 as well as the contexts contained in the external contextual databases are involved in the conversion of the successive voice messages MVk into textual voice messages MTk in the voice recognition module 1 and also in the translation of each text message MTk into parallel translated text messages MTT1 to MTTI in I-1 languages in the translation module 2. The contexts written in the contextual database 9 are thus gradually improved during the processing of text messages to facilitate voice recognition in the voice recognition module 12 and translation in the translation module 2. At the start of the audio conference or the first intervention of a speaker, no o0 context has not been determined. The processing of the current voice message is therefore delayed by a time necessary for determining the first contexts. This necessary time is reduced when at least one of the speakers informs the context of the audio conference 15 during the initial phase of determining the languages of the second preferred embodiment.

  As a variant, the activity detector 11 in the multilingual translation and audio conference management module 20 MTG periodically detects the most active terminals including the voice signals SV1 to SVK having a power greater than said predetermined threshold in order to process messages in parallel current voices of several speakers 25 speaking at the same time, unlike the three embodiments above, in which the activity detector 11 selects only the most active telephone line LT1 to LTI. In this variant, the MTG module is capable of processing a maximum of K voice messages SV1 to SVK simultaneously.

  Each DMk of the K multiplexers DM1 to DMK presents K sets of I inputs which are respectively connected to K timer units 4 in order to select and mix at most K current timed messages relating to the K speakers in the respective transmission channel EMk In another variant , a group of several speakers are gathered around the same terminal. In this case, a group voice analyzer succeeds the activity detector 11 or precedes the voice recognition module 12, and distinguishes the different voices in the group so that they are processed separately by the multilingual translation module 10 and audio conference management, before being recombined for a message translated to the other terminals.

  As a variant, a speaker specifies at the start of the audio conference or during the phase of determining the languages, the different languages which he masters, in order to send to the terminal associated with this speaker the voice messages of the other untranslated speakers when these voicemails are in a language that he speaks.

  According to a variant of the architecture of the system, components of the multilingual translation and audio conference management module MTG or a part thereof are installed in the terminals T1 to TK * The translation system according to the invention can be integrated into a videoconferencing system with closed captioning. The voice synthesis phase of the system according to the invention is then deleted and the text messages translated MTT1 to MTTI are directly displayed respectively on screens of the terminals of the videoconference speakers in the languages chosen by them.

  The translation system for multilingual audioconferencing according to the invention is also usable for learning foreign languages. All the voice messages of the audio conference are systematically translated into the same predetermined language and the speakers speak in a language of their choice, that of learning or another. Thus all the speaker terminals receive the voice messages 10 in the same language. This variant offers, for example, the possibility for pupils with little talent in a certain language to express themselves in their mother tongue while listening to voice messages in the language of learning. 1 5

Claims (7)

  1 - System for translating voice messages transmitted by K terminals (Tk) of speakers during 5 a multilingual audio conference, comprising a voice recognition means (1) converting a current voice message (MVk) transmitted from a speaker terminal into a current text message (MTk), characterized in that it comprises - means (2) for translating the current text message (MTk) into current translated text messages (MTT1 - MTTI) in I-1 languages other than the language (Lk) of the speaker whose terminal has transmitted the current voice message, I being a number of 15 different languages spoken by the speakers, - a means (3) for synthesizing vocally the current translated text messages respectively into current synthesized messages ( MS1 - MSI), and - a means (4) for delaying the I-1 synthesized messages (MS1 MSI) and the current voice message (MVk), for predetermined durations respectively in I current timed messages (Ml - M I), so that the terminals (Tl - TK), other than the terminal having transmitted the current voice message 25 (MVk), receive substantially simultaneously the ends of the current timed messages (Ml - MI) respectively in their languages (Ll - LK 2) The system as claimed in claim 1, in which the means for delaying (4) delays the synthesized messages respectively according to the current synthesized messages, so that the following synthesized messages and the current timed messages do not overlap.   3 - System according to claim 1 or 2, comprising means (11) for periodically detecting the terminal which is the most active (Tk) among the terminals and which constitutes the speaker terminal having transmitted the current voice message (MVk).   4 - System according to any one of claims 1 to 3, comprising means (11) for periodically detecting the terminals which are the most active in order to process in parallel common voice messages from several speakers speaking at the same time .   - System according to any one of Claims 1 to 4, comprising means (11) for progressively segmenting the current voice message (MVk) into segmented messages having a duration equal to or less than a predetermined duration, when the duration of the voice message current (MVk) is greater than said predetermined duration, each of the segmented messages being processed successively in the voice recognition means (12), the means for translating (2), the means for synthesizing (3) and the means for delaying ( 4).   6 - System according to any one of claims 1 to 5, comprising means (6) for analyzing the current voice message (MVk) in order to continuously determine voice parameters (PVk) of the current voice message (MVk) to apply by means of synthesis (3).   7 - System in accordance with any of the   claims 1 to 6, comprising means (3; 4)   to equalize the durations of current synthesized messages (MS1 - MSI).   8 - System according to any one of claims 1 to 7, in which a speaker terminal having a language identical to that of the speaker of the current voice message (MVk) receives the current voice message (MVk) timed by the means for delay (4).   9 - System according to any one of claims 1 to 8 comprising means (5, 7) for determining the languages (L1 - LK) of voice signals (SV1 - SVK) transmitted respectively by the terminals 15 in order to identify the language (Lk) of the current voice message (MVk).   - System according to any one of claims 1 to 9, comprising means (8) for segmenting the current text message (MTk) into periodic time text segments, means (8) for determining a context (CSn) d a current text segment as a function of the content and the language (Lk) of the current text segment and as a function of averages of voice parameters (PVk) of the current voice message continuously supplied by the means for analyzing (6), on the current text segment (5n) in order to determine the contexts of the current text message (MTk) and of the translated text messages (MTT1 30 - MTTI).