EP0734555A1

EP0734555A1 - Method and device for converting a first voice message in a first language into a second message in a predetermined second language

Info

Publication number: EP0734555A1
Application number: EP95903389A
Authority: EP
Inventors: Jean Gachot
Original assignee: Individual
Current assignee: Individual
Priority date: 1993-12-15
Filing date: 1994-12-09
Publication date: 1996-10-02
Also published as: WO1995016968A1; FR2713800B1; CA2178925A1; FR2713800A1

Abstract

A method and device for converting a first voice message in a first language into a second message in a predetermined second language, wherein: a) a first voice message spoken in a first language is received by means of a receiver; b) the first voice message is voice analysed by automatic voice recognition using a computer to convert it into a first written message in the first language; c) the first written message in the first language is converted by automatic computer translation into a second written message in the second language; d) a second message corresponding to the second written message is output; e) the first written message corresponding to the first voice message is displayed before the automatic computer translation conversion; and/or f) the second written message is displayed after said automatic computer translation conversion. The reply follows the same path.

Description

"Method and device for transforming a first voice message in a first language, into a second message in a predetermined second language"

The present invention relates to a method and a device for transforming a first voice message spoken in a first language, into a second voice message or written in a predetermined second language.

The present invention also relates to a method and a device for allowing the dialogue of at least two interlocutors speaking different languages, by implementing the aforementioned method.

Obviously, simultaneous translation is known by specialized interpreters who know both the original language and the language into which translation is required. In such a method, the following steps are implemented: a) a first voice message spoken in the first language is received by means of a microphone; b) translating this first voice message into a second voice message spoken in the second language; c) this second voice message is transmitted.

Such simultaneous translation requires the permanent presence throughout the duration of the operation of several interpreters who take turns to ensure such permanence. On the other hand, it is not always easy to find an interpreter for certain little-known languages.

Furthermore, methods and devices are known which allow automatic translation by computer of a written text written in a first language into a written text written in a second language.

It even exists in France, on the network of MINITEL, a multilingual dialogue service by transmission of written messages between two interlocutors of different languages addressing each other via pseudonyms: each interlocutor types on his keyboard a written message which is immediately and automatically translated by a computer in the language of the second caller.

It is also known to carry out vocal analyzes with a computer, to transform a vocal message into a written message, and vocal syntheses to transform a written message into a vocal message.

Finally, devices are known which make it possible to transform a first voice message spoken in a first language into a second voice message spoken in a second language.

Document EP-A-0 311 416 describes in particular this transformation using a translation communication system comprising a central translation device. Documents EP-A- 0 262 938, COMPUTER vol. 24, n ° 6, June 1991, p. 36 to 50, ICASSP 91, vol. 2,

May 14, 1991, p.793 to 796, and US-A-4,882,681 present similar systems, that is to say which are based on a device and a method in which a recording of a first message is carried out voice, a recognition of this first voice message, an automatic translation of this first recognized message into a second message, a voice synthesis and therefore a transformation of this second message into a second voice message and an emission of this second voice message.

However, none of these devices plans to go through a written phase as part of the transformation of a voice message into a first language in any message, i.e. voice or written in a second language. So much so that in the case of a conversation between two interlocutors, there is no way to keep track of what has been said, and above all to visualize what is said, and how it is translated. in order to make a correction if necessary.

On the other hand, simultaneous voice-to-voice translation, in the context of conferences for example, has the disadvantage of a cacophony: we hear in the headset the translation of what is said by the lecturer, but we hear also from the outside what the lecturer says, the sound insulation is not always perfect. This cacophony can cause confusion in understanding and is not very comfortable.

The invention therefore aims to overcome these drawbacks.

It relates to a transformation process as previously described which makes it possible to memorize and view the first voice message and its translation, and also possibly the second voice message and the following voice messages and their translations, and thus the content and the sequence of a dialogue.

It also relates to such a method and a device implementing this method, which gives another solution for simultaneous translation in particular in the context of conferences, for example, and which makes it much more user-friendly. They give in particular the possibility to the interlocutors to follow on a screen the different speeches with a sentence by sentence translation while possibly having the choice of the translated language. Finally, it relates to such a device which allows rapid transformation and therefore normal conversation.

To this end, the invention relates to a method for transforming a first voice message spoken in a first language, into a second message in a second predetermined language, this method consisting of the combination of the following steps: a) receiving by means of a microphone or the like a first voice message spoken in the first language; b) by automatic voice recognition by computer, a voice analysis of this first voice message is made to transform it into a first message written in the first language; c) this first message written in the first language is transformed, by automatic computer translation, into a second message written in the second language; d) a second message is sent corresponding to the second written message,

with the following additional steps:

e) before the transformation by automatic computer translation, the first written message corresponding to the first voice message is displayed by means of first storage and display means, and / or f) after the transformation by automatic translation by computer, the second written message is displayed, by means of second storage and display means.

Recording and text-to-speech can be performed simultaneously. Thus, one can visualize the message that one says as well as its translation, which makes it possible to modify his speech if one notes an error or a possible incomprehension on the part of the other interlocutor.

On the other hand, given that the time for speech recognition is very short and practically immediate, and that the translation of a sentence by machine translation by computer requires only one to two seconds, the time necessary to implement the method of the invention is limited to a few seconds.

The second message can be a written message, but also a voice message. In this case, a voice synthesis by computer of the second written message is provided in a second voice message which is then transmitted.

According to another aspect of the invention, the device for transforming a first message in a first language, into a second voice message spoken in a second predetermined language, for implementing the method according to the invention, consists of the combination the following means: a) means for receiving and storing the first voice message; b) automatic voice recognition means by computer, adapted to make a voice analysis of this first voice message to transform it into a first message written in the first language; c) means of automatic computer translation to transform this first message written in the first language into a second message written in the predetermined second language; d) means for transmitting a second message corresponding to the second written message; e) first means for storing and viewing the first message written in the first language, and / or f) second means for storing and viewing the second message written in the second predetermined language.

The means for receiving and the voice recognition means can be integrated together.

The invention also relates to a method for enabling the dialogue of at least two interlocutors speaking different languages, consisting in transforming the voice message spoken in the language of one of the interlocutors into a vocal message spoken in the language of the other. interlocutor and in the reverse transformation, characterized in that at least one of the transformations is carried out by the implementation of the method described above and for which it is expected that the second message sent is a voice message.

Finally, it relates to a system for allowing the dialogue of at least two interlocutors speaking different languages, comprising first means for transforming the voice message spoken in the language of one of the interlocutors into a voice message spoken in the language of the '' other interlocutor, connected to second means for the reverse transformation, characterized in that at least the first transformation means comprise a first device according to the invention, the link between this device and the second transformation means being a link allowing the transmission not only of voice messages but also of written messages, in particular by modem.

Other features and advantages of the present invention will appear in the detailed description below.

In the accompanying drawings, given solely by way of nonlimiting examples:

- Figure 1 is a block diagram illustrating the implementation of the method according to one invention;

- Figure 2 is a schematic view of an embodiment of the system according to the invention to allow the dialogue of two interlocutors speaking in different languages, in particular by long distance telephone link.

FIG. 1 shows schematically two devices according to the invention each adapted to transform a first voice message spoken in a first language into a second voice message spoken in a second predetermined language.

The languages transmitted are preferably modern languages of all types, including languages using ideograms or hieroglyphs or any other alphabet comprising drawings.

According to the invention each device 1 _A , ¹ n ₁ comprises, connected in series: a) means M _A , M _β , for example a microphone, for receiving and storing the first voice message; b) means R _A , R _β , of automatic voice recognition by computer, adapted to make a voice analysis of this first voice message to transform it into a first message written in the first language; c) of the first storage and display means _A1 , Vg-, of the first messenger written in the first language and d) means T,, Tg, of automatic computer translation to transform this first message written in the first language into a second message written in the predetermined second language; e) second storage and display means V _A2 / ^V B2 ^ ^{U second} message written in the second predetermined language. f) means S _A , S _β , of voice synthesis by computer to transform this second written message into a second voice message spoken in the second language; g) means E _A , E _β , for example a loudspeaker or an earphone or a headset, for transmitting this second voice message.

The means _A or Mr, for receiving and the means R _A or Rg of voice recognition can be integrated together.

Thus, in the context of a conference for example, the lecturer who speaks in a first language can be understood by all the listeners who can also view the terms of his speech, after translation, on their device 1 _A , 1 _B.

To avoid a cacophony, one can also plan to transmit only the message written in the first language.

In this case, the devices 1A and l _β do not comprise means S _A and S _β for voice synthesis by computer.

The invention also relates to a system allowing the dialogue of at least two interlocutors speaking different languages. This system comprises first LV1 transformation means for transforming a voice message spoken in the language of one of the interlocutors into a voice message spoken in the language of the other interlocutor, connected to second LV2 transformation means for the reverse transformation .

According to a first embodiment, the first LVl transformation means comprise a first device 1 _A as described above, the link between this device 1 _A and the second transformation means being a link 3 allowing the transmission not only of voice messages, but also written messages.

In this case, the second LV2 transformation means are means of known type allowing reception, voice recognition, translation and voice synthesis, and the reverse link between the second LV2 storage means and the first device 1 _A can be a link 2 carrying only voice messages.

According to a second embodiment, the second storage and display means LV2 also include a second device l _β of the type previously described.

The link between said first 1 _A and second l _β devices is then a bidirectional link 7 allowing the transmission not only of voice messages, but also of written messages.

As shown in FIG. 2, each of the devices 1 _A , l _β comprises a microcomputer 4 provided with a screen 8, as well as a microphone 5 and a speaker 6 or earphone connected to the corresponding microcomputer 4.

Each microcomputer 4 comprises means for receiving the voice message, and software for voice recognition, automatic translation and voice synthesis necessary to transform the voice message spoken in its language by the corresponding interlocutor into a voice message spoken in the 'other language and transmitted to the other interlocutor, the screen 8 serving as first and second display means V _A1 and V _A2 or V _B1 and V _β2 .

Each microcomputer 4 is thus coupled to the telephone link 7, which can be a wire and / or radio link or the like, and comprise as many wires or channels as necessary.

the written message translated and transmitted by one of the devices then appears on the screen 8 of the microcomputer of the other device. Screen 8 therefore consists of two areas, an area for the message as it has been said and an area for the corresponding translated message. The keyboard 9 of each microcomputer makes it possible to enter into memory and to display on screen 8 all the instructions necessary for carrying out the dialogue operation, telephone number of the other party, choice of language in which the message must be translated, possible modifications of the text to be transmitted if a translated term is not suitable, etc. The assembly consisting of the keyboard 9, the microphone 5 and the speaker 6 can of course be replaced by a telephone handset connected to the corresponding microcomputer 4 and comprising a microphone of the quality required for voice recognition, and a dial call allowing the introduction of the aforementioned order data. In the preferred embodiment shown, the microcomputer 4 of each interlocutor directly develops the voice message spoken in the other language, and transmits this voice message in addition to the message written by the link 7 to the microcomputer of the other interlocutor. In this embodiment, each device preferably comprises means for receiving and using learning words spoken in its language by the corresponding interlocutor to facilitate the step of voice recognition of the latter's words. These learning words for the computer are for example recorded, by being transformed into a program allowing the piloting of the voice recognition, on a floppy disk which can be read by the microcomputer and which each interlocutor can keep with him.

Speech recognition, automatic translation and text-to-speech software are software programs known in themselves and which need not be described in detail here.

The voice recognition software is thus, for example, software developed in the United States by the company DRAGON, or software developed by the company

IBM. We know that such software is capable of transforming direct and real-time dictated text into written text with a speech recognition quality index of around 97 or 98%. The software for automatic computer translation of a text written from one language to another is for example the software developed by the company SYSTRAN. Text to speech software is software of any known type available on the market.

The microcomputer 4 is in each case a computer having the operating speed and the memory capacity necessary to contain and use the various software required and to make them work if later. This microcomputer also includes all the interface cards necessary to connect it to the microphone 5, to the speaker 6 and to the telephone link 7. The microphone 5 and the speaker 6 can also be replaced by a headset equipped with '' a built-in microphone (not shown), or by a handset.

The microcomputer 4 may also not include one or the other of the aforementioned software and be connected to a central computer, or to a network of computers and servers, adapted to make available to the user (s) corresponding time-sharing software.

One can also provide a printing means, such as a printer, connected to one of the devices 1 _A or lg, or both, and for printing the corresponding written messages.

It will be understood that the method for transforming a first voice message spoken in a first language, into a second message in a second predetermined language, comprises the following steps:

a) a first voice message spoken in the first language is received by means of a microphone or the like; b) by automatic voice recognition by computer, a voice analysis of this first voice message is made to transform it into a first message written in the first language; c) this first message written in the first is transformed by automatic computer translation language in a second message written in the second language; d) a second message is sent corresponding to the second written message,

with the following additional steps: e) before the transformation by automatic translation by computer, the first written message corresponding to the first voice message is displayed, via the first display means, and / or f) after the transformation by translation automatically by computer, the second written message is viewed, via the second viewing means.

Recording and voice recognition can be performed simultaneously.

To allow the dialogue of at least two interlocutors speaking different languages, we translate successively and alternately the words of each of the interlocutors locally to transmit them to the other interlocutor in the form of a voice message spoken in the latter's language .

Of course, the present invention is not limited to the embodiments which have just been described, and many modifications and modifications can be carried out on the latter without departing from the scope of the invention.

It is thus possible, for example, to integrate the means of voice recognition and speech synthesis in the two languages in a single computer 4, the telephone link 7 being used only for transmitting in one direction and in the other voice or written messages. spoken in the second predetermined language.

It is of course possible to replace the conventional telephone handset with a telephone apparatus comprising a handset and a conventional base equipped with a viewing window, a loudspeaker and a call dial. It is also possible to produce a particular base comprising all the means of the microcomputer for implementing all or part of the functions described above.

It will be understood that the letters inscribed in the diagram of FIG. 1 respectively symbolize the following functions:

E = Transmitter

M = Micro

R = Speech recognition

T = Translation S = Text-to-speech

V = Visualization

Claims

1. method for transforming a first voice message spoken in a first language, into a second message in a second predetermined language, this process consisting of the combination of the following steps: a) a first spoken voice message is received by means of a microphone in the first language; b) by automatic voice recognition by computer, a voice analysis of this first voice message is made to transform it into a first message written in the first language; c) this first message written in the first language is transformed by automatic computer translation into a second message written in the second language; d) a second message is sent corresponding to the second written message,

with the following additional steps:

e) before the transformation by automatic translation by computer, one visualizes, by means of first visualization means ( _A ,, ^v Bl) ' ^e First written message corresponding to the first voice message, and / or f) after the transfer ^or mation by automatic computer translation, we visualize, by means of second visualization means (V _A2 , ^V B2 ^ '^ ^{e second} of written message.

2. Method according to claim 1, characterized in that after viewing the second written message, this second written message is transformed by voice synthesis by computer into a second voice message spoken in the second language and this second voice message is transmitted.

3. Method for enabling the dialogue of at least two interlocutors speaking different languages consisting in transforming the voice message pronounced in the language of one of the interlocutors into a vocal message pronounced in the language of the other interlocutor and in reverse transformation, characterized in that at least one of the transformations takes place by the implementation of the method according to claim 2.

4. Method according to claim 3, characterized in that all the transformations are carried out by the implementation of the method according to claim 2.

5. Method according to any one of claims 1 to 4, characterized in that previously recorded training words for the computer spoken by each interlocutor.

6. Device for transforming a first voice message pronounced in a first language, into a second message in a second predetermined language, for implementing the method according to claim 1, consisting of the combination of the following means: a) means ( E _A , M _A , E _β , M _β ) to receive and store the first voice message; b) automatic voice recognition means (R _A , R _β ) by computer, adapted to make a voice analysis of this first voice message to transform it into a first message written in the first language; c) means of automatic translation (T _A , T _β ) by computer to transform this first message written in the first language into a second message written in the second predetermined language; d) means for transmitting a second message corresponding to the second written message;

with the following means: e) first means of memorization and display (V _A1 , V _βl ) of the first message written in the first language, and / or f) second means of memorization and display (V _A2 , V _β2 ) of the second written message in the second predetermined language;

7. Device according to claim 6, characterized in that it further comprises voice synthesis means (S _A , S _β ) by computer to transform the second written message into a voice message spoken in the second language, the means for transmitting the second signal corresponding to the second written message being means for transmitting voice signals.

8. System for enabling the dialogue of at least two interlocutors speaking different languages, comprising first transformation means (LVl) for transforming the voice message spoken in the language of one of the interlocutors into a voice message spoken in the language from the other party, connected to second transformation means (LV2) for the reverse transformation, characterized in that at least the first transformation means (LVl) comprise a first device (1A) according to claim 7, the connection (3) between this device and the second transformation means being a link allowing the transmission not only of voice messages but also of corresponding written messages.

9. System according to claim 8, characterized in that the second transformation means (LV2) comprise a second device (1g) according to claim 7, the link (7) between the first and second devices being a bidirectional link allowing the transmission not only messages voice, but also corresponding written messages.

10. System according to claim 8 or 9, characterized in that each of said first and / or second devices comprises a microcomputer (4) equipped with means for recording voice message, voice recognition, automatic translation and synthesis voice necessary to transform the voice message pronounced in his language by the corresponding interlocutor into a vocal message pronounced in the other language and transmitted to the other interlocutor, as well as means (5) adapted to receive and to send in a receiver loudspeaker (6), headset or handset type, the voice message of the other interlocutor translated by the microcomputer (4) of the latter into the language of the corresponding interlocutor, the first and second means of storage and display ( _A1 , V _A2 ; V _βl , V _β2 ) being a screen (8) on which appears the written message and / or the message translated and transmitted by the other device.

11. System according to one of claims 8 to 10, characterized in that the first (1 _A ) and / or the second (I _β device comprises means for receiving and using learning words for the computer spoken in its language by the corresponding interlocutor to facilitate the voice recognition stage of the latter's words.

12. System according to any one of claims 8 to 11, characterized in that it further comprises at least one printing means connected to at least one of said devices (1 _A or l _β ) for printing at least one of said written messages.