GB2469329A

GB2469329A - Combining an interpreted voice signal with the original voice signal at a sound level lower than the original sound level before sending to the other user

Info

Publication number: GB2469329A
Application number: GB0906201A
Authority: GB
Inventors: Benjamin Cohen; Maciek Kaminski; Patrick Smarzynski
Original assignee: WEBINTERPRET Sas
Current assignee: WEBINTERPRET Sas
Priority date: 2009-04-09
Filing date: 2009-04-09
Publication date: 2010-10-13
Also published as: GB0906201D0

Abstract

A method for interpreting a conversation comprising conversation signals between at least two users speaking at least two different languages through a communication network comprising the steps of sending a first and second input signal (voice) in a first language from a first and second user module to an interface module; interpreting a selected one of first or second input signal from the first language into the second; determining during a predetermined time delay the existence of the interpreted selected input signal; decreasing the sound level of the first or second input signal (voice) to a predetermined ratio of the original sound level; sending the second processed signal to the other user module. Thus, a combined voice signal is sent to each user, made up of the interpreter voice signal and the voice signal of the corresponding user at a sound level lower than the original sound level. A level between 10-30% of the first signal has been found to provide the best combination of interpreter signal and original signal. The user can clearly hear the interpreter signal in a preferred language whilst the emotional and background noise content can be heard from the original signal.

Description

Method and System for Interpreting a Conversation

Field of the invention

The present invention relates to a method and system for interpreting a conversation between users speaking different languages over a communication network, particularly but not exclusively over an Internet network.

Background of the invention

Nowadays, the use of an interpreter for interpreting a conversation between two or more parties speaking different languages is very common. The interpretation is a generally consecutive interpretation i.e. the interpretation occurs after the speaker finishes his speech. The interpretation can take place during a conference or any meeting where the parties speak different languages. In this situation, the interpreter and the parties are located at the same place during the interpretation.

With the development of communication technologies, improved interpretation systems have appeared. Improved interpretation systems allow an interpretation of a conversation even when parties are located remotely from one another through a communication network.

The patent application US 2006/0165225 discloses a telephone interpretation system comprising two parties and an interpreter all of whom are connected through the telephone network. US 2006/0165225 provides a method for improving the quality of the interpreter voice signal during the interpretation of the interpreter. The method provides a suppression of overlapping voice signals while the interpreter is interpreting the conversation between the two parties. This means that the method suppresses any voice signal from the first party or from the second party while the interpreter is interpreting the first party speech or the second party speech. Such a method supposedly provides a better quality for the interpreter voice signal as there is no other signals which overlap with the interpreter voice signal during the interpretation.

During certain interpretations where there is no visual communication between parties talking to each other, there are yet more issues. When the interpretation occurs, each party only hears the voice of the interpreter without seeing the other party. One party does not hear the voice of the other party. The interpreter interprets only the content of the speech from one party to another. The interpreter does not interpret the context of the speech such as the tone of the voice of one party or the emotions of the party such as anger, enthusiasm or sadness etc. Such a lack of context during the interpretation provides a great loss for the quality of the interpretation. Indeed, one party is not informed of the manner in which the other party is speaking. The absence of context can introduce misunderstandings between both parties during the discussion.

Such misunderstandings can have important negative effects when the discussion deals with a business matter.

Summary of the invention

An object of the present invention is to alleviate at least some of the

problems associated with the prior art systems.

A further object of the present invention is to provide a method and system for interpreting a conversation comprising conversation signals between at least two users speaking at least two different languages each at a respective original sound level wherein the conversation occurs through a communication network with a respective user module for each user, characterized in that the method comprises the steps of sending a first input signal in a first language from a first user module to an interface module located between the first user module and the second user module; processing the first input signal from the first user module with a second input signal from the second user module to the interface module, to produce a first processed signal; selecting one of the first and second input signals in the first processed signal; interpreting the selected input signal from the first language into the second language to produce a corresponding selected input signal; determining during a predetermined time delay the existence of the interpreted selected input signal; decreasing the sound level of the first input signal to a predetermined ratio of the original sound level according to the duration of existence of the interpreted selected input signal in order to produce the second processed signal with the interpreted first input signal and the predetermined ratio of original sound level of the first input signal; sending the second processed signal to the other user module.

Another object of the present invention is to provide a system for a system for interpreting a conversation comprising conversation signals between at least two users speaking at least two different languages each at a respective original sound level wherein the conversation occurs through a communication network with a respective user module for each user, characterized in that the system comprises a first user module and a second user module for respectively sending a first input signal and a second input signal; an interface module located between the first user module and the second user module for receiving the first input signal and the second input signal and for sending an interpreted first input signal to the second user module; an adaptation module for combining the interpreted first input signal with the first input signal at a sound level lower than the original sound level.

Brief description of drawings

Reference will now be made, by way of example, to the accompanying drawings, in which: Figure 1 is a schematic overview of the system of interpretation, by way of example, in accordance with one embodiment of the present invention; Figure 2 is a flow chart of the method steps, by way of example, in accordance with one embodiment of the present invention; Figure 3 is a schematic graph representing a specific step of the method as shown in Figure 2, by way of example, in accordance with one embodiment of the present invention; Figure 4 is a schematic view of another embodiment of the system of interpretation, by way of example, in accordance with one embodiment of the present invention.

Detailed description of the embodiments

In a first instance, the present invention will be described in the context where two users and one interpreter are involved in an interpretation process. The two users do not speak the same language.

Figure 1 shows a first user speaking a first language and using a first user module 100. Figure 1 also shows a second user speaking a second language and using a second user module 200. The user modules 100, 200 can be an electronic device connected to the Internet network. The user module 100, 200 comprises a communication module 102, 202. The communication module 102, 202 provides transmission of the voice signal of the corresponding user of the user module over the Internet. The communication module 102 can also receive voice signals transmitted from the Internet to the corresponding user of the user module. The communication module 102 comprises for example a microphone, a loudspeaker and/or a headset and communication software. In order to determine the preferred language for each user during the discussion, each user has to define the language before the discussion takes place.

Thus, each user has to indicate a so-called user language criterion. The user language criterion either refers to the language spoken by the user or a preferred language when the first language is unavailable.

Figure 1 shows an interpreter located remotely from the first and the second users. The interpreter can interpret the speech of the first or the second user. This means that the interpreter has to decide from which language to which other language the interpretation will occur based on the speech from the parties. The interpreter is using an interpreter module 300. The interpreter module can have the same characteristics as a user module 102, 202 as described above. Similar to the user module 102, 202, the interpreter module 300 comprises a communication module 302.

The interpreter module also comprises a switch module 304 in order to allow the interpreter to choose into which language the interpret will interpret.

The interpreter listens to the conversation and determines who is speaking (the first user). The interpreter interprets the speech of the first user from the first language to the second language. Then, the interpreter processes the switch module in a first manner before interpreting the speech of the first user. Thus, during the interpretation process, the interpreter module sends the interpreter voice signal to the second user module. In this situation, the interpreter voice signal refers to the interpreted speech of the first user. As the conversation progresses, the interpreter will also interpret the speech of the second user from the second language to the first language. When this occurs, the interpreter processes the switch module 304 in a second manner before interpreting the speech of the second user. Thus, during the interpretation process, the interpreter module sends the interpreter voice signal to the first user module. In this situation, the interpreter voice signal refers to the interpreted speech of the second user.

As shown in figure 1, the first user module 100, the second user module and the interpreter module 300 are all connected to a conference server 400. The conference server 400 relates to an Internet platform.

This means that the user modules 100, 200 and the interpreter module 300 can access the conference server 400 at a specific website on the Internet at a specific Internet address.

The conference server 400 can both receive and transmit voice signals.

The conference server 400 receives voice signals from the first user module 100, from the second user module 200 and from the interpreter module 300. The conference server 400 also transmits voice signals to the first user module 100, to the second module 200 and to the interpreter module 300.

The conference server comprises specific modules. As shown in figure 1, the conference server 400 comprises a comparator module 401. The comparator module 401 stores the user language criterion as defined by each user. The comparator module 401 also compares all the user language criteria of all users. Thus, the comparator module 401 determines which users can hear and talk each other without needing an interpretation from the interpreter. The conference server 400 also comprises a processing module 402. The processing module 402 can combine two different voice signals received at the conference server 400 from the first user module 100 and from the second user module 200.

Thus, the processing module 402 provides a processed signal comprising distinct voice signals from the users.

As shown in figure 1, the conference server 400 also comprises a detection module 404. The detection module 404 detects the presence of the interpretation voice signal received from the interpreter module 300.

As shown in figure 1, the conference server 400 comprises an adaptation module 406. The adaptation module 406 produces the interpreter voice signal referring to the interpretation of a speech from a user in combination with the voice signal of the corresponding user. Such a combination occurs only when the detection module 404 detects an interpretation voice signal. Thus, in the present example, the combination provides a combined signal comprising the interpretation voice signal and the voice signal of the first user at a sound lever lower than the original sound level.

The determination of the correct or optimal percentage sound level of the first signal to be combined with the interpreted voice signal is not straightforward. In fact, the precise value is dependent on a number of factors. These factors include: the rate at which any change occurs in the sound levels; the ease with which the listener can hear the combined interpreted voice signal and percentage sound level of the first signal without being confused; and the general listening ease for a user. In the prior art the objective of the inventions is to avoid any contamination or combination of the interpreted signal with another voice signal.

Accordingly the present invention is counterintuitive to the prior art. The object of the present invention is to combine a certain level of the first signal to enable and listener to have a greater understanding of the conversation, where that understanding is not associated with the words but more with emotions and other feelings which are usually present in a voice signal. After considerable experimentations, a level around less than 50% or between about 10% and 30% for the first signal has been found to provide the best combination of interpreter signal and original signal. The user can clearly hear the interpreter signal in a preferred language whilst the emotional and background noise content can be heard from the original signal. The overall effect is a considerably improved listening experience when compared to traditional interpreter systems and methods.

Therefore, the adaptation module 406 provides a combined voice signal made up of the interpreter voice signal at the original sound level combined with the voice signal of the corresponding user at a sound level lower than the original sound level. The rate of change of the sound level can be predetermined which is a further important feature in the present invention. The rate can be for example of less than 50% or between about 10% and 30% of the original sound level. The adaptation module 406 then sends a combined signal to the other user waiting for the interpretation through the interpreter voice signal. Thus, in the present invention, the second user can hear both the interpreter who interprets the speech of the first user and the voice signal of the first signal at a sound level of less than 50% and preferably less than 10% and 30% of the original sound level of the first user voice signal.

In the above described example where there are only two users speaking different languages, neither user can directly listen to the other user.

The steps of the process will now be described based on the example where two users speak in different languages and one interpreter interprets the conversation between the two users.

At the beginning of the process, each user module 100, 200 and the interpreter module 300 are connected to the conference server 400 through a dedicated web site located on an Internet Protocol network (not shown). As shown in figure 2, in a step 500, each user has to define the user language criterion in order to determine the language the user wants to use for speaking and for listening to other users. For example, the first user defines French as the language for the first user and the second user defines English as the language for the second user. It will be appreciated there may be other language combinations. In addition, if there are restrictions on the number of available translators, the user language criterion may be restricted in some way. Further, a user may wish to speak in one language and listen in others. Each user module 100, 200 sends the information of the user language criterion to the comparator module 401 of the conference server 400. Then, in a step 502, the comparator module 401 compares all the user language criteria in order to determine whether the first user language criterion is different from the second user language criterion. In a step 504, each user has to establish a specific connection for joining the discussion. When the discussion begins, the conference server receives each voice signal from each user module in a step 506. Then, in a step 508, the conference server uses the result from the comparator module 401 to determine where to send each voice signal from each user module. If the comparator module 401 determines that all the users' language criteria are different, then the conference server 400 has to send all users' voice signals to the interpreter module 300. If the comparator module 401 determines that there are some identical user's language criteria, then the conference server 400 can directly transmit the voice signal from a user to another user having the same user language criterion as indicated in step 509. In the present situation, the French first user and the English second user do not have the same user language criterion. This means that both users need an interpretation in order for the discussion to progress. Therefore, the processing module 402 processes both signal voices from the first and the second user and sends them to the interpreter module 300 as indicated in step 510 The interpreter has to chose which voice signal to interpret based on the discussion. The interpreter can decide at any time to select a specific voice signal by processing the switch module 304. For the following steps it is assumed that the interpreter is interpreting the speech from the French user.

As shown in figure 2, in a step 512, the interpreter module sends the interpreter voice signal back to the conference server 300. The interpreter has to process the switch module 404 to specify which user has to receive the interpreter voice signal. In a step 514, the detection module 404 of the conference server 400 detects the arrival of the interpreter voice signal at the conference server 400. In a step 516, before sending the interpreter voice signal to the English user module 200, the adaptation module 406 combines the interpreter voice signal with the French voice signal at a predetermined sound level. The sound level can represent for example between about 10% and 30% of the original sound level of the French user. Then in a step 518, the conference server sends the combined signal made of the interpreter voice signal and a ratio of less than 50% or between 10% to 30% of the sound level of the French user voice signal to the English user module 200.

The process occurring at step 514 will now be described in greater details.

In step 514, the detection module 404 detects the arrival of the interpreter voice signal which refers to the interpretation of the first user's speech.

Then, the detection module 404 measures the duration of the interpreter voice signal. As shown in figure 3, before the arrival of the interpreter voice signal, the first user voice signal is at an original sound level of 100%, for example. The detection module 404 combines the interpreter voice signal with the corresponding first user voice signal according to the determination of the duration of the interpreter voice signal. As shown in figure 3, the detection module 404 measures the duration of the interpreter voice signal during a predetermined time delay. The time delay is based on technical parameters to provide a decrease or an increase of the sound level of a voice signal. The time delay is also based on the consideration of the physical capacity of a human user to hear both the interpreter voice signal and the corresponding user voice signal in a natural way. After considerable experimentations, the calculation of the time delay provides a predetermined time delay between 3 and 6 seconds for decreasing the sound level of the user voice signal for combination with the interpreter voice signal and a predetermined time delay between Sand 10 seconds for increasing the sound level of the user voice signal. Thus, the detection module 404 needs about twice the time for increasing the sound level of the user voice signal than for the decreasing part. A predetermined time delay of 5 seconds is selected for a specific example as follows. In a specific example, as shown in figure 3, in part 1 of the graph, the duration of the interpreter voice signal exceeds 5 seconds. As the duration of the interpreter voice signal exceeds the predetermined time delay, the detection module 404 starts decreasing the sound level of the user voice signal as shown in part 2 of the graph in figure 3. The duration of such a decrease is also between 3 and 6 seconds as mentioned above. After such a decrease, the adaptation module 406 can then combine the interpreter voice signal with the corresponding user voice signal to produce a combined signal. Meanwhile, the detection module 404 goes on regularly measuring the duration of the interpreter voice signal on a time basis related to the predetermined time delay. This means that in the present example, the detection module 404 determines each 5 seconds if the interpreter voice signal is still present. As soon as the detection module 404 detects that there is no more interpreter voice signal, the detection module 404 starts increasing the sound level of the user voice signal in order to reach the original sound level of the user voice signal as shown in part 4 of the graph in figure 3. The duration of such an increase is longer than the duration of the decrease as above mentioned.

As shown in figure 4, in another embodiment there may be three French users, two Russian users and one interpreter who interprets from French into Russian and from Russian into French. In the example where the French user speaks first, the French user voice signal 1 directly goes through the conference server to the two others French users and to the interpreter. The two French users and the interpreter receive the French user voice signal 2 at the original sound level. Then, the interpreter sends the interpreter voice signal 3 to the conference server. The conference server sends the combined voice signal 4 comprising the interpreter voice signal combined with the French user voice signal at a sound level lower than the original sound level to the two Russian users. Other features of this example are based on the earlier described example.

For different language combination and for more languages, the arrangement may become more complex. However, the main principles are unaffected. The voice signals of each party are combined with the interpreted signal as described above to add greater content for the listening experience such as emotion, expression and back ground noise from each party or location.

The present invention provides a method and system for processing a simultaneous interpretation which is more convenient and accurate than a consecutive interpretation. In addition, such a simultaneous interpretation occurs through a web browser, without any specific hardware such as interpreter booth or headsets which are needed in the consecutive interpretation. Such a simultaneous interpretation may occur remotely.

It will be appreciated that this invention may be varied in many different ways and still remain within the intended scope and spirit of the invention.

Furthermore, a person skilled in the art will understand that some or all of the functional entities as well as the processes themselves may be embodied in software, or one or more software enabled modules and/or devices.

Claims

Claims 1. A method for interpreting a conversation comprising conversation signals between at least two users speaking at least two different languages each at a respective original sound level wherein the conversation occurs through a communication network with a respective user module (100, 200) for each user, characterized in that the method comprises the steps of: -sending a first input signal in a first language from a first user module (100) to an interface module (300) located between the first user module (100) and the second user module (200); -processing the first input signal from the first user module (100) with a second input signal from the second user module (200) to the interface module (300), to produce a first processed signal; -selecting one of the first and second input signals in the first processed signal; -interpreting the selected input signal from the first language into the second language to produce a corresponding selected input signal; -determining during a predetermined time delay the existence of the interpreted selected input signal; -decreasing the sound level of the first input signal to a predetermined ratio of the original sound level according to the duration of existence of the interpreted selected input signal in order to produce the second processed signal with the interpreted first input signal and the predetermined ratio of original sound level of the first input signal; -sending the second processed signal to the other user module (200).
2. A method as claimed in claim 1, wherein the step of sending a first input signal in a first language to an interface module (300) comprises the step of sending the first input signal to a server (400).
3. A method as claimed in any of claims 1 or 2, wherein the step of processing the first input signal with a second input signal comprises the step of sending the second input signal to the server (400).
4. A method as claimed in any preceding claims, wherein the step of determining during a predetermined delay the existence of the interpreted first input signal comprises the step of measuring the time duration of the interpreted first input signal.
5. A method as claimed in any preceding claims, wherein the step of decreasing the sound level of the first input signal to a predetermined ratio comprises the step of decreasing the sound level to a ratio of less than 50 percent of the original sound level of the first input signal.
6. A system for interpreting a conversation comprising conversation signals between at least two users speaking at least two different languages each at a respective original sound level wherein the conversation occurs through a communication network with a respective user module for each user, characterized in that the system comprises: -a first user module (100) and a second user module (200) for respectively sending a first input signal and a second input signal; -an interface module (300) located between the first user module (100) and the second user module (200) for receiving the first input signal and the second input signal and for sending an interpreted first input signal to the second user module (200); -an adaptation module (406) for combining the interpreted first input signal with the first input signal at a sound level lower than the original sound level.
7. A system as claimed in claim 6, wherein the system further comprises a server (400) for receiving and processing the first input signal and the second input signal.
8. A system as claimed in any of claims 6 or 7, wherein the system further comprises a processing module (402) for combining the first input signal and the second input signal.
9. A system as claimed in any of claims 6 to 8, wherein the system further comprises a detection module (404) for detecting the interpreted first input signal.
10. A system as claimed in any of claims 6 to 9,wherein the system further comprises a switch module (304) for selecting one of the first and the second input signals to interpret.