JP2002320037A - Translation telephone system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a translation telephone system that can output a synthesized voice translated in the order of uttered sounds even when the uttered sequence of uttered sounds differs from the completion sequence of voice processing. SOLUTION: In the case of applying processes of voice recognition, language translation and synthesis voice to a series of uttered voices given to a telephone set 20 (30) and outputting the result to the telephone set 30 (20), the translation phone system 10 captures the voice of utterers. The translation phone system 10 stores utterance time information of the uttered voice to a memory by each utterance. Then the translation phone system 10 acquires the synthesis voice from the memory in the sequence according to the utterance time information and provides an output when outputting the translated synthesis voice.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a translation telephone system, and more particularly to, for example, a translation telephone system for translating a conversation between two speakers using different languages via a telephone line.

[0002]

2. Description of the Related Art As a conventional translation telephone system of this type, there has been a system in which voice processing of speech recognition, language translation, and speech synthesis is performed on a speaker's uttered voice using a distributed network system.

[0003]

However, in the prior art, the order of speech is not taken into account, and the speech processing does not end in the order of speech depending on the length of the inputted speech voice, and the output of the speech processing result is not obtained. There is a possibility that the order of the uttered voices may not be in order.

[0004] Therefore, a main object of the present invention is to provide a translation telephone system capable of outputting voice processing results in the utterance order of uttered voices regardless of the ending order of voice processing.

[0005]

SUMMARY OF THE INVENTION The present invention relates to a translation telephone system for translating a conversation carried out via a telephone line among a plurality of speakers using different languages. Capturing means for capturing, a detecting means for detecting a capturing time of the uttered voice, a creating means for generating a synthesized voice representing a translation content of the uttered voice, and an output means for outputting the synthesized voice in an order according to the capturing time. , A translation telephone system.

[0006]

According to the present invention, in a translation telephone system for translating a conversation performed via a telephone line between a plurality of speakers using different languages, the translated synthesized speech is arranged in the order of speech of each speaker. Is output. That is, the capturing means captures the uttered voice of each speaker, and the detecting means detects the capturing time of the uttered voice by the capturing means. The creating unit creates a synthesized speech in which the content of the uttered speech captured by the capturing unit is translated. Then, the output means outputs the synthesized voices in the order of capturing the uttered voices based on the captured time.

The plurality of uttered voices fetched by the fetching means are processed by the creator to perform speech recognition processing for recognizing the contents of the uttered voices, language translation processing for translating the processing results of the speech recognition processing, and processing results of the language translation processing. Are performed in parallel with each other to generate a synthesized voice representing the contents of the above.

[0008]

According to the present invention, even if the end of the voice processing is not in the utterance order of the utterance voice, the processing result of the voice can be read out in the utterance order.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0010]

Referring to FIG. 1, a translation telephone system 10 of this embodiment includes a telephone line 40 having a plurality of telephones 20 and 3 connected thereto.
Connected with 0. Here, the telephone 20 is installed in Japan, for example, and is used by a Japanese A resident in Japan. The telephone 30 is installed in the United States, for example, and is used by a German B living in the United States.

When Japanese A wants to have a conversation with German B using the translation telephone system 10, Japanese A first accesses the translation telephone system 10. The call processing for the German B is performed by the translation telephone system 10, and when the German B answers, the call becomes available. Japanese A
Is translated from Japanese to German, and the translated synthesized speech is transmitted to German B. Similarly, the uttered voice of German B is translated from German to Japanese, and the translated synthesized voice is transmitted to Japanese A.

Referring to FIG. 2, translation telephone system 10
Is the integrated management system 10a, the database server 10
b, a plurality of speech recognition systems 10c, 10c,... 10
c, a plurality of language translation systems 10d, 10d,..., 10
d, a plurality of speech synthesis systems 10e, 10e,... 10e
And a modem 10f. The translation telephone system 10 is configured by a distributed network, and the translation of the uttered voice is realized by each voice processing of voice recognition by the voice recognition system 10c, language translation by the language translation system, and voice synthesis by the voice synthesis system 10e.

Incidentally, speech recognition, language translation and speech synthesis are performed by a plurality of speech recognition systems 10c and 10c, respectively.
c,... 10c, language translation systems 10d, 10d,.
Since the voice processing is performed by the voice synthesis system 10d, the voice synthesis system 10e, 10e,... 10e, the end time of the voice processing may be longer or shorter than the input order of voices due to the load on the system. That is, as shown in FIG. 3, the voice processing (the process of creating a synthesized voice to be output) may not always end in the order in which the voice is input to the translation telephone system 10.
In the example of FIG. 3, first, the voice 1 with a longer utterance time is input, and then the voice 2 with a shorter utterance time is input. However, the creation of the synthesized speech 2 corresponding to the speech 2 with a smaller load on the system is finished first, and then the creation of the synthesized speech 1 corresponding to the speech 1 is finished. Therefore, if the output is performed in the order of completion of the creation, the reverse occurs in the output order of the synthesized speech. In order to prevent the output order from being reversed, the utterance time of the speaker is managed, and the synthesized speech is output in the utterance order.

Next, the mechanism of the translation telephone system 10 will be described. First, the utterance times of the uttered voices of the respective speakers (Japanese A and German B) input to the translation telephone system 10 are acquired, and utterance time information indicating the utterance times is stored in the database server 10b. The content of the uttered voice of each speaker is recognized by the voice recognition system 10c. The voice recognition system 10 also extracts features of the uttered voice, and specifies the native language of the utterance source (Japanese for Japanese A, German for German B), voice quality, and gender. The specified native language information, voice quality information, and gender information are registered in the database server 10b.

The language translation system 10d translates the contents of the recognized speech voice based on the native language information registered in the database server 10b. For Japanese A, the native language information "Japanese" is registered in the database server 10b, and for Japanese A, the native language information "Japanese" is registered in the database server 10b. For this reason, the content of the speech voice of Japanese A is translated into German by the language translation system 10d, and the content of the speech voice of German B is translated into Japanese by the language translation system 10d.

The speech synthesizing system 10e is composed of the contents translated by the language translation system 10d and the database server 10b.
A synthesized speech is created based on the voice quality information and the gender information registered in. The created synthesized speech is stored in the database server 10b. The integrated management system 10a refers to the utterance time information stored in the database server 10b, acquires a synthesized voice corresponding to the earliest utterance time information from the database server, and transmits the synthesized speech to each speaker. As a result, as shown in FIG. 3, even when the synthesized speech 2 is created (stored in the database server 10b) before the synthesized speech 1, the synthesized speech 1 is output first in the utterance order, and then the synthesized speech 2 is output. Is done. If the Japanese A is a woman who speaks with a high voice, a German synthesized voice with a feminine tone and high key is transmitted to the German B. In addition, if German B is a man who speaks in a low voice, Japanese A with a masculine tone and low key is transmitted to Japanese A.

The speech recognition system 10c, language translation system 10d, speech synthesis system 10e, and database server 10b are managed by an integrated management system 10a. When accessed through the modem 10f, the integrated management system 10 controls these systems and servers and provides translation services to users. Specifically, the integrated management system 10a processes the flowcharts shown in FIGS. The flow charts shown in FIGS. 4 to 9 show the respective processes of the comprehensive management system 10a, the speech recognition system 10c, the language translation system 10d, and the speech synthesis system 10e in one flow chart. 10 and 11
Shows a part of a flowchart of a processing routine processed by the integrated management system 10a.

The access request to the translated telephone system 10 includes telephone number information of the caller. Therefore, first, in step S1, telephone number information is obtained from the access request,
In a succeeding step S3, country information of the caller is detected based on the detected telephone number information. When the Japanese A accesses the translation telephone system 10 using the telephone 20, the telephone number information includes the country code "81". Step S3
Then, the country of the caller is specified as Japan from the country code "81". In step S5, the telephone number information and country information of the caller obtained in this manner are stored in the internal memory. In step S7, the native language of the caller (the official language of the country to which the caller belongs) is stored from the stored country information. ) Is provisionally determined. When the caller is Japanese A, in step S7, Japanese is provisionally determined as the native language.

In step S9, voice guidance is provided to the caller in the provisionally determined native language. If the tentatively-determined native language is Japanese, a synthesized speech with the content "This is an automatic speech translation service provided by the ATR Spoken Language Communication Laboratory. Is your language spoken in Japanese?" Communicated to caller. When the voice guidance is completed, it is determined in step S11 whether a voice input has been made from the caller, and in step S13, it is determined whether a predetermined time has elapsed. If there is no voice input within the predetermined period, it is assumed that the caller's native language is different from the provisionally determined native language,
In step S15, the temporarily determined native language is changed to, for example, English, and the process returns to step S9. In step S9, the voice guidance in the changed language, that is, in this case, English is performed.

When there is a voice input from the caller, the voice recognition system 10c is controlled to execute steps S17 to S21.
Is performed. First, in step S17, features of the input uttered voice are extracted. In step S19, the native language, voice quality, and gender of the speaker are specified from the extracted features, and the specified native language information, voice quality information, and gender information are associated with the telephone number information and country information of the caller and transmitted to the database server 10b. register. In step S21, the input uttered voice is recognized and a reply to the voice guidance is obtained.

If the obtained reply is an affirmative reply such as "Yes, yes", "YES" is determined in the step S23, and the process proceeds to a step S31. On the other hand, if the answer is negative, the process proceeds to step S25, and it is determined whether or not the language of the speaker can be determined from the answer. If the content of the reply is simply "No", it is determined NO in step S25,
After changing the provisionally determined native language in step S15, the process returns to step S9. On the other hand, if the reply includes a word that can determine the native language, such as “No, English”, step S
25 is determined to be YES, the native language information of the database server 10b is updated in step S27, and then step S3
Proceed to 1.

In step S31, voice guidance for prompting the input of the telephone number of the other party is performed. Specifically, a voice message "Please say the telephone number of the other party from the country code?" Is output in a language according to the native language information registered in the database server 10b. In a step S33, it is determined whether or not a voice input is made to the voice message, and in a step S35, it is determined whether or not a predetermined period has elapsed. If the predetermined period has elapsed without a response from the caller, “YES” is determined in the step S35, and the process returns to the step S31. Therefore, the same voice message is output again.

On the other hand, when the voice of the caller is input,
It is determined as YES in step S33, and the voice recognition system 1
The processing of steps S37 to S41 is performed using 0c. In step S37, features of the uttered voice are extracted, and in step S3
In step 9, the native language information, voice quality information and gender information of the speaker identified from the extracted features are stored in the database server 10.
Register in b. That is, the native language information, the voice quality information, and the gender information registered in the database server 10b are updated each time a speech sound is input.

In step S41, the input uttered voice is recognized to acquire the telephone number information of the other party. In step S43, the validity of the acquired telephone number information is determined.
If the country code is not included in the telephone number information, NO is determined in this step, and the process returns to step S31.
On the other hand, if the telephone number information is valid, the process proceeds to step S45, and voice guidance for confirming the telephone number is performed. Specifically, in the language according to the native language information registered in the database server 10b, "the telephone number of the destination is
○○ − ×××× − △△△△△ In step S47, it is determined whether or not a voice input has been made to this voice message, and in step S49, it is determined whether or not a predetermined period has elapsed. If there is no response for a predetermined period, the process proceeds to step S49. YE
S is determined, and the process returns to step S45. On the other hand, if there is a voice response from the caller, “YES” is determined in the step S47, and the steps S51 to S51 are performed using the voice recognition system 10c.
55 is performed. In step S51, features are extracted from the voice forming the reply. In step S53, the speaker's native language information, voice quality information, and gender information are specified from the extracted features, and the database server 10b is updated with the specified information. Upon completion of the process in the step S53, the content of the reply is recognized in a step S55.

In step S57, it is determined whether or not the recognized voice is affirmative. If the answer is negative such as "no", the process returns to step S31. Thereby, a series of processes from the voice guidance for guiding the input of the telephone number is repeated. On the other hand, when an affirmative reply such as "Yes, yes" is obtained, the process proceeds from the step S57 to the step S59, and the country information of the call partner is detected from the telephone number information obtained in the step S41. If the country code included in the telephone number information is "1", the country to which the other party belongs is the United States. Subsequently, in step S61, the telephone number information and the country information of the other party are held in the internal memory.
In step S63, the native language of the call partner (the official language of the country to which the call partner belongs) is provisionally determined. If the country to which the other party belongs is the United States, the native language of the other party is provisionally determined to be English.

In step S65, voice guidance for prompting input of the name of the other party is performed. The caller receives a voice message in the native language of the caller saying "Please say the name of the other party?" In a step S67, it is determined whether or not there is a voice response to the voice message, and in a step S69, it is determined whether or not a predetermined period has elapsed. If the predetermined period has elapsed without a reply, YES is determined in the step S69, and the process returns to the step S65 to perform the voice guidance again.

On the other hand, if there is a reply from the caller, YES is determined in the step S67, and the voice recognition system 10
The processing of steps S71 to S75 is performed using c. In step S71, features of the uttered voice are extracted, and in step S73.
Then, the database server 10b is updated with the native language information, voice quality information, and gender information specified from the extracted features, and in step S75, the name of the other party is obtained by speech voice recognition processing.

When the acquisition of the telephone number information and the name of the called party is completed, it is determined in steps S77 and S79 whether or not the called party is registered in the database server 10b. Specifically, the database server 10b is searched using the telephone number information of the other party as a keyword, and it is determined in step S79 whether the same telephone number information has been found. When the telephone number information cannot be found, the process directly proceeds to step S85, but when the telephone number information can be found, the native language information of the other party is obtained from the database server 10b in step S81, and the other party's native language information is obtained in step S83. After changing the provisionally determined native language, the process proceeds to step S85. For example, if the native language provisionally determined in step S63 is English and the native language information acquired from the database server 10b is German, the provisionally determined native language is changed from English to German.

In step S85, a call is made to the other party. In step S87, it is determined whether the line state is OK. In step S89, it is determined whether there is a response from the other party. If the line state is busy (busy) or there is no response from the other party, step S8
7 or S89 proceeds to step S107. In step S107, voice guidance for guiding that the call cannot be made is performed. More specifically, if the line status is busy, a voice message saying "the other party's phone is busy" is output, and if there is no response, a voice message saying "the other party has not responded" is output. . On the other hand, if the caller disconnects the telephone line, step S10
It is determined as YES at 9 and the process is terminated.

On the other hand, if the line status is OK and there is a response from the other party, Y is determined in steps S87 and S89.
It is determined to be ES, and voice guidance for service guidance is performed in step S91. At this time, the name of the other party obtained in step S75 is used. Specifically, a voice message saying "This is an automatic speech translation service provided by the ATR Spoken Language Communication Research Institute. Are you Schrader?" Is delivered to the other party in the provisionally determined native language.

In a step S93, it is determined whether or not there is a voice response to the voice message, and in a step S95, it is determined whether or not a predetermined period has elapsed. If the predetermined period has elapsed without reply, YES is determined in the step S95, the temporarily determined native language is changed in a step S97, and the process returns to the step S91. On the other hand, if there is a reply from the other party, YES is determined in step S93, the content of the reply is recognized using the voice recognition system 10c in step S99, and whether the reply recognized in step S101 is positive or not is determined. Judge whether it is appropriate.

Here, the reply is "Schrader is absent."
If the answer is negative, the process proceeds from step S101 to step S107, and a voice message "Mr. Schrader seems to be out of the office" is transmitted to the caller. On the other hand, if the reply is an affirmative reply such as “Yes, yes”, then YES is determined in the step S101, and the steps S103 and S1 are performed using the speech recognition system 10c.
05 is performed. In step S103, features of the uttered voice are extracted, and in step S105, native language information, voice quality information, and gender information specified from the extracted features are registered or updated in the database server 10b.

That is, the communication partner is the database server 1
0b, the database server 10b is updated with the specified native language information, voice quality information, and gender information. If the other party is not registered, the specified native language information, voice quality information, and gender information are updated in step S61. Is registered in the database server 10b in association with the telephone number information and the country information of the other party held in step.

When the native language of both the calling party and the called party is specified and the communication is enabled, voice guidance for guiding the start of the service is provided in step S111. Specifically, a voice message saying, "Please speak now" is given to both the caller and the other party. This starts a conversation between the two parties.

In step S113, the uttered voice of one speaker is fetched, the utterance time of the uttered voice is determined in step S115, and this utterance time information is recorded in the database server 10b in step S117. Step S11
In step 9, features of the uttered voice are extracted using the voice recognition system 10c. In the following step S121, the speaker's native language, voice quality, and gender are specified based on the extracted features, and the registered information of the database server 10b is updated with the specified native language information, voice quality information, and gender information. That is, even after the conversation is started, the native language information, the voice quality information, and the gender information of the database server 10b are updated each time the uttered voice is input.

In step S123, the contents of the uttered voice taken in are recognized using the voice recognition system 10c.
The recognized content is translated into the native language of the other speaker using the language translation system 10d. At this time, the language of the translation destination is determined by the native language information of the other speaker registered in the database server 10b.

When the translation process is completed, step S125
Then, a synthesized speech of the translated content is created using the speech synthesis system 10e. In step S127, the generated synthesized speech is recorded in the database server 10b. Note that the processing of steps S115 to S127 is simultaneously and frequently executed by a plurality of speech recognition systems 10c, language translation systems 10d, and speech synthesis systems 10e. Therefore, the utterance time information and the synthesized voice are sequentially stored in the database server 10b. The storage of the synthesized speech in the database server 10b is in the order of creation of the synthesized speech, and is not limited to the order of input to the translation telephone system (the order of speech of the speaker) as shown in FIG.

In step S129, it is determined whether any of the speakers has disconnected the telephone line. If NO, the process returns to step S113. As a result, the spoken voice is translated each time each speaker speaks, and the translated synthesized voice is stored in the database server 10b.

When both conversations are started, the routine shown in the flowchart of FIG. 10 is executed in parallel in the integrated management system 10a. In step S141, it is determined whether utterance time information exists with reference to the database server 10b. When the utterance time information exists, the utterance time information is read from the database server 10b. In step S145, the synthesized speech corresponding to the read utterance time information is stored in the database server 10b.
, And in step S147, the translated synthesized voice is reproduced by the other party. Then, step S149
Then, it is determined whether one of the speakers has disconnected the line, and if it is determined that the line has been disconnected, the process ends. On the other hand, if the line has not been disconnected, step S141
And the next synthesized voice is reproduced. In this way, the synthesized speech translated in the utterance order is stored in the database server 10b.
3 and reproduced, the translated synthesized speech is output in the order of speech, regardless of the length of the utterance content, that is, regardless of the load on the speech recognition system 10c, the language translation system 10b, and the speech synthesis system 10e, as shown in FIG. Will be played. By repeatedly executing the routine in FIG. 10, the translated speech is appropriately transmitted to the other party each time each speaker speaks.

Referring again to FIG. 9, in step S129, if it is determined that one of the speakers has disconnected the telephone line, the flow advances from step S129 to step S131 to provide voice guidance for guiding the disconnection of the line. That is,
A voice message is sent to the other speaker saying, "The other party has hung up and will end the service, thank you for using." On the other hand, if the other speaker disconnects the line, YES is determined in step S133, and the process ends.

As can be seen from the above description, in the translated telephone system of this embodiment, when outputting the synthesized speech obtained by translating the input uttered speech, the synthesized speech is output based on the utterance time. Therefore, even when the load on the system differs depending on the length of the uttered voice, the synthesized voice is output in the utterance order, so that the input voice and the translated output voice can be kept in synchronization.

[Brief description of the drawings]

FIG. 1 is an illustrative view showing a state where a translation telephone system and a plurality of telephones are connected to a telephone line;

FIG. 2 is an illustrative view showing one example of a configuration of a translation telephone system;

FIG. 3 is an illustrative view showing a processing time of the translation telephone system;

FIG. 4 is a flowchart showing a part of the operation of the translation telephone system.

FIG. 5 is a flowchart showing another part of the operation of the translation telephone system.

FIG. 6 is a flowchart showing another part of the operation of the translation telephone system.

FIG. 7 is a flowchart showing yet another portion of the operation of the translation telephone system.

FIG. 8 is a flowchart showing another part of the operation of the translation telephone system.

FIG. 9 is a flowchart showing another part of the operation of the translation telephone system.

FIG. 10 is a flowchart showing yet another portion of the operation of the translation telephone system.

[Explanation of symbols]

 10 Translation telephone system 10a Integrated management system 10b Database server 10c Speech synthesis system 10d Language translation system 10e Speech recognition system 10f Modem 20, 30 Telephone 40 Telephone line

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) H04M 3/50 G10L 3/00 551A 3/60 R 531K (72) Inventor Atsushi Nishino Kodaiji Seikacho, Soraku-gun, Kyoto Prefecture AT-2 Co., Ltd. AT R Spoken Language Communication Research Laboratories (72) Inventor Liner Green 2-2-2 Kodai, Seika-cho, Soraku-gun, Kyoto Prefectural A.T. ) Inventor Satoshi Nakamura 2-2-2 Kodai, Seika-cho, Soraku-gun, Kyoto F-term in ATR Spoken Language Communication Research Laboratories (reference) 5D015 AA06 KK02 KK04 5D045 AB03 AB04 5K015 AA06 AA07 AA10 AB02 AD05 AF07 GA02 5K024 AA72 AA74 BB01 BB02 BB05 BB07 CC01 DD01 DD04 EE09 FF06 GG01 GG03 GG10 GG12

Claims (2)

[Claims] 1. A translation telephone system for translating a conversation carried out via a telephone line between a plurality of speakers using different languages, a capturing means for capturing the uttered voice of each of said speakers, A translation telephone, comprising: a detection unit that detects a capture time of the speech; a creation unit that creates a synthesized voice representing the translation content of the uttered voice; and an output unit that outputs the synthesized voice in an order according to the capture time. system. 2. The method according to claim 1, wherein the generating unit is configured to perform a speech recognition process for recognizing the content of the uttered speech, a language translation process for translating a result of the speech recognition process, and the synthesized speech based on a result of the language translation process. 2. The translation telephone system according to claim 1, wherein the voice synthesizing processes for generating the texts are performed in parallel with each other.