JP2002304392A - Voice translation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To establish a conversation by translating voice without an operator. SOLUTION: When two speakers using different languages as mother languages (used languages) talk to each other through a communication network 40, the voices uttered by the respective speakers are taken in by this voice translation system 10. The voice translation system 10 identifies the mother language of each speaker based on the voices uttered and taken in, translates the uttered voices based on the identified mother language and transmits the translated synthetic sounds to the respective speakers.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech translation system, and more particularly to, for example, speech translation for translating a conversation between two speakers using different languages via a communication line. About the system.

[0002]

2. Description of the Related Art As a conventional speech translation system of this type, there is a speech translation system which performs a series of translation processes of speech recognition, language translation, and speech synthesis on the speech of each speaker, and transmits the synthesized speech to the other party.

[0003]

However, the prior art,
Translation between two languages determined in advance is premised, and it is necessary to intervene an operator to change the translation destination language.

[0004] Therefore, a main object of the present invention is to provide a speech translation system capable of translating between arbitrary languages without the intervention of an operator.

[0005]

SUMMARY OF THE INVENTION The present invention relates to a speech translation system for translating a conversation carried out via a communication line between a plurality of speakers using different languages. Capturing means for capturing, extracting means for extracting characteristics of the uttered voice, specifying means for specifying the language used by each speaker based on the characteristics of the uttered voice, recognition means for recognizing the content of the uttered voice, specifying the content of the uttered voice Translation means for translating based on the specified result by the means, creation means for creating a synthesized speech based on the translation result by the translation means,
And an output means for outputting synthesized speech.

[0006]

The uttered voices of a plurality of speakers using different languages are captured by capturing means. The features of the uttered voice are extracted by the extracting means, and the specifying means specifies the language used by each speaker based on the extracted characteristics.
Further, the content of the uttered voice is recognized by the recognition means, and the translation means translates the recognized content based on a result specified by the specifying means. The synthesizing unit generates a synthesized speech based on the translation result, and the generated synthesized speech is output by the output unit. As a result, a conversation is established between the speakers via the communication line without the intervention of the operator.

[0007] Preferably, the telephone number of each speaker is detected by the telephone number detecting means, and the country to which each speaker belongs is detected by the country detecting means based on the telephone number. The voice guidance unit performs voice guidance in the official language of the detected country before capturing the utterance voice. At this time, the specifying means specifies the language used by each speaker based on the answer to the voice guidance.

[0008] More preferably, if there is no reply to the voice guidance, the language of the voice guidance is changed by the changing means. If there is a positive answer to the voice guidance, the determining means determines the language of the voice guidance at the present time as the language used by the speaker who has answered.

When the language information indicating the specified language is stored in the memory, the translation means translates the language in accordance with the language information stored in the memory.

In the case where the language used as the utterance source is specified each time the uttered voice is captured, the memory may be updated with the used language information specified each time the uttered voice is captured.

[0011]

According to the present invention, the language used by each speaker is specified on the basis of the characteristics of the uttered speech taken in, so that the translated speech can be transmitted via the communication line without the intervention of an operator. Conversation can be held.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIG. 1, a speech translation system 10 of this embodiment is connected to a communication network 40 such as the Internet. A plurality of telephones 20 and 30 are also connected to the communication network 40. Here, telephone 2
0 is installed in Japan, for example, and is used by Japanese A living in Japan. The telephone 30 is installed in the United States, for example, and is used by a German B living in the United States.

When Japanese A wants to have a conversation with German B using the speech translation system 10, the Japanese A first accesses the speech translation system 10. The calling process for the German B is performed by the speech translation system 10, and when the German B answers, the call becomes available. Japanese A
Is translated from Japanese to German, and the translated synthesized speech is transmitted to German B. Conversely, the uttered voice of German B is translated from German into Japanese, and the translated synthesized voice is transmitted to Japanese A.

Referring to FIG. 2, speech translation system 10
Is the integrated management system 10a, the database server 10
b, a plurality of speech recognition systems 10c, 10c, ..., a plurality of language translation systems 10d, 10d, ... and a plurality of speech synthesis systems 10e, 10e, ... and a modem 10f.
Consists of The speech translation system 10 is configured by a distributed network, and performs speech recognition by a speech recognition system 10c, language translation by a language translation system 10d, and speech synthesis by a speech synthesis system 10e.
Translation of the uttered voice is realized.

Specifically, the contents of the uttered voices of the speakers (Japanese A and German B) are described in the speech recognition system 1.
0c. The voice recognition system 10 also extracts features of the uttered voice, and specifies the native language of the utterance source (Japanese for Japanese A, German for German B), voice quality, and gender. The specified native language information, voice quality information and gender information are stored in the database server 1
0b.

The language translation system 10d translates the contents of the recognized speech voice based on the native language information registered in the database server 10b. For Japanese A, the native language information "Japanese" is registered in the database server 10b, and for Japanese A, the native language information "Japanese" is registered in the database server 10b. For this reason, the content of the speech voice of Japanese A is translated into German by the language translation system 10d, and the content of the speech voice of German B is translated into Japanese by the language translation system 10d.

The speech synthesizing system 10e is composed of the contents translated by the language translation system 10d and the database server 10b.
A synthesized speech is created based on the voice quality information and the gender information registered in, and the created synthesized speech is transmitted to each speaker.
If the Japanese A is a woman who speaks with a high voice, a German synthesized voice with a feminine tone and high key is transmitted to the German B. In addition, if German B is a man who speaks in a low voice, Japanese A with a masculine tone and low key is transmitted to Japanese A.

The speech recognition system 10c, language translation system 10d, speech synthesis system 10e, and database server 10b are managed by an integrated management system 10a. When accessed through the modem 10f, the integrated management system 10 controls these systems and servers and provides translation services to users. Specifically, the integrated management system 10a processes the flowcharts shown in FIGS.

The access request to the voice translation system 10 includes the telephone number information of the caller. Therefore, first, in step S1, telephone number information is obtained from the access request,
In a succeeding step S3, country information of the caller is detected based on the detected telephone number information. When Japanese A accesses the speech translation system 10 using the telephone 20, the telephone number information includes the country code "81". Step S3
Then, the country of the caller is specified as Japan from the country code "81". In step S5, the telephone number information and country information of the caller obtained in this manner are stored in the internal memory. In step S7, the native language of the caller (the official language of the country to which the caller belongs) is stored from the stored country information. ) Is provisionally determined. When the caller is Japanese A, in step S7, Japanese is provisionally determined as the native language.

In step S9, voice guidance is given to the caller in the provisionally determined native language. If the tentatively-determined native language is Japanese, a synthesized speech with the content "This is an automatic speech translation service provided by the ATR Spoken Language Communication Laboratory. Is your language spoken in Japanese?" Communicated to caller. When the voice guidance is completed, it is determined in step S11 whether a voice input has been made from the caller, and in step S13, it is determined whether a predetermined time has elapsed. If there is no voice input within the predetermined period, it is considered that the caller's native language is different from the provisionally determined native language, and the provisionally determined native language is changed to, for example, English in step S15, and the process returns to step S9. In step S9, the voice guidance in the changed language, that is, English is performed.

When there is a voice input from the caller, the voice recognition system 10c is controlled to execute steps S17 to S21.
Is performed. First, in step S17, features of the input uttered voice are extracted. In step S19, the speaker's native language, voice quality, and gender are specified from the extracted features.
The specified native language information, voice quality information, and gender information are registered in the database server 10b in association with the caller's telephone number information and country information. In step S21, the input uttered voice is recognized and a reply to the voice guidance is obtained.

If the obtained reply is an affirmative reply such as "Yes, so", YES is determined in the step S23, and the process proceeds to a step S31. On the other hand, if the answer is negative, the process proceeds to step S25, and it is determined whether or not the language of the speaker can be determined from the answer. If the content of the reply is simply "No", it is determined NO in step S25,
After changing the provisionally determined native language in step S15, the process returns to step S9. On the other hand, if the reply includes a word that can determine the native language, such as “No, English”, step S
25 is determined to be YES, the native language information of the database server 10b is updated in step S27, and then step S3
Proceed to 1.

In step S31, voice guidance for prompting the input of the telephone number of the other party is performed. Specifically, a voice message "Please say the telephone number of the other party from the country code?" Is output in a language according to the native language information registered in the database server 10b. In a step S33, it is determined whether or not a voice input is made to the voice message, and in a step S35, it is determined whether or not a predetermined period has elapsed. If the predetermined period has elapsed without a response from the caller, “YES” is determined in the step S35, and the process returns to the step S31. Therefore, the same voice message is output again.

On the other hand, when the voice of the caller is input,
It is determined as YES in step S33, and the voice recognition system 1
The processing of steps S37 to S41 is performed using 0c. In step S37, the feature of the uttered voice is extracted, and
At 39, the native language information, voice quality information and gender information of the speaker identified from the extracted features are stored in the database server 10
Register in b. That is, the native language information, the voice quality information, and the gender information registered in the database server 10b are updated each time a speech sound is input.

In step S41, the input uttered voice is recognized to acquire the telephone number information of the other party, and in the following step S43, the validity of the acquired telephone number information is determined.
If the country code is not included in the telephone number information, NO is determined in this step, and the process returns to step S31.
On the other hand, if the telephone number information is valid, the process proceeds to step S45, and voice guidance for confirming the telephone number is performed. Specifically, in the language according to the native language information registered in the database server 10b, "the telephone number of the destination is
○○ − ×××× − △△△△△ The voice message "" is output.

In step S47, it is determined whether or not a voice input has been made in response to the voice message.
At 9, it is determined whether a predetermined period has elapsed. When there is no reply for a predetermined period, YES is determined in the step S49, and the process returns to the step S45. On the other hand, if there is a voice response from the caller, YES is determined in step S47,
Steps S51 to S55 using the voice recognition system 10c
Is performed. In step S51, features are extracted from the voice forming the reply. In step S53, the speaker's native language information, voice quality information, and gender information are specified from the extracted features, and the database server 10b is updated with the specified information. Upon completion of the process in the step S53, the content of the reply is recognized in a step S55.

In the step S57, it is determined whether or not the recognized voice is affirmative. If the answer is negative such as "No", the process returns to the step S31. Thereby, a series of processes from the voice guidance for guiding the input of the telephone number is repeated. On the other hand, when an affirmative reply such as "Yes, yes" is obtained, the process proceeds from the step S57 to the step S59, and the country information of the call partner is detected from the telephone number information obtained in the step S41. If the country code included in the telephone number information is "1", the country to which the other party belongs is the United States. Subsequently, in step S61, the telephone number information and the country information of the other party are held in the internal memory.
In step S63, the native language of the call partner (the official language of the country to which the call partner belongs) is provisionally determined. If the country to which the other party belongs is the United States, the native language of the other party is provisionally determined to be English.

In step S65, voice guidance for prompting the input of the name of the other party is performed. The caller receives a voice message in the native language of the caller saying "Please say the name of the other party?" In a step S67, it is determined whether or not there is a voice response to the voice message, and in a step S69, it is determined whether or not a predetermined period has elapsed. After a predetermined period without a reply,
YES is determined in the step S69, and the process returns to the step S65 to perform the voice guidance again.

On the other hand, if there is a reply from the caller, YES is determined in the step S67, and the voice recognition system 10
The processing of steps S71 to S75 is performed using c. In step S71, features of the uttered voice are extracted, and in step S7
In step 3, the database server 10b is updated with the native language information, voice quality information, and gender information specified from the extracted features. In step S75, the name of the other party is acquired by speech voice recognition processing.

When the acquisition of the telephone number information and the name of the called party is completed, it is determined in steps S77 and S79 whether or not the called party is registered in the database server 10b. Specifically, the database server 10b is searched using the telephone number information of the other party as a keyword, and it is determined in step S79 whether the same telephone number information has been found. When the telephone number information cannot be found, the process directly proceeds to step S85, but when the telephone number information can be found, the native language information of the other party is obtained from the database server 10b in step S81, and the other party's native language information is obtained in step S83. After changing the provisionally determined native language, the process proceeds to step S85. For example, if the native language provisionally determined in step S63 is English and the native language information acquired from the database server 10b is German, the provisionally determined native language is changed from English to German.

In step S85, a call is made to the other party. In step S87, it is determined whether the line state is OK. In step S89, it is determined whether there is a response from the other party. If the line state is busy (busy) or there is no response from the other party, step S8
7 or S89 proceeds to step S107. In step S107, voice guidance for guiding that the call is not possible is performed. More specifically, if the line status is busy, a voice message saying "the other party's phone is busy" is output, and if there is no response, a voice message saying "the other party has not responded" is output. . On the other hand, when the caller disconnects the telephone line, step S1
09 is determined as YES, and the process ends.

On the other hand, if the line status is OK and there is a response from the other party, Y is determined in steps S87 and S89.
It is determined to be ES, and voice guidance for service guidance is performed in step S91. At this time, the name of the other party obtained in step S75 is used. Specifically, a voice message saying "This is an automatic speech translation service provided by the ATR Spoken Language Communication Research Institute. Are you Schrader?" Is delivered to the other party in the provisionally determined native language.

In a step S93, it is determined whether or not there is a voice response to the voice message, and in a step S95, it is determined whether or not a predetermined period has elapsed. If the predetermined period has elapsed without reply, YES is determined in the step S95, the temporarily determined native language is changed in a step S97, and the process returns to the step S91. On the other hand, if there is a reply from the other party, YES is determined in step S93, the content of the reply is recognized using the voice recognition system 10c in step S99, and whether the reply recognized in step S101 is positive or not is determined. Judge whether it is appropriate.

Here, the answer is "Schrader is absent."
If the answer is negative, the process proceeds from step S101 to step S107, and a voice message "Mr. Schrader seems to be out of the office" is transmitted to the caller. On the other hand, if the reply is an affirmative reply such as “Yes, yes”, then YES is determined in the step S101, and the steps S103 and S1 are performed using the speech recognition system 10c.
Step 05 is performed. In step S103, features of the uttered voice are extracted, and in step S105, native language information, voice quality information, and gender information specified from the extracted features are registered or updated in the database server 10b.

That is, the communication partner is the database server 1
0b, the database server 10b is updated with the specified native language information, voice quality information, and gender information. If the other party is not registered, the specified native language information, voice quality information, and gender information are updated in step S61. Is registered in the database server 10b in association with the telephone number information and the country information of the other party held in step.

When the native language of both the caller and the called party is specified and the communication is enabled, voice guidance for guiding the start of the service is performed in step S111. Specifically, “Tell me now.”
To both the caller and the other party. This starts a conversation between the two parties.

In step S113, the uttered voice of one speaker is captured, and in step S115, the features of the uttered voice are extracted using the voice recognition system 10c. In the following step S117, the speaker's native language, voice quality, and gender are specified based on the extracted features, and the specified native language information,
Database server 1 based on voice quality information and gender information
0b is updated. That is, even after the conversation is started, the native language information, the voice quality information, and the gender information of the database server 10b are updated each time the uttered voice is input.

In step S119, the contents of the uttered speech fetched are recognized using the speech recognition system 10c.
The recognized content is translated into the native language of the other speaker using the language translation system 10d. At this time, the language of the translation destination is determined by the native language information of the other speaker registered in the database server 10b.

When the translation process is completed, steps S121 and S123 are processed using the speech synthesis system 10e. In step S121, a synthesized speech of the translated content is created, and in step S123, the created synthesized speech is reproduced. The reproduced synthesized voice is transmitted to the other speaker.

In step S125, it is determined whether any of the speakers has disconnected the telephone line. If NO, the process returns to step S113. As a result, the spoken voice is translated each time each speaker speaks, and the translated synthesized voice is transmitted to the other party. That is, conversation using the translated synthesized speech is continuously performed. When one speaker disconnects the telephone line, the process proceeds from step S125 to step S127, and voice guidance for guiding the disconnection of the line is performed. In other words, a voice message saying "the service is terminated because the other party has hung up the phone, thank you for using" is transmitted to the other speaker. On the other hand, when the other speaker disconnects the line, step S129 is performed.
Is determined as YES, and the process ends.

As can be understood from the above description, when two speakers whose different languages are their native languages (languages used) have a conversation via the communication network 40, the uttered voice of each speaker is the voice Captured by the translation system 10. The speech translation system 10 specifies the native language of each speaker based on the captured uttered speech, translates the uttered speech based on the identified native language, and provides the translated synthesized speech to each speaker. Tell

When specifying the native language of each speaker, first, the country to which each speaker belongs is detected based on the telephone number,
Voice guidance is provided in the official language of the detected country.
The native language is specified based on a response to the voice guidance. However, since the official language of the detected country and the speaker's native language may not match, if there is no response to the voice guidance, the language of the voice guidance is changed. When a positive reply is given to the voice guidance, the language of the current voice guidance is specified as the native language.

The specified native language information is registered in the database server 10b, and the language translation is stored in the database server 1b.
0b is performed according to the native language information registered. The native language of each speaker is specified each time the uttered voice is captured. The native language information registered in the database server 10b is updated every time the native language is specified.

Therefore, it is possible to have a conversation between speakers having different native languages without the intervention of an operator,
Further, it is possible to cope with a sudden change of the speaker during the conversation.

[Brief description of the drawings]

FIG. 1 is an illustrative view showing a state in which a speech translation system and a plurality of telephones are connected to a communication network;

FIG. 2 is an illustrative view showing one example of a configuration of a speech translation system;

FIG. 3 is a flowchart showing a part of the operation of the speech translation system.

FIG. 4 is a flowchart showing another part of the operation of the speech translation system.

FIG. 5 is a flowchart showing another part of the operation of the speech translation system.

FIG. 6 is a flowchart showing yet another portion of the operation of the speech translation system.

FIG. 7 is a flowchart showing another part of the operation of the speech translation system.

FIG. 8 is a flowchart showing another part of the operation of the speech translation system.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Speech translation system 10a ... Integrated management system 10b ... Database server 10c ... Speech synthesis system 10d ... Language translation system 10e ... Speech recognition system 10f ... Modem 20,30 ... Telephone 40 ... Communication network

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat 参考 (Reference) G10L 3/00 551C 531K (72) Inventor Atsushi Nishino 2-2-1 Kodai, Seikacho, Soraku-gun, Kyoto Stock AT R Spoken Language Communication Research Laboratories (72) Inventor Rainer Greene 2-2-2 Kodai, Seika-cho, Soraku-gun, Kyoto Prefecture AT R Spoken Language Communication Research Laboratories (72) Inventor Satoshi Nakamura F-term (reference) 5A093 AA03 CB12 CB32 DA02 5D015 AA06 KK02 KK04 5D045 AB03

Claims (5)

[Claims] 1. A speech translation system for translating a conversation performed between a plurality of speakers using different languages via a communication line, comprising: a fetching unit for fetching uttered voices of the respective speakers; Extracting means for extracting the characteristics of the uttered voice, specifying means for specifying the language used by each speaker based on the characteristics of the uttered voice, recognition means for recognizing the content of the uttered voice, and specifying means for the content of the uttered voice A translation unit that translates based on a result specified by the translation unit, a creation unit that creates a synthesized speech based on a translation result by the translation unit, and an output unit that outputs the synthesized speech. 2. A telephone number detecting means for detecting a telephone number of each of said speakers, a country detecting means for detecting a country to which each of said speakers belongs based on said telephone number, and said country detecting means. 2. The voice translation device according to claim 1, further comprising: voice guidance means for providing voice guidance in an official language of the country where the voice guidance is provided, wherein the specification means includes first language specification means for specifying the language to be used based on a reply to the voice guidance. system. 3. The first language specifying means includes: changing means for changing the language of the voice guidance when there is no response to the voice guidance; and current time when a positive response is received to the voice guidance. 3. The speech translation system according to claim 2, further comprising a determination unit that determines a language of said voice guidance as said language to be used. 4. The method according to claim 1, wherein the specifying unit further includes a storage unit configured to store, in a memory, use language information indicating the use language specified by the first language specification unit, and the translation unit stores the use language information stored in the memory. 4. The speech translation system according to claim 2, wherein translation is performed according to language information. 5. The second language specifying means for specifying the language used as the utterance source each time the uttered voice is captured, and a usage indicating the used language specified by the second language specifying means. 5. The speech translation system according to claim 4, further comprising an updating unit that updates said memory with language information.