JP6250209B1 - Speech translation device, speech translation method, and speech translation program - Google Patents

Abstract

In a conversation using speech translation, a questioner can ask an appropriate question to an answerer, thereby achieving smooth communication between the two. An audio translation apparatus according to an aspect of the present invention includes an input unit for inputting a voice of a user who has a conversation, and a scene selection unit for a user to select a scene in the conversation from a plurality of scenes. A scene selection means presenting unit that presents a plurality of fixed syntaxes assumed for a specific phrase, a storage unit that stores the specific phrase and each scene in advance, and recognizes the input speech. When the content of the recorded voice is a question sentence and includes a specific phrase, a fixed form is presented so that the user can select a fixed syntax stored in association with the specific phrase and the selected scene. A syntax presentation unit is provided. [Selection] Figure 5

Description

  The present invention relates to a speech translation device, a speech translation method, and a speech translation program.

  In order to enable conversations between people who do not understand each other's language, for example, conversations between store clerk and foreign customers, the speech of the speaker is converted into text and the content of the text is machine translated into the language of the listener There has been proposed a speech translation technique for displaying on a screen or playing back the text content using a speech synthesis technique (for example, Patent Document 1). A speech translation application that operates on an information terminal such as a smartphone that embodies such speech translation technology has also been put into practical use (for example, Non-Patent Document 1).

Japanese Patent Laid-Open No. 9-34895

U-STAR homepage [Search March 8, 2017], Internet <URL: http://www.ustar-consortium.com/qws/slot/u50227/app/app.html>

  In general, in the conventional speech translation technology, a spoken speech is recognized to obtain a reading (character) of the speech, and the character is translated into another language using a dictionary. In this case, an acoustic model or a language model built in advance is applied to the speech recognition process, and a database such as a corpus of each language prepared in advance is used for the translation process. By the way, when a speaker (questioner) such as a store clerk makes an inquiry to a foreign customer (respondent), the questioner's way of asking may be inappropriate or insufficient. In this case, the respondent may not be able to communicate smoothly because he / she does not know the true meaning of the questioner's question, returns a question that reflects the true meaning, or gives a non-targeted answer. possible.

  Therefore, the present invention has been made in view of such circumstances, and in a conversation using speech translation, a questioner can make an appropriate question to an answerer, thereby enabling smooth communication between the two. An object of the present invention is to provide a speech translation device, a speech translation method, and a speech translation program capable of achieving the above.

  In order to solve the above-described problem, a speech translation apparatus according to an aspect of the present invention includes an input unit for inputting speech of a user who has a conversation, and a scene (scene or situation) in which the user has a conversation among a plurality of scenes. A scene selection means presenting section for presenting a scene selection means for selecting a scene, and a storage section for storing a plurality of standard syntaxes assumed for a specific phrase in advance in association with the specific phrase and each scene, and When the content of the recognized speech is a question sentence and includes a specific phrase, the user can determine the fixed syntax stored in association with the specific phrase and the selected scene. The fixed syntax presentation section that presents the selected fixed syntax, the translation section that translates the content of the selected fixed syntax into the content of a different language, and the content that has been translated into a different language Beauty / or an output unit for outputting the text. The “specific phrase” includes a sentence, a clause, a phrase, a word, and a number, and the “standard syntax” may include an image or a symbol accompanying the sentence.

  Moreover, you may comprise so that an input part may present the input change means for a user to change the content of the selected fixed form syntax.

  Furthermore, it is preferable that the fixed syntax presentation unit presents the fixed syntax together with the recognized speech content.

  Furthermore, a fixed syntax input unit for a user to add and input a fixed syntax may be further provided.

  A speech translation method according to an aspect of the present invention uses a speech translation device including an input unit, a scene selection unit presentation unit, a storage unit, a fixed syntax presentation unit, a translation unit, and an output unit. A step of inputting a user's voice, a step of presenting a scene selection means for a user to select a scene in a conversation from a plurality of scenes, and a storage section for a specific phrase A step of storing a plurality of assumed fixed syntaxes in advance in association with the specific phrase and each scene, and a fixed syntax presenting unit recognizes the input speech, and the content of the recognized speech is a question sentence. When there is a specific phrase, the standard syntax stored in association with the specific phrase and the selected scene is presented so that the user can select it. A step in which the translation unit translates the content of the selected standard syntax into a content in a different language, and a step in which the output unit outputs the content translated into a different language in speech and / or text. Including.

  In addition, the speech translation program according to one aspect of the present invention includes a computer (not limited to a single type or a single type, and may be a plurality or a plurality of types; the same shall apply hereinafter), an input unit for inputting a voice of a user having a conversation, A scene selection means presenting section for presenting a scene selection means for a user to select a scene in a conversation from among a plurality of scenes, and a plurality of fixed syntaxes assumed for a specific phrase, for the specific phrase and each scene A storage unit that associates and stores in advance, and recognizes the input voice, and when the content of the recognized voice is a question sentence and includes a specific phrase, the specific phrase and the selected scene The fixed syntax presentation unit that presents the fixed syntax stored in association so that the user can select it, and the content of the selected fixed syntax is translated into the content of a different language A translation unit that, what is translated into different languages, to function as an output unit for outputting voice and / or text.

  According to the present invention, the content of the voice input by a questioner who is a user who has a conversation using speech translation is not translated into another language and transmitted to the other user (respondent). And select the content of the question that the questioner intended or in line with the questioner's intention from the specific phrases included in the content of the speech input by the user and the fixed syntax associated with the conversation scene can do. And since the selected standard syntax is translated and communicated to the respondent, the questioner can ask the respondent an appropriate question, thereby enabling smooth communication between the two. .

1 is a system block diagram schematically showing a preferred embodiment of a network configuration and the like related to a speech translation apparatus according to the present invention. It is a flowchart which shows an example of the flow (part) of the process in suitable one Embodiment of the speech translation apparatus by this invention. (A) thru | or (D) are top views which show an example of the transition of the display screen in an information terminal. (A) thru | or (D) are top views which show an example of the transition of the display screen in an information terminal. It is a flowchart which shows an example of the flow (part) of the process in suitable one Embodiment of the speech translation apparatus by this invention. (A) And (B) is a top view which shows an example of the transition of the display screen in an information terminal.

  Hereinafter, embodiments of the present invention will be described in detail. The following embodiments are examples for explaining the present invention, and are not intended to limit the present invention only to the embodiments. The present invention can be variously modified without departing from the gist thereof. Furthermore, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent ones, and such embodiments are also included in the scope of the present invention. Furthermore, positional relationships such as up, down, left, and right shown as needed are based on the display shown unless otherwise specified. Furthermore, various dimensional ratios in the drawings are not limited to the illustrated ratios.

(Device configuration)
FIG. 1 is a system block diagram schematically showing a preferred embodiment such as a network configuration related to a speech translation apparatus according to the present invention. In this example, the speech translation apparatus 100 includes a server 20 that is electronically connected to the information terminal 10 (user apparatus) used by the user via the network N (but is not limited to this).

  The information terminal 10 employs a user interface such as a touch panel and a display with high visibility, for example. The information terminal 10 here is a portable tablet terminal device including a mobile phone represented by a smartphone having a communication function with the network N. The information terminal 10 further includes a processor 11, a storage resource 12, a voice input / output device 13, a communication interface 14, an input device 15, a display device 16, and a camera 17. In addition, the information terminal 10 operates by the installed speech translation application software (at least a part of the speech translation program according to the embodiment of the present invention), so that a part of the speech translation apparatus according to the embodiment of the present invention or It functions as a whole.

  The processor 11 includes an arithmetic logic unit and various registers (program counter, data register, instruction register, general-purpose register, etc.). Further, the processor 11 interprets and executes speech translation application software, which is the program P10 stored in the storage resource 12, and performs various processes. The speech translation application software as the program P10 can be distributed from the server 20 through the network N, for example, and may be installed and updated manually or automatically.

  The network N includes, for example, a wired network (a short-range communication network (LAN), a wide-area communication network (WAN), a value-added communication network (VAN), etc.) and a wireless network (mobile communication network, satellite communication network, Bluetooth ( Bluetooth (registered trademark)), WiFi (Wireless Fidelity), HSDPA (High Speed Downlink Packet Access), etc.).

  The storage resource 12 is a logical device provided by a storage area of a physical device (for example, a computer-readable recording medium such as a semiconductor memory), and an operating system program, a driver program, various data, etc. used for processing of the information terminal 10 Is stored. Examples of the driver program include an input / output device driver program for controlling the audio input / output device 13, an input device driver program for controlling the input device 15, and a display device driver program for controlling the display device 16. Can be mentioned. Furthermore, the voice input / output device 13 is, for example, a general microphone and a sound player capable of reproducing sound data.

  The communication interface 14 provides a connection interface with the server 20, for example, and is configured from a wireless communication interface and / or a wired communication interface. The input device 15 provides an interface for accepting an input operation by a tap operation such as an icon, a button, a virtual keyboard, or a text displayed on the display device 16. Various input devices to be attached can be exemplified.

  The display device 16 provides various kinds of information to a user (speaker and listener) as an image display interface, and examples thereof include an organic EL display, a liquid crystal display, and a CRT display. The camera 17 is for capturing still images and moving images of various subjects.

  The server 20 is constituted by, for example, a host computer having a high arithmetic processing capability, and expresses a server function by operating a predetermined server program in the host computer, for example, a speech recognition server, a translation server, And a single or a plurality of host computers functioning as a speech synthesis server (in the drawing, it is indicated by a single, but is not limited thereto). Each server 20 includes a processor 21, a communication interface 22, and a storage resource 23.

  The processor 21 is composed of an arithmetic and logic unit for processing arithmetic operations, logical operations, bit operations and the like and various registers (program counter, data register, instruction register, general-purpose register, etc.), and is stored in the storage resource 23. P20 is interpreted and executed, and a predetermined calculation processing result is output. The communication interface 22 is a hardware module for connecting to the information terminal 10 via the network N. For example, the communication interface 22 is a modulation / demodulation device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

  The storage resource 23 is a logical device provided by, for example, a storage area of a physical device (a computer-readable recording medium such as a disk drive or a semiconductor memory), and each includes one or a plurality of programs P20, various modules L20, and various databases. D20 and various models M20 are stored. In addition, the storage resource 23 stores a plurality of question phrases prepared in advance for one user (speaker) of the conversation to speak to the other user (listener) of the conversation, history data of input speech, and data for various settings. Phrase data and the like described later are also stored.

  The program P20 is the above-described server program that is the main program of the server 20. In addition, the various modules L20 perform a series of information processing related to requests and information transmitted from the information terminal 10, so that they are appropriately called and executed during the operation of the program P10 (moduleized subprograms). ). Examples of the module L20 include a speech recognition module, a translation module, and a speech synthesis module.

  The various databases D20 include various corpora required for speech translation processing (for example, in the case of Japanese and English speech translation, a Japanese speech corpus, an English speech corpus, a Japanese character (vocabulary) corpus, an English character) (Vocabulary) Corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.), voice database, management database for managing information about users, phrase database having a hierarchical structure described later, etc. It is done. Examples of the various models M20 include acoustic models and language models used for speech recognition.

(Normal speech translation conversation)
An example of processing operations and operations in the speech translation apparatus 100 configured as described above will be described below. Here, an example of normal speech translation processing in conversation between users (questioner and answerer: both speakers) and / or conversation preparation will be described. FIG. 2 is a flowchart showing an example of (a part of) the processing flow in the speech translation apparatus 100. FIGS. 3A to 3D and FIGS. 4A to 4D are plan views illustrating examples of display screen transition in the information terminal. In the present embodiment, a conversation is assumed in which the language of one speaker (questioner) is Japanese and the language of the other speaker (answerer) is Chinese (however, the language or Situations are not limited to this).

  First, when the user activates the application (step SU1), the language selection for selecting the language of the other user on the display device 16 of the information terminal 10 by the processor 21 of the server 20 and the processor 11 of the information terminal 10 is performed. A screen is displayed (FIG. 3A; step SJ1). In this language selection screen, Japanese text T1 for prompting the user to ask the partner user for the language, English text T2 for asking the partner user for the language, and a plurality of assumed representatives A language button 31 is displayed that indicates a typical language (here, English, Chinese (for example, two types depending on the typeface), and Korean). Further below that, a cancel button B1 for closing the language selection screen and ending the application is also displayed.

  At this time, as shown in FIG. 3A, the Japanese text T1 and the English text T2 are divided by the processor 11 and the display device 16 into different areas on the screen of the display device 16 of the information terminal 10, and Are displayed in opposite directions (different directions; upside down in the figure). Accordingly, when a conversation is performed with the users facing each other, one user can easily confirm the Japanese text T1, while the other user can easily confirm the English text T2. In addition, since the Japanese text T1 and the English text T2 are displayed separately, there is an advantage that both are clearly distinguished and can be visually recognized more easily.

  When the user presents the display of the English text T2 on the language selection screen to the listener and the partner user taps the “Chinese” button, or the partner user himself / herself is in the language used “Chinese” can be selected. When the language of the other user is thus selected, the processor 21 of the server 20 and the processor 11 of the information terminal 10 display a standby screen for voice input in Japanese and Chinese on the display device 16 as the home screen ( FIG. 3B; Step SJ2).

  On this voice input standby screen, an input button 32a which is designed as a microphone for inputting Japanese speech and an input button 32b which is designed as a microphone for inputting Chinese speech are displayed. Further, on the edge side of the screen from the input buttons 32a and 32b, a Japanese text T3 indicating conversion of Japanese into Chinese and a Chinese text indicating conversion of Chinese into Japanese, respectively. Text T4 is displayed. Further, in the center of the screen from the input buttons 32a and 32b, there are a Japanese text T5 and a Chinese text T6 prompting to start a conversation by tapping the input buttons 32a and 32b each having a microphone. Is displayed.

  The voice input standby screen further includes a registered phrase button B2 for displaying a group of phrases registered in advance by the user, a text input button B3 for inputting text instead of voice input, and the application software. A setting button B4 for performing various settings and a scene selection button BS for selecting a conversation scene are also displayed.

  Next, on the voice input standby screen shown in FIG. 3B, when the user (questioner) taps the Japanese input button 32a and selects Japanese voice input, the user's Japanese utterance content is accepted. The voice input screen is displayed (FIG. 3C). When this voice input screen is displayed, voice input from the voice input / output device 13 is enabled.

  In addition, this voice input screen is designed with Japanese text T6 prompting voice input to the microphone of the information terminal 10, Chinese text T7 indicating that the other party is inputting voice, and a microphone. The input button 32a and the multiple circular design 33 surrounding the input button 32a are displayed. The multiple circular design 33 indicates that the voice input state is present, and the size of a circle portion to be displayed changes in accordance with the voice volume so as to schematically and dynamically represent the magnitude of the voice volume. Thereby, the voice input level is visually fed back to the user.

  Further, a cancel button B1 is also displayed on this voice input screen, and when this is tapped, the application can be terminated, or the voice input standby screen (FIG. 3B) can be returned to perform voice input again. it can. Further, in the vicinity of the input button 32a, Japanese text T8 for performing later-described speech recognition processing and multilingual translation processing is displayed after the speech input is completed.

  In this state, when the user (questioner) utters a matter to be transmitted to the other user (answerer) (step SU2), voice input is performed through the voice input / output device 13 (step SJ3). The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N. As described above, the information terminal 10 itself, or the processor 11 and the voice input / output device 13 function as an “input unit”.

  Then, when the utterance is finished and the Japanese text T8 shown in FIG. 4C is tapped (touched), the processor 11 finishes accepting the utterance content. The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N.

  Next, the processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing (step SJ4). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains “sound” of the input speech. Convert to "reading" (character). As described above, the server 20 also functions as a “voice recognition server” as a whole. In addition, the processor 21 stores the recognized content in the storage resource 23 (storage unit) (in an appropriate database as necessary) as voice input history data.

  Subsequently, the processor 21 proceeds to multilingual translation processing for translating the recognized “reading” (characters) of the recognized speech into a plurality of other languages (step SJ5). Here, since Chinese is selected as the language of the other user, the processor 21 uses the storage resource 23 to obtain the necessary module L20 and database D20 (translation module, Japanese character corpus, Japanese dictionary, Chinese dictionary). , Japanese / Chinese Bilingual Dictionary, Japanese / Chinese Bilingual Corpus, etc.), and appropriately sort the “reading” (character string) of the input speech that is the recognition result. The Chinese corresponding to the conversion result is extracted, rearranged according to the Chinese grammar, and converted into natural Chinese phrases, clauses, sentences, and the like.

  As described above, the processor 21 also functions as a “translation unit” that translates the content of the input speech into the content of the second language (Chinese) different from the first language (Japanese). It also functions as a “translation server”. If the input voice is not correctly recognized, the voice can be re-input (not shown). Further, the processor 21 can also store those Japanese and Chinese phrases, clauses, sentences, and the like in the storage resource 23.

  Further, during the translation process, the processor 11 of the information terminal 10 displays a translation process in progress screen shown in FIG. In this translation processing in progress screen, Japanese text T9 and Chinese text T10 indicating that translation processing is in progress, and a part of an arc for indicating that translation processing is in progress rotate. The displayed circular design 34 is also displayed. Furthermore, the cancel button B1 is also displayed on this translation processing screen, and when this is tapped, the application is terminated, or the voice input standby screen (FIG. 3B) is returned to perform voice input again. Can do.

  Next, when the multilingual translation processing is completed, the processor 21 proceeds to speech synthesis processing (step SJ6). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, Chinese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and the translation result of Chinese. Convert phrases, clauses, sentences, etc. to natural speech. As described above, the processor 21 also functions as a “speech synthesizer”, and the server 20 also functions as a “speech synthesizer” as a whole.

  Next, the processor 21 generates a text signal for text display based on the translation result in Chinese (or a corresponding Chinese conversation corpus), and transmits the text signal to the information terminal 10. The processor 11 which has received the text signal displays the Japanese text T11 indicating the content of the input speech that has been speech-recognized, and the translation result (translation) text T12 in Chinese as a translation result display shown in FIG. Display on the screen.

  In addition, on this translation result display screen, a Japanese input button 32a and a Chinese input button 32b, which are designed microphones also displayed on the home screen of FIG. 3B, are displayed, and in the vicinity thereof. , Texts T13 and T14 indicating Japanese and Chinese, respectively, are displayed.

  Furthermore, the processor 21 generates a voice signal for voice output based on the synthesized voice, and transmits the voice signal to the information terminal 10. The processor 11 that has received the speech signal outputs (reads out) the speech of the content of the Chinese text T12 using the speech input / output device 13 (output unit) together with the display of the texts T13 and T14 (step SJ7). .

  Further, a text input button B5 for inputting text instead of voice input is displayed near the text T11, and a check button B7 for returning to the home screen in FIG. Then, a mistranslation notification button B6 for reporting an error in the translation result is displayed. Furthermore, a display button B8 for displaying emotion characters or emotion symbols for expressing the user's emotion is also displayed on the translation result display screen.

  Here, when the text input button B5 is tapped, a text input screen shown in FIG. 4B is displayed. On this text input screen, the Japanese text T11 displayed in FIG. 4A is displayed in a light gray scale. When text input is started from the Japanese keyboard K, the text T11 is erased and the content of the newly input text is displayed at that location. Further, an erase button B9 for canceling and erasing the input text is displayed at the top of the text input screen, and a translation button B10 for translating the input text is displayed immediately above the keyboard K. . When this translation button B10 is tapped, the above translation process and speech synthesis process are performed, and a screen similar to the translation result display screen shown in FIG. 4A is displayed.

  After that, when the other user answers, when the Chinese input button 32b displayed on the translation result display screen in FIG. A voice input screen for accepting the utterance content is displayed (FIG. 4C). When this voice input screen is displayed, a voice input from the voice input / output device 13 is possible as in the voice input screen shown in FIG. In addition, this voice input screen is designed with a Chinese text T15 that prompts the user to input voice into the microphone of the information terminal 10, Japanese text T16 indicating that the other party is inputting voice, and a microphone. The input button 32b and the multiple circular design 33 surrounding the input button 32b are displayed.

  Further, a cancel button B1 is also displayed on this voice input screen, and when this is tapped, the application can be terminated, or the voice input standby screen (FIG. 3B) can be returned to perform voice input again. it can. Further, in the vicinity of the input button 32b, Chinese text T17 for performing speech recognition processing and multilingual translation processing described later is displayed after the speech input is completed.

  In this state, when the partner user (respondent) utters an answer item or the like to the user (questioner) (step SU2), voice input is performed through the voice input / output device 13 (step SJ3). The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N. Then, when the utterance is finished and the Chinese text T14 is tapped (touched), the processor 11 finishes accepting the utterance content. The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N.

  Next, the processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing (step SJ4). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Chinese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains the “sound” of the input speech. Convert to "reading" (character). In addition, the processor 21 stores the recognized content in the storage resource 23 (in an appropriate database as necessary) as voice input history data.

  Subsequently, the processor 21 proceeds to multilingual translation processing for translating the recognized “reading” (characters) of the recognized speech into a plurality of other languages (step SJ5). The processor 21 receives the necessary module L20 and database D20 from the storage resource 23 (translation module, Chinese character corpus, Chinese dictionary, Japanese dictionary, Chinese / Japanese bilingual dictionary, Chinese / Japanese bilingual corpus, etc.) , And appropriately sort the “reading” (character string) of the input speech that is the recognition result to convert it into Chinese phrases, clauses, sentences, etc., extract the Japanese corresponding to the conversion result, and extract them Rearrange according to Japanese grammar and convert to natural Japanese phrases, clauses, sentences, etc. If the input voice is not correctly recognized, the voice can be re-input (not shown). Further, the processor 21 can also store those Chinese and Japanese phrases, clauses, sentences and the like in the storage resource 23.

  Further, during the translation process, the processor 11 of the information terminal 10 displays a translation process in progress screen shown in FIG. On this translation processing screen, Japanese text T9 and Chinese text T10 indicating that translation processing is in progress, and a part of an arc that rotates to indicate that translation processing is in progress are displayed. An annular design 34 is also displayed. Furthermore, the cancel button B1 is also displayed on this translation processing screen, and when this is tapped, the application is terminated, or the voice input standby screen (FIG. 3B) is returned to perform voice input again. Can do.

  Next, when the multilingual translation processing is completed, the processor 21 proceeds to speech synthesis processing (step SJ6). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and translates the Japanese translation result. Convert phrases, clauses, sentences, etc. to natural speech.

  Next, the processor 21 generates a text signal for text display based on the translation result in Japanese (or a corresponding Japanese conversation corpus), and transmits the text signal to the information terminal 10. The processor 11 that has received the text signal converts the Chinese text indicating the content of the input speech that has been voice-recognized and the text of the translation result (translation) in Japanese into the translation result display screen shown in FIG. Display in the same way.

(Conversation by speech translation considering the conversation scene)
Next, in conversation between users (speakers) and / or conversation preparation, the user selects a scene of the conversation, and processing operations and operations when performing speech translation in consideration of the selected conversation scene An example will be described below. FIG. 5 is a flowchart showing an example of (part of) the processing flow in the speech translation apparatus 100. 6A and 6B are plan views showing an example of display screen transition in the information terminal. In the present embodiment, a conversation is assumed when the language of one user (questioner) is Japanese and the language of the other user (answerer) is English (however, the language and situation are Not limited to).

  The processing procedure here is to select a conversation scene prior to the utterance (step SU2) shown in FIG. 2 (step SU3), and between speech recognition (step SJ4) and multilingual translation (step SJ5). A determination as to whether or not the content of the recognized voice is a question sentence and includes a specific phrase (step SJ8); a determination as to whether or not the specific phrase is associated with the selected scene (step SJ9); The processing procedure in the “conversation by normal speech translation” shown in FIG. 2 is the same except that the fixed phrase stored in association with the specific phrase and the selected scene is executed (step SJ10).

  That is, the user (questioner) starts the application (step SU1), and displays a language selection screen for selecting the language of the other user (respondent) (FIG. 3A; step SJ1). Further, the language of the other user (respondent) is selected and a Japanese and English voice input standby screen is displayed on the display device 16 (similar to FIG. 3B; step SJ2). Then, on the voice input standby screen shown in FIG. 3B, the user (questioner) taps a scene selection button BS for selecting a conversation scene. Then, a list display screen of conversation scenes is displayed on the display device 16 by the processor 21 of the server 20 and the processor 11 of the information terminal 10 (FIG. 6A).

  In the example shown in FIG. 6A, a large classification of scenes (for example, “food”, “shopping”, “tourism”, “business”, etc.) is displayed as scene heading tabs S1 to S4. When the user (questioner) taps the desired scene heading tabs S1 to S4 from among them, a small classification of a specific scene belonging to the scene tab (when the large classification is “food”, for example, “customer service”, “ “Guidance”, “Order”, “Account”, etc.) are displayed as a list as scene bars S11 to S15.

  Here, as an example, if the user (questioner) is a clerk of a restaurant that speaks Japanese, and the user (respondent) is an English-speaking foreign customer who has visited the restaurant, the clerk who is the questioner Tap the “Drink” scene tab from the scene tabs S1 to S4, and tap the “Customer Service” or “Guidance” scene bar from the scene bars S11 to S15 because it is a response when visiting the store. Then, the scene of the conversation at that time can be selected (step SU3).

  When a conversation scene is selected by the user, the voice input standby screen shown in FIG. 3B is displayed again on the display device 16 of the information terminal 10 by the processor 21 of the server 20 and the processor 11 of the information terminal 10. . In this state, when the questioner utters, for example, “What kind of seats are you sure?” As a question to the respondent, unlike the phrase shown in FIG. 4A (step SU2), the voice input / output device A voice input is performed through 13 (step SJ3). When the utterance is finished and the Japanese text T8 shown in FIG. 4C is tapped (touched), the processor 11 finishes accepting the utterance content, and the processor 21 of the server 20 performs the speech recognition process. Perform (step SJ4).

  In this case, specifically, the processor 21 converts “sound” of the input speech into “reading” (characters), performs morphological analysis, and determines, for example, “what”, “seat”, “ Phrases such as “,” “OK,” and “Is it?” Are extracted. Next, the processor 21 determines whether or not the content of the input voice is a question sentence and includes a specific phrase (step SJ8). More specifically, the processor 21 asks whether the content of the input voice is a question from the fact that the question of “what” is included in the content of the input voice, or in addition to the expression of the ending of “?”. Judge that it is a sentence.

  On the other hand, in the storage resource 23 (storage unit), for example, a single phrase “seat” or a combination of the phrases “what” + “seat” is stored in advance as a specific phrase. Then, it is determined that the content of the input voice includes a specific phrase (Yes in step SJ8). If the content of the input voice is not a question sentence or does not include a specific phrase (No in step SJ8), the process proceeds to a normal multilingual translation process (step SJ5).

  Next, the processor 21 determines whether the selected scene (“customer service” or “guidance”) is associated with a specific phrase (“seat” or “what” + “seat”) included in the content of the input voice. Is determined (step SJ9). Here, since the scene “customer service” is associated with “seat” or “what” + “seat” and stored in advance in the storage resource 23, a specific phrase included in the content of the input voice is It is determined that the selected scene is associated (Yes in step SJ9). If the selected phrase included in the content of the input voice is not associated with the selected scene (No in step SJ9), the process proceeds to a normal multilingual translation process (step SJ5).

  Then, the processor 21 selects a specific phrase (“seat” or “what” + “seat”) and a selected scene (“customer service” or “guidance”) from the information stored in the storage resource 23. If there is a fixed syntax stored in association with both, it is extracted and presented as a list on the display device 16 (Yes in step SJ10; FIG. 6B). In the example shown in FIG. 6B, the storage resource 23 is associated with both a specific phrase (“seat” or “what” + “seat”) and the selected scene (“customer service” or “guidance”). Memorized canonical syntax is: “Would you like a seat indoors or a terrace?”, “Would you like a non-smoking seat or a smoking seat?”, “Would you like a counter or a table? The syntax list PL displays syntaxes such as “Which seat do you want, 1st floor or 2nd floor?” And “Do you have any seats?”

  Here, for example, the foreign customer who is the respondent who was asked by the clerk, who is the questioner, what kind of seats are there in the first place? Often, the store clerk may not know what kind of seat is assumed as “what kind of seat” is the question. In that case, it is also assumed that the foreign customer who has been questioned asks the store clerk for confirmation of the content as exemplified in the syntax list PL of FIG. In addition, since such exchange is performed via speech translation, it tends to be difficult to achieve smooth communication.

  On the other hand, as shown in FIG. 6B, when the syntax list PL is displayed, the clerk who is the questioner, for example, has a question content with higher accuracy than “what seat is right?” Can I ask a foreign customer to choose a counter or a table? ”), So you can ask appropriate questions to the foreign customer and make the foreigner ’s intention more accurate It is possible to facilitate smooth communication between the two while grasping.

  In this way, the clerk who is the questioner taps and selects a desired syntax from the syntax list PL shown in FIG. 6B, so that the content of the syntax is translated (step SJ5). Then, voice synthesis (step SJ6) and voice output (step SJ7) are performed. In addition, on the syntax list display screen shown in FIG. 6B, a text input button B5 similar to that displayed on the translation result display screen of FIG. 4A is also displayed. When the speaker taps the text input button B5, a text input screen similar to that shown in FIG. 4B is displayed. For example, referring to the syntax list PL, other contents can be input as text. The contents may be additionally registered in the storage resource 23 as a fixed syntax.

  Note that, as described above, each of the above embodiments is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art can replace the resources (hardware resources or software resources) described in the embodiments with equivalents, and such replacements are also included in the scope of the present invention.

  Further, step SU3 for selecting a conversation scene may be performed, for example, after application activation (step SU1), after language selection screen display (step SJ1), or after speech (step SU2). Or you may comprise so that the scene of a conversation can be selected (input) in the arbitrary timings when the conversation progressed to some extent. Further, the syntax list PL may be displayed together with the recognized voice content (for example, “What kind of seats are you sure?” Which is the voice input content described above).

  Moreover, although the example which performs each process, such as speech recognition, translation, speech synthesis, by server 20, was described, you may comprise so that these processes may be performed in the information terminal 10. FIG. In this case, the module L20 used for these processes may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. Furthermore, the database D20 that is a voice database and / or a model M20 such as an acoustic model may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. . As described above, the speech translation apparatus may not include the network N and the server 20.

  Of course, a gateway server for converting a communication protocol between the information terminal 10 and the network N may be interposed. The information terminal 10 is not limited to a portable device, and may be a desktop personal computer, a notebook personal computer, a tablet personal computer, a laptop personal computer, or the like.

  According to the present invention, a questioner answers in a conversation using speech translation by presenting a fixed syntax stored in association with a specific phrase and a selected conversation scene so that the user can select it. For example, a program in the field of providing services related to conversations between people who cannot understand each other's language. It can be widely used in activities such as designing, manufacturing, providing, selling, etc., devices, systems, and methods.

  DESCRIPTION OF SYMBOLS 10 ... Information terminal, 11 ... Processor, 12 ... Storage resource, 13 ... Voice input / output device, 14 ... Communication interface, 15 ... Input device, 16 ... Display device, 17 ... Camera, 20 ... Server, 21 ... Processor, 22 ... Communication interface 23 ... Memory resource 31 ... Language button 32a, 32b ... Input button 33 ... Multiple circular design 34 ... Circular design 100 ... Speech translation device B1 ... Cancel button B2 ... Registered phrase button B3 ... Text input button, B4 ... Setting button, B5 ... Text input button, B6 ... Check button, B7 ... Mistranslation notification button, B8 ... Display button, B9 ... Erase button, B10 ... Translation button, BS ... Scene selection button, D20 ... Database , E1, E2 ... face mark, K ... keyboard, L20 ... module, M2 ... model, N ... network, P10, P20 ... program, PL ... syntax list, S1~S4 ... scene heading tab, S11~S15 ... scene bar, SJ1~SJ10, SU1~SU3 ... step, T1~T17 ... text.

Claims (6)

An input unit for inputting the voice of the user who is having a conversation;
A scene selection means presenting section for presenting a scene selection means for the user to select a scene in the conversation from a plurality of scenes;
A storage unit that stores a plurality of standard syntaxes assumed for a specific phrase in advance in association with the specific phrase and the scene;
Recognizing the input voice, and when the content of the recognized voice is a question sentence and includes the specific phrase, the specific phrase and the selected scene are stored in association with each other. A fixed syntax presenting section that presents a fixed syntax so that the user can select it;
A translation unit that translates the content of the selected fixed syntax into content of a different language;
An output unit for outputting the content translated into the different languages in audio and / or text;
A speech translation apparatus comprising: The input unit presents input changing means for the user to change the content of the selected fixed syntax,
The translation unit translates the changed content into content of a different language;
The speech translation apparatus according to claim 1. The fixed syntax presentation unit presents the fixed syntax together with the recognized speech content;
The speech translation apparatus according to claim 1 or 2. A standard syntax input unit for the user to add and input the standard syntax;
The speech translation apparatus according to any one of claims 1 to 3. Using a speech translation device including an input unit, a scene selection means presentation unit, a storage unit, a fixed syntax presentation unit, a translation unit, and an output unit,
The input unit inputting a voice of a user having a conversation;
The scene selection means presenting unit presenting a scene selection means for the user to select a scene in the conversation from a plurality of scenes;
A step of storing in advance a plurality of fixed syntaxes assumed for a specific phrase in association with the specific phrase and the scene;
When the fixed syntax presenting unit recognizes the input voice and the content of the recognized voice is a question sentence and includes the specific phrase, the specific phrase and the selected scene Presenting the boilerplate syntax stored in association with the user for selection by the user;
The translating unit translating the content of the selected fixed syntax into content of a different language;
The output unit outputting the content translated into the different languages in voice and / or text;
Speech translation method including Computer
An input unit for inputting the voice of the user who is having a conversation;
A scene selection means presenting section for presenting a scene selection means for the user to select a scene in the conversation from a plurality of scenes;
A storage unit that stores a plurality of standard syntaxes assumed for a specific phrase in advance in association with the specific phrase and the scene;
Recognizing the input voice, and when the content of the recognized voice is a question sentence and includes the specific phrase, the specific phrase and the selected scene are stored in association with each other. A fixed syntax presenting section that presents a fixed syntax so that the user can select it;
A translation unit that translates the content of the selected fixed syntax into content of a different language;
An output unit for outputting the content translated into the different languages in audio and / or text;
A speech translation program that makes it work.