JP6383748B2 - Speech translation device, speech translation method, and speech translation program - Google Patents

Description

  The present invention relates to a speech translation device, a speech translation method, and a speech translation program.

  In order to enable conversation between people who cannot understand each other's language, for example, conversation between a user (store employee, etc.) and a conversation person (foreign customer, etc.) Speech translation technology has been proposed in which the content is machine-translated into the partner language and displayed on the screen, or the text content is played back using speech synthesis technology (for example, Patent Documents 1 and 2). reference).

Japanese Patent Laid-Open No. 9-34895 Japanese Unexamined Patent Publication No. 2014-16475

  By the way, as shown in, for example, the operation screen of the application shown in Patent Document 2, a conventional speech translation application or device is configured to first select the language used by the user and the conversation person during conversation. However, such a language selection operation is necessary for the conversation between speakers using different languages, but in the conversation, the user asks a conversation person who does not understand the language to ask the speaker to select a language. It must be made. This is a psychological burden on the user and an obstacle for smooth conversation.

  Therefore, the present invention has been made in view of such circumstances, and a speech translation apparatus that can naturally start and smoothly advance a conversation between a user and a conversation person without performing language selection at the start of the conversation, An object is to provide a speech translation method and a speech translation program.

  In order to solve the above-described problem, a speech translation apparatus according to an aspect of the present invention provides an input unit for inputting a voice of a user and / or a dialoguer and one input voice (one phrase) (for input voice). A translation unit that acquires parallel translations in a plurality of different languages (other than a language) and an output unit that outputs the parallel translations in the plurality of different languages in text and / or speech.

  Specifically, the output unit may display a parallel translation text in a plurality of different languages at one time. The screen display in this case is not limited to a single screen, and may be divided into a plurality of screens. Furthermore, the voice output by the output unit may not output all of the parallel translations in a plurality of different languages, and may output the voice of at least one parallel translation designated by, for example, the user or the dialogue person among them.

  In addition, a language selection unit may be further provided that selects a language used by the conversation person after a parallel translation in a plurality of different languages is output. Furthermore, a storage unit for storing or acquiring the selection frequency or selection frequency of each of a plurality of different languages is further provided, and the output unit is a predetermined number of higher-order languages having a relatively high selection frequency or selection frequency among a plurality of different languages. It is also preferable to configure so as to output the bilingual translation. In this case, the output unit may output the translations in the predetermined number of languages in the order of selection frequency or selection frequency. The “selection frequency” may be a value based on the number of times selected by the language selection unit after bilingual translation in a plurality of different languages is output, or may be a value based on the number of times selected in advance, or both It may be a value based on the total number of times.

  Furthermore, it further includes an index acquisition unit that acquires a compatibility index (an index indicating the accuracy or accuracy of translation) between the content of one input speech and each of parallel translations in a plurality of different languages, and the output unit includes a plurality of different languages It is also preferable to output a predetermined number of high-order parallel translations having a relatively high compatibility index.

  A speech translation method according to an aspect of the present invention is a method using a speech translation apparatus including an input unit, a translation unit, and an output unit. That is, in the method, the input unit inputs the voice of the user and / or the conversation person, the translation unit acquires parallel translations in a plurality of different languages for one input voice, and the output unit Outputting a translation in a plurality of different languages as text and / or voice.

  In addition, the speech translation program according to one aspect of the present invention is a computer (not limited to a single type or a single type, but may be a plurality or a plurality of types; the same shall apply hereinafter), and an input unit for inputting a voice of a user and / or a conversation person And a translation unit that acquires parallel translations in a plurality of different languages for one input voice, and an output unit that outputs parallel translations in a plurality of different languages as text and / or speech.

  According to the present invention, in a conversation between a user and a conversation person, for example, parallel translations in a plurality of different languages are obtained for one input speech uttered by the user, and the plurality of parallel translations are output as text and / or speech. By doing so, even if the language used by the interlocutor is unknown, it is possible to obtain a conversation start (timing) and to convey the content of the user's utterance to the interlocutor. Therefore, the conversation can be started naturally without selecting the language used by the interlocutor prior to the conversation, and the psychological burden on the user can be reduced. In addition, since the language used by the interlocutor is determined from the parallel translation that the interlocutor can understand, the subsequent utterances do not require translation into different languages, and the subsequent conversation can proceed smoothly. .

1 is a system block diagram schematically showing a preferred embodiment of a network configuration and the like related to a speech translation apparatus according to the present invention. It is a flowchart which shows an example of the flow (part) of the process in suitable one Embodiment of the speech translation apparatus by this invention. It is a flowchart which shows an example of the flow (part) of the process in suitable one Embodiment of the speech translation apparatus by this invention. (A) thru | or (D) are top views which show an example of the transition of the display screen in an information terminal. (A) And (B) is a top view which shows an example of the transition of the display screen in an information terminal.

  Hereinafter, embodiments of the present invention will be described in detail. The following embodiments are examples for explaining the present invention, and are not intended to limit the present invention only to the embodiments. The present invention can be variously modified without departing from the gist thereof. Furthermore, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent ones, and such embodiments are also included in the scope of the present invention. Furthermore, positional relationships such as up, down, left, and right shown as needed are based on the display shown unless otherwise specified. Furthermore, various dimensional ratios in the drawings are not limited to the illustrated ratios.

(Device configuration)
FIG. 1 is a system block diagram schematically showing a preferred embodiment such as a network configuration related to a speech translation apparatus according to the present invention. In this example, the speech translation apparatus 100 includes a server 20 that is electronically connected to the information terminal 10 (user apparatus) used by the user via the network N (but is not limited to this).

  The information terminal 10 employs a user interface such as a touch panel and a display with high visibility, for example. The information terminal 10 here is a portable tablet terminal device including a mobile phone represented by a smartphone having a communication function with the network N. The information terminal 10 further includes a processor 11, a storage resource 12, a voice input / output device 13, a communication interface 14, an input device 15, a display device 16, and a camera 17. In addition, the information terminal 10 operates by the installed speech translation application software (at least a part of the speech translation program according to the embodiment of the present invention), so that a part of the speech translation apparatus according to the embodiment of the present invention or It functions as a whole.

  The processor 11 includes an arithmetic logic unit and various registers (program counter, data register, instruction register, general-purpose register, etc.). Further, the processor 11 interprets and executes speech translation application software, which is the program P10 stored in the storage resource 12, and performs various processes. The speech translation application software as the program P10 can be distributed from the server 20 through the network N, for example, and may be installed and updated manually or automatically.

  The network N includes, for example, a wired network (a short-range communication network (LAN), a wide-area communication network (WAN), a value-added communication network (VAN), etc.) and a wireless network (mobile communication network, satellite communication network, Bluetooth ( Bluetooth (registered trademark)), WiFi (Wireless Fidelity), HSDPA (High Speed Downlink Packet Access), etc.).

  The storage resource 12 is a logical device provided by a storage area of a physical device (for example, a computer-readable recording medium such as a semiconductor memory), and an operating system program, a driver program, various data, etc. used for processing of the information terminal 10 Is stored. Examples of the driver program include an input / output device driver program for controlling the audio input / output device 13, an input device driver program for controlling the input device 15, and a display device driver program for controlling the display device 16. Can be mentioned. Furthermore, the voice input / output device 13 is, for example, a general microphone and a sound player capable of reproducing sound data.

  The communication interface 14 provides a connection interface with the server 20, for example, and is configured from a wireless communication interface and / or a wired communication interface. The input device 15 provides an interface for accepting an input operation by a tap operation such as an icon, a button, a virtual keyboard, or a text displayed on the display device 16. Various input devices to be attached can be exemplified.

  The display device 16 provides various information as an image display interface to a user or a conversation person (conversation partner), and examples thereof include an organic EL display, a liquid crystal display, and a CRT display. The camera 17 is for capturing still images and moving images of various subjects.

  The server 20 is constituted by, for example, a host computer having a high arithmetic processing capability, and expresses a server function by operating a predetermined server program in the host computer, for example, a speech recognition server, a translation server, And a single or a plurality of host computers functioning as a speech synthesis server (in the drawing, it is indicated by a single, but is not limited thereto). Each server 20 includes a processor 21, a communication interface 22, and a storage resource 23.

  The processor 21 is composed of an arithmetic and logic unit for processing arithmetic operations, logical operations, bit operations and the like and various registers (program counter, data register, instruction register, general-purpose register, etc.), and is stored in the storage resource 23. P20 is interpreted and executed, and a predetermined calculation processing result is output. The communication interface 22 is a hardware module for connecting to the information terminal 10 via the network N. For example, the communication interface 22 is a modulation / demodulation device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

  The storage resource 23 is a logical device provided by, for example, a storage area of a physical device (a computer-readable recording medium such as a disk drive or a semiconductor memory), and each includes one or a plurality of programs P20, various modules L20, and various databases. D20 and various models M20 are stored. The storage resource 23 also includes a plurality of standard questions prepared for the user to speak to the conversation person, history data of input speech, data for various settings, selection frequency (or number of selections) of each language, and the like. It is remembered.

  The program P20 is the above-described server program that is the main program of the server 20. In addition, the various modules L20 perform a series of information processing related to requests and information transmitted from the information terminal 10, so that they are appropriately called and executed during the operation of the program P10 (moduleized subprograms). ). Examples of the module L20 include a speech recognition module, a translation module, and a speech synthesis module.

  The various databases D20 include various corpora required for speech translation processing (for example, in the case of Japanese and other language speech translation, a Japanese speech corpus, each other language speech corpus, a Japanese character (vocabulary) corpus). , Each other language character (vocabulary) corpus, Japanese dictionary, each other language dictionary, Japanese / each other language parallel translation dictionary, Japanese / each other language parallel corpus, etc.), voice database, for managing information about users Examples include a management database, a compatibility index (an index indicating the accuracy or accuracy of translation) between corpus of different languages, and the like. Examples of the various models M20 include acoustic models and language models used for speech recognition.

(First embodiment)
An example (first embodiment) of speech translation processing operations and operations in the speech translation apparatus 100 configured as described above will be further described below. FIG. 2 is a flowchart illustrating an example of (a part of) a processing flow in the speech translation apparatus 100 according to the first embodiment. 4A to 4D are plan views illustrating an example of transition of display screens in the information terminal. Here, it is assumed that the user of the information terminal 10 is a Japanese clerk (store employee) and the conversation person (conversation partner) is a foreign customer who speaks French (however, Languages and situations are not limited to this).

  First, when the user (clerk) starts the application (step SU1), a voice input standby screen in Japanese and English as default languages is displayed on the display device 16 as a home screen on the display device 16 of the information terminal 10. (FIG. 4A; step SJ1). In this voice input standby screen, the Japanese text T1 asking which of the user's language and the talker's language is to be spoken, the input button 42a for performing Japanese voice input, and the English voice input are performed. The input button 42b is displayed.

  In addition, the voice input standby screen includes a voice button 43 for selecting a list display of a plurality of preset canned sentences, a language selection button 44 for manually selecting a language of a conversation person, The history button 45 for selecting the history display of the voice input content made so far, and the conversation can be advanced by selecting a desired fixed phrase from a plurality of prepared fixed phrases (recommended phrases) group. A suggest button 46 for executing the suggest function and a setting button 47 for performing various settings of the application software are also displayed.

  Next, in the voice input standby screen shown in FIG. 4A, when the user taps the Japanese input button 42a and selects Japanese voice input, a voice input screen for accepting the user's Japanese utterance content; (FIG. 4B). When this voice input screen is displayed, voice input from the voice input / output device 13 is enabled. Further, on this voice input screen, a text T2 for prompting the user to input voice, a microphone design 48 indicating that the voice input state is set, and an input switching button 41 for switching to the text input are displayed. Further, a cancel button B1 is also displayed on this voice input screen. By tapping this button, the conversation can be ended or the voice input can be performed again by returning to the voice input standby screen (FIG. 4A). .

  In this state, when the user utters an item to be communicated to the interlocutor (for example, a phrase such as “Is there any use?”) (Step SU2), the voice volume is schematically and dynamically expressed with the text T2. Multiple circular designs 49 are displayed and the voice input level is visually fed back to the user who is the speaker. Then, when the utterance is finished and the user taps the microphone design 48, the processor 11 finishes accepting the utterance content by the user. The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N. As described above, the information terminal 10 itself, or the processor 11 and the voice input / output device 13 function as an “input unit”.

  Next, the processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing. At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains “sound” of the input speech. Convert to "reading" (character). As described above, the processor 21 or the server 20 functions as a “voice recognition server” as a whole. In addition, the processor 21 stores the recognized content in the storage resource 23 (in an appropriate database as necessary) as voice input history data.

  Next, the processor 21 transmits the recognition result of the input voice to the information terminal 10, and the processor 11 displays it on the screen as Japanese text (not shown). At this time, the recognition result of the input voice may be displayed as it is, or the Japanese speech corpus stored in the storage resource 23 in advance is called and displayed corresponding to the content of the actual input voice. May be.

  Subsequently, the processor 21 proceeds to multilingual translation processing for translating the recognized “reading” (characters) of the speech into a plurality of other languages. At this time, the processor 21 reads the necessary module L20 and database D20 (translation module, Japanese character corpus, Japanese dictionary, each other language dictionary, each Japanese / each other language parallel dictionary, each Japanese / each language from the storage resource 23. Call other languages parallel corpus, etc.), and sort the “reading” (character string) of the input speech, which is the recognition result, and convert it into Japanese phrases, clauses, sentences, etc. The language is extracted and rearranged according to the grammar of each other language to be converted into natural phrases, clauses, sentences, etc. of each other language. As described above, the processor 21 also functions as a “translation unit” that acquires parallel translations in a plurality of different languages for one input speech, and the server 20 also functions as a “translation server” as a whole. If the input voice is not correctly recognized, the voice can be re-input (not shown). The processor 21 can also store those Japanese and English phrases, clauses, sentences, and the like in the storage resource 23.

  Then, the processor 21 generates an output signal of a parallel translation (for example, a parallel translation of “Is there any use?”) Obtained by multilingual translation processing and transmitted to the information terminal 10. Based on these output signals, the processor 11 of the information terminal 10 displays a list screen of parallel texts T3 and T4 shown in FIGS. 4C and 4D on the display device 16, for example. The list screen of these translation texts T3 and T4 is switched by wiping the screen of the display device 16 left and right with a finger, for example. As described above, the processors 11 and 21 and the display device 16 function as an “output unit”, and bilingual texts T3 and T4 in different languages are displayed on the display device 16 at a time.

  Here, parallel translations in English and Asian languages (Chinese, Korean, Vietnamese, Tagalog, etc.) are displayed on the list screen of the parallel translation text T3 shown in FIG. In addition, the translation screen in English and other Western languages (Italian, Spanish, German, French, etc.) is displayed on the list screen of the parallel text T4 shown in FIG. Further, buttons B11 to B15 and B21 to B25 for tapping and inputting answers in the respective languages (corresponding to “yes” and “no”) are displayed immediately below the respective parallel translation texts T3 and T4.

  Next, if there is a bilingual text in a language used by the interlocutor or a language that the interlocutor can understand, the interlocutor presented with the list screen of the bilingual texts T3 and T4 below the bilingual translation. Can respond by tapping the button (button B25 displayed under the translation in French if the conversation person is French) (step SU3).

  When the dialogue person taps the “Oui” (Yes) portion of the button B25 in response to the user's inquiry, the selection signal is transmitted from the processor 11 of the information terminal 10 to the server 20, and the selection signal is received. The processor 21 determines that the language of the conversation person is French, and selects French as the language used by the conversation person in the conversation (step SJ3).

  Next, the processor 21 displays the voice input standby screen on the display device 16 again (step SJ1 again). This voice input standby screen is the same as the Japanese voice input standby screen shown in FIG. 4A except that an input button for performing French voice input is displayed instead of the English input button 42b. (Not shown). Next, on the voice input standby screen, for example, when the user taps a French input button and selects French voice input, the voice input screen is displayed that accepts the content of the conversational person in French. This French speech input screen is configured in the same manner as the Japanese speech input screen shown in FIG. 4B except that the Japanese text display is replaced with the French text display (not shown). .

  In this state, the conversation person utters a matter to be transmitted to the user (step SU2 again), and when the conversation person taps the microphone design 48, the processor 11 ends the acceptance of the utterance content by the conversation person, and the processor 21 Performs speech recognition processing for input speech and translation processing from French content to Japanese content.

  Then, the processor 21 proceeds to speech synthesis processing. At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, English speech corpus, acoustic model, language model, etc.) from the storage resource 23, and the Japanese phrase that is the translation result. , Paragraphs, sentences, etc. are converted into natural speech. As described above, the processor 21 also functions as a “speech synthesizer”, and the server 20 also functions as a “speech synthesizer” as a whole.

  Then, the processor 21 generates a voice signal for voice output based on the synthesized voice, and transmits the voice signal to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the audio signal through the communication interface 14 and performs Japanese audio output processing using the audio input / output device 13 (step SJ4 so far). As described above, the processors 11 and 21 and the voice input / output device 13 also function as “output units”. Prior to the voice output, the voice recognition result and the translation result of the conversation person may be temporarily displayed on the information terminal 10 and the voice may be outputted after confirmation by the talker (not shown).

  Then, by repeating steps SJ1, SU2, and SJ4 as necessary, the conversation between the user and the conversation person can be advanced. After the conversation is finished, the user can end the application as appropriate (step). SU4).

(Second Embodiment)
Next, another example (second embodiment) of operations and operations of speech translation processing in the speech translation apparatus 100 will be described. FIG. 3 is a flowchart illustrating an example of (a part of) a processing flow in the speech translation apparatus 100 according to the second embodiment. 5A and 5B are plan views showing an example of display screen transition in the information terminal. In the second embodiment, in step SJ2, in addition to displaying a list of bilingual texts T3 and T4 in a plurality of languages, the same processing as in the first embodiment is executed except that the bilingual speech is output. To do.

  As the speech output processing of the parallel translation at this time, for example, the user designates the parallel translation in at least one language estimated by the user based on the appearance or behavior of the conversation person, and only the designated parallel translation is output by voice. Alternatively, some or all of the parallel translations in a plurality of languages obtained as a translation result may be automatically output as voice. As a method of designating a parallel translation to be output, for example, a configuration in which a desired parallel translation portion is tapped in the parallel translation texts T3 and T4 displayed in FIGS. 4C and 4D, FIGS. As shown in FIG. 9, there is a configuration in which a check box 51 for selection is provided for each translation and designation (checking) is performed by tapping. When the parallel translation to be output in this way is designated, the information terminal 10 and the server 20 perform speech synthesis of the content of the parallel translation, and the speech input / output device 13 is used to perform parallel speech output processing in each language (sequentially). Reproduction) is performed (step SJ2).

(Third embodiment)
Next, another example (third embodiment) of speech translation processing operations and operations in the speech translation apparatus 100 will be described. In the third embodiment, in step SJ2, parallel translations in a plurality of languages having a relatively high frequency or frequency of selection (use) in a conversation between a user and a conversation person at a user's store or a store of the same industry, that is, The same processing as that in the first embodiment or the second embodiment is executed except that a parallel translation in a predetermined number of languages having a relatively high selection frequency or selection frequency among a plurality of different languages is output.

  In this case, in the conversation using the speech translation application at the user's store or the store of the same industry, the processor 21 of the server 20 stores the number or frequency of the language selection at step SJ3 in an appropriate database in the storage resource 23. Remember. Alternatively, the processor 21 acquires the selection frequency or selection frequency of each language from the information terminals 10 of a plurality of users via the network N at an appropriate timing, and stores them in an appropriate database in the storage resource 23. As described above, the processor 21 and the storage resource 23 function as a “storage unit”.

  Then, in step SJ2, the processor 21 selects a predetermined number of parallel translations of a language having a relatively high selection frequency or selection frequency from the parallel translations in a plurality of languages, and the selected parallel translations are, for example, shown in FIG. And it displays like the list screen of bilingual text T3, T4 shown to (D). Alternatively, in addition to or instead of it, the processor 21 converts a parallel translation of a language having a relatively high selection frequency or selection frequency from a plurality of languages into parallel translation texts shown in FIGS. In the list screen of T3 and T4, it is displayed in a higher order (for example, the upper part of the screen).

(Fourth embodiment)
Furthermore, another example (fourth embodiment) of speech translation processing operations and operations in the speech translation apparatus 100 will be described. In the fourth embodiment, in step SJ2, a plurality of parallel translations having a relatively high translation accuracy or accuracy among a plurality of parallel translations in different languages, that is, the compatibility index between the contents of the input speech and the parallel translation is relatively high. A process similar to that of the first embodiment or the second embodiment is executed except that a predetermined number of higher-order translations are output.

  In this case, for example, the processor 21 acquires in advance a compatibility index for each parallel phrase recorded in the corpus of each language from a web page or a commercial database connected to the network N, or, for example, Evaluation or collection is performed in advance by accuracy evaluation using crowdsourcing. The processor 21 stores the acquired suitability index in the storage resource 23 in association with each bilingual phrase. As described above, the processor 21 and the storage resource 23 also function as an “index acquisition unit”. In addition, as a conformity evaluation method, for example, the conformity evaluation (Walker, K., et al: Multiple-Translation Arabic (MTA) Part 1, Linguistic Data Consortium, Philadelphia used for semantic comparison between two languages is used. (2003)) is used (but not limited to).

  Then, in step SJ2, the processor 21 selects a predetermined number of parallel translations of languages having a relatively high suitability index from the parallel translations in a plurality of languages, and the selected parallel translations are, for example, shown in FIGS. It is displayed as a list screen of parallel translation texts T3 and T4 shown in D). Alternatively, in addition to or instead of it, the processor 21 converts a parallel translation of a language having a relatively high consistency index from a plurality of parallel translations into, for example, a parallel text T3 shown in FIGS. Displayed in a higher order (for example, at the top of the screen) on the list screen of T4.

  According to the speech translation apparatus 100 configured as described above, the speech translation method using the speech translation device, and the speech translation program, when a conversation between the user and the conversation person is started, a plurality of input speech uttered by the user The parallel translations in different languages are acquired, the plurality of parallel translation texts T3 and T4 are displayed as shown in FIGS. 4C and 4D, for example, and presented to the conversation person, and / or the parallel translations are spoken. To output.

  Therefore, even if the language used by the interlocutor is unknown, it is possible to obtain a conversation start (timing) and to correctly convey the user's utterance content to the interlocutor. Therefore, the conversation can be started naturally without selecting the language used by the interlocutor prior to the conversation, and the psychological burden on the user can be reduced. In addition, the language used by the conversation person (French in the example of the embodiment) is determined from the parallel translation that the conversation person can understand, and is selected as the use language. Therefore, in the subsequent utterances, a plurality of different languages are used. Translation into is no longer necessary, and subsequent conversations can proceed smoothly.

  Further, as shown in FIGS. 4C and 4D, if the bilingual texts T3 and T4 in a plurality of different languages are displayed at a time, the visibility is improved, so that the conversation person can easily grasp the contents of the user's utterance. . Further, as shown in FIGS. 4C and 4D, if bilingual translations in Asian languages and Western languages are displayed on different screens, the visibility and convenience at that time are further improved. In addition, such display is particularly useful when the user can narrow down the language used by the user to some extent from the appearance and behavior of the user.

  Furthermore, by outputting parallel translations in a predetermined number of languages having a relatively high selection frequency or selection frequency among a plurality of different languages, and outputting those parallel translations in the order of selection frequency or selection frequency. Thus, the language used by the conversation person can be estimated more reliably, and the corresponding translation can be presented to the conversation person.

  Furthermore, by outputting a predetermined number of parallel translations having a relatively high compatibility index (translation accuracy or accuracy) among parallel translations in a plurality of different languages, for example, a dialogue person uses or understands a plurality of languages. In such a case, even if the user's utterance content cannot be understood in one language, the possibility that the user's utterance content can be understood in another language is increased. More specifically, for example, for a conversation person who can speak Chinese and understand English, depending on the content of the user's utterance, the compatibility index of the English translation may be higher than the compatibility index of the Chinese translation. If it is high, the interlocutor may not understand the user's utterance with the Chinese translation, but may be able to understand it with the English translation (that is, the case where it can be understood in English but not in Chinese) .

  Note that, as described above, each of the above embodiments is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art can replace the resources (hardware resources or software resources) described in the embodiments with equivalents, and such replacements are also included in the scope of the present invention.

  Further, in step SJ2, instead of displaying a list of the parallel translation texts T3 and T4 in a plurality of languages, only the parallel translation audio output in a plurality of languages may be used. In this case, those translations may be read out sequentially. Furthermore, when the dialoguer does not ask for a simple answer but asks for the utterance by the dialoguer, the answers shown in FIGS. 4 (C) and (D) and FIGS. 5 (A) and (B) are tapped. Instead of the buttons B11 to B15 and B21 to B25, a speech button may be displayed in the vicinity of each language parallel translation. Specifically, for example, in the case of English, a tapping button (icon) displaying a text such as “Please talk.” Indicating “Please speak” is displayed below the English translation. For example, in the case of German, a tap button (icon) displaying a text such as “Bitte sprench.” Indicating “Please speak” is displayed below the translation in German.

  Moreover, although the example which performs each process, such as speech recognition, translation, speech synthesis, by server 20, was described, you may comprise so that these processes may be performed in the information terminal 10. FIG. In this case, the module L20 used for these processes may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. Furthermore, the database D20 that is a voice database and / or a model M20 such as an acoustic model may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. . As described above, the speech translation apparatus may not include the network N and the server 20.

  Of course, a gateway server for converting a communication protocol between the information terminal 10 and the network N may be interposed. The information terminal 10 is not limited to a portable device, and may be a desktop personal computer, a notebook personal computer, a tablet personal computer, a laptop personal computer, or the like.

  According to the present invention, since the conversation between the user and the conversation person can be started naturally and smoothly without performing language selection at the start of the conversation, for example, the conversation between persons who cannot understand each other's language. The present invention can be widely used for activities such as design, manufacture, provision, and sales of programs, apparatuses, systems, and methods in the field of service provision.

DESCRIPTION OF SYMBOLS 10 Information terminal 11 Processor 12 Storage resource 13 Voice input / output device 14 Communication interface 15 Input device 16 Display device 17 Camera 20 Server 21 Processor 22 Communication interface 23 Storage resource 41 Input switch button 42a, 42b Input button 43 Voice button 44 Language Selection button 45 History button 46 Suggest button 47 Setting button 48 Microphone design 49 Multiple circular design 51 Check box 100 Speech translation device B1 Cancel button B11-B15, B21-B25 Answer button D20 Database L20 Module M20 Model N Network P10, P20 Program T1, T2 Text T3, T4 Parallel text

Claims (7)

An input unit for inputting voices of a user and a dialogue person;
A translation unit for acquiring parallel translations in a plurality of different languages for one input voice by the user;
An output unit that outputs the parallel translations in the plurality of different languages in text or text and voice;
A language selection unit that selects a language used by the dialog after the bilingual texts in the plurality of different languages are output;
With
When the input speech is a question item, the output unit displays the text of the translation of the question item in the plurality of different languages at a time on the screen, and the text of each of the parallel translations Display a button indicating the answer to the question in the language of the translation,
The language selection unit selects a language corresponding to an answer of the tapped button as a language used by the conversation person when any of the buttons is tapped.
Speech translation device. The output unit displays the text of the parallel translation of the question items in the plurality of different languages and the answer to the questionnaire in the respective parallel languages without displaying the language type itself on the screen. Display a button on the screen,
The speech translation apparatus according to claim 1. A storage unit for storing or acquiring the number of selections or the selection frequency of each of the plurality of different languages;
The output unit outputs a parallel translation in a predetermined number of languages having a relatively high number of selections or the selection frequency among the plurality of different languages;
The speech translation apparatus according to claim 1 or 2. The output unit outputs the translations in the predetermined number of languages in the order of the selection frequency or the selection frequency.
The speech translation apparatus according to claim 3. An index acquisition unit for acquiring a compatibility index between the content of the one input voice and each of the parallel translations in the plurality of different languages;
The output unit outputs a predetermined number of parallel translations having a relatively high compatibility index among the translations in the plurality of different languages.
The speech translation apparatus according to claim 1. Using a speech translation device comprising an input unit, a translation unit, an output unit, and a language selection unit,
The input unit inputting voices of a user and a dialogue person;
The translation unit acquiring parallel translations in a plurality of different languages for one input voice by the user;
The output unit outputting the bilingual translation in the plurality of different languages as text or text and voice;
The language selection unit, after the bilingual texts in the plurality of different languages are output, selecting the language used by the dialogue person;
Including
When the input speech is a question item, the output unit displays the text of the translation of the question item in the plurality of different languages at a time on the screen, and the text of each of the parallel translations Display a button indicating the answer to the question in the language of the translation,
The language selection unit selects a language corresponding to an answer of the tapped button as a language used by the conversation person when any of the buttons is tapped.
Speech translation method. Computer
An input unit for inputting voices of a user and a dialogue person;
A translation unit for acquiring parallel translations in a plurality of different languages for one input voice by the user;
An output unit that outputs the parallel translations in the plurality of different languages in text or text and voice;
A language selection unit that selects a language used by the dialog after the bilingual texts in the plurality of different languages are output;
To function,
When the input speech is a question item, the output unit displays the text of the translation of the question item in the plurality of different languages at a time on the screen, and the text of each of the parallel translations Display a button indicating the answer to the question in the language of the translation,
The language selection unit selects a language corresponding to an answer of the tapped button as a language used by the conversation person when any of the buttons is tapped.
Speech translation program.