JP6174746B1 - Speech translation device, speech translation method, and speech translation program - Google Patents

Abstract

The present invention aims to optimize customer service by naturally and smoothly proceeding with customer service conversations with foreigners. A speech translation apparatus according to an aspect of the present invention includes an input unit for inputting speech of a user or the like, a translation unit for translating the content of the input speech into content of a different language, the translation content of the input speech as speech, etc. An output unit that outputs in the above, a high-level phrase group including at least one high-level phrase, and a storage unit that hierarchically stores a plurality of low-level phrase groups including at least one low-level phrase associated with each high-level phrase, and a high-level A display unit is provided that displays a group of phrases and sequentially executes a process of displaying a group of lower phrases including a lower phrase associated therewith when a specific phrase is selected from the upper phrases. [Selection] Figure 3

Description

  The present invention relates to a speech translation device, a speech translation method, and a speech translation program.

  In order to enable conversations between people who do not understand each other's language, for example, conversations between store clerk and foreign customers, the speaker's speech is converted into text and the text content is machine-translated into the partner's language. A speech translation technique has been proposed in which the text is displayed on the screen or the text content is played back using a speech synthesis technique (for example, Patent Document 1). A speech translation application that operates on an information terminal such as a smartphone that embodies such speech translation technology has also been put into practical use (for example, Non-Patent Document 1). Furthermore, there is also known a translation application in which a conversation pattern for each purpose is displayed in a list by selecting a situation in which the user wants to have a conversation (for example, Non-Patent Document 2).

Japanese Patent Laid-Open No. 9-34895

U-STAR Consortium homepage [Search on February 23, 2016], Internet <URL: http://www.ustar-consortium.com/app_en/app.html> Conversation simulation is possible! Japanese-Chinese-Korean-compatible translation application “[Travel App No. 1] TS Conversation Translator [CJK]” [searched February 23, 2016], Internet <URL: http://andronavi.com/2012/08/ 208376>

  By the way, in the translation application described in Non-Patent Document 2, for example, both a plurality of question sentences and a plurality of answer sentences for each question sentence are listed and displayed as a conversation pattern for each purpose. The speaker selects a desired one of those example sentences. However, in that case, since only one example sentence is selected and uttered, the conversation becomes one-shot. In addition, when example sentences are listed on one screen as described above, it is necessary to scroll the screen and search each time in order to select a desired one from the example sentences. As a result, for example, at the time of customer service, a series of conversations cannot be performed naturally and smoothly, resulting in inconvenience that it is difficult to perform appropriate customer service.

  Therefore, the present invention has been made in view of such circumstances, and can naturally and smoothly advance conversation between a user and a talker (foreign customer) during customer service, thereby optimizing customer service. It is an object to provide a speech translation device, a speech translation method, and a speech translation program that can contribute.

  In order to solve the above problems, a speech translation apparatus according to an aspect of the present invention includes an input unit for inputting a voice of a user and / or a dialoguer, a translation unit for translating the content of the input speech into content of a different language, and And an output unit that outputs the translation content (translation) of the input speech as speech and / or text. And the said speech translation apparatus WHEREIN: The memory | storage part which memorize | stores hierarchically the high-order phrase group containing at least 1 high-order phrase, and the several low-order phrase group containing the at least 1 low-order phrase linked | related with each high-order phrase, A display unit that displays the upper phrase group, and when the specific phrase is selected from the upper phrase, the process of displaying the lower phrase group associated with the specific phrase in a hierarchical manner. Prepare. The “phrase” includes sentences, clauses, phrases, words, and numbers, and may include images or symbols accompanying them. In other words, the speech translation apparatus according to an aspect of the present invention prepares a so-called tree diagram of the plurality of phrase groups, and sequentially (repetitively) executes hierarchical display and phrase selection thereof. Provide the flow processing.

  More specifically, the display unit may be configured to display the upper phrase group and the lower phrase group as separate screens.

  In addition, the display unit includes a specific question as an upper phrase, and when the specific question is selected, the display unit displays a screen for inputting an answer to the specific question. Good.

  Further, the upper phrase and the lower phrase may be automatically or manually set in advance for each type of business to which the user belongs or for each store of the user.

  Further, the storage unit stores the number of selections of each upper phrase and each lower phrase, and the display unit displays the higher phrase having a higher number of selections in a higher order on the display screen of the upper phrase group, and the number of selections. The lower phrase having more may be displayed in a higher order on the display screen of the lower phrase group.

  Furthermore, if the specific phrase is an order inquiry and the sub-phrase group including the sub-phrase associated with the specific phrase is a list of a plurality of order items, the storage unit selects the number of times each order item is selected, Alternatively, the profit ratio of each ordered item is stored, and the display unit can display an ordered item having a higher number of selections or an ordered item having a higher profit rate in a higher rank on the display screen of the lower phrase group. .

  Furthermore, the display unit may display a translation of each upper phrase and each lower phrase in a different language, or the output unit may output a translation of each upper phrase and each lower phrase in a different language by voice. Good. The “translation” may be stored in advance in the storage unit, or may be translated by the translation unit each time an upper phrase or a lower phrase is selected.

  A speech translation method according to an aspect of the present invention is a method using a speech translation apparatus including an input unit, a translation unit, an output unit, a storage unit, and a display unit. That is, in this method, the step of inputting the voice of the user and / or the conversation person, the step of the translation unit translating the content of the input voice into the content of a different language, and the output unit And / or outputting in text, and the storage unit hierarchically includes a plurality of lower phrase groups including at least one upper phrase including at least one upper phrase and at least one lower phrase associated with each upper phrase. The storing step and the display unit displays the upper phrase group, and when a specific phrase is selected from the upper phrases, the lower phrase group including the lower phrase associated with the specific phrase is displayed. Are sequentially executed in a hierarchical manner.

  In addition, the speech translation program according to one aspect of the present invention is a computer (not limited to a single type or a single type, but may be a plurality or a plurality of types; the same shall apply hereinafter), and an input unit for inputting a voice of a user and / or a conversation person A translation unit that translates the content of the input speech into content of a different language, an output unit that outputs the translation content of the input speech in speech and / or text, a group of upper phrases including at least one upper phrase, and each A storage unit that hierarchically stores a plurality of lower phrase groups including at least one lower phrase associated with the upper phrase, and the upper phrase group. When a specific phrase is selected from the upper phrases, The process of displaying the lower phrase group including the lower phrase associated with the specific phrase is made to function as a display unit that sequentially executes hierarchically. .

  According to the present invention, when a specific phrase is selected from the upper phrases included in the upper phrase group in the conversation between the user and the interlocutor, the lower phrase group including the lower phrase stored in association therewith is displayed. A series of processing is executed sequentially (repeatedly) hierarchically. Thereby, in a predetermined assumed situation, it is possible to continue the conversation without delay without considering the contents of the question and the answer each time an utterance is made. Therefore, the conversation between the user and the interlocutor can be proceeded naturally and smoothly, which makes it possible to optimize customer service.

1 is a system block diagram schematically showing a preferred embodiment of a network configuration and the like related to a speech translation apparatus according to the present invention. It is a flowchart which shows an example of the flow (part) of the process in suitable one Embodiment of the speech translation apparatus by this invention. It is a flowchart which shows an example of the flow (part) of the process in suitable one Embodiment of the speech translation apparatus by this invention. (A) thru | or (D) are top views which show an example of the transition of the display screen in an information terminal. (A) thru | or (D) are top views which show an example of the transition of the display screen in an information terminal. It is a schematic diagram which shows an example of the data structure of a phrase database.

  Hereinafter, embodiments of the present invention will be described in detail. The following embodiments are examples for explaining the present invention, and are not intended to limit the present invention only to the embodiments. The present invention can be variously modified without departing from the gist thereof. Furthermore, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent ones, and such embodiments are also included in the scope of the present invention. Furthermore, positional relationships such as up, down, left, and right shown as needed are based on the display shown unless otherwise specified. Furthermore, various dimensional ratios in the drawings are not limited to the illustrated ratios.

(Device configuration)
FIG. 1 is a system block diagram schematically showing a preferred embodiment such as a network configuration related to a speech translation apparatus according to the present invention. In this example, the speech translation apparatus 100 includes a server 20 that is electronically connected to the information terminal 10 (user apparatus) used by the user via the network N (but is not limited to this).

  The information terminal 10 employs a user interface such as a touch panel and a display with high visibility, for example. The information terminal 10 here is a portable tablet terminal device including a mobile phone represented by a smartphone having a communication function with the network N. The information terminal 10 further includes a processor 11, a storage resource 12, a voice input / output device 13, a communication interface 14, an input device 15, a display device 16, and a camera 17. In addition, the information terminal 10 operates by the installed speech translation application software (at least a part of the speech translation program according to the embodiment of the present invention), so that a part of the speech translation apparatus according to the embodiment of the present invention or It functions as a whole.

  The processor 11 includes an arithmetic logic unit and various registers (program counter, data register, instruction register, general-purpose register, etc.). Further, the processor 11 interprets and executes speech translation application software, which is the program P10 stored in the storage resource 12, and performs various processes. The speech translation application software as the program P10 can be distributed from the server 20 through the network N, for example, and may be installed and updated manually or automatically.

  The network N includes, for example, a wired network (a short-range communication network (LAN), a wide-area communication network (WAN), a value-added communication network (VAN), etc.) and a wireless network (mobile communication network, satellite communication network, Bluetooth ( Bluetooth (registered trademark)), WiFi (Wireless Fidelity), HSDPA (High Speed Downlink Packet Access), etc.).

  The storage resource 12 is a logical device provided by a storage area of a physical device (for example, a computer-readable recording medium such as a semiconductor memory), and an operating system program, a driver program, various data, etc. used for processing of the information terminal 10 Is stored. Examples of the driver program include an input / output device driver program for controlling the audio input / output device 13, an input device driver program for controlling the input device 15, and a display device driver program for controlling the display device 16. Can be mentioned. Furthermore, the voice input / output device 13 is, for example, a general microphone and a sound player capable of reproducing sound data.

  The communication interface 14 provides a connection interface with the server 20, for example, and is configured from a wireless communication interface and / or a wired communication interface. The input device 15 provides an interface for accepting an input operation by a tap operation such as an icon, a button, a virtual keyboard, or a text displayed on the display device 16. Various input devices to be attached can be exemplified.

  The display device 16 provides various information as an image display interface to a user or a conversation person (conversation partner), and examples thereof include an organic EL display, a liquid crystal display, and a CRT display. The camera 17 is for capturing still images and moving images of various subjects.

  The server 20 is constituted by, for example, a host computer having a high arithmetic processing capability, and expresses a server function by operating a predetermined server program in the host computer, for example, a speech recognition server, a translation server, And a single or a plurality of host computers functioning as a speech synthesis server (in the drawing, it is indicated by a single, but is not limited thereto). Each server 20 includes a processor 21, a communication interface 22, and a storage resource 23.

  The processor 21 is composed of an arithmetic and logic unit for processing arithmetic operations, logical operations, bit operations and the like and various registers (program counter, data register, instruction register, general-purpose register, etc.), and is stored in the storage resource 23. P20 is interpreted and executed, and a predetermined calculation processing result is output. The communication interface 22 is a hardware module for connecting to the information terminal 10 via the network N. For example, the communication interface 22 is a modulation / demodulation device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

  The storage resource 23 is a logical device provided by, for example, a storage area of a physical device (a computer-readable recording medium such as a disk drive or a semiconductor memory), and each includes one or a plurality of programs P20, various modules L20, and various databases. D20 and various models M20 are stored. The storage resource 23 also stores a plurality of standard questions prepared for the user to speak to the talker, input voice history data, data for various settings, phrase data to be described later, and the like.

  The program P20 is the above-described server program that is the main program of the server 20. In addition, the various modules L20 perform a series of information processing related to requests and information transmitted from the information terminal 10, so that they are appropriately called and executed during the operation of the program P10 (moduleized subprograms). ). Examples of the module L20 include a speech recognition module, a translation module, and a speech synthesis module.

  The various databases D20 include various corpora required for speech translation processing (for example, in the case of Japanese and English speech translation, a Japanese speech corpus, an English speech corpus, a Japanese character (vocabulary) corpus, an English character) (Vocabulary) Corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.), voice database, management database for managing information about users, phrase database having a hierarchical structure described later, etc. It is done. Examples of the various models M20 include acoustic models and language models used for speech recognition.

(processing)
An example of operations and operations of speech translation processing in the speech translation apparatus 100 configured as described above will be further described below. 2 and 3 are flowcharts showing an example of (a part of) the processing flow in the speech translation apparatus 100. FIG. FIGS. 4A to 4D and FIGS. 5A to 5D are plan views illustrating an example of (a part of) transitions in the screen display of the information terminal 10. Here, it is assumed that the user of the information terminal 10 is a clerk of a restaurant that speaks Japanese, and the conversation person (conversation partner) is a foreign customer who speaks English (however, the language or Situations are not limited to this).

  First, when the user (clerk) activates the application (step SU1), the language selection screen for the conversation person shown in FIG. 4A is displayed on the display device 16 of the information terminal 10 (step SJ1). In this language selection screen, the Japanese text T1 for prompting the user to ask the conversation person about the language, the English text T2 for asking the conversation person about the language, and a plurality of typical languages assumed. Here, a language button 41 indicating English, Chinese (for example, two types depending on the typeface), and Korean is displayed. Further below that, a cancel button B1 for closing the language selection screen and ending the application is also displayed.

  At this time, as shown in FIG. 4A, the Japanese text T1 and the English text T2 are divided by the processor 11 and the display device 16 into different areas on the screen of the display device 16 of the information terminal 10, and Are displayed in opposite directions (different directions; upside down in the figure). Thereby, when a conversation is performed in a state where the user and the interlocutor face each other, the user can easily confirm the Japanese text T1, while the interrogator can easily confirm the English text T2. In addition, since the Japanese text T1 and the English text T2 are displayed separately, there is an advantage that both are clearly distinguished and can be visually recognized more easily.

  The user presents the display of the English text T2 on the language selection screen to the conversation person, and the conversation person taps the English button, for example, or the user himself selects the conversation person's language. can do. When the language of the conversation person is selected in this way, the standby screen for voice input in Japanese and English is displayed on the display device 16 as the home screen by the processor 21 of the server 20 and the processor 11 of the information terminal 10 (FIG. 4). (B); Step SJ2). On this standby screen, Japanese text T3 asking which of the user's language and the speaker's language is to be spoken, an input button 42a for performing Japanese speech input, and an input for performing English speech input Button 42b is displayed.

  Further, on this standby screen, a voice button 43 for selecting a list display of a plurality of preset question phrases, and the language selection screen of FIG. Language selection button 44 for redoing language selection, history button 45 for selecting history display of voice input contents made so far, and selecting a desired phrase from a plurality of recommended phrase groups prepared in advance Then, a suggest button 46 for executing a suggest function capable of proceeding with the conversation and a setting button 47 for performing various settings of the application software are also displayed.

[Conversation by normal speech translation]
Here, with reference to FIG. 2, an example of a normal speech translation process in a conversation between a user and a conversation person and / or a conversation preparation will be described. First, on the standby screen shown in FIG. 4B, when the user taps the Japanese input button 42a and selects Japanese voice input, the voice input is enabled. In this state, when the user utters a matter to be communicated to the conversation person (step SU2), voice input is performed through the voice input / output device 13 (step SJ3). The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N. As described above, the information terminal 10 itself, or the processor 11 and the voice input / output device 13 function as an “input unit”.

  The processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing (step SJ4). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains “sound” of the input speech. Convert to "reading" (character). As described above, the processor 21 or the server 20 functions as a “voice recognition server” as a whole.

  Next, the processor 21 proceeds to multilingual translation processing for translating the recognized “reading” (characters) of the recognized speech into another language (step SJ5). At this time, the processor 21 calls the necessary module L20 and database D20 (translation module, Japanese character corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.) from the storage resource 23 and recognizes them. The resulting input speech “reading” (character string) is properly sorted and converted into Japanese phrases, clauses, sentences, etc., the English corresponding to the conversion result is extracted, and these are sorted according to the English grammar. To natural English phrases, clauses, sentences, etc. As described above, the processor 21 also functions as a “translation unit”, and the server 20 also functions as a “translation server” as a whole. If the input voice is not correctly recognized, the voice can be re-input (screen display is not shown).

  Further, the processor 21 stores the content of the recognized input voice in the storage resource 23. Next, when the multilingual translation processing and the input speech content storage processing are completed, the processor 21 proceeds to speech synthesis processing (step SJ6). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, English speech corpus, acoustic model, language model, etc.) from the storage resource 23, and the English phrase that is the translation result, Convert clauses, sentences, etc. to natural speech. As described above, the processor 21 also functions as a “speech synthesizer”, and the server 20 also functions as a “speech synthesizer” as a whole.

  Next, the processor 21 generates a voice signal for voice output based on the synthesized voice, and transmits the voice signal to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the audio signal through the communication interface 14, and performs an audio output process using the audio input / output device 13 (step SJ7). As described above, the processor 11 and the voice input / output device 13 function as an “output unit”.

[Conversation using the suggest function]
Next, with reference to FIG. 3, an example of processing in the case where the suggest function is used in the conversation between the user and the conversation person and / or conversation preparation will be described. For example, when a dialog person (foreign customer) enters a user's store (for example, a store providing a food service), the user taps the suggest button 46 on the standby screen shown in FIG. SU3). Then, the processor 11 of the information terminal 10 transmits to the server 20 a command signal for displaying a phrase group frequently used in customer service at a store of the industry to which the user belongs. The processor 21 of the server 20 that has received the command signal accesses the phrase database D60 included in the database D20 stored in the storage resource 23.

  Here, FIG. 6 is a schematic diagram showing an example of the data structure of the phrase database D60. As shown in the figure, the phrase database D60 includes a plurality of hierarchized phrase groups (data) 61, 62, 63 (for convenience of illustration, up to three hierarchies are described, but not limited thereto; the same applies hereinafter). ing. Each of these phrase groups 61, 62, and 63 includes a plurality of Japanese phrases X11 to X55, Y11 to Y55, Z11 to Z55, and their translations in different languages, and the phrases included in the higher-level phrase group Are associated with a plurality of phrases and lower-level phrase groups including them. In this way, a so-called tree diagram of phrases is constructed from the phrase groups 61, 62, 63, etc., thereby providing a continuous flow of phrases. Moreover, when a phrase is a specific question item, the question item flag is attached | subjected to the said phrase. Then, screen data for inputting an answer to the question is stored in the phrase database D60 or other appropriate database as a part of the phrase group in association with the phrase. As described above, the storage resource 23 functions as a “storage unit”.

  More specifically, the phrase group 61 and the phrase group 62 have a relationship of “upper phrase group” and “lower phrase group”, respectively. For example, the phrase X11 (upper phrase) included in the phrase group 61 includes: The phrases Y11 to Y55 (subordinate phrases) and the phrase group 62 including them are associated with each other. Similarly, the phrase group 62 and the phrase group 63 also have a relationship of “upper phrase group” and “lower phrase group”, respectively. For example, a phrase Y11 (upper phrase) included in the phrase group 62 includes phrases Z11 to Z11. Z55 (subordinate phrase) and a phrase group 63 including them are associated with each other. Further, the question item flag F is attached to the phrase X11 which is a specific question item, and screen data for inputting an answer to the specific question item is part of the phrase group 62 in association with the phrase X11. Is remembered as

  The processor 21 of the server 20 that has accessed the phrase database D60 first calls the phrases X11 to X55 included in the phrase group 61 of the upper hierarchy, creates display image data of those lists, and sends it to the processor 11 of the information terminal 10. Send. The processor 11 displays, for example, the phrase group screen shown in FIG. 4C on the display device 16 based on the display image data (step SJ8 so far).

  In the phrase group screen of FIG. 4C, a plurality of Japanese phrase texts (upper phrases) and English phrase texts indicating their translations in English are written together as phrases list P1 in a state where each phrase is written together. Is displayed. As shown in FIG. 4C, the phrase list P1 includes a plurality of phrases that are often spoken when a restaurant clerk speaks to a customer who first visits the store. The user can select a desired specific phrase by tapping those text portions. In the phrase group screen, above and below the phrase list P1, the language selected as the language of the conversation person (ie, the parallel language) on the language selection screen in FIG. 4A is English. The Japanese text T4 shown and a close button B2 for closing the phrase group screen and returning to the standby screen of FIG. 4B are also displayed (the same applies hereinafter).

  Here, the text T5 (for example, the phrase X11 in the phrase group 61) of a phrase (“how many people?”: A specific phrase and a specific question) that the user inquires about the number of people from the phrase list P1, for example. If it is selected by tapping (corresponding) (step SU4), the processor 11 of the information terminal 10 transmits the selection command signal to the processor 21 of the server 20. The processor 21 that has received it returns voice output data of the translation of the text T5 in English to the processor 11, and the processor 11 outputs the voice from the voice input / output device 13.

  Further, the processor 21 accesses the phrase database D60 again, and determines whether or not the phrase X11 corresponding to the selected text T5 is a specific question (step SJ9). Here, since the question item flag F is attached to the phrase X11 (“Yes” in step SJ9), the processor 21 stores the answer input screen data stored as a part of the phrase group 62 associated with the phrase X11. Is returned to the processor 11. The processor 11 that has received the message displays, for example, an answer input screen shown in FIG. 4D on the display device 16 as a screen for the interlocutor to input the number of visitors (step SJ10 so far). In the answer input screen of FIG. 4D, a numeric key 48 for inputting the number of persons is displayed. The user presents the answer input screen to the dialog person, and the dialog person taps the screen to display the number of visitors ( Here, for example, when two persons are entered (step SU5), the number is displayed in the column 49.

  Thus, when an answer (number of people) to the phrase X11 (text T5), which is a specific question, is input, the processor 11 of the information terminal 10 transmits an input completion signal to the processor 21 of the server 20. The processor 21 having received it accesses the phrase database D60 again, calls the phrases Y11 to Y55 included in the phrase group 62 in the lower hierarchy associated with the phrase X11, creates display image data of those lists, and creates information. It transmits to the processor 11 of the terminal 10. Based on the display image data, the processor 11 displays, for example, the phrase group screen shown in FIG. 5A on the display device 16 (step SJ11 so far). As described above, the processors 11 and 21 and the display device 16 function as a “display unit”.

  On the phrase group screen of FIG. 5A, a phrase list P2 including a plurality of phrase texts (lower phrases) is displayed in the same form as the phrase list P1 shown in FIG. As shown in FIG. 5A, the phrase list P2 includes phrases that are often uttered after the number of customers who have visited the restaurant is confirmed. Here, similarly to the phrase list P1, the user can select a desired specific phrase by tapping the text portion in the phrase list P2. Hereinafter, a series of processes such as display of the phrase list of the upper phrase group, selection of a specific phrase from them, and display of the lower phrase group including the lower phrase associated with the specific phrase are hierarchically performed. Therefore, the conversation between the user and the conversation person can be advanced by executing sequentially (repetitively). As described above, the phrase list P2 displayed as the lower phrase group of the phrase list P1, which is the upper phrase group, corresponds to the upper phrase group for the further lower phrase group to be displayed next.

  Next, the text T6 (corresponding to, for example, the phrase Y11 in the phrase group 62) of the phrase ("Guide to the seat.": A specific phrase) that the user guides, for example, to a vacant seat from the phrase list P2. Is selected by tapping (step SU4), the processor 11 of the information terminal 10 transmits the selection command signal to the processor 21 of the server 20. The processor 21 that has received it returns voice output data of the translation of the text T6 in English to the processor 11, and the processor 11 outputs the voice from the voice input / output device 13.

  Further, the processor 21 accesses the phrase database D60 again, and determines whether or not the phrase Y11 corresponding to the selected text T6 is a specific question (step SJ9). Here, since the question flag F is not attached to the phrase Y11 (“No” in step SJ9), the processor 21 includes the phrases Z11 to Z55 included in the phrase group 63 of a lower hierarchy associated with the phrase Y11. And display image data of those lists is created and transmitted to the processor 11 of the information terminal 10. Based on the display image data, the processor 11 displays, for example, the phrase group screen shown in FIG. 5B on the display device 16 (step SJ11 so far).

  On the phrase group screen of FIG. 5B, a phrase list P3 including a plurality of phrase texts (lower phrases) is displayed in the same form as the phrase list P1 shown in FIG. As shown in FIG. 5B, the phrase list P3 includes phrases that are often spoken at the timing when the customer is seated.

  Next, the user makes a text T7 (corresponding to, for example, the phrase Z11 in the phrase group 63) of a phrase ("Would you like a drink?": A specific phrase) to inquire about an order for drinks from the phrase list P3 Is selected by tapping (step SU4), the processor 11 of the information terminal 10 transmits the selection command signal to the processor 21 of the server 20. The processor 21 that has received it returns voice output data of the translation of the text T7 in English to the processor 11, and the processor 11 outputs the voice from the voice input / output device 13.

  Furthermore, the processor 21 accesses the phrase database D60 again, and determines whether or not the phrase Z11 corresponding to the selected text T7 is a specific question item (step SJ9). Since the question flag F is not attached to this phrase Z11 (“No” in step SJ9), the processor 21 calls the phrases included in the lower-level phrase group associated with the phrase Z11 and lists them. Display image data is generated and transmitted to the processor 11 of the information terminal 10. Based on the display image data, the processor 11 displays, for example, the phrase group screen shown in FIG. 5C on the display device 16 (step SJ11 so far).

  On the phrase group screen of FIG. 5C, a phrase list P4 including a plurality of phrase texts (lower phrases) is displayed in the same manner as the phrase list P1 shown in FIG. As shown in FIG. 5C, the phrase list P4 includes menu names of a plurality of drinks, and also includes text T8 indicating that the interrogator requests a menu from the user. The user can place an order by presenting this phrase group screen to the interlocutor and tapping on the desired drink menu name. Alternatively, when the menu name of the drink desired by the dialog person is not in the phrase list P4, the dialog person can request the user who is a store clerk to browse the menu by tapping the text T8 portion (step SU4).

  In this way, the dialog person taps and selects the phrase representing the desired menu name from the phrase list P4 or the text of the phrase requesting to browse the menu (both are specific phrases) (step SU4). ) And the processor 11 of the information terminal 10 transmits the selection command signal to the processor 21 of the server 20. The processor 21 that has received it returns the Japanese voice output data of the text to the processor 11, and the processor 11 outputs the voice from the voice input / output device 13.

  Further, the processor 21 accesses the phrase database D60 again, and determines whether or not the phrase corresponding to the selected text is a specific question (step SJ9). Here, since the question item flag F is not attached to the phrase included in the phrase list P4 (“No” in step SJ9), the processor 21 is included in the lower-level phrase group associated with the phrase. A plurality of phrases are called, and display image data of those lists is created and transmitted to the processor 11 of the information terminal 10. Based on the display image data, the processor 11 displays, for example, the phrase group screen shown in FIG. 5D on the display device 16 (step SJ11 so far).

  On the phrase group screen of FIG. 5D, a phrase list P5 including a plurality of phrase texts (lower phrases) is displayed in the same form as the phrase list P1 shown in FIG. As shown in FIG. 5D, the phrase list P5 includes phrases that are often spoken when orders or requests from customers are received. When the user taps and selects the text portion of the desired phrase from the phrase list P5 (step SU4), the English translation of the phrase is output in the same manner as the processing so far. The user can end the application as appropriate (step SU6). The phrase list P5 shown in FIG. 5D is effective as a lower phrase group for any phrase included in the phrase list P4 shown in FIG. Thus, a phrase group including a plurality of the same lower phrases may be associated with different upper phrases.

  According to the speech translation apparatus 100 configured as described above, the speech translation method using the speech translation device, and the speech translation program, in a conversation between a user and a conversation person, a specific phrase is selected from the upper phrases included in the upper phrase group. Is selected, a lower phrase group including lower phrases stored in association therewith is displayed on the screen, and such a series of processes are executed sequentially (repeatedly) hierarchically. For example, a desired phrase according to the hierarchical transition of the screen display shown in FIG. 4 (C) → FIG. 4 (D) → FIG. 5 (A) → FIG. 5 (B) → FIG. 5 (C) → FIG. By selecting, a conversation in a situation such as customer service at a restaurant can be advanced. As described above, according to the present invention, in a predetermined assumed situation, each time an utterance is made, it is possible to continue the conversation without considering the contents of the question or the answer, so that the conversation between the user and the conversation person can be performed. This makes it possible to optimize the customer service.

  Also, as shown in FIGS. 4C and 4D and FIGS. 5A to 5D, the upper phrase group and the lower phrase group are sequentially displayed on the display device 16 as separate screens, so that the conversation progresses. It becomes easy to select a desired phrase according to the user easily and accurately, and the conversation between the user and the interlocutor can be made more natural and smooth.

  Furthermore, a specific question (for example, the phrase X11 of the text T5 shown in FIG. 4C) is included as an upper phrase, and when it is selected, an answer to the specific question is input. For example, the screen shown in FIG. 4D is displayed. Therefore, compared with the case where only phrases are sequentially displayed, the choice of conversation or the width of conversation can be expanded, and this makes it possible to flexibly deal with conversations in more situations.

  Furthermore, as described above, a plurality of phrases (that is, upper phrases and lower phrases) set in each of the phrase groups 61, 62, 63, etc. (phrase lists P1 to P5) are set in advance for each industry to which the user belongs. Therefore, conversations specialized for customer service in the industry can be carried out more smoothly and appropriately, and customer service can be further optimized. In addition, such phrases can be set in advance for each user's store. In this case, finer customer service reflecting the features of each store and the situation of the store is possible.

  Further, the setting of these upper phrases and lower phrases may be performed automatically or manually. As an example of automatic setting, first, when using the translation application, the user's business type is registered as one of the user information, and the processor 21 of the server 20 uses a corpus of phrases frequently appearing in conversations of the business type. There is a form in which phrases that are particularly frequently used are selected from the history and hierarchized as a phrase group. Alternatively, phrases uttered by a plurality of users in the same industry are stored in an appropriate database together with the utterance frequency, and the processor 21 of the server 20 selects a phrase that is particularly frequently used from those phrases, and a phrase group It may be arranged as a hierarchy. On the other hand, as an example of setting manually, there is a form in which the user selects a desired phrase and customizes it as a hierarchical phrase group.

  At that time, phrases included in each of the phrase groups 61, 62, 63, etc. (phrase lists P1 to P5) may be maintained as originally set (fix the phrase list), or phrases included in them may be stored. These may be changed as necessary. That is, in the latter case, for example, the number of times each phrase (upper phrase and lower phrase) is selected is stored in the storage resource 23, and the processor 21 of the server 20 selects a phrase with a larger number of selections in each phrase group 61. , 62, 63, etc. (phrase lists P1 to P5) may be displayed in a higher order (for example, displayed at the top of the screen, or displayed with emphasis or enlargement). As a result, it is possible to easily display a phrase in accordance with the user's type of business and the actual situation of the store, and it is easy to select, and the communication between the user and the conversation person can be further speeded up.

  Also, as shown in FIGS. 5B and 5C, the specific phrase is an order (drink) inquiry (phrase Z11 of text T7), and a lower phrase group including lower phrases associated with the specific phrase Is a list of a plurality of order items (menu names), the following processing may be performed. That is, the number of times each order item is selected in the past predetermined period (the number of orders) or the profit rate of each order item is stored in the storage resource 23, and the processor 21 of the server 20 determines the number of selection times. More ordered items or ordered items with a higher profit rate may be displayed in a higher order on the phrase group display screen shown in FIG. 5C, for example. As a result, it is possible to actively recommend a popular menu or a menu with a high customer unit price to a conversation person (foreign customer), and as a result, it is possible to improve the sales and profit of the user's store. .

  Furthermore, as shown in FIGS. 4C and 4D and FIGS. 5A to 5D, on the phrase group screen, there is a Japanese phrase text for each phrase and an English phrase text indicating an English translation. In the phrase lists P1 to P5, translations of the phrases in different languages are output by voice. Therefore, the user and the conversation person can confirm the content of the other party's utterance more reliably in addition to viewing the screen or without viewing the screen.

  Note that, as described above, each of the above embodiments is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art can replace the resources (hardware resources or software resources) described in the embodiments with equivalents, and such replacements are also included in the scope of the present invention.

  4 (C) and 4 (D) and FIGS. 5 (A) to 5 (D), the Japanese phrase text and the English phrase text are mutually converted into the texts T1 and T2 in FIG. 4 (A). They may be displayed in opposite directions (different directions; upside down in the figure). Furthermore, these Japanese phrase text and English phrase text may not be written together, and only one of them may be displayed. Furthermore, the answer input shown in FIG. 4D may be configured to be performed by voice input. Furthermore, regardless of the user's business type, a phrase group set for other business types may be selected. In addition, on the phrase group display screen (menu name) shown in FIG. 5C, a menu name using a material with a large amount of stock at that time or a menu name uniquely recommended by the user or the user's store is displayed. You can also.

  Moreover, although the example which performs each process, such as speech recognition, translation, speech synthesis, by server 20, was described, you may comprise so that these processes may be performed in the information terminal 10. FIG. In this case, the module L20 used for these processes may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. Furthermore, the database D20 that is a voice database and / or a model M20 such as an acoustic model may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. . As described above, the speech translation apparatus may not include the network N and the server 20.

  Of course, a gateway server for converting a communication protocol between the information terminal 10 and the network N may be interposed. The information terminal 10 is not limited to a portable device, and may be a desktop personal computer, a notebook personal computer, a tablet personal computer, a laptop personal computer, or the like.

  According to the present invention, a conversation between a user and a talker (foreign customer) at the time of customer service can be promoted naturally and smoothly, thereby contributing to optimization of customer service. It can be widely used for activities such as design, manufacture, provision, and sales of programs, devices, systems, and methods in the field of providing services related to conversations between people who cannot understand.

10 Information terminal 11 Processor 12 Storage resource 13 Voice input / output device 14 Communication interface 15 Input device 16 Display device 17 Camera 20 Server 21 Processor 22 Communication interface 23 Storage resource 41 Language button 42a Japanese input button 42b English input button 43 Voice button 44 Language selection button 45 History button 46 Suggest button 47 Setting button 48 Number keys 49 Columns 61, 62, 63 Phrase group 100 Spoken translation device B1 Cancel button B2 Close button D20 Database D60 Phrase database F Question item flag L20 Module M20 Model N Network P1-P5 Phrase list P10 Program P20 Program T1-T8 Text X11-X55 Phrases Y11-Y5 5 phrases Z11-Z55 phrases

Claims (6)

An input unit for inputting a voice of a user and / or a conversation person, a translation unit for translating the contents of the input voice into contents of different languages, and an output unit for outputting the translation contents of the input voice as voice and / or text A speech translation device comprising:
A storage unit that hierarchically stores an upper phrase group including at least one upper phrase, and a plurality of lower phrase groups including at least one lower phrase associated with each upper phrase;
Display the upper phrase group, and when the specific phrase is selected from the upper phrases, the process of displaying the lower phrase group including the lower phrase associated with the specific phrase is sequentially executed hierarchically A display unit to
Further comprising
The upper phrase group includes a specific question as the upper phrase,
Wherein the display unit, when the particular questionnaire is selected, a different display form and the display of the lower phrase, displays a screen for inputting any answers to the specific questions,
The upper phrase and the lower phrase are preset automatically or manually for each type of business to which the user belongs or for each store of the user,
The storage unit stores the number of times of selection of each upper phrase and each lower phrase,
The display unit displays the higher order phrase having a higher number of selections in a higher order on the display screen of the first phrase group, and the lower order phrase having a higher selection number is displayed on the display screen of the lower order phrase group. In a higher ranking,
If the specific phrase is an order inquiry and the sub-phrase group including the sub-phrase associated with the specific phrase is a list of a plurality of orders,
The storage unit stores the number of times each order item is selected, or the profit rate of each order item,
The display unit displays an ordered item with a larger number of selections or an ordered item with a higher profit margin in a higher order on the display screen of the lower phrase group.
Speech translation device. The display unit displays the upper phrase group and the lower phrase group as separate screens,
The speech translation apparatus according to claim 1. The storage unit stores a high-level phrase that is the specific question item with a question flag, and stores screen data for inputting the arbitrary answer to the specific question item. Remember it associated with the high-level phrase that is the question,
The display unit determines whether or not the specific question is selected based on the presence or absence of the question flag, and determines that the specific question is selected when the specific question is selected. Based on the screen data for inputting an arbitrary answer, a screen for inputting the arbitrary answer is displayed.
The speech translation apparatus according to claim 1 or 2. The display unit displays a translation of each upper phrase and each lower phrase in a different language, or the output unit outputs a translation of each upper phrase and each lower phrase in a different language by voice;
The speech translation apparatus according to any one of claims 1 to 3 . Using a speech translation device including an input unit, a translation unit, an output unit, a storage unit, and a display unit,
The input unit inputs a voice of a user and / or a conversation person;
The translation unit translating the content of the input speech into content of a different language;
The output unit outputting the content of translation of the input speech as speech and / or text;
The storage unit hierarchically stores a plurality of lower phrase groups including an upper phrase group including at least one upper phrase and at least one lower phrase associated with each upper phrase;
The display unit displays the upper phrase group, and when a specific phrase is selected from the upper phrases, a process of displaying a lower phrase group including a lower phrase associated with the specific phrase, Steps to be executed sequentially in a hierarchy;
Including
The upper phrase group includes a specific question as the upper phrase,
Wherein the display unit, when the particular questionnaire is selected, a different display form and the display of the lower phrase, displays a screen for inputting any answers to the specific questions,
The upper phrase and the lower phrase are preset automatically or manually for each type of business to which the user belongs or for each store of the user,
The storage unit stores the number of times of selection of each upper phrase and each lower phrase,
The display unit displays the higher order phrase having a higher number of selections in a higher order on the display screen of the first phrase group, and the lower order phrase having a higher selection number is displayed on the display screen of the lower order phrase group. In a higher ranking,
If the specific phrase is an order inquiry and the sub-phrase group including the sub-phrase associated with the specific phrase is a list of a plurality of orders,
The storage unit stores the number of times each order item is selected, or the profit rate of each order item,
The display unit displays an ordered item with a larger number of selections or an ordered item with a higher profit margin in a higher order on the display screen of the lower phrase group.
Speech translation method. Computer
An input unit for inputting a voice of a user and / or a dialogue person;
A translation unit that translates the content of the input speech into content of a different language;
An output unit that outputs the translated content of the input speech as speech and / or text;
A storage unit that hierarchically stores an upper phrase group including at least one upper phrase, and a plurality of lower phrase groups including at least one lower phrase associated with each upper phrase;
Display the upper phrase group, and when the specific phrase is selected from the upper phrases, the process of displaying the lower phrase group including the lower phrase associated with the specific phrase is sequentially executed hierarchically A display unit to
To function,
The upper phrase group includes a specific question as the upper phrase,
Wherein the display unit, when the particular questionnaire is selected, a different display form and the display of the lower phrase, displays a screen for inputting any answers to the specific questions,
The upper phrase and the lower phrase are preset automatically or manually for each type of business to which the user belongs or for each store of the user,
The storage unit stores the number of times of selection of each upper phrase and each lower phrase,
The display unit displays the higher order phrase having a higher number of selections in a higher order on the display screen of the first phrase group, and the lower order phrase having a higher selection number is displayed on the display screen of the lower order phrase group. In a higher ranking,
If the specific phrase is an order inquiry and the sub-phrase group including the sub-phrase associated with the specific phrase is a list of a plurality of orders,
The storage unit stores the number of times each order item is selected, or the profit rate of each order item,
The display unit displays an ordered item with a larger number of selections or an ordered item with a higher profit margin in a higher order on the display screen of the lower phrase group.
Speech translation program.

Images