JP6141483B1 - Speech translation device, speech translation method, and speech translation program - Google Patents

Abstract

Order information can be acquired easily and effectively by analyzing the contents of a dialogue while facilitating smooth communication. A speech translation apparatus according to an aspect of the present invention includes an input unit for inputting speech of a user or the like, a translation unit for translating the content of the input speech into content of a different language, the translation content of the input speech as speech, etc. The output unit, the display unit that displays a fixed phrase so that the user can select it, the determination unit that determines whether or not the order by the interrogator for the order inquiry from the user is confirmed, and the order is confirmed When the determination is made, a storage unit is provided for storing the confirmed contents of the order. The determination unit determines that the order has been completed when, for example, the content of the input voice or the content of the selected fixed phrase includes the product name, the order quantity, and the closing. [Selection] Figure 2

Description

  The present invention relates to a speech translation device, a speech translation method, and a speech translation program.

  Conventionally, in a chain or an independent retail store, etc., using a cash register (hereinafter referred to as “POS register”) having a point-of-sale information management (POS) function, order information and accounting information for each store and product are obtained. The technique which performs is known (for example, each patent document described in patent document 1 and patent document 1). In addition, recently, an order entry system (hereinafter referred to as “the order entry system”) that obtains order information by operating a mobile terminal device and entering a food order menu and quantity ordered by a customer, including restaurants. OES ”is also introduced (for example, Patent Document 2). The order information and accounting information acquired in this way can be used for sales analysis and inventory management for each store or product.

  On the other hand, for example, by selecting a speech translation application (for example, Non-Patent Document 1) for carrying out a conversation between a store clerk and a foreign customer, or a situation (restaurant, shopping, etc.) that the user wants to carry out a conversation, A translation application (for example, Non-Patent Document 2) that displays a list of conversation patterns is also known.

JP 2013-137657 A JP 2011-204227 A

U-STAR Consortium homepage [Search on February 23, 2016], Internet <URL: http://www.ustar-consortium.com/app_en/app.html> Conversation simulation is possible! Japanese-Chinese-Korean-compatible translation application “[Travel App No. 1] TS Conversation Translator [CJK]” [searched February 23, 2016], Internet <URL: http://andronavi.com/2012/08/ 208376>

  By the way, by combining the above conventional techniques, for example, using a translation application, a foreign customer's order is heard, the contents of the order are entered into the OES or POS register, and the order information thus obtained is stored for each store or product. It is recalled for use in sales analysis and inventory management. However, such a technique requires a terminal device that executes a translation application and a terminal device such as an OES or POS register, and the operation of operating them to acquire order information becomes complicated. In addition, it may be possible to operate an application having an OES or POS register function on the same terminal device as the translation application. However, even in this case, it is necessary to operate a plurality of applications, and the work is still complicated. End up.

  In addition, by sales analysis for each store or product, for example, it is possible to extract popular products and food and drink menus at that time and recommend them to customers as recommended products and recommended food and drink menus. However, for foreign customers, it is necessary to obtain the analysis results of such recommended products and recommended food and drink menus, and then recommend them to customers using the translation application described above. As a result, smooth communication with foreign customers becomes difficult, and the quality of customer service may be reduced.

  Therefore, the present invention has been made in view of such circumstances, and it is possible to simplify the order information by analyzing the contents of the dialogue while facilitating smooth communication when the user interacts with the conversation person (foreign customer). An object of the present invention is to provide a speech translation device, a speech translation method, and a speech translation program that can be acquired effectively and thereby contribute to the optimization of customer service and the improvement of sales and profits of user stores. .

  In order to solve the above problems, a speech translation apparatus according to an aspect of the present invention includes an input unit for inputting a voice of a user and / or a dialoguer, and a translation unit for translating the content of the input speech into content of a different language An output unit that outputs the translated content of the input speech as a voice and / or text, a display unit that displays a fixed phrase so that the user and / or the dialogue can be selected, and a dialogue by the user regarding an order inquiry from the user A determination unit that determines whether or not the order has been confirmed, and a storage unit that stores the confirmed content of the order when it is determined that the order has been confirmed. And when the determination unit includes (1) the content of the input voice or the content of the selected fixed phrase includes the product name, the order quantity, and the closing, or (2) the content of the input voice, Alternatively, when the product name is included in the content of the selected fixed phrase and there is an operation indicating the end of the conversation, it is determined that the order is confirmed.

  The “standard phrase” includes sentences, clauses, phrases, words, and numbers, and may include images or symbols accompanying them. “Closing” includes a phrase meaning the end of the conversation (for example, a phrase indicating or implying that the order has been accepted and confirmed). Further, the “operation indicating the end of the conversation” means an intention to end the conversation (for example, the termination of the speech translation application according to the present invention operating on the speech translation device, or a predetermined process in the speech translation application). Operation and operation).

  Further, the display unit may be configured to display a list of product names and display product names having a larger total number of order quantities stored in the storage unit in a higher order in the list.

  Still further, the storage unit stores the accounting quantity for each product name, the display unit displays a list of product names stored in the storage unit, and the product name having a larger total number of accounting quantities in the list. It is also preferable to display the images in a higher order.

  Furthermore, the storage unit stores the profit rate for each product name, the display unit displays a list of product names, and the product name having a higher profit rate is displayed in a higher order in the list. Even it is suitable.

  Further, the fixed phrase may be set automatically or manually in advance for each type of business to which the user belongs or for each store of the user.

  Further, the display unit may display a translation of each fixed phrase in a different language, or the output unit may output a translation of each fixed phrase in a different language by voice.

  A speech translation method according to an aspect of the present invention is a method using a speech translation apparatus including an input unit, a translation unit, an output unit, a display unit, a determination unit, and a storage unit. That is, in the method, the input unit inputs the voice of the user and / or the conversation person, the translation unit translates the content of the input voice into different language content, and the output unit The step of outputting the translation contents by voice and / or text, the step of displaying a fixed phrase so that the display unit can be selected by the user and / or the dialog, and the determination unit interacting with the inquiry about the order from the user A step of determining whether or not the order by the person has been confirmed, and a step of storing the confirmed content of the order when the storage unit determines that the order has been confirmed. In the determining step, (1) the content of the input voice or the content of the selected fixed phrase includes the product name, the order quantity, and the closing, or (2) the input voice When the product name is included in the content or the content of the selected fixed phrase and there is an operation indicating the end of the conversation, it is determined that the order is confirmed.

  In addition, the speech translation program according to one aspect of the present invention is a computer (not limited to a single type or a single type, but may be a plurality or a plurality of types; the same shall apply hereinafter), and an input unit for inputting a voice of a user and / or a conversation person A translation unit that translates the content of the input speech into a different language content, an output unit that outputs the translation content of the input speech as speech and / or text, and a fixed phrase that can be selected by the user and / or the interlocutor , A determination unit for determining whether or not the order by the interrogator for the order inquiry from the user is confirmed, and when the order is determined to be determined, the determined content of the order is stored It functions as a storage unit. And when the determination unit includes (1) the content of the input voice or the content of the selected fixed phrase includes the product name, the order quantity, and the closing, or (2) the content of the input voice, Alternatively, when the product name is included in the content of the selected fixed phrase and there is an operation indicating the end of the conversation, it is determined that the order is confirmed.

  According to the present invention, it is determined whether or not an order by a conversation person in response to an order inquiry from a user is confirmed in a conversation when the user interacts with a conversation person (foreigner customer), and the confirmed content is stored. be able to. In particular, (1) when the content of the spoken input speech or the content of the selected fixed phrase includes a product name, order quantity, and closing, or (2) the content of the input speech, Alternatively, when the product name is included in the content of the selected fixed phrase and there is an operation indicating the end of the conversation, it is determined that the order has been confirmed. That is, according to the present invention, order information can be acquired easily and effectively by analyzing the contents of a dialog while conducting smooth communication through a conversation when a user interacts with a dialog person (foreign customer). it can. In addition, this makes it possible to contribute to optimizing customer service and improving sales and profits of user stores.

1 is a system block diagram schematically showing a preferred embodiment of a network configuration and the like related to a speech translation apparatus according to the present invention. It is a flowchart which shows an example of the flow (a part) of the process in the 1st thru | or 3rd embodiment of the speech translation apparatus by this invention. It is a flowchart which shows an example of the flow (a part) of the process in 4th thru | or 6th embodiment of the speech translation apparatus by this invention. (A) thru | or (C) are top views which show an example of the transition of the display screen in an information terminal. (A) thru | or (D) are top views which show an example of the transition of the display screen in an information terminal.

  Hereinafter, embodiments of the present invention will be described in detail. The following embodiments are examples for explaining the present invention, and are not intended to limit the present invention only to the embodiments. The present invention can be variously modified without departing from the gist thereof. Furthermore, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent ones, and such embodiments are also included in the scope of the present invention. Furthermore, positional relationships such as up, down, left, and right shown as needed are based on the display shown unless otherwise specified. Furthermore, various dimensional ratios in the drawings are not limited to the illustrated ratios.

(Device configuration)
FIG. 1 is a system block diagram schematically showing a preferred embodiment such as a network configuration related to a speech translation apparatus according to the present invention. In this example, the speech translation apparatus 100 includes a server 20 that is electronically connected to the information terminal 10 (user apparatus) used by the user via the network N (but is not limited to this).

  The information terminal 10 employs a user interface such as a touch panel and a display with high visibility, for example. The information terminal 10 here is a portable tablet terminal device including a mobile phone represented by a smartphone having a communication function with the network N. The information terminal 10 further includes a processor 11, a storage resource 12, a voice input / output device 13, a communication interface 14, an input device 15, a display device 16, and a camera 17. In addition, the information terminal 10 operates by the installed speech translation application software (at least a part of the speech translation program according to the embodiment of the present invention), so that a part of the speech translation apparatus according to the embodiment of the present invention or It functions as a whole.

  The processor 11 includes an arithmetic logic unit and various registers (program counter, data register, instruction register, general-purpose register, etc.). Further, the processor 11 interprets and executes speech translation application software, which is the program P10 stored in the storage resource 12, and performs various processes. The speech translation application software as the program P10 can be distributed from the server 20 through the network N, for example, and may be installed and updated manually or automatically.

  The network N includes, for example, a wired network (a short-range communication network (LAN), a wide-area communication network (WAN), a value-added communication network (VAN), etc.) and a wireless network (mobile communication network, satellite communication network, Bluetooth ( Bluetooth (registered trademark)), WiFi (Wireless Fidelity), HSDPA (High Speed Downlink Packet Access), etc.).

  The storage resource 12 is a logical device provided by a storage area of a physical device (for example, a computer-readable recording medium such as a semiconductor memory), and an operating system program, a driver program, various data, etc. used for processing of the information terminal 10 Is stored. Examples of the driver program include an input / output device driver program for controlling the audio input / output device 13, an input device driver program for controlling the input device 15, and a display device driver program for controlling the display device 16. Can be mentioned. Furthermore, the voice input / output device 13 is, for example, a general microphone and a sound player capable of reproducing sound data.

  The communication interface 14 provides a connection interface with the server 20, for example, and is configured from a wireless communication interface and / or a wired communication interface. The input device 15 provides an interface for accepting an input operation by a tap operation such as an icon, a button, a virtual keyboard, or a text displayed on the display device 16. Various input devices to be attached can be exemplified.

  The display device 16 provides various information as an image display interface to a user or a conversation person (conversation partner), and examples thereof include an organic EL display, a liquid crystal display, and a CRT display. The camera 17 is for capturing still images and moving images of various subjects.

  The server 20 is constituted by, for example, a host computer having a high arithmetic processing capability, and expresses a server function by operating a predetermined server program in the host computer, for example, a speech recognition server, a translation server, And a single or a plurality of host computers functioning as a speech synthesis server (in the drawing, it is indicated by a single, but is not limited thereto). Each server 20 includes a processor 21, a communication interface 22, and a storage resource 23.

  The processor 21 is composed of an arithmetic and logic unit for processing arithmetic operations, logical operations, bit operations and the like and various registers (program counter, data register, instruction register, general-purpose register, etc.), and is stored in the storage resource 23. P20 is interpreted and executed, and a predetermined calculation processing result is output. The communication interface 22 is a hardware module for connecting to the information terminal 10 via the network N. For example, the communication interface 22 is a modulation / demodulation device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

  The storage resource 23 is a logical device provided by, for example, a storage area of a physical device (a computer-readable recording medium such as a disk drive or a semiconductor memory), and each includes one or a plurality of programs P20, various modules L20, and various databases. D20 and various models M20 are stored. The storage resource 23 also includes a plurality of standard questions prepared for the user to speak to the interlocutor, history data of input speech, data for various settings, order confirmation details of a product (menu) described later, and the like. It is remembered.

  The program P20 is the above-described server program that is the main program of the server 20. In addition, the various modules L20 perform a series of information processing related to requests and information transmitted from the information terminal 10, so that they are appropriately called and executed during the operation of the program P10 (moduleized subprograms). ). Examples of the module L20 include a speech recognition module, a translation module, and a speech synthesis module.

  The various databases D20 include various corpora required for speech translation processing (for example, in the case of Japanese and English speech translation, a Japanese speech corpus, an English speech corpus, a Japanese character (vocabulary) corpus, an English character) (Vocabulary) Corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.), speech database, management database for managing information related to users, order history database to be described later, and the like. Examples of the various models M20 include acoustic models and language models used for speech recognition.

(First embodiment)
An example (first embodiment) of speech translation processing operations and operations in the speech translation apparatus 100 configured as described above will be further described below. FIG. 2 is a flowchart illustrating an example of (a part of) a processing flow in the speech translation apparatus 100 according to the first to third embodiments. 4A to 4C are plan views illustrating an example of display screen transition in the information terminal. Here, it is assumed that the user of the information terminal 10 is a clerk of a restaurant that speaks Japanese, and the conversation person (conversation partner) is a foreign customer who speaks English (however, the language or Situations are not limited to this).

  First, when the user (clerk) activates the application (step SU1), the language selection screen for the conversation person shown in FIG. 4A is displayed on the display device 16 of the information terminal 10 (step SJ1). In this language selection screen, the Japanese text T1 for prompting the user to ask the conversation person about the language, the English text T2 for asking the conversation person about the language, and a plurality of typical languages assumed. Here, a language button 41 indicating English, Chinese (for example, two types depending on the typeface), and Korean is displayed. Further below that, a cancel button B1 for closing the language selection screen and ending the application is also displayed.

  At this time, as shown in FIG. 4A, the Japanese text T1 and the English text T2 are divided by the processor 11 and the display device 16 into different areas on the screen of the display device 16 of the information terminal 10, and Are displayed in opposite directions (different directions; upside down in the figure). Thereby, when a conversation is performed in a state where the user and the interlocutor face each other, the user can easily confirm the Japanese text T1, while the interrogator can easily confirm the English text T2. In addition, since the Japanese text T1 and the English text T2 are displayed separately, there is an advantage that both are clearly distinguished and can be visually recognized more easily.

  The user presents the display of the English text T2 on the language selection screen to the conversation person, and the conversation person taps the English button, for example, or the user himself selects the conversation person's language. can do. When the language of the conversation person is selected in this way, the processor 21 of the server 20 and the processor 11 of the information terminal 10 display the Japanese and English voice input standby screen on the display device 16 as the home screen (FIG. 4 ( B); Step SJ2). On this voice input standby screen, the Japanese text T3 asking which of the user's language and the talker's language is spoken, the input button 42a for inputting Japanese voice, and the English voice input are performed. The input button 42b is displayed.

  Also, on this voice input standby screen, a voice button 43 for selecting a list display of a plurality of preset question phrases, the language selection screen shown in FIG. Language selection button 44 for switching the language (re-selecting the language), history button 45 for selecting the history display of the voice input content made so far, and a plurality of standard phrases (recommended phrases) prepared in advance Also displayed are a suggest button 46 for executing a suggest function that allows a user to select a desired fixed phrase and proceed with a conversation, and a setting button 47 for performing various settings of the application software.

  Here, an outline of a main procedure of normal speech translation processing (step SJ3 in FIG. 2) in the conversation between the user and the conversation person and / or the conversation preparation will be described. First, on the voice input standby screen shown in FIG. 4 (B), when the user taps the Japanese input button 42a and selects Japanese voice input, the voice input screen accepts the user's Japanese utterance content. (FIG. 4C). When this voice input screen is displayed, voice input from the voice input / output device 13 is enabled. In addition, the voice input screen displays text T4 that prompts the user to input voice, a microphone design 48 indicating that the user is in a voice input state, and an input switching button 50 for switching to text input. Further, a cancel button B1 is also displayed on this voice input screen. By tapping this button, the conversation can be ended or the voice input can be performed again by returning to the voice input standby screen (FIG. 4B). .

  In this state, when the user utters an item to be communicated to the talker (step SU2), a multi-circular pattern 49 that schematically and dynamically represents the volume of the voice is displayed together with the text T4, and the voice input level Is visually fed back to the user who is the speaker. Then, when the utterance is finished and the user taps the microphone design 48, the processor 11 finishes accepting the utterance content by the user. The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N. As described above, the information terminal 10 itself, or the processor 11 and the voice input / output device 13 function as an “input unit”.

  Next, the processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing. At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains “sound” of the input speech. Convert to "reading" (character). As described above, the processor 21 or the server 20 functions as a “voice recognition server” as a whole. In addition, the processor 21 stores the recognized content in the storage resource 23 (in an appropriate database as necessary) as voice input history data.

  Next, the processor 21 transmits the recognition result of the input voice to the information terminal 10, and the processor 11 displays it on the screen as Japanese text (not shown). At this time, the recognition result of the input voice may be displayed as it is, or the Japanese speech corpus stored in the storage resource 23 in advance is called and displayed corresponding to the content of the actual input voice. May be.

  Subsequently, the processor 21 proceeds to multilingual translation processing for translating the recognized “reading” (characters) of the speech into another language. At this time, the processor 21 calls the necessary module L20 and database D20 (translation module, Japanese character corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.) from the storage resource 23 and recognizes them. The resulting input speech “reading” (character string) is properly sorted and converted into Japanese phrases, clauses, sentences, etc., the English corresponding to the conversion result is extracted, and these are sorted according to the English grammar. To natural English phrases, clauses, sentences, etc. As described above, the processor 21 also functions as a “translation unit”, and the server 20 also functions as a “translation server” as a whole. If the input voice is not correctly recognized, the voice can be re-input (not shown). The processor 21 can also store those Japanese and English phrases, clauses, sentences, and the like in the storage resource 23.

  Then, the processor 21 proceeds to speech synthesis processing. At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, English speech corpus, acoustic model, language model, etc.) from the storage resource 23, and the English phrase that is the translation result, Convert clauses, sentences, etc. to natural speech. As described above, the processor 21 also functions as a “speech synthesizer”, and the server 20 also functions as a “speech synthesizer” as a whole.

  Then, the processor 21 generates a voice signal for voice output based on the synthesized voice, and transmits the voice signal to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the audio signal through the communication interface 14 and performs an audio output process using the audio input / output device 13 (step SJ3 so far). As described above, the processor 11 and the voice input / output device 13 function as an “output unit”. Prior to the voice output, the user's voice recognition result and the translation result thereof may be temporarily displayed on the information terminal 10 and the voice output may be performed after confirmation by the user (not shown).

  Next, as a more specific processing flow in the present embodiment, an example of processing in conversation when the user places an order for a conversation person will be further described. First, the user speaks an order inquiry (for example, the phrase “Are you sure you want to hear your order?”) Toward the voice input screen (FIG. 4C), and the voice is sent to the information terminal. 10 (step SU2).

  The processor 21 of the server 20 that has received the voice signal generates voice recognition and multilingual translation (for example, “May I have your order? And the processing up to the voice and / or text output is performed (step SJ3). Then, the processor 21 determines whether or not the order of the interrogator in response to the user's order inquiry is confirmed (completed) (step SJ4). Specifically, (1) When the content of the conversation (input voice) so far includes the product name (menu name), order quantity, and closing, or (2) the content of the conversation so far If the product name is included and there is an operation indicating the end of the conversation, it is determined that the order has been confirmed.

  The first embodiment corresponds to the case of (1) above, and more specifically, for example, as follows. First, the processor 21 performs morphological analysis on the content of the input speech that has been speech-recognized to obtain a morpheme, and if necessary, phrases, clauses, sentences, etc. of the input speech stored in the storage resource 23 in multilingual translation processing Call. Then, it is determined whether or not those morphemes, phrases, clauses, sentences, and the like match preset product names (menu names), order quantities, and closings. That is, if all the items that match the product name, the order quantity, and the closing are all included in the conversation contents up to that point, it is determined that the order has been confirmed (Yes in step SJ4). In the case of (2), more specifically, when there is an item that matches the product name in the previous conversation content and there is an operation indicating the end of the conversation, such as tapping the cancel button B1, It is determined that the order has been confirmed (Yes in step SJ4). On the other hand, if neither of the above (1) and (2) is applicable, it is determined that the order has not been confirmed (No in step SJ4).

  Here, the order inquiry ("Are you sure you want to ask me?") Has just been uttered, and the content of the conversation includes product name, order quantity, and closing. It is not included (not applicable to either (1) or (2)), the processor 21 determines that the order has not been confirmed (No in step SJ4), and proceeds to the utterance of the conversation person (step SU2). Transition.

  Next, as an answer to the inquiry, the dialog person utters the menu name and quantity (for example, “Two draft beer, please.” Corresponding to “Two draft beer please.”) And places an order (step SU2). The processor 21 performs speech recognition, multilingual translation, and processing up to the speech and / or text output (step SJ3) for the utterance content, and subsequently determines the final order (step SJ4).

  Here, the order details (“Two draft beers please.”) Are spoken, and the content of the conversation so far includes the product name (“draft beer”) and the order quantity (“two”). ) Is included, but the closing is not yet included, and there is no operation indicating the end of the conversation such as tapping the cancel button B1, so the processor 21 determines that the order has not been confirmed (step SJ4). No) and the process again proceeds to the user utterance (step SU2).

  Next, the user utters a closing phrase (for example, “Thank you”, “I have it now”) in response to the order of the conversation person (step SU2). The processor 21 performs speech recognition, multilingual translation, and processing up to the speech and / or text output (step SJ3) for the utterance content, and subsequently determines the final order (step SJ4).

  At this stage, closing ("Thank you", "I'll have it now", etc.) is spoken, so the content of the conversation so far includes the product name ("Draft beer"), the order quantity ( “Two”) and closing are all included. Therefore, the processor 21 determines that the order has been confirmed (Yes in step SJ4). Then, the processor 21 stores the menu name “Draft beer” and its quantity “2” in an appropriate database in the storage resource 23 as a history (order history) of the order contents acquired in the conversation (step). SJ5). In this case, the processor 21 may hold the order history data in, for example, an order history database as one of the databases D20. Then, the user can end the application as appropriate (step SU3).

(Second Embodiment)
Next, another example (second embodiment) of operations and operations of speech translation processing in the speech translation apparatus 100 will be described. In the second embodiment, after the dialog person places an order (menu name and quantity) in response to the user's order inquiry, the user does not utter the closing phrase and the voice shown in FIG. On the input screen, the same processing as in the first embodiment is executed except that the conversation is ended by pressing the cancel button B1. The second embodiment corresponds to the case (2) above.

  That is, for example, after a dialoguer utters the contents of an order (“Thank you for two draft beers.”), The user does not enter a closing phrase, and an appropriate greeting (for example, End the inquiry about the order by verbally or telling the interlocutor (in English indicating “Thank you.”) (Although this is not necessary), and tap the cancel button B1 to apply the speech translation application. End the conversation by (operation indicating the end of the conversation). Then, since the operation for ending the conversation by tapping the cancel button B1 has been performed, the processor 21 skips step SJ3 which is a process related to the input voice, and performs order confirmation determination (step SJ4).

  At this stage, since the content of the conversation so far includes the product name (“draft beer”) and there is an operation indicating the end of the conversation such as tapping the cancel button B1, the processor 21 determines that the order has been placed. It is determined that it has been confirmed (Yes in step SJ4). Then, the processor 21 stores the menu name “Draft beer” and its quantity “2” in the storage resource 23 as the history of the order contents acquired in the conversation (order history) (step SJ5). Also in this case, the processor 21 may hold the order history data in, for example, an order history database as one of the databases D20. Then, the user can end the application as appropriate (step SU3).

(Third embodiment)
Next, another example (third embodiment) of speech translation processing operations and operations in the speech translation apparatus 100 will be described. In the third embodiment, the voice input shown in FIG. 4C is performed without the user uttering the closing phrase after the dialog person places an order (menu name only) in response to the user's order inquiry. On the screen, the same processing as in the second embodiment is executed except that the conversation is ended by pressing the cancel button B1. The third embodiment also corresponds to the case (2).

  That is, for example, after a dialog person who visited the store alone uttered the contents of the order ("draft beer"), the user did not enter a closing phrase, but an appropriate greeting (for example, "I was clever". (English, etc.) "is sent to the dialogue person verbally or in an alternative action, and the inquiry for the order is terminated (this is not necessary), and the conversation by the speech translation application is terminated by tapping the cancel button B1. (Operation indicating end of conversation). Then, since the operation for ending the conversation by tapping the cancel button B1 has been performed, the processor 21 skips step SJ3 which is a process related to the input voice, and performs order confirmation determination (step SJ4).

  At this stage, since the content of the conversation so far includes the product name (“draft beer”) and there is an operation indicating the end of the conversation such as tapping the cancel button B1, the processor 21 determines that the order has been placed. It is determined that it has been confirmed (Yes in step SJ4). Then, the processor 21 stores the “draft beer” of the menu name in the storage resource 23 as a history of order contents (order history) acquired in the conversation (step SJ5). At this time, the order quantity may not be stored, or the default value of the order quantity may be set as “one” in advance, and the default value may be stored as the order quantity in the conversation. Then, the user can end the application as appropriate (step SU3).

(Fourth embodiment)
Subsequently, another example (fourth embodiment) of the operation and operation of the speech translation process in the speech translation apparatus 100 will be further described below. FIG. 3 is a flowchart illustrating an example of (a part of) a processing flow in the speech translation apparatus 100 according to the fourth to sixth embodiments. 5A to 5D are plan views illustrating an example of display screen transition in the information terminal. The fourth embodiment is the same as the first embodiment except that steps SU4 and SJ6 are executed after step SJ2, step SU5 is executed instead of step SU2, and step SJ7 is executed instead of step SJ3. A similar process is executed. Note that step SJ7 instead of step SJ3 is executed after step SJ4.

  That is, when the user taps the suggest button 46 on the voice input standby screen shown in FIG. 4B (step SU4), the processor 11 of the information terminal 10 is frequently used in customer service at the store of the industry to which the user belongs. The command signal for displaying the fixed phrase group is transmitted to the server 20. The processor 21 of the server 20 that has received the command signal accesses, for example, a phrase database included in the database D20 stored in the storage resource 23, calls the corresponding fixed phrase group, and creates display image data of those lists. To the processor 11 of the information terminal 10. Based on the display image data, the processor 11 displays, for example, an initial phrase group screen shown in FIG. 5A on the display device 16 (step SJ8).

  On the initial phrase group screen of FIG. 5 (A), a plurality of Japanese phrase texts and English phrase texts indicating their translations in English are displayed as a phrase list P1 in a state where they are written together for each fixed phrase. The As shown in FIG. 5A, the phrase list P1 includes, for example, fixed phrases that are often uttered at the timing when the customer arrives at the seat. In the initial phrase group screen, the language selected as the language of the conversation person (that is, the parallel language) on the language selection screen of FIG. 4A is English above and below the phrase list P1, respectively. A Japanese text T4 indicating “” and a close button B2 are also displayed (the same applies hereinafter). By tapping this close button B2, it is possible to end the conversation or close the phrase group screen and return to the voice input standby screen of FIG.

  Next, the user taps and selects the text T5 of a phrase (for example, “Would you like a drink?”) From the phrase list P1 to inquire about an order for a drink, for example (step SU5). The ten processors 11 transmit the selection command signal to the processor 21 of the server 20. The processor 21 that has received it returns voice output data of the translation of the text T5 in English to the processor 11, and the processor 11 outputs the voice from the voice input / output device 13. Further, the processor 21 determines whether or not the order of the interrogator in response to the user's order inquiry has been confirmed (completed) (step SJ4).

  Specifically, (1) If the content of the standard phrase selected so far includes the product name (menu name), order quantity, and closing, or (2) selected so far When the content of the standard phrase includes a product name (menu name) and there is an operation indicating the end of the conversation, it is determined that the order has been confirmed. The fourth embodiment corresponds to the case of (1) above.

  Here, the order inquiry ("Would you like a drink?") Has just been selected, and the contents of the fixed phrase selected so far include the product name (menu name), order quantity Neither closing nor closing is included, and the processor 21 determines that the order has not been confirmed (No in step SJ4), and shifts to a phrase group screen display (step SJ7) that can be selected by the interlocutor. To do.

  Next, the processor 21 accesses the phrase database again, calls a plurality of other fixed phrases associated with the fixed phrase of the text T5, creates display image data of a list of those fixed phrases, and creates the information terminal 10. To the processor 11. Based on the display image data, the processor 11 displays, for example, a phrase group screen shown in FIG. 5B on the display device 16 (step SJ7).

  On the phrase group screen of FIG. 5B, a phrase list P2 including a plurality of phrase texts is displayed in the same manner as the phrase list P1 shown in FIG. As shown in FIG. 5B, the phrase list P2 includes a plurality of product names (drink menu names), and also includes text T6 indicating that the interrogator requests a menu from the user. It is. The user can present an order by presenting the phrase group screen to the interlocutor, and the interrogator taps the menu name of the desired drink. Alternatively, if the menu name of the drink desired by the dialogue person is not in the phrase list P2, the dialogue person can request the user who is a store clerk to browse the menu by tapping the text T6 portion.

  Thus, when the dialog person taps and selects the text of a phrase representing a desired menu name (for example, “draft beer”) from the phrase list P2 (step SU5), the processor 11 of the information terminal 10 The selection command signal is transmitted to the processor 21 of the server 20. The processor 21 that has received it returns the Japanese voice output data of the text to the processor 11, and the processor 11 outputs the voice from the voice input / output device 13. Further, the processor 21 determines whether or not the order of the interrogator in response to the user's order inquiry has been confirmed (completed) (step SJ4).

  Here, at the stage where the menu name (“Draft beer”) is selected, the contents of the fixed phrase selected so far include neither the order quantity nor the closing, and close. Since there is no operation for ending the conversation by tapping the button B2, the processor 21 determines that the order has not been confirmed (No in step SJ4).

  Next, the processor 21 accesses the phrase database again, and when the menu name is selected, the processor 21 calls or creates the display image data of the quantity input screen for inputting the quantity to the dialog person. To the processor 11 of the information terminal 10. Based on the display image data, the processor 11 displays, for example, an order quantity input screen shown in FIG. 5C on the display device 16 (step SJ7). In the quantity input screen of FIG. 5C, a numeric key 51 for inputting a number is displayed. The user presents this quantity input screen to the dialog person, and the dialog person taps the screen to order quantity ( Here, for example, when two are input (step SU5), the numbers are displayed in the column 52. Next, the processor 21 determines whether or not the order of the interrogator in response to the user's order inquiry is confirmed (completed) (step SJ4).

  Here, the order quantity (“2”) of the menu name (“Draft beer”) is entered, and the content of the fixed phrase selected so far does not include closing, Since there is no operation for ending the conversation by tapping the close button B2, the processor 21 determines that the order has not been confirmed (No in step SJ4).

  Next, the processor 21 accesses the phrase database again, calls a plurality of other fixed phrases associated with the quantity input, creates display image data of a list of these fixed phrases, and processes the processor of the information terminal 10. 11 to send. Based on the display image data, the processor 11 displays, for example, a phrase group screen shown in FIG. 5D on the display device 16 (step SJ7).

  Also on the phrase group screen of FIG. 5D, a phrase list P3 including a plurality of phrase texts is displayed in the same manner as the phrase list P1 shown in FIG. As shown in FIG. 5D, the phrase list P3 includes closing phrases that are often spoken when orders or requests from customers are received. Then, when the user taps and selects the text portion of the desired phrase from the phrase list P3 (step SU5), the English translation of the phrase is output in the same manner as the processing so far. . Further, the processor 21 determines whether or not the order of the interrogator in response to the user's order inquiry has been confirmed (completed) (step SJ4).

  At this stage, closing phrases (such as “I've got it right”, “I will bring you right now”) are selected, so the content of the standard phrase selected so far includes the product name (“Draft beer” "), Order quantity (" 2 "), and closing. Therefore, the processor 21 determines that the order has been confirmed (Yes in step SJ4). Then, the processor 21 stores the menu name “Draft beer” and its quantity “2” in the storage resource 23 as the history of the order contents acquired in the conversation (order history) (step SJ5). In this case, the processor 21 may hold the order history data in, for example, an order history database as one of the databases D20. Then, the user can end the application as appropriate (step SU3).

(Fifth embodiment)
Next, another example (fifth embodiment) of speech translation processing operations and operations in the speech translation apparatus 100 will be described. In the fifth embodiment, after the dialog person places an order (menu name and quantity) in response to the user's order inquiry, the user does not utter the closing phrase, and FIG. 5 (C) or FIG. In the phrase group display screen shown in (D), the same process as in the fourth embodiment is executed except that the close button B2 is pressed to end the conversation. The fifth embodiment corresponds to the case (2) above.

  That is, for example, after a dialoguer utters the contents of an order (“Thank you for two draft beers.”), The user does not enter a closing phrase, and an appropriate greeting (for example, End the inquiries about the order by telling the interlocutor verbal or alternative actions (such as English indicating “Thank you.”), And tap the close button B2 to tap the speech translation application. End the conversation by (operation indicating the end of the conversation). Then, the processor 21 performs order confirmation determination (step SJ7).

  At this stage, the contents of the fixed phrase selected so far include the product name (“draft beer”), and there has been an operation indicating the end of the conversation, such as tapping the close button B2, so the processor 21 Determines that the order has been confirmed (Yes in step SJ7). Then, the processor 21 stores the menu name “Draft beer” and its quantity “2” in the storage resource 23 as the history of the order contents acquired in the conversation (order history) (step SJ5). Also in this case, the processor 21 may hold the order history data in, for example, an order history database as one of the databases D20. Then, the user can end the application as appropriate (step SU3).

(Sixth embodiment)
Next, another example (sixth embodiment) of operations and operations of speech translation processing in the speech translation apparatus 100 will be described. In the sixth embodiment, after the dialog person places an order (only the menu name) in response to the user's order inquiry, the user does not utter the closing phrase, and FIG. 5 (B) or FIG. In the phrase group display screen shown in C), the same process as in the fifth embodiment is executed except that the close button B2 is pressed to end the conversation. The sixth embodiment also corresponds to the case (2).

  That is, for example, after a talker who visited the store alone uttered the order details ("draft beer"), the user did not ask the talker to input the quantity, and an appropriate greeting (for example, "I'm sorry") (In English, etc.) indicating verbal or alternative actions to the dialogue person, the order inquiry is terminated (this may not be necessary), and the close button B2 is tapped to conduct the conversation by the speech translation application. End (operation indicating the end of the conversation). Then, the processor 21 performs order confirmation determination (step SJ7).

  At this stage, the contents of the fixed phrase selected so far include the product name (“draft beer”), and there has been an operation indicating the end of the conversation, such as tapping the close button B2, so the processor 21 Determines that the order has been confirmed (Yes in step SJ4). Then, the processor 21 stores the “draft beer” of the menu name in the storage resource 23 as a history of order contents (order history) acquired in the conversation (step SJ5). At this time, the order quantity may not be stored, or the default value of the order quantity may be set as “one” in advance, and the default value may be stored as the order quantity in the conversation. Then, the user can end the application as appropriate (step SU3).

  According to the speech translation apparatus 100 configured as described above, the speech translation method using the speech translation device, and the speech translation program, (1) conversation content or selection at the time of customer service to a talker (foreign customer) If there are items that match the product name (menu name: for example, “draft beer”), order quantity (for example, “two”), and closing in the contents of the fixed phrases that have been made, the order is confirmed judge. Moreover, when the product name is included in the content of the conversation or the content of the selected fixed phrase, and there is an operation indicating the end of the conversation (for example, tap of the cancel button B1 or the close button B2). It is also determined that the order has been confirmed. As described above, according to the present invention, when a user interacts with a conversation person (foreigner customer), it is possible to analyze whether or not a predetermined element is included in the contents of the conversation while performing smooth communication through conversation. By doing so, the order information can be acquired easily and effectively. As a result, it is possible to contribute to optimizing customer service and improving sales and profits of user stores.

  In addition, as shown in FIGS. 5A to 5D, the fixed phrases associated with the fixed phrases selected previously are sequentially displayed on the display device 16 as separate screens, so that the conversation progresses. It becomes easy to select a desired phrase easily and accurately, and the conversation between the user and the conversation person can be performed naturally and smoothly. Furthermore, if a plurality of phrases set in each of the phrase lists P1 to P3 are preset for each type of industry to which the user belongs, conversations specialized for customer service in the type of industry can be made more smoothly and more smoothly. Can be implemented appropriately. Further, such a fixed phrase can be set in advance for each user's store, and in this case, a finer customer service reflecting the features of each store and the situation of the store becomes possible.

  Here, the fixed phrase may be set automatically or manually. As an example of automatic setting, first, when using the translation application, the user's business type is registered as one of the user information, and the processor 21 of the server 20 uses a corpus of fixed phrases that frequently appear in conversations of that business type. In addition, there is a form in which a fixed phrase that is particularly frequently used is selected from history and set as a phrase list. Alternatively, the fixed phrases uttered by a plurality of users in the same industry are stored in an appropriate database together with the utterance frequency, and the processor 21 of the server 20 selects those frequently used from those fixed phrases. It may be set as a phrase list. On the other hand, as an example of setting manually, there is a form in which the user selects a desired fixed phrase and customizes it as a phrase list.

  At that time, for example, the fixed phrases included in the phrase lists P1 to P3 may be maintained at the initial settings (fixed phrase list), or the fixed phrases included in them may be appropriately changed as necessary. May be. In particular, in the latter case, for example, the number of times each fixed phrase is selected is stored in the storage resource 23, and the processor 21 of the server 20 selects a phrase with a larger number of selections on the display screen of the phrase lists P1 to P3. It may be displayed in a higher order (for example, displayed at the upper part of the screen or displayed with emphasis or enlargement). Thereby, there is an advantage that it becomes easy to display a fixed phrase according to the user's type of business and the actual situation of the store, and it is easy to select, and the communication between the user and the conversation person can be further speeded up.

  In addition, the number of times each selected product (menu) is selected in the past predetermined period (order quantity by the speech translation application, accounting quantity at the user's store, order quantity at a plurality of stores including the user's store, (Accounting quantity) or the profit rate of each product (menu) is stored in the storage resource 23, and the processor 21 of the server 20 selects a product with a higher number of selections or a product with a higher profit rate. For example, it may be displayed in a higher order on the display screen of the phrase list P2 shown in FIG. As a result, it is possible to actively recommend popular products and products with a high unit price per visitor to foreigners, and as a result, it is possible to further improve the sales and profits of the user's store. it can.

  Furthermore, on the display screen of the phrase lists P1 to P3, the Japanese phrase text for each fixed phrase and the English phrase text indicating the translated sentence in English are written together, and each fixed phrase is translated in a different language (for example, English). Is output by voice. Therefore, the user and the conversation person can confirm the content of the other party's utterance more reliably in addition to viewing the screen or without viewing the screen.

  Note that, as described above, each of the above embodiments is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art can replace the resources (hardware resources or software resources) described in the embodiments with equivalents, and such replacements are also included in the scope of the present invention.

  5 (A), (B) and (D), the Japanese phrase text and the English phrase text are opposite to each other (in different directions, like the texts T1 and T2 in FIG. 4 (A)). ; May be displayed in the reverse direction in the figure). Furthermore, these Japanese phrase text and English phrase text may not be written together, and only one of them may be displayed. Furthermore, it may be configured such that the quantity input shown in FIG. 5C can be performed by voice input. Furthermore, it may be possible to select the display of the phrase list set for other industries regardless of the user's industry. In addition, in the phrase list of the product name (menu name) shown in FIG. 5B, a menu name using a material with a large stock at that time or a menu name uniquely recommended by the user or the user's store is displayed. You can also.

  Moreover, although the example which performs each process, such as speech recognition, translation, speech synthesis, by server 20, was described, you may comprise so that these processes may be performed in the information terminal 10. FIG. In this case, the module L20 used for these processes may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. Furthermore, the database D20 that is a voice database and / or a model M20 such as an acoustic model may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. . As described above, the speech translation apparatus may not include the network N and the server 20.

  Of course, a gateway server for converting a communication protocol between the information terminal 10 and the network N may be interposed. The information terminal 10 is not limited to a portable device, and may be a desktop personal computer, a notebook personal computer, a tablet personal computer, a laptop personal computer, or the like.

  According to the present invention, order information can be acquired easily and effectively by analyzing the contents of a dialogue when a user interacts with a conversation person (foreigner), so that, for example, the languages of each other cannot be understood. The present invention can be widely used for activities such as design, manufacture, provision, and sales of programs, devices, systems, and methods in the field of providing services related to conversation between people.

10 Information terminal 11 Processor 12 Storage resource 13 Voice input / output device 14 Communication interface 15 Input device 16 Display device 17 Camera 20 Server 21 Processor 22 Communication interface 23 Storage resource 41 Language button 42a Japanese input button 42b English input button 43 Voice button 44 Language selection button 45 History button 46 Suggest button 47 Setting button 48 Microphone design 49 Multiple circular design 50 Input switch button 51 Number key 52 Column 100 Speech translation device B1 Cancel button B2 Close button D20 Database L20 Module M20 Model N Network P1-P3 Phrase list P10 Program P20 Program T1-T6 Text

Claims (8)

An input unit for inputting the voices of the clerk who makes an order inquiry and the foreign customer who makes the order;
A translation unit that translates the content of the input speech into content of a different language;
An output unit that outputs the translated content of the input speech as speech and / or text;
A display unit that displays a list of fixed phrases that include only product names and fixed phrases that do not include product names so that the store clerk and / or the foreign customers can select,
A determination unit for determining whether or not an order by the foreign customer in response to an order inquiry from the store clerk is confirmed;
When it is determined that the order is confirmed, a storage unit that stores the confirmed content of the order;
With
The determination unit is (1) the content of the input voice or the content of the selected fixed phrase includes a product name, an order quantity, and a closing by the store clerk , or (2) the If the content of the input voice or the content of the selected fixed phrase includes at least a product name and there is an operation by the store clerk indicating the end of the conversation other than the closing, the order is Judge that it is confirmed,
Speech translation device. The display unit, said in a case where the by foreigners trade name is selected, displays the quantity input screen that does not include the quotient product name for the foreign visitors to enter the order quantity,
The speech translation apparatus according to claim 1. The display unit displays a list of the product names, and displays the product names having a larger total number of the order quantities stored in the storage unit in a higher rank in the list.
The speech translation apparatus according to claim 1 or 2. The storage unit stores an accounting quantity for each product name and / or a profit rate for each product name,
The display unit displays a list of the product names, and the product names having a larger total number of the accounting quantities stored in the storage unit and / or the product names having a higher profit rate are displayed. Display higher in the list,
The speech translation apparatus according to any one of claims 1 to 3. The fixed phrase for each shop industries or per the clerk the clerk belongs, in which preset automatically or manually,
The speech translation apparatus according to claim 1. The display unit displays a translation of each fixed phrase in a different language, or the output unit outputs a translation of each fixed phrase in a different language by voice.
The speech translation apparatus according to claim 1. Using a speech translation device including an input unit, a translation unit, an output unit, a display unit, a determination unit, and a storage unit,
The input unit inputs a voice of a clerk who makes an order inquiry and / or a foreign customer who makes an order;
The translation unit translating the content of the input speech into content of a different language;
The output unit outputting the content of translation of the input speech as speech and / or text;
A step of displaying a list of standard phrases including only product names and standard phrases not including product names so that the display unit can be selected by the store clerk and / or the foreign customer ;
The step of determining whether or not the order by the foreign customer for the inquiry of the order from the store clerk is confirmed by the determination unit;
Storing the confirmed content of the order when the storage unit determines that the order has been confirmed;
Including
In the determining step, (1) if the content of the input voice or the content of the selected fixed phrase includes a product name, order quantity, and closing by the store clerk , or (2 ) When the content of the input voice or the content of the selected fixed phrase includes at least a product name and there is an operation by the store clerk indicating the end of the conversation other than the closing, Determine that the order has been confirmed,
Speech translation method. Computer
An input unit for inputting the voice of a clerk who makes an order inquiry and / or a foreign customer who makes an order;
A translation unit that translates the content of the input speech into a different language
An output unit that outputs the translated content of the input speech as speech and / or text;
A display unit that displays a list of fixed phrases that include only product names and fixed phrases that do not include product names so that the store clerk and / or the foreign customers can select,
A determination unit for determining whether or not an order by the foreign customer in response to an order inquiry from the store clerk is confirmed;
When it is determined that the order is confirmed, a storage unit that stores the confirmed content of the order;
To function,
The determination unit is (1) the content of the input voice or the content of the selected fixed phrase includes a product name, an order quantity, and a closing by the store clerk , or (2) the If the content of the input voice or the content of the selected fixed phrase includes at least a product name and there is an operation by the store clerk indicating the end of the conversation other than the closing, the order is Judge that it is confirmed,
Speech translation program.

Images