JP2012164172A - Text input device, translation device including the same, text input method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device that is easy in handling and is capable of translation with high reliability.SOLUTION: A translation device having a touch panel display includes an icon display unit for displaying icons 118. An icon is selected so that an icon sequence is input and displayed in an icon sequence field 120. When a user presses a source generation button 126, a source sentence in a complete form is predicted by a word sequence corresponding to the icon sequence. When the source sentence is a satisfied one, the source sentence is translated into a target sentence by selecting a translation button 124 so as to be displayed in a translation sentence field 130.

Description

  The present invention relates to natural language processing, and more particularly to a picture or icon-based text processing apparatus and machine translation including the same.

  Recently, the demand for machine translation (MT) services has increased greatly because the quality of translation has improved to a practical level in many language combinations. One common platform for machine translation applications is portable devices that can be used anywhere when needed. As the MT system has become widespread, the MT has been focused on peripheral issues other than translation quality. A significant problem faced by real-world MT applications is that there is a great variety of what a user can provide as input to an MT system. For example, for input, a sentence that is wrong, deviating from the standard, or that does not follow the grammar, a shortened word, a pictogram, a word replacement (for example, “u” instead of “you”, etc.), etc. It is included. Such a phenomenon reduces the performance of the MT system, but is essentially a peripheral matter to the central problem that the MT system is trying to address, which is already a sufficiently difficult problem.

  One way to deal with these problems is to collect a large corpus of irregular usages and the corresponding exact usages and train them to regularize the language in a supervised way. Let's go. However, this strategy clearly has its limitations. This is because the way a real user deviates from the standard depends on the user's communication mode or code, content, and even the user's wit.

  In the MT domain, two communication channels have been studied so far. As shown in FIG. 1, the first communication channel 40 is a translation between the source language 42 and the target language 46, which is an intermediate language placed between the source language 42 and the target language 46. This can be done via some intermediate language (interlinger) 44. If the intermediate language 44 is a natural language, the communication channel may be a concatenation of two MT systems. The first is from the source language 42 to the intermediate language 44, and the second is from the intermediate language 44 to the target language 46. Translation can also be achieved via the communication channel 50. This is a direct translation process from the source language 52 to the target language 54. This can be accomplished using the latest statistical machine translation systems that have been extensively studied.

picTrans. 2010. A simple picture-based translation system. 7 Zillion. [Online] http://www.7zillion.com/iPhone/PicTrans/, access date January 18, 2011. Pointing. 2010. Pointing. Information Center Publishing Bureau. [Online], http://www.yubisashi.com/free/t/iphone/, access date January 18, 2011. TexTra. 2010. (Text translator by NICT). NICT. [Online], http://mastar.jp/translation/textra-en.html, access date January 18, 2011. VoiceTra. 2010. (Nicct speech translator). NICT. [Online], http://mastar.jp/translation/voicetra-en.html, access date January 18, 2011.

  MT is a useful tool for people who want to communicate with people who speak different native languages. But you can't carry a complete MT system. Mobile devices such as mobile phones or mobile tablet computers with wireless communication capabilities are useful as long as they can communicate with a complete MT system. However, entering text on a mobile device is cumbersome and requires many key presses, making it easy to make mistakes. Even a very short sentence is difficult to input on a mobile device. In such a case, a conventional method using means other than the portable device is convenient. For example, in many cases, a person trying to communicate has communication means such as images, symbols, and gestures, and often uses these means when other means do not work. In fact, icon-based communication has a long history among diplomats trying to communicate with people in areas where there is no means of communication. Communication is carried out using books mainly using pictures. This book includes icons that indicate the meaning of the content that you want to express, and diplomat can communicate with foreign people just by pointing to them.

  For example, as shown in FIG. 4, the picture book includes two pictures 80 and 82. The picture 80 has an English text “I want to go to” and a corresponding Japanese text. The picture 82 includes an English text “restaurant”, a corresponding Japanese text, and a knife and fork picture. Suppose the user wants to translate the expression “I want to go to the restaurant” (I want to go to a restaurant). With this picture book, the user will probably point to the two pictures described above, as shown in FIG. That is, it points to the picture 80 (“I want to go to”) and then points to the picture 82 (“restaurant”).

  For picture-based communication, various applications have been proposed for portable devices. In the former area, PictTrans (pic-Trans, 2010: Non-Patent Document 1) indicates only a picture icon, and pointing (2010, Non-Patent Document 2) (meaning pointing with a finger) taps the icon. A voice is then produced, but these systems do not generate any language, which is left to human users.

  Conversely, a significant number of machine translation systems have also been proposed for handheld devices. For example, there is a texTra (texTra, 2010: Non-Patent Document 3) text translation system and a voiceTra (VoiceTra, 2010: Non-Patent Document 4) utterance translation system. None have a user interface.

  Picture-based applications are easy to handle, but the possibilities for expressing source sentences are limited. Speech-based machine translation applications are easy to handle, but speech recognition seems to be error prone. Text-based machine translation has the highest translation reliability, but text input depends on the input system of the mobile device and is usually cumbersome.

  Accordingly, an object of the present invention is to provide an apparatus and method for generating a translation that is easy to handle and highly reliable.

  Another object of the present invention is to provide an apparatus and method for easily inputting source text and generating a reliable translation.

  According to a first aspect of the present invention, a text input device used in connection with a touch panel display is connected to the touch panel display, means for displaying a set of icons on the touch panel display, and the touch panel display Connected to the touch panel display, input receiving means for receiving an input of one or more sequences of the icons displayed on the touch panel display by the user, and using the input receiving means Means for displaying a sequence of icons input by a user on the touch panel display; and translation means for translating the sequence of icons into a complete sentence in a predetermined language.

  Preferably, the text input device further includes means connected to the touch panel display and the translating means, and displaying the complete sentence translated by the translating means on the touch panel display.

  More preferably, the text input device is connected to the storage unit for storing the phrase in the predetermined language, each of which is associated with the icon, and the storage unit and the input receiving unit. Means for generating text in an external language by concatenating phrases derived from the storage means. The translation means may include a phrase-based statistical machine translation unit that is statistically trained to translate the text in the external language into a sentence in the predetermined language in a corpus of sentence pairs, each of the pairs Includes a sequence of phrases including words in a first vocabulary of the predetermined language and a sentence of the predetermined language including words in a second vocabulary greater than the first vocabulary.

  More preferably, the first vocabulary includes content words of the predetermined language.

  The statistical machine translation unit outputs a search graph that represents the bilingual phrase pairs used to build the sentence hypothesis, along with the model probabilities associated with adding these phrase pairs to the end of the preceding partial hypothesis. May be. The sentence may be a hypothesis corresponding to the most likely path in the search graph.

  The text input device further includes means for receiving selection by one user of an icon in the icon sequence and up to a node corresponding to the icon selected by the user in the search graph in response to the user selection. Means for indicating a plurality of partial hypotheses drawn from the part.

  According to a second aspect of the present invention, a translation device translates a text input device according to any of the text input devices described above and the sentence output by the text input device into a translated sentence in a target language. And a statistical machine translation device for displaying the translated sentence on the touch panel display.

  Preferably, the translation device further includes means for receiving a predefined command from the touch panel display, and back-translates the translated sentence into the predetermined language in response to the predefined command, and the back-translated Means for displaying a sentence on the touch panel display.

  According to a third aspect of the present invention, a method for inputting text in relation to a touch panel display includes a step of displaying a set of icons on the touch panel display, and the icon displayed on the touch panel display by a user. Receiving one or more sequences of input, displaying the sequence of icons input in the receiving step on the touch panel display, and translating the sequence of icons into a complete sentence in a predetermined language Including the steps of:

  According to a fourth aspect of the present invention, a computer program connects a computer having a touch panel display to the touch panel display and displays a set of icons on the touch panel display, and the touch panel display. Input receiving means for receiving an input of one or more sequences of the icons displayed on the touch panel display by the user, and connected to the touch panel display, and using the input receiving means, the user And a means for displaying the sequence of icons input by the touch panel display and a translation unit for translating the sequence of icons into a complete sentence in a predetermined language.

1 shows a prior art first communication channel 40; FIG. FIG. 2 shows a second communication channel 50 of the prior art. It is a figure which shows the communication channel 60 of this invention. It is a figure which shows the example of the picture in a picture book. It is a figure which shows a mode that a user operates the picture book shown in FIG. It is a figure which shows the user interface (User Interface: UI) screen 100 of the icon-based translation application of 1 embodiment of this invention. It is a figure which shows the toggle of translation and its reverse translation. It is a block diagram of the icon-based translation system of 1st Embodiment of this invention. It is a functional block diagram of the translation server of the translation system shown in FIG. It is a flowchart which shows the flow of control of the program which trains a source language-source language (source-to-source: S-TO-S) SMT engine. It is a figure which shows the example of alignment of an icon base source sentence and a complete type source sentence. It is a flowchart which shows the flow of control of the icon base translation application performed with the portable apparatus according to this invention. It is a figure which shows the detailed control flow of an icon addition routine. It is a figure which shows the detailed control flow of the routine which produces | generates a complete type source sentence from the sequence of icon text. It is a figure which shows the detailed control flow of the routine which improves a complete type | mold source sentence. It is a figure which shows the flow of the detailed control of translation operation. It is a graph which shows the range of the unexpected data with respect to the size of the set of icons when there is a human operation. It is a graph which shows the range of the un-encountered data with respect to the size of the set of icons when there is no human operation. It is a figure which shows the source sentence production | generation quality using the SMT approach with a tabular form with respect to the hidden n-gram method for recovering the missing invariant. FIG. 3 is a perspective view of a mobile device on which an icon-based translation application is executed. It is a block diagram of the apparatus shown in FIG. It is a front view of the computer which functions as a translation server of one embodiment of this invention. It is a block diagram of the computer shown in FIG.

  The present invention will be described with reference to the above-mentioned drawings. In the following description and drawings, the same reference numerals are assigned to the same components. Therefore, detailed description thereof will not be repeated. Since their functions are also the same, detailed description thereof will not be repeated.

1. External Language This embodiment uses an external language (Extralingua) that is visible to any of the interlocutors. Referring to FIG. 3, the user can interactively operate the external language 62 with the help of the following three MT systems. The first is between the external language 62 and the source language 64, the second is between the source language 64 and the target language 66, and the third is between the external language 62 and the target language 66. If such an external language exists, you may be wondering why MT is necessary in the first place. In fact, this is important. There is no common language that can communicate with each other. So far, only the method of filling this gap using a single MT channel as shown in FIGS. 1 and 2 has been considered. However, in many situations, the interlocutor has communication means such as images, symbols, and gestures, and often uses these means when other means do not work. Although another mode of this communication can be employed in parallel independently of the MT channel, the idea of the present invention is to seek to couple the second communication channel directly to the machine translation system. It is.

  This approach has many advantages. First, above all, the quality of communication between users is improved. By adopting the external language 62, the user can communicate with each other through two different types of channels. Since the MT output cannot be expected to be complete, a second independent communication mode that reinforces or contradicts this will lead to better mutual understanding. Second, this proposal also improves the quality of MT. This is because user input processing can be supported by a user interface to an external language, thereby making the input regular and reducing the number of unexpected entries.

  The concept of external language 62 presents two major problems.

-What form should an external language take?
-How can this external language be effectively integrated into the MT system?
In principle, the external language can be any language-type communication medium that can be shared by interlocutors. It may be images, symbols, official or international natural languages. In this embodiment, in order to show that such an external language approach is effective even with a simple abstract language, an icon is first adopted as an example.

2. Icon-based translation system Picture-based translation aids have been used in the form of paper books and are now integrated into handheld devices, but as shown in the prior art Non-Patent Document 1 or 2, It is not combined with a machine translation system. Briefly, in this embodiment, the user taps the picture icon that appears on the touch screen, just like a picture-based translation aid. The system automatically generates possible sentences from these selected sequences of icons and feeds them to machine translation so that the translation results can be displayed. Unlike a picture book, the sequence of icons is maintained on the display and can be viewed by the user and manipulated as needed. When input is complete, the system generates a complete sentence in the source language, either automatically or on demand, which is then translated by machine translation software and displayed on the screen with a sequence of icons.

  As will be described later, in this embodiment, the sequence of icons is input on the touch screen of the portable device. A remote translation server generates a complete source sentence search tree. Using the search tree, the user refines the complete source sentence on the mobile device. This improved source sentence is sent to the translation server. The translation is sent back to the mobile device and presented to the user.

  In this embodiment, communicating through this user interface allows the user to combine much more pictures than can be done with a picture book designed for combinations only from the same book spread. Because of this, the application can achieve a richer expression than a book. Machine translation systems can provide detailed and accurate translations, supported by a picture-based mode, which not only provides a quicker way to convey basic concepts, It also provides a “second opinion” of machine translation output that allows the user to retry the sentence and avoid misunderstandings.

  Such a system is very advantageous when applied to MT on portable devices. The user's input on these form devices will be troublesome in the case of text input, and there will be many errors in voice input. As a result, users have come to prefer simpler and more reliable interlanguage communication methods such as picture book translation support tools, which are now widely used both in paper and electronic translation support applications. It has become like this.

2.1 User Interface A diagram of the complete user interface of an application running on a mobile device is shown in FIG. Simply put, this application allows the user to enter what they want to represent in a sequence of bilingual annotated icons. This is essentially the same idea as a picture book.

  Referring to FIG. 6, the UI of this embodiment includes a UI language toggle button 110 that toggles the language between the source language and the target language, a category button 112 that allows the user to select a desired icon category, Of the icon categories, a subcategory button 114 that allows the user to select a desired subcategory, an icon sort button 116 that rearranges the order of the icons, and each holds a pair of a source language phrase and a target language phrase, and the meaning of the phrase is graphically illustrated And a set of icons 118 with icons represented by. In this embodiment, the text on the icon is not essential. If the figure is well designed, the text may be omitted from the icon. Each icon 118 is realized as an object in a program for realizing this system. Each icon object has its own graphic data for display and a source language and target language word or phrase associated with the icon. The icon data may be a part of the program and is stored in a storage device such as a hard disk drive.

  The UI further triggers an icon sequence field 120 for an input icon sequence to display a sequence of icons (eg, icons 140, 142, and 144) in the order of input and generation of a complete source sentence from the icon sequence. A source generation button 126; a source sentence field 122 for displaying a complete source sentence generated from a sequence of icons; a translation sentence field 130 for displaying a translation sentence of the source sentence in the source sentence field 122; and a source sentence A translation button 124 that activates translation of the complete source sentence shown in the field 122 and causes the translated sentence to appear in the translated sentence field 130, an icon sequence field 120, a source sentence field 122, and a translated sentence field 130 Clear to clear It includes a Abotan 128, a check button 132 to start the other the language translation of the translation field translation shown in 130. When the user selects one of the icons 118, the selected icon is added to the sequence of icons in the icon sequence field 120. This means that not only is a sequence of graphical representations of icons generated, but by concatenating words or phrases according to the icon sequence, a sequence of words or phrases associated with the icon is also generated.

  The source generation button 126 is not always necessary. For example, if two or more icons are selected, the complete source text may be estimated automatically. Or, depending on the source language, for example, the estimation may be automatically triggered by a specific class of words or phrases.

  The user can switch the user interface to his / her language by pressing the UI language toggle button 110. The translation process proceeds as follows.

  (1) When the user presses any of the category buttons 112, the category of the concept that the user wants to express is selected.

  (2) The user selects one of the subcategories by pressing one of the subcategory buttons 114.

  (3) When the user selects an icon from among the icons 118, the icon is added to the icon sequence in the icon sequence field 120.

  (3a) If the user wants to continue input, return to (1) and select another icon for the sequence.

  (3b) If the icon sequence is completed, the source generation button 126 is pressed, and the process proceeds to step (4). The icon sequence in icon sequence field 120 is translated into a complete source sentence and the source language sentence appears in source sentence field 122. This translation is a translation from the source language to the source language. The complete source language is inferred at the remote server by the phrase-based source language-source language SMT, described below, based on a sequence of icons or more specifically on a sequence of icon text.

  (4) The user improves the source sentence by operating the system interactively (described later). The user can refine the source sentence by selecting any one of the icons in the icon sequence field 120.

  (5) If the user is satisfied with the complete source sentence, the user clicks the translation button 124. This causes the SMT system in the remote server to translate the sentence in the source sentence field 122 into the target language.

  (6) A translated sentence appears in the translated sentence field 130.

  Referring to FIG. 7, once the translation is completed, pressing the check button 132 indicates reverse translation of the target sentence in the translated sentence field 130 into the source language, and the user can verify the translation. When the check button 132 is pressed again, the reverse translation is replaced with the translation.

2.2 System Architecture Referring to FIG. 8, translation system 150 includes portable device 152 and server computer 154 connected by network 156. The portable device 152 is connected to a server computer 154 via a network 156, and the server computer 154 incorporates three different machine translation server processes. The first server process translates the icon sequence into a complete source sentence, and returns a search process search graph to the UI application running on the mobile device 152. This processing is realized by statistical machine translation (SMT), and in this embodiment, this processing is called a source language-source language SMT. The other two server processes perform translation from the source language to the target language and reverse translation from the target language to the source language. These processes are realized by the latest SMT and are called source language-target language SMT and target language-source language SMT, respectively.

  By toggling the UI language, two users 160 and 162 can both use the UI application on the mobile device 152. Therefore, as shown in FIG. 9, two source languages—source languages SMT 174 and 176 are built in the computer 154. If this system is between Japanese and English, SMT 174 is a day-to-day (from Japanese to Japanese) SMT, with Japanese icon text sequences being fully sourced in Japanese SMT 176 is an English-English (English to English) SMT 176 that translates an English icon text sequence into a full English source sentence.

  Therefore, in this embodiment, the server computer 154 is a network interface card that provides a connection to the network 156 for the day-day SMT 174, the English-English SMT 176, the Japanese-English SMT 178, the English-Japanese SMT 180, and the server computer 154. (Network Interface Card: NIC) 170 and requests from a remote portable device (not shown) such as portable device 152 of FIG. 8 are sent to Japanese-Japanese SMT174, English-English SMT176, Japanese-English SMT178, and English-Japanese SMT180. And a request distribution process 172 that returns a response to the portable device.

3. Conversion from an external language to a source sentence 3.1 Machine translation The process of converting an external language into a complete source sentence is very similar to the transliteration generation process, which is a one-way constraint in word reordering. This can be done using the phrase-based statistical machine translation system (SMT) used (Finch and Sumita, 2008; Rama and Gali, 2009). Again, a similar approach was taken, but the Bayesian co-segmentation approach (described in the next section) was used to derive the phrase table for the SMT system.

  In order to train the SMT system to translate the icon text sequence into a complete source sentence, a training corpus is generated by word deletion. In this embodiment and experiment, Japanese was used as the source language. That is, as shown in FIG. 10, a Japanese corpus 200 is prepared. The Japanese corpus 200 is subjected to morphological analysis using a morphological analysis tool such as publicly available MeCab (Kudo, 2008). A set of part-of-speech (POS) tags representing the type of content words (eg, nouns, verbs, adjectives, etc.) that will be represented by icons in this system are collected by hand and the rest is left at step 206 The word classes (invariants, formal nouns, auxiliary verbs) are deleted from the source sentence. In addition, all the lexemes are reduced to the headword in this step. This process results in a bilingual (from Japanese to Japanese) training corpus 210, which is a sequence of source-side content words (stem shape) indicating the external language and a target-side, complete type. It consists of a source sentence. The source-side content words of the training corpus 210 are limited in vocabulary, while the complete source word on the target sentence side consists of a larger vocabulary.

  In the experiments described below, this model was integrated with a log-linear framework (Och and Ney, 2002) using a phrase-based machine translation decoder (day-day SMT174). Discovery and extraction of phrase pairs in the training corpus 210 was performed using a Bayes bilingual alignment unit (Finch and Sumita, 2010). In this processing, a 5-gram language model constructed by Knesser-Ney smoothing was used. The system uses the minimum error rate training (MERT) procedure (Och) on the BLEU score (Papineni et al., 2001) on the development data set aside to optimize the weight of the log-linear model. , 2003) and was trained in a standard way.

  The machine translation system was trained on approximately 700,000 bilingual sentence pairs consisting of expressions typically found in travel conversation collections. This is a very limited domain and the sentences in this domain tend to be very short (on average 7 words on the English side of the corpus), so translation was very easy. The machine translation system is the latest system, and as a result of being applied only to short sentences of limited domains, high-quality translation was possible.

3.2 Bayes co-segmentation The central part of all phrase-based statistical machine translation systems is the phrase table. This table is the basic set of components used to build a translation.

  Creation of a phrase table during the normal training process of a phrase-based SMT system often involves a word alignment step using GIZA ++ (Och and Ney, 2003) followed by heuristics (eg, MOSES (Koehn et al., 2007) Phrase pair extraction step using toolkit grow-diag-final-and). This approach works very well in practice, but is based on a maximum likelihood method that is asymmetric between the source and target and has a high risk of overlearning the data.

  The model used for co-segmentation here is a Dirichlet process model similar to Finch and Sumita, 2010. The reason for using the Bayesian approach here not only showed that the results were more effective than using GIZA ++ / MOSES heuristics (Finch and Sumita, 2010) for unidirectionally alignable sequences. This results in a consistent, single bilingual segmentation of the corpus. This consistency is considered a very desirable property to build this model. This is because this system generates a natural language only by combining these phrase pairs.

  FIG. 11 illustrates the co-segmentation process. Referring to FIG. 11, the sequence of Japanese icon text 220 is aligned with complete Japanese sentence 222. The sequence of Japanese icon text 220 is considered an external language sentence, while the Japanese sentence 222 is a Japanese sentence. Therefore, these two sentences or word sequences can be aligned as in the case of English sentences and Japanese sentences. In the example of FIG. 11, the icon texts 230 (taxi), 232 (restaurant) and 234 (go) are aligned with the phrases 240 (by taxi), 242 (to the restaurant) and 244 (I want to go).

3.3 User Operations Referring to FIG. 12, the main program for realizing the UI screen 100 of the icon-based translation application shown in FIG. 6 has the following control structure. The program begins at step 260 where system initialization is performed. In this step, a memory location is assigned to the program, a base address for executing the program is determined, the memory location is initialized, and the screen is initialized.

  In the next step 262, the system waits for user input. When a user input is detected, the system determines which button or tab has been touched by the user and branches control to steps 264-282.

  When the user touches UI language toggle button 110 (FIG. 6), the flow of control proceeds to step 264 where the UI language toggles between Japanese and English in this embodiment, after which control proceeds to step 262. Return to. When the user touches one of the category button 112 and the subcategory button 114, the corresponding icon set is read from an icon storage (not shown) and arranged on the touch screen, and control returns to step 262. If the user touches one of the icon sort buttons 116, control proceeds to step 270 where the icons 118 are rearranged in the selected order and control returns to step 262. These steps are considered normal processing and can be easily realized by those skilled in the art. Therefore, the details of these steps are not described here.

  If the user touches one of the icons 118, control proceeds to step 272. Referring to FIG. 13, step 272 includes a step 300 for adding an icon to the end of the icon sequence for the icon in icon sequence field 120, a step 302 for adding icon text to the end of the text sequence (not shown), And updating the icon sequence field 120 to exit 304 from this routine.

  If the user touches source generation button 126, control proceeds to step 274. Referring to FIG. 14, step 274 sends an icon text sequence corresponding to the icon sequence in icon sequence field 120 to day-day SMT 174 (see FIG. 9), and a search graph is sent back from day-day SMT 174. Step 322 for waiting for the search, step 324 (not shown) for storing the search graph in the storage unit, step 326 for searching the route with the highest likelihood in the search graph, and the maximum likelihood found in step 326 Storing 328 the source text corresponding to the high path in the storage for source sentence field 122 and updating 330 source sentence field 122; After step 330, control returns to step 262 (FIG. 12).

  The output of a machine translation system is a search graph that relates to adding bilingual phrase pairs used to build a complete source sentence hypothesis to the end of the preceding partial hypothesis. It is expressed with the model probability. This graph is provided from the machine translation system to the user interface client, which uses the information in the graphic to give the user satisfactory results without having to re-decode the input continuously during the manipulation process.

  In this system, following the generation process from the external language, as a result of the process in step 274 described above, the hypothesis with the highest likelihood of the complete source sentence is presented to the user for the given input sentence. If this sentence does not match the user's intended meaning, the user can improve the generated sentence by interactively manipulating the icon sequence. This is done in step 276 of FIG.

  The user may tap any icon in the icon sequence displayed in the icon sequence field 120. Referring to FIG. 12, control proceeds to step 276.

  Referring to FIG. 15, step 276 includes step 340, in which the user interface examines the search graph and includes to the user translation of the selected icon and to the translation of the selected icon. Present an n-best list of partial translation hypotheses. In response, the user can select one of the partial translation hypotheses. In this embodiment, text cannot be entered directly into the system, but it is possible, and this is probably necessary in a real world system. The consideration to be paid by limiting the input in this way is expressiveness, so in the following section we will consider this system experimentally with this in mind.

  Step 276 further includes generating a new complete source text 342 based on the user input at step 340 and storing the new complete source text in the storage for source sentence field 122. 344 and a step 346 of updating the source sentence field 122. After step 346, control returns to step 262.

  Referring again to FIG. 12, if the user touches the clear button 128 (see FIG. 6), control proceeds to step 278 where the icon sequence field 120, the source sentence field 122 and the translated sentence field 130 are cleared, Control returns to step 262.

  If the user touches translation button 124 (see FIG. 6), control proceeds to step 280. Referring to FIG. 16, step 280 sends a complete source sentence in source sentence field 122 to Japanese-English SMT 178 shown in FIG. 9, and waits for a response (translation) from Japanese-English SMT 178 step 362. And a step 366 for receiving the translated sentence and storing it in the storage unit for the translated sentence field 130 and a step 368 for updating the translated sentence field 130, and the control returns to step 262 (see FIG. 12).

  When the user touches check button 132 (see FIG. 6), control proceeds to step 282. In this embodiment, in step 282, the text shown in the translated text field 130 depends on whether the text shown in the translated text field 130 is a Japanese text (reverse translation) or an English translation. Are sent to Japanese-English SMT178 or English-Japanese SMT180. A translation or reverse translation is sent back from the Japanese-English SMT 178 or the English-Japanese SMT 180, shown in the translated text field 130, and control returns to step 262.

  While the above embodiment has been described with reference to translation from a source language (eg, Japanese) to a target language (eg, English), the above described device translates backwards, ie, from English to Japanese. It will be apparent to those skilled in the art that it can also be used. Furthermore, even if the server computer 154 (see FIG. 9) does not have the English-English SMT 176, it still functions as an icon-based translation device from Japanese to English.

4). Evaluation 4.1 Expressiveness One of the main concerns about icon-driven user input is its expressive power within the domain. This is because the sentence must be expressed using only icons that can be used on the portable device. Therefore, the system was evaluated to determine the ratio of sentences in the domain that the UI screen 100 can represent. For this purpose, a sample of 100 sentences is taken from a set of data extracted from the same sample as the training corpus 210 (see FIG. 10), and an icon-driven interface and its Judgment was made as to whether a semantically equivalent form can be generated using source sentence generation processing. Development of the current prototype is not yet sufficient, so it does not include a set of icons that deal with numerical representations (price, phone number, date, time, etc.). For this reason, the sentence including these was excluded from the evaluation group (the size of the evaluation group after removing the sentence including the numerical expression was 100 sentences). However, the treatment of numerical expressions is relatively straightforward, so there seems to be no difficulty in adding this function in the future. The icon set used in the evaluation is the 2010 content words most often used on the English side of the training corpus used to train the Japanese-English SMT178 and English-Japanese SMT180, and more than 28 times in the training corpus It is a content word that has appeared. This value was chosen so that the number of icons in the user interface would be around 2000, which is an approximation of the number of icons needed to build a useful application in the real world.

  It was found that 74% of the sentences in the evaluation data can generate semantically equivalent sentences. This is shown in the graph 380 of FIG. 17 along with statistics (based on 30 sample sentences taken randomly from 100 evaluation sentences) for an example where a smaller number of icons was used. Given the simplification of the user interface, this range of defense seems to be high. A comparison of the two methods when there is no human correction to the output is shown in FIG. Again, the MT method (shown by graph 390) showed higher performance than the n-gram method (shown by graph 392). In combination with this and the fact that this method can generally be expected to be applicable to all languages, it would be no exaggeration to say that this method is a good generation technique for the purpose of the present invention.

4.2 Efficiency The number of key presses required to enter these sentences using an external language is the number of times that would have been required if entered using the device's text input interface. A comparative study was conducted. Here we assume that it is necessary to press the key three times to select each of the icons, but in many cases icons from the same icon subcategory can be used, so such icons press the key once. Just do it. Therefore, the estimation here represents the upper limit of the number of times of pressing the required key. The time it takes to press a key once is not the same for icon input and text input, but this was not measured in the experiment. Furthermore, the effect of user input errors in the input process was not measured. Included in the measurement was the additional action of the user pressing the necessary keys to interact with the system so that the source sentence had the intended meaning.

  In this experiment, it was found that the external language input system according to this embodiment requires only 57% of the number of times of pressing the key of the text input method. The number of times of pressing the key is 1,650 times for text input, and 941 times for the input method of the external language. This means that an external language input system can be used as an efficient text input system.

4.3 Quality of source sentence generation In the first version of the system of the present invention, a simple language model based approach was used to recover missing function words. This approach is very regular in shape and uses invariants in close proximity to the content word to indicate function, and is appropriate for Japanese that does not contain a qualifier. Bigrams containing a pair of content words that have functional words on the left or right are extracted from the training data, and these bigrams are inserted into the corresponding content word locations in the generation process. The model of the present invention scores the set of hypotheses resulting from all of these possible substitutions with a 5-gram language model and selects the hypothesis with the highest language model score as the best candidate. The beam search method was used to reduce the search space to a manageable size.

  The BLEU score (Papineni et al., 2001) and NIST score (Doddington, 2002), which are common methods for measuring machine translation performance based on n-gram accuracy, both of the quality of the source generation component of the system of the present invention. Was evaluated using version 13a of the NIST machine translation evaluation score script. Removed sentences containing numerical expressions from the evaluation data. With this removal, the initial set of 510 Japanese sentences was reduced to 455 sentences. These sentences were supplied to the MeCab morphological analyzer and words that were not related to the icons of the inventive system were removed. The experimental results are shown in table form in FIG.

  The scores for hidden n-grams (headworded) and SMT generation are derived from exactly the same input. The reason why the score of the hidden n-gram is low in this example is that an inflected form cannot be generated. Therefore, in the second experiment, a hidden n-gram can be generated from the correct surface layer format of the utilized word. This has given this model an unfair advantage over the SMT generation model that needs to be predicted for use. Nevertheless, the source sentence generated by the SMT process had a higher score for both evaluation criteria used.

5. Realization by Computer As is apparent from the above description of the embodiment of the present invention, the portable device 152 and the server computer 154 can be realized by computer hardware. Since the mobile device 152 is carried by the user, its resources are limited, but the server computer 154 has no such limitation. Therefore, their configurations are different from each other.

  Referring to FIG. 20 and FIG. 21, the portable device 152 of this embodiment is a tablet computer, and includes a touch panel 418 formed on a liquid crystal display (LDC) 420, and the user Can input text or touch any button displayed on the LCD 420. The portable device 152 further includes a CPU (Central Processing Unit) 410, a flash ROM (Read-Only Memory) 412, a RAM (Random Access Memory) 414, and an input / output interface. (I / F) 416, a network interface 422, and a hardware button 424 are included. All these components are connected to the bus 426. The CPU 410 can access any of these via the bus 426.

  Referring to FIG. 22, a computer system 430 that provides hardware resources to the server computer 154 of this embodiment includes a computer 440. Computer system 430 further includes a monitor 442, keyboard 446 and mouse 448, all connected to computer 440. A computer 440 reads data from a DVD (Digital Versatile Disc) 462 or writes data to the DVD 462, and a memory port for reading data from or writing data to the removable memory 464. 452.

  Referring to FIG. 23, computer 440 further includes a CPU 456 and a bus 466 connected to CPU 456. The DVD drive 450 and the memory port 452 are connected to the bus 466 and can access the CPU 456 via the bus 466. The computer 440 further includes a ROM 458 that stores a boot program for the computer 440, a RAM 460 that provides a work area used by the CPU 456 and a storage area for programs executed by the CPU 456, and corpus data, translation models, and languages used by the SMT. And a hard disk drive (HDD) 454 that stores the model and other data used between the training process and the translation process.

  The computer 440 further includes a NIC 170, which is also shown in FIG. 9, but is connected to the bus 466 to the computer 440, typically a LAN (Local Area Network) (not shown) and wireless communication. Give connection to.

  The software that implements the server computer 154 of the above-described embodiment may be distributed in the form of an object code recorded on a recording medium such as a DVD 462, or stored in the removable memory 464 and stored in the DVD drive 450 or the memory port 452. Or the like, may be provided to the computer 440 via a reading device such as, and stored in the HDD 454. When CPU 456 executes the program, the program is read from HDD 454 and stored in RAM 460. An instruction is fetched by the CPU 456 from an address specified by a program counter (not shown) of the CPU 456 and processed by the CPU 456. The CPU 456 reads the data to be processed from the register in the CPU 456, the RAM 460 or the HDD 454 to be processed from the address specified by the instruction operand, and the result of the processing is the register in the CPU 456, which is also specified by the instruction operand. The data is stored in the RAM 460 or the HDD 454.

  Since the general operation of computer system 430 is well known, its details are not described here.

  As for the software distribution method, it is not necessarily fixed to the recording medium. For example, the software may be transmitted from another system to the computer 440 via the network and NIC 170. A part of the software may be stored in the HDD 454, and the remaining part of the software may be taken from the network into the HDD 454 and integrated at the time of execution.

  Modern computers typically utilize functions provided by a computer operating system (OS) to perform functions in a controlled manner according to the desired purpose. Therefore, a program that does not include a function provided by the OS or a third party and that specifies only a combination of instructions that execute a general function also has a control structure that achieves a desired object as a whole. Included in the range.

6). CONCLUSION The present invention provides a new machine translation input framework that uses an abstract language called an external language to clarify the semantic content that both parties are trying to convey during communication. This external language not only acts as a transparent and reliable second opinion for the machine translation system, but can also be used as a primary user input method that fits well with applications running on mobile devices.

  An embodiment of the present invention, a specific example of a system for carrying out the idea of the invention, and some experimental explanations of its characteristics have been presented. This embodiment combines the ideas of two different translation methods for travelers. Picture book and statistical machine translation. This approach provides all the benefits of a simple but powerful representation of a picture book, while at the same time generating a natural language in the target language that can clearly express the user's meaning in the source sentence. According to the inventor's evaluation, the icon-based input system covers about 74% of sentences in this basic travel expression corpus domain, and also inputs expressions to be translated for text-only input methods. The number of key presses required for is significantly reduced.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

References Doddington. 2002. Automatic evaluation of machine translation quality using N-gram co-occurrence statistics. Proceedings of the HLT Conference, San Diego, California.

  Andrew Finch and Eiichiro Sumita. 2008. Phrase-based machine translation. Proceedings of the 3rd NLP International Joint Conference, Volume 1. Hyderabad, India.

  Andrew Finch and Eiichiro Sumita. 2010. A bilingual segmented Bayesian model for transliteration. IWSLT Proceedings, Paris, France.

  Philip Koehn et al., 2007. Moses: An open source toolkit for statistical machine translation. ACL 2007: Proceedings of Demos and Paper Sessions, pp. 177-180, Prague, Czech Republic, June.

  T.A. Kudo. 2008. MeCab. [Online] Available at http://mecab.sourceforge.net/

  Franz Josef Och and Hermann Ney. 2002. Discriminative training and maximum entropy model for statistical machine translation. Proceedings of the 40th Annual Meeting of the Computer Linguistic Society (ACL2002), pp. 295-302.

  Franz Josef Och and Hermann Ney 2003. Systematic comparison of various statistical alignment models. Computer language, 29 (1): 19-51.

  Kishore Papineni et al., 2001. Bleu: Automatic machine translation evaluation method. ACL'02: Proceedings of the 40th Annual Meeting of the Computer Linguistic Society, 311-318, Morristown, NJ, USA, Computer Linguistic Society.

  Taraka Rama and Karthik Gali. 2009. Machine translation model as a phrase-based statistical machine translation task. NEWS'09: Specific Expression Workshop: Transliteration Sharing Task, pages 124-127, Morristown, NJ, USA, Computer Linguistic Society.

40, 50, 60 Communication channels 42, 52 and 64 Source language 44 Intermediate language 46, 56 and 66 Target language 62 External language 80 and 82 Picture 100 UI screen 110 UI language toggle button 112 Category button 114 Sub category button 116 Icon sort button 118 , 140, 142, and 144 Icon 120 Icon sequence field 122 Source sentence field 124 Translation button 126 Source generation button 128 Clear button 130 Translation sentence field 132 Check button 150 Translation system 152 Portable device 154 Server computer 156 Network

Claims (10)

A text input device used in connection with a touch panel display,
Means connected to the touch panel display and displaying a set of icons on the touch panel display;
Input receiving means connected to the touch panel display and receiving an input of one or more sequences of the icons displayed on the touch panel display by a user;
Means for displaying on the touch panel display a sequence of icons connected to the touch panel display and input by the user using the input receiving means;
Translation means for translating the sequence of icons into a complete sentence in a predetermined language. The text input device according to claim 1, further comprising a unit connected to the touch panel display and the translation unit, and displaying the complete sentence translated by the translation unit on the touch panel display. Storage means for storing phrases of the predetermined language, each of which is associated with the icon;
Means for connecting to said storage means and said input receiving means, and for generating a text in an external language by concatenating phrases extracted from said storage means for each of said icons;
The translation means includes a phrase-based statistical machine translation unit that is statistically trained to translate the text in the external language into a sentence in the predetermined language in a corpus of sentence pairs, each of the pairs A sequence of phrases including words in a first vocabulary of the predetermined language and a sentence of the predetermined language including words in a second vocabulary greater than the first vocabulary. Item 3. The text input device according to Item 2. The text input device according to claim 3, wherein the first vocabulary includes content words of the predetermined language. The statistical machine translation unit outputs a search graph that represents the bilingual phrase pairs used to build the sentence hypothesis along with the model probabilities associated with adding these phrase pairs to the end of the preceding partial hypothesis Is what
The text input device according to claim 3 or 4, wherein the sentence is a hypothesis corresponding to a route having the highest likelihood in the search graph. Means for receiving selection by one user of the icons in the icon sequence;
Means for indicating a plurality of partial hypotheses drawn from a portion of the search graph up to a node corresponding to the icon selected by the user in response to the user selection. The text input device described. A text input device according to any one of claims 1 to 6,
A statistical machine translation device that translates the sentence output by the text input device into a translation in a target language;
Means for displaying the translated sentence on the touch panel display. Means for receiving a predefined command from the touch panel display;
The translation device according to claim 7, further comprising means for reverse-translating the translated sentence into the predetermined language in response to the predetermined command and displaying the reverse-translated sentence on the touch panel display. . A method of entering text related to a touch panel display,
Displaying a set of icons on the touch panel display;
Receiving a user input of one or more sequences of the icons displayed on the touch panel display;
Displaying a sequence of icons input in the receiving step on the touch panel display;
Translating the sequence of icons into a complete sentence in a predetermined language. A computer program comprising a touch panel display,
Means connected to the touch panel display and displaying a set of icons on the touch panel display;
Input receiving means connected to the touch panel display and receiving an input of one or more sequences of the icons displayed on the touch panel display by a user;
Means for displaying on the touch panel display a sequence of icons connected to the touch panel display and input by the user using the input receiving means;
A computer program that functions as translation means for translating the sequence of icons into a complete sentence in a predetermined language.

Images