JP2011053922A - Code-based multilingual interpretation device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a code-based multilingual interpretation device and method capable of rapidly interpreting into multiple languages. <P>SOLUTION: The code-based multilingual interpretation device and method capable of rapidly interpreting multiple languages by defining words and set phrases for communication common to languages as codes and then mapping the languages to the codes are provided. The device comprises a multiple language standard database in which words and set phrases of one's own language are mapped to the words and set phrases of other languages based on the codes; a one's own language speech recognition device for recognizing the speech of the one's own language; an interpretation search engine that receives the one's own language recognized by the one's own language speech recognition device, retrieves the corresponding codes, searches for another language corresponding to the codes retrieved, and outputs the result; a word order converter which, if the other language received from the interpretation search engine is a set phrase, directly outputs it and which, if the language received is a plurality of words, converts the word order of the one's own language into the word order of the other language and outputs the sentence of the language converted; a speech output means; and a display means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a multilingual interpreting apparatus and method, and more specifically, a word for communication and a simple idiom common to each language are defined by a code (index), and then the language of each country is mapped to the code. The present invention relates to a code-based multilingual interpreting apparatus and method capable of quickly translating multilingual languages.

  Recently, overseas travel has become frequent due to the development of transportation and improved economic standards, and the number of foreign visitors has also increased. However, if you are not good at foreign languages, you often cannot communicate when you travel abroad or meet foreigners. In order to solve such problems, the present applicant has filed a patent application with the Korean Patent Office for a multilingual interpreting / learning apparatus and method using speech recognition, and has been registered as Korean Patent No. 10-0593589. . According to the technique of the patent registered No. 10-0593589, after constructing multilingual content divided into various situations (episodes) and cases (units), interpretation is processed based on the situations and cases.

Korean Registered Patent No. 10-0593589

  In the conventional multilingual interpreter system, the contents were constructed in advance for each situation and case, and then the contents according to the situation and case were searched and converted into the language of the partner country, so the situation and case were accurate. If it does not match, there is a problem that interpretation is not possible and natural interpretation is difficult, and the procedure for searching for situations and cases is complicated and time is delayed.

  The present invention is proposed in order to solve the above-mentioned problems, and the object of the present invention is to define words and simple idioms for communication that are common to the languages of each country after code. A code-based multilingual interpreter that builds a database for each country's language based on the code, recognizes the native language through speech recognition, and then searches for other languages based on the code to quickly translate to other languages It is to provide an apparatus and method.

  In order to achieve the above-described object, the apparatus of the present invention inputs a multilingual standard database in which native language words and idiomatic sentences are mapped to other language words and idiomatic sentences based on a code, via a microphone. A native language speech recognizer that recognizes the recognized native language, and retrieves the corresponding native language recognized by the native language speech recognizer, searches for a corresponding code, and searches for another language corresponding to the retrieved code. Interpretation search engine to be output, and if the other language received from the interpreter search engine is idiomatic, it is output as it is. A word order converter for outputting a sentence in Japanese, a TTS (Text to Speech) for converting another language text of the word order converter into a voice signal, and a voice output means for amplifying the voice signal of the TTS and outputting it to a speaker The above Display means for displaying the text output of the word order converter in characters.

  In order to achieve the above-mentioned object, the method of the present invention includes a step of defining a code with words and idioms for communication, constructing a multilingual standard database based on the code, A step of setting another language, a step of receiving voice of the set native language and recognizing the native language, a step of searching for a code corresponding to the recognized native language, and Searching the mapped foreign language in the native language order, converting the foreign language arranged in the native language order into the language of the foreign language to form a foreign language text sentence, and the foreign language text sentence Converting to a foreign language audio signal.

  According to the present invention, after defining words and simple idioms for communication common to the languages of each country using codes based on sign language (sign language codes), a database for each language is constructed based on the sign language codes. After recognizing the native language by voice recognition, it is possible to search for the other language based on the sign language code and quickly and accurately interpret the native language into the other language. In addition, according to the present invention, the native language can be simultaneously translated into N languages, and voices in other languages can be immediately output using the TTS technology together with the text.

It is the schematic which showed the concept which builds a multilingual standard database by this invention. [BRIEF DESCRIPTION OF THE DRAWINGS] It is a figure which shows 1st Embodiment of the standard code base multilingual interpretation apparatus by this invention. It is a flowchart which shows the operation | movement procedure of 1st Embodiment by this invention. It is a figure which shows 2nd Embodiment of the standard code base multilingual interpreting apparatus by this invention. It is a flowchart which shows the operation | movement procedure of 2nd Embodiment by this invention.

  The technical problems achieved by the present invention and the practice of the present invention will become clearer by the preferred embodiments of the present invention described below. The following embodiments are merely examples for explaining the present invention and are not intended to limit the scope of the present invention.

  FIG. 1 is a schematic diagram illustrating the concept of constructing a multilingual standard database according to the present invention.

  The present invention generates a standard code by coding common words and idiomatic sentences for communication, and then maps the national language to the standard code and uses the standard code as a reference for the multilingual standards as shown in Table 1 below. Build a database. In the embodiment of the present invention, the sign (14) is defined on the basis of the sign language, focusing on the fact that the sign language of each country is the optimum means necessary for communication, but simple sentences frequently used are also defined by the code. To do. In this way, the sign language code (14) defined by coding the sign language word of each country and the frequently used simple idiomatic sentence according to the present invention so as to indicate a common meaning is defined.

に該当する意味を手話コード(14)「00000001」で定義した後、この手話コード(14)と同じ意味の英語である「ｍｏｕｎｔａｉｎ」をマッピングさせ、日本語の「やま」をマッピングさせる。このように本発明の多国語標準データベース(16)には各国の単語と簡単な慣用文が手話コード(14)を基準にマッピングされている。 In Table 1, the first language (12-1) is Korean, the second language is English, and the nth language (12-N) is Japanese. In the present invention, Korean

After defining the meaning corresponding to the sign language code (14) “00000001”, “mountain” which is English having the same meaning as the sign language code (14) is mapped, and “yama” in Japanese is mapped. As described above, the multilingual standard database (16) of the present invention maps the words of each country and simple idioms based on the sign language code (14).

  FIG. 2 is a first embodiment of a code-based multilingual interpreting apparatus according to the present invention, which is an example implemented in the form of an independent terminal. As such a terminal, a dedicated terminal can be used, but it can be additionally installed in a portable terminal such as a mobile phone, an electronic dictionary, and a portable multimedia player (PMP).

  As shown in FIG. 2, a code-based multilingual interpreting apparatus (100) according to the present invention includes a microphone (102), an analog / digital (A / D) converter (104), and a digital / analog (D / A) converter (106). ), Amplifier (108), speaker (110), key input unit (112), speech recognizer (121), interpreter search engine (122), word order converter (123), TTS (124), multilingual standard database ( 16) and LCD (130). Here, the speech recognizer (121), the interpretation search engine (122), the word order converter (123), and the TTS (124) may be software executed by the MCU (120).

  As shown in FIG. 2, the multilingual standard database (16) is stored in a storage medium such as an HDD or a flash memory, and local language words and idiomatic sentences are mapped to other language words and idiomatic sentences based on sign language codes. Has been.

  The analog-to-digital converter (104) receives the analog voice signal from the microphone (102) and converts it into digital, and the native-language speech recognizer (121) recognizes the native language input via the microphone (102). The sentence and the word text recognized are output.

  The interpreter search engine (122) receives the native language sentence recognized by the speech recognizer (121), and if there is a sentence that matches the native language data in the multilingual standard database (16), the sign language of the sentence A code is searched, a foreign language text mapped to the sign language code is searched, and the foreign language text is output to the TTS (124) without changing the word order. Also, when there is no matching sentence, the interpreter search engine (122) searches each word constituting the sentence to find the sign language code of the word, and then searches for another language word mapped to the sign language code. Output to the word order converter (123).

  The word order converter (123) inputs sentences and words from the interpreter search engine (122) in the native language order and transmits them to the TTS (124) without converting the word order in the case of sentences, and in the case of words According to the sentence end signal, the native language word order is converted to the other language word order, and the converted foreign language sentence is output to the TTS (124).

  The TTS (124) converts the other language text of the word order converter (123) into speech data, and the digital-analog converter (124) converts the digital data into an analog signal and outputs it to the amplifier (108). The amplifier (108) receives and amplifies the analog audio signal, and then outputs it through the speaker (110).

  The LCD (130) receives the foreign language text data input from the word order converter (123) and displays the foreign language text corresponding to the native language in characters. In some cases, a sign language code can be received via the word order converter 130 and a sign language operation in another language can be output together. Further, the LCD (130) can be integrated with a key input unit function by a touch panel.

  When the power is turned on, the control unit (125) outputs an initial screen for setting a multilingual interpretation to the LCD (130) to guide the languages that can be interpreted, and via the key input unit (112). The user's operation is received and the native language and the foreign language are set. Here, the native language is a language that the user inputs by voice, and the other language is the language of the other party that the other party can hear and understand. According to the present invention, when N languages are stored in the multilingual standard database (16), simultaneous interpretation of N languages is possible. After constructing a database of three languages such as Japanese and English, the interpretation processing can be performed as shown in Table 2 below according to the setting of the native language and other languages. When the output part such as word order converter (123), TTS (124), digital-analog converter (106), amplifier (108), speaker (110) is doubled, the native language is changed to two other languages. Simultaneous interpretation is possible, and in the case of N-fold configuration, simultaneous interpretation into N languages is also possible.

  Next, the operation of the multilingual interpreting apparatus (100) configured as described above will be described as follows.

  FIG. 3 is a flowchart showing the operation procedure of the first embodiment according to the present invention.

  First, the sign language code-based multilingual interpreter (100) according to the present invention defines a sign language code with words and idioms for communication, and constructs a multilingual standard database (16) based on the sign language code (S101). ). The storage device of the sign language code-based multilingual interpreter is equipped with a multilingual standard database (16) and software for operation.

  When the power is turned on, the MCU (120) reads the software stored in the flash memory or HDD and outputs a menu screen for multilingual interpretation to the LCD (130). The menu screen has an item for setting the native language and the other language, and the native language and the other language are set using the direction keys, the enter key, and the like (S102).

  When the native language and the foreign language are set, the MCU (120) drives the speech recognizer (121) for recognizing the native language, informs the interpreter search engine (122) of the setting of the native language and the foreign language, and sets it. The word order converter (123) for converting the word order of the native language thus set is driven, and the TTS (124) for converting the set foreign language text into the foreign language speech is driven.

  After the setting is completed, when the user speaks in his / her native language, the microphone (102) converts the native language voice into an electrical signal, and the analog-digital converter (104) converts it into a digital signal and converts it into a voice recognizer (121). To communicate. The speech recognizer (121) recognizes the beginning and end of one sentence, recognizes the contents of the sentence, recognizes the words constituting the sentence, and outputs the native language text (S103).

  The interpreter search engine (122) first compares the sentence transmitted from the speech recognizer (121) with the corresponding native language data of the multilingual standard database (16), and if there is a matching idiomatic sentence, the sign language of the sentence The code is searched (S104). Then, a sentence in another language mapped to the searched sign language code is extracted and transmitted to the TTS (124) through the word order converter (123) (S111) and (S112).

  If there is no matching sentence in the sentence level search, the words constituting the sentence are searched in the same way, and the sign language code of the word is searched from the multilingual standard database (16) and then mapped to the sign language code. The foreign language word is searched and transmitted to the word order converter (123) (S105 to S107).

  The word order converter (123) converts the word order of other languages arranged in the native language word order to form a foreign language sentence, and then outputs the converted foreign language sentence text to the LCD (130) as it is In addition to displaying the text, it is transmitted to the TTS (124) to convert the foreign language text into the foreign language speech (S108 to S110).

  Therefore, according to the present invention, it is possible to search for a foreign language sentence or word corresponding to the native language from the multilingual standard database based on the sign language code, and to perform the interpretation process quickly and accurately.

  FIG. 4 shows a second embodiment of the code-based multilingual interpreting apparatus according to the present invention, which is an example implemented in a client-server structure. FIG. 5 shows the operation procedure of the second embodiment according to the present invention. It is the shown flowchart. Such a client-server interpreting apparatus is implemented by a web server and can provide an interpreting service according to the present invention to a plurality of clients via the Internet.

  As shown in FIG. 4, the multilingual interpreting service site (210) according to the present invention includes a multilingual interpreting server (212) equipped with software for a multilingual interpreting service, and a multilingual standard database (16). And an installed database server (214). The multilingual interpretation server (212) is a web application server that can be connected to a wired terminal (PC) (220) via the Internet (202) and moved via a gateway (G / W) (204). The mobile terminal (230) can be connected via the communication network (206). The wired terminal (220) is a PC or notebook computer equipped with a microphone input and speaker output function and equipped with a web browser, and the portable terminal (230) is a notebook computer or web book that can be connected to a wireless LAN or mobile A smartphone, a mobile phone, and the like connected via a communication network.

  As shown in FIG. 5, the multilingual interpretation service site (210) is connected through the wired terminal (220) and the portable terminal (230) (S201).

  The multilingual interpreting service site (210), when the terminals (220) and (230) are connected, provides a web page for the multilingual interpreting service, and then sets the native language and other languages (S202). When the user selects the native language and other languages through the terminals (220) and (230), the multilingual interpreter server (212) interprets the selected native language into the other languages. An interpreter software file including a search engine, a multilingual standard database, a word order converter, and a TTS is downloaded (S203).

  In this way, when the native language and the foreign language are set in advance, the multilingual interpreting server (212) can select and transmit only the software module and database required by the user, thereby reducing the transmission capacity. it can. Also, the multilingual standard database loaded with only the selected native language and other languages also transmits the data frequently used according to the usage frequency without transmitting all files, as shown in Table 3 below. Data that is transmitted first in a file format and less frequently used data can be additionally transmitted when the client requests it.

  Next, after the terminal (220), (230) executes the downloaded software, the native language is received by receiving the native language voice via the microphone installed in the terminal (220), (230). Recognize (S204) and (S205).

  Then, a sign language code corresponding to the recognized native language is searched from the downloaded multilingual standard database, and if there is no corresponding sign language code as a result of the search, another multilingual standard database is stored in the multilingual interpretation server (212). Request and download a copy (S206 to S208). Next, a sign language code corresponding to the recognized native language is searched from the newly downloaded multilingual standard database (S206).

  Then, the foreign language mapped to the searched sign language code is searched, and when one sentence is completed, the language order of the foreign language arranged in the native language order is converted into the language order of the foreign language to form the foreign language text sentence. (S210 to S212). Next, the other-language text sentence is displayed on the LCD screen, and the other-language text sentence is converted into an audio signal in the TTS and output through the speaker (S213) (S214).

  As described above, the multilingual interpretation service according to the present invention can be serviced in a client-server structure via the Internet. In addition, the present invention can be applied to a language dictionary with pronunciation functions, and can be implemented in mobile devices such as mobile phones, smartphones, portable multimedia players, and navigation.

  Although the present invention has been described above with reference to the embodiments illustrated in the drawings, various modifications and other equivalent embodiments can be made by those skilled in the art. You should be able to understand that.

  14 ... sign language code, 16 ... multilingual standard database, 102 ... microphone, 104 ... analog-to-digital converter, 106 ... digital-to-analog converter, 108 ... amplifier, 110 ... speaker, 112 ... key input unit, 120 ... MCU, 121 ... Speech recognizer, 122 ... interpreting search engine, 123 ... word order converter, 124 ... TTS, 130 ... LCD.

Claims (7)

A multilingual standard database in which native language words and idiomatic sentences are mapped to other language words and idiomatic sentences based on codes,
A native language speech recognizer that recognizes the native language input via the microphone,
An interpreter search engine that receives the native language recognized by the native language speech recognizer, searches for a corresponding code, and searches for and outputs another language corresponding to the retrieved code;
A word-order converter that outputs a translated foreign language sentence by converting the native language word order to the foreign language word order if the foreign language received from the interpreter search engine is an idiomatic sentence, or if it is a plurality of words When,
TTS (Text to Speech) for converting the other language text of the word order converter into an audio signal;
Audio output means for amplifying the audio signal of the TTS and outputting it to a speaker;
A code-based multilingual interpreting apparatus comprising: display means for displaying text output of the word order converter in characters. The multilingual interpreter is
The user's operation is received, the native language and the foreign language are set, the native language set in the speech recognizer is recognized by speech, the native language and the foreign language are informed to the interpreting search engine, and the word order converter A control unit that converts the language order of the native language set to the language order of the set other language and controls the TTS to convert the selected foreign language text into a foreign language speech. The code-based multilingual interpreter according to claim 1. The multilingual interpreter is
The interpreting search engine transmits the N language data mapped to the code to N word order converters, and the N word order converters respectively transmit the data to N TTSs, and then N speech output means. The code-based multilingual interpreting apparatus according to claim 1, wherein the local language can be simultaneously translated into N languages by outputting to The multilingual interpreter is
The base multilingual interpreting apparatus according to claim 1, wherein the base multilingual interpreting apparatus is a stand-alone portable terminal or a server-client structure. The code is
2. The code-based multilingual interpreting apparatus according to claim 1, wherein the sign-based multilingual interpreting apparatus is a sign language code defined so as to encode a sign language word of each country and a frequently used simple sign language idiomatic sentence to express a common meaning. Defining a code with words and idioms for communication and constructing a multilingual standard database based on the code;
Setting a native language and a foreign language,
Receiving a voice of the set native language and recognizing the native language;
Searching for a code corresponding to the recognized native language;
Searching for other languages mapped in the searched code in native language order;
Converting the foreign language arranged in the native language order into the language of the foreign language to form a foreign language text sentence;
A code-based multilingual interpretation method comprising: converting the foreign language text sentence into a foreign language speech signal. Connecting to a multilingual interpreter server to set the native language and other languages;
Downloading and executing a part of a multilingual standard database and interpreting software from the multilingual interpreting server;
Receiving the native language speech and recognizing the native language,
Retrieving the code corresponding to the recognized native language from the downloaded multilingual standard database;
If there is no corresponding code as a result of the search, requesting and downloading another part of the multilingual standard database to the multilingual interpretation server;
Retrieving a code corresponding to the recognized native language from the newly downloaded multilingual standard database;
Searching other languages mapped in the searched code in their native language order;
Converting the word order of the other language words arranged in the word order of the native language to form the other language text sentence by converting the word order of the other language words;
A code-based multilingual interpretation method comprising: converting the foreign language text sentence into an audio signal.

Images