KR102069697B1 - Apparatus and method for automatic interpretation - Google Patents

Abstract

Reveal automatic interpretation equipment and methods. According to an aspect of the present invention, an interface unit for receiving a source language-based voice to be interpreted by a user command from a user and outputting a source language text for displaying a sound of a target language-based text in which the source language-based voice is translated in the source language is provided. In response to the command, a source language-based voice is recognized and converted into source language-based text through a setting unit and an interface unit for setting information on the source language and the target language, and the source language-based text is translated into the target language-based text. And an interpreter for converting the utterance of the target language-based text into the source language text in addition to the translated target language-based text and transmitting it to the interface unit.

Description

Automatic interpretation device and method {APPARATUS AND METHOD FOR AUTOMATIC INTERPRETATION}

The present invention relates to an automatic interpretation apparatus and method, and more particularly, to an automatic interpretation apparatus and method capable of displaying the sound of each of the speech recognition text and the automatic translation text output at the time of automatic interpretation in a partner station language or phonetic notation.

The automatic interpreter is a device that facilitates communication among people who speak different languages. In general, when a user of the source language speaks in the source language, the user recognizes the spoken source language. Automatically translate the target language to be used, and output the translated target language as a synthesized voice.

That is, in the existing automatic interpreter, the sound of the source language is displayed as the text of the source language by voice recognition and translated into the text of the target language by the automatic translation. The text of the target language is synthesized by the voice of the target language and output. However, since the synthesized voice is immediately extinguished, there is a problem that it is difficult for a user to directly reproduce the utterance of the other country's language and use it for communication.

In addition, most of the existing automatic interpreters show a phenomenon in which speech recognition performance is rapidly deteriorated in the case of inputting a proper noun with low frequency of use or in a noisy environment. When the speech recognition performance is deteriorated, the user is required to communicate by directly inputting text into the automatic interpretation device, or to speak the language of the other country directly without the automatic interpretation. Therefore, various interfaces should be provided as much as possible for smooth communication of the user.

An object of the present invention is to automatically display the target language according to the phonetic notation of the source language so that the user can directly pronounce the translated target language while simultaneously outputting the voice by automatically recognizing the source language and converting it into the target language. To provide an interpreter.

Another object of the present invention to provide an automatic interpretation method of the automatic interpretation device for achieving the above object.

In order to achieve the above object, an automatic interpretation apparatus according to an embodiment of the present invention receives a source language-based voice to be interpreted by a user command from a user, and generates a sound of the target language-based text in which the source language-based voice is translated. An interface unit for outputting source language text displayed in the source language; A setting unit configured to set information about the source language and the target language in response to the user command; And receiving the source language-based voice through the interface unit, and recognizing the voice, converting the source language-based text into the target language-based text, and adding the translated source language-based text to the target language-based text. An interpreter for converting a voice of a target language-based text into the source language text and transmitting the converted sound to the interface unit; It includes.

The translator includes: an acoustic and language database unit for storing a recognition network integrating a language model, an acoustic model, and a pronunciation dictionary for the source language and the target language; A speech recognition unit for analyzing the source language-based voice received from the interface unit based on the recognition network of the sound and language database unit and converting the source language-based speech into source language-based text; A text translation unit which receives the source language-based text from the speech recognition unit and translates the source language-based text into the target language-based text; A voice synthesizer configured to receive the target language-based text from the text translator, synthesize a voice corresponding to the target language-based text, generate a synthesized sound, and transmit the synthesized sound to the interface unit; And a voice sound conversion unit for receiving the target language-based text, converting the utterance of the target language-based text into the source language text, and outputting the sound to the interface unit. Characterized in that it comprises a.

The speech tones converting unit receives the source language-based text from one of the speech recognition unit and the text translator, and the sound of the source language-based text received using the recognition network of the language database unit is the target language. A first phonetic sound converting unit converting the text into the text and outputting the converted text to the interface unit; And a second phonetic sound converter configured to receive the target language-based text from one of the text translator and the speech synthesizer, convert the utterance of the target language-based text into the source language text, and output the converted source language text to the interface unit. Characterized in that it comprises a.

Each of the first and second voice conversion units analyzes and corrects a grammatical error in the source language-based text and the target language-based text, and corresponds to a symbol included in the source language-based text and the target language-based text. Characterized in that it further comprises a pre-processing unit for converting to language-based text.

The sound and language database may include at least one of a speech variation database, a graph to phoneme (g2p) conversion table, a pronunciation band database, and a band word database according to the type of the source language and the target language. It is characterized by including one.

Automatic interpretation method according to an embodiment of the present invention for achieving the other object in the automatic interpretation method of the automatic interpretation device having an interface unit, the setting unit and the interpretation unit, the user is applied through the interface unit Storing the automatic interpretation setting in response to the command; Determining whether a source language based voice is applied through the interface unit; When the source language-based voice is applied, speech recognition is performed by using a recognition network integrating a language model, an acoustic model, and a pronunciation dictionary for the source language and the target language stored in the sound and language database of the interpreter. Generating language-based text; Translating the source language-based text into target language-based text using the recognition network; And converting and outputting utterances of the translated target language-based text into source language text. It includes.

The source language is Korean, and the target language is Japanese.

The recognition network may include at least one of a pronunciation variation database, a graph to phoneme (g2p) conversion table, a pronunciation band database, and a band word database according to the type of the source language and the target language. It is done.

The converting of the target language text into the target language text may include: performing a phonetic variation on the source language-based text corresponding to a phonetic variation according to the characteristics of the source language; Sequentially dividing the source language-based text on which the pronunciation shift is performed in units of words, syllables, and phonemes; Converting the phoneme separated using the g2p conversion table into phonetic symbols of phoneme units; Combining the converted phonetic unit phonetic symbols in syllable units; Converting into syllables of the target language corresponding to the combined syllables; Combining the converted syllables to generate the target language text indicating the sound of the source language based text by restoring the word; And outputting the target language text through the interface unit. Characterized in that it comprises a.

The converting of the source language text into the source language text may include determining whether a target language-based text includes a band word that is a vocabulary commonly used in a source language; Applying a pronunciation of a band word to a vocabulary in which the band word exists; Sequentially separating words having no band word into words, syllables, and phonemes; Converting the phoneme separated using the g2p conversion table into phonetic symbols of phoneme units; Combining the converted phonetic unit phonetic symbols in syllable units; Converting into syllables of the source language corresponding to the combined syllables; Generating the source language text indicating the sound of the target language based text by restoring the word by combining the pronunciation of the band word and the converted syllables; And outputting the source language text through the interface unit. Characterized in that it comprises a.

Therefore, the automatic interpretation apparatus and method of the present invention, like the existing automatic interpretation apparatus, receives the user's source language utterance and recognizes the speech, and automatically translates and outputs the speech to the target language as well as adding to the speech recognized text. The sound is displayed in the target language and the pronunciation of the translated target language is displayed according to the pronunciation notation of the source language, so that the user can directly pronounce the speech recognition text or pronounce the translated target language. Therefore, the user can directly speak the speech recognition target text or the interpreted target language according to the situation, so that the user can not only communicate smoothly with the conversation partner but also easily recognize the pronunciation of the foreign language that is difficult to understand and understand the language of the other country. Because you can follow the pronunciation, you can significantly increase the foreign language learning achievement. In addition, by displaying the voice recognition results of the user's voice in both the source language and the target language, it is possible to quickly and accurately determine and cope with errors of the automatic interpreter.

1 shows a configuration of an automatic interpretation device according to an embodiment of the present invention.
Figure 2 shows an automatic interpretation method of the automatic interpretation device according to an embodiment of the present invention.
FIG. 3 illustrates in detail the first voice conversion step in the automatic interpretation method of FIG.
FIG. 4 illustrates in detail the second voice conversion step in the automatic interpretation method of FIG.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

Throughout the specification, when a part is said to "include" a certain component, it means that it may further include other components, without excluding other components unless otherwise stated. In addition, the terms "... unit", "... group", "module", "block", etc. described in the specification mean a unit for processing at least one function or operation, which is hardware or software or hardware. And software in combination.

1 shows a configuration of an automatic interpretation device according to an embodiment of the present invention.

Referring to FIG. 1, the automatic interpreter of the present invention includes an interface unit, a setting unit 200, and an interpreter 300.

The interface unit is an input / output unit of the automatic interpretation device, receives a user's command, and outputs an interpretation result to the user. The interface unit includes a voice sensing unit 110, a user input unit 120, a display unit 130, and a voice output unit 140. The voice detector 110 includes a voice sensor such as a microphone, and detects a voice signal (in) input by the user and transmits it to the interpreter 300. The user input unit 120 is implemented as a user command input means such as a keyboard, a mouse, a touch pad, a touch screen, and the like so that a user can apply a user command or text to an automatic interpreter. The user input unit 120 transmits the text to the interpreter 300 when the user command is applied, and transmits the text to the interpreter 300 when the text to be interpreted is input.

The display unit 130 may be implemented as a display means such as a screen or a monitor. In some cases, the display unit 130 may be implemented in combination with the user input unit 120 such as a touch screen or a touch panel. The display 130 displays a voice recognition result for the source language input by the user, text input by the user, and translated target language text. In particular, the display unit 130 of the automatic interpreter of the present invention displays the speech recognition result of the source language or the pronunciation of the text input by the user in the target language, and the pronunciation of the interpreted target language text in the source language. Display. This allows the user to recognize the pronunciation of the other's language and pronounce it directly, unlike the voice that is extinguished immediately. When the interpreted target language is output as a voice and the pronunciation of the target language text is displayed together with the source language, the user can understand the pronunciation of the interpreted language more easily than when the voice is spoken only. You can get great results.

The voice output unit 140 outputs the synthesized sound generated when the target language interpreted by the interpreter 300 is generated as the synthesized sound. The voice output unit 140 may be implemented by a voice output means such as a speaker.

The setting unit 200 sets and stores source language information, target language information, and output of synthesized sound in response to a user command applied through the user input unit 120. Here, the source language information includes information on which language is a source language to be interpreted by a user inputting by voice or text. Similarly, the target language information includes information on which language to translate the input source language into. For example, the source language information and the target language information may be set to Korean, English, Japanese, Chinese, and the like.

The interpreter 300 translates the source language-based voice or text input by the user into a target language-based text, and generates the translated target language-based text as a synthesized sound to perform interpretation. In particular, in the present invention, the interpreter 300 displays the sound of the source language based text in the target language, and displays the translated sound of the target language based text in the source language. That is, as the voices are displayed in the languages of the counterpart countries for each of the source language and the target language, a plurality of users who use different languages can directly pronounce each other's languages.

The interpreter 300 includes a speech and language database 310, a speech recognizer 320, a text translator 330, a speech synthesizer 340, a first voice conversion unit 350, and a second voice. It includes a conversion unit 360.

The voice and language database unit 310 stores data about a source language and a target language. In general, the speech and language database 310 stores a language model, a pronunciation dictionary, and an acoustic model used by the speech recognition apparatus for speech recognition. The language model is provided to find any regularity about grammar, phrases, words, etc. in the natural language, and is used to use the regularity. The acoustic model separates the speech into recognition units and models the speech to recognize phonemes of recognition units. Is provided to convert. Pronunciation dictionary provides language notation for each language, including pronunciation notation and pronunciation characteristic information of each language. The language model, the acoustic model, and the pronunciation dictionary may be provided for the source language and the target language, respectively, and may be provided for other languages. The voice and language database unit 310 may form and store a recognition network integrating a language model, a pronunciation dictionary, and an acoustic model.

In particular, in the present invention, the automatic interpretation device not only converts the input source language into a target language and outputs it, but also displays a voice sound for the text converted into the target language as the source language or input source language, unlike the existing interpretation device. Allows you to display the phonetic sounds of the text in the target language. Each language also has its own writing system and pronunciation system. Accordingly, the speech and language database 310 of the present invention may include a phonetic dictionary, a phonetic variation database, a phoneme to phoneme (g2p) conversion table, a pronunciation band database, and a bandword database according to the type of language model. At least one may be provided.

As an example, assuming that Korean is translated into Japanese and Japanese is translated into Korean, Korean is a syllable and phonemic letter system, whereas Japanese is a syllable unit letter system.

First, a case in which a Korean voice is displayed in Japanese is described. The pronunciation dictionary stores a rule for converting a character into a phonetic symbol of a phoneme and is the basis of a g2p conversion table. In other words, when a phoneme is input, the phoneme may be converted into phonetic symbols of the corresponding phoneme and output.

The pronunciation band database stores the combined structure of phonemes corresponding to Korean syllables, and stores Japanese syllables corresponding to the combination of phonemes to convert Korean syllables into Japanese syllables.

In the case of Korean, there are various pronunciation variations such as consonant fairy tales, palate phonetics, and abbreviations. The pronunciation variation database stores the pronunciation variation information so that the pronunciation corresponding to the actual pronunciation of each vocabulary can be extracted. For example, when Korean "Silla" is to be displayed in Japanese, Japanese corresponding to "Sila" which is the actual pronunciation of "Silla" can be displayed. Allows the display of Japanese corresponding to "Kuji."

On the other hand, referring to the case of displaying Japanese phonetic sounds in Korean, the g2p conversion table operates in the same way as displaying the pronunciation of Korean in Japanese. The pronunciation band database allows the phonetic symbols of Japanese syllables to be converted into Korean syllables. However, in Japanese, pronunciation variations such as Korean are relatively small, and thus the pronunciation variation database may be omitted. Instead, there are a number of vocabulary in which Korean notation is already customary in Japanese. The bandword database provides a customary bandword for Japanese to cope with the customary vocabulary.

The voice recognition unit 320 receives the voice signal in through the voice detection unit 110 and converts the text into text using a recognition network set based on an acoustic model and a language model stored in the voice and language database 310. Convert. The converted text is source language based text.

The text translator 330 receives the source language-based text converted from the voice signal in from the speech recognizer 320 or receives the source language-based text input by the user using the user input unit 120 based on the target language. Translate into text. If the Korean voice signal in is voice-recognized as Korean "Goodbye", the text translation unit 330 may convert the Japanese voice signal "さ よ う な ら" corresponding to "Goodbye". The text translation unit 330 translates the source language-based text into the target language-based text based on the language model of the speech and language database 310. Techniques for translating texts are well known in the art, so a detailed description thereof will be omitted.

The text translation unit 330 may include a preprocessor (not shown). The preprocessor may perform a preprocessing operation to convert symbols such as Arabic numerals into text or check and correct spelling errors before text translation. However, numbers and symbols are often used in many languages, so they may not need to be converted to text. However, since the pronunciation of numbers or symbols is different in each language, the preprocessor may be included in the first and second voice conversion converters 350 and 360 for converting the text according to the voice.

The speech synthesizer 340 synthesizes the translated target language-based text and generates a synthesized sound so that the user can listen to the speech output unit 140. In this case, the voice synthesizer 340 may set the synthesizer sound in various ways, such as a male voice or a female voice or an adult or child voice, depending on the setting of the setting unit 200.

The first speech sound conversion unit 350 converts the source language-based text recognized by the speech recognition unit 320 into a target language based on the pronunciation dictionary of the speech and language database. That is, when the voice signal in is input and converted into the source language-based text by the speech recognition unit 320, the voice of the converted source language-based text is displayed in the target language. As in the above example, when Korean is translated into Japanese, the voice signal in is input as a Korean-based voice signal in which the source language is Korean. In addition, the Korean-based speech signal is converted into Korean-based text by the speech recognizer 320. Accordingly, the first voice conversion unit 350 converts the voice of the Korean-based text into Japanese text that is a target language. If the input signal in is voice recognized as "Hello," the first voice conversion unit 350 displays "Hello," using the data from the speech and language database unit 310 to display Japanese Katakana. " Convert it to ANNION GEESE. This is unlike the semantic translation "さ よ う な ら" performed by the text translation unit 330 by converting the Korean utterance into "ア ン ニ ョ ン ヒ ギ ェ セ ヨ" so that the Japanese as a partner can easily pronounce the Korean pronunciation by looking at their native language.

In addition, the first voice conversion unit 350 transmits the source language-based text recognized by the speech recognition unit 320 and the target language text indicating the voice sound of the source language-based text to the display 130, and then displays the display unit 130. In addition to the speech-recognized source language-based text, the 130 may display the target language text indicating the voice of the source language-based text. Accordingly, the user may determine whether the automatic interpreter correctly recognizes the voice spoken by the user, and check the target language notation corresponding to the corresponding voice.

In the above description, the first voice conversion unit 350 receives the source language-based text from the voice recognition unit 320, but the source language-based text may be received from the text translation unit.

The second voice conversion unit 360, as opposed to the first voice conversion unit 350, converts the voice of the target language-based text translated by the text translation unit 330 into text of the source language based on the language database. Convert. The second voice conversion unit 360 receives the text translated into the target language applied to the speech synthesis unit 340 and converts the voice sound of the translated target language text into the source language. In the above example, the second voice conversion unit 360 converts the Japanese "さ よ う な ら" into which the Korean "Goodbye" is translated into the Korean "Sayo-Nara". Where "-" is a long note notation.

As in the first voice conversion unit 350, the second voice conversion unit 360 transmits the Korean voice indicating the translated Japanese voice to the display unit 130 along with the Japanese text for display. Since the pronunciation of the text translated in the target language can be confirmed in Korean, it is possible to easily utter the directly translated text with a high understanding of the pronunciation.

At the same time, since the synthesized sound corresponding to the text translated into the target language is output through the voice output unit 140, it is possible to have a high understanding of the pronunciation, thereby improving achievement even during language learning.

In the above description, the first voice conversion unit 350 and the second voice conversion unit 360 are separately illustrated for convenience of description, but the first voice conversion unit 350 and the second voice conversion unit 360 are illustrated. May be integrated. In addition, in FIG. 1, although the second voice conversion unit 360 receives the target language-based text translated from the voice synthesis unit 340, the second voice conversion unit 360 is the text translation unit 330. It is also possible to receive target language-based text from.

In addition, although the setting unit 200 is illustrated separately from the interpreter 300 in FIG. 1, the setting unit 200 may be included in the interpreter 300.

The automatic interpretation device of FIG. 1 may be implemented as a separate device for interpretation, but since the setting unit 200 and the interpretation unit 300 may be implemented in software, various devices having an interface unit may be used as the automatic interpretation device. Can be. For example, various information communication devices such as smart phones, smart pads, PDAs, PCs, etc. may be used as the automatic interpretation device.

Figure 2 shows an automatic interpretation method of the automatic interpretation device according to an embodiment of the present invention.

The automatic interpretation method of FIG. 2 will also be described taking the case of interpreting Korean as Japanese as shown in FIG. Referring to FIG. 1, the automatic interpretation method of FIG. 2 will be described below. The automatic interpretation apparatus first receives and stores an automatic interpretation setting as a user command authorized through the user input unit 120 (S10). Here, the automatic interpretation setting stores source language and target language information, output setting of the synthesized sound, and the like. The automatic interpretation setting defaults, and even if not set by the user can be interpreted by the default setting.

If the automatic interpretation setting is specified, the automatic interpretation device determines whether a source language voice, which is a voice signal in, is input (S20). The source language voice may be input through the voice sensor 110 as a source language based voice used by the user, and the user inputs the voice input command as a user command through the user input unit 120. In some cases, an automatic interpreter can automatically detect whether a voice is input. If it is determined that the source language voice is input, the speech recognition unit 320 integrates the acoustic model, the pronunciation dictionary and the language model in the speech and language database 310 to perform speech recognition using a stored recognition network. Generate language-based text (S30).

On the other hand, if the source language voice is not input, it is determined whether the source language text is input (S40). A user may apply a sentence to be interpreted to an automatic interpretation device by voice. However, in a special case such as a lot of noise or an environment in which speech is difficult to speak, the sentence to be interpreted using the user input unit 120 may be used as a text. You can also enter it manually. In this case, since speech recognition is unnecessary, the speech recognition step S30 is omitted.

When the source language-based text is obtained by speech recognition or source text input, the first voice conversion unit 350 converts the pronunciation of the source language-based text into the text of the target language of the target language, and the source language-based text and the source. The target language text is displayed through the display unit 130 to display the spoken sound of the language text (S50).

The source language-based text is automatically translated according to the automatic interpretation setting and converted into the target language text (S60).

When the automatic translation is performed and the target language text is obtained, the second phonetic sound conversion unit 260 of the automatic interpretation device converts the pronunciation of the translated target language text into the source language text, and the speech of the target language text and the target language text is spoken. The source language text displaying the sound is output through the display 130 (S70).

On the other hand, the speech synthesizer 340 synthesizes a voice corresponding to the target language-based text to generate a synthesized sound (S80). The voice output unit 140 receives and outputs the synthesized sound (S90).

After outputting the synthesized sound, the automatic interpretation device, on the contrary, determines whether to interpret the target word according to a user command applied through the user input unit 120 (S100). If an interpreter command is applied to the target word, an interpreter setting switch for exchanging the source word and the target word in the preset automatic interpreter setting is performed (S110).

As described above, in the automatic interpretation method of the automatic interpretation apparatus according to the present invention, when the source language is input as voice or text, the target language text corresponding to the pronunciation of the input source language and the source language to which the source language is translated are added to the spoken sound. By outputting the displayed source language text, it is possible to confirm both the pronunciation representation of the target word for the voice spoken by the user and the pronunciation representation of the source language for the interpreted target word. Therefore, the user can easily recognize the pronunciation of the interpreted target language, so that the user can speak directly, as well as help in language learning.

In the above description, the first voice conversion step S50 of converting the pronunciation of the source language-based text into the target language text is described, but the actual user may not need the target language text. In this case, the first voice conversion step S50 may be omitted, and the first voice conversion unit 350 may also be removed from the automatic interpreter.

FIG. 3 shows the first voice conversion step in detail in the automatic interpretation method of FIG. 2, and FIG. 4 shows the second voice conversion step in detail in the automatic interpretation method in FIG.

3 and 4 also illustrate the interpretation from Korean to Japanese as an example, and it is assumed that the preprocessor is provided in the first and second phonetic sound converters 350 and 360, respectively, rather than the text translation unit.

The first speech conversion step S50 of FIG. 3 is a step of converting the speech sound of the source language-based text into the target language text, which is first pre-processed for the source language-based text obtained by speech recognition or input of the source language text. Perform the operation (S51). As described above, the preprocessing operation may be performed by correcting a grammatical error such as spelling check and converting numbers or symbols into source language-based text. Then, the pronunciation variation is converted using the pronunciation variation database of the speech and language database 310 (S52). The phonetic variation transformation allows various phonetic variation, such as consonant assimilation, palatalization, and abbreviation, which are characteristic of Korean, to be applied to the source language-based text. In other words, the source language-based text is partially converted into pronunciation-based text.

After the phonetic mutation conversion is performed, the source language-based text is separated into word units (S53). Segmentation is intended to reflect spacing regardless of language type. When the word is separated, syllable separation is performed (S54). Then, when the syllables are separated, phoneme separation is performed for each syllable into primary, neutral, and final (S55). Since phoneme separation is a phoneme-based language, phoneme separation can be omitted in the case of Japanese or Chinese, where phonemes cannot be separated.

When the phoneme separation is performed, the phonemes are separated into phonetic symbols of phoneme units using the g2p conversion table of the speech and language database unit 310 (S56). Table 1 shows an example of a g2p conversion table for converting Korean into phonetic symbols.

Then, the phoneme unit phonetic symbols converted using the phonetic band database of the speech and language database unit 310 are combined in syllable units, and then converted into syllable target words (here, Japanese) syllables corresponding to the combined syllables (S57). Table 2 shows an example of a pronunciation band database for converting Korean into Japanese syllables.

When all of the separated syllables are converted to the target word syllable, the converted syllables are combined again to restore the word (S58). The restored word is obtained by converting a utterance of the source language-based text into the target language text, and the automatic interpreter displays the target language text through the display 130 (S59). In this case, the target language text for the sound of the source language-based text and the source language-based text may be added and displayed together.

Meanwhile, the second voice conversion step S70 of FIG. 4 is a step of converting the voice of the translated target language-based text into the source language text, and similarly to the first voice conversion step S50, a preprocessing step may be performed. It may be (S71). Then, the band word search step is performed (S72). As described above, when the phonetic sound of Japanese-based text is converted into Korean text, a large number of band words, which are already conventionalized vocabularies, must be reflected. In order to apply the band word to the pronunciation, the band word database included in the speech and language database unit 310 is used to determine whether the band word exists in the source language in the target language-based text.

If it is determined that the band word exists, the band word stored in the speech and language database unit 310 is called and applied (S74). In the case of applying a band word, a separate conversion process is not necessary for the corresponding vocabulary.

Meanwhile, similar to FIG. 3, the syllable separation step S75, the g2p conversion step S76, and the pronunciation band conversion step S77 are performed on words having no band word. However, in the case of the g2p conversion table and the pronunciation band database for g2p conversion, the conversion of Korean pronunciation to Japanese is different from the conversion of Japanese pronunciation to Korean, so different g2p conversion tables and pronunciation band databases can be used. have.

Table 3 shows an example of a g2p conversion table for converting Japanese into phonetic symbols.

Table 4 shows an example of a pronunciation band database for converting Japanese into Korean syllables.

In Tables 1 to 4, the capital letters B and L of the alphabet reflect utterances pronounced differently depending on the vocabulary, the language, and the ending of the Japanese pronunciation.

When the band word is applied to the vocabulary in which the band word exists, and the syllable separation, the g2p conversion, and the pronunciation band conversion are performed on the vocabulary in which the band word does not exist, the vocabulary in which the band word and the pronunciation band are performed is combined into a sentence again (S78). .

The combined sentence is a voice of the translated target language-based text is converted into the source language text, the automatic interpreter displays the source language text through the display 130 (S79). In addition to the target language-based text translated therein, the source language text in which the phonetic sound is displayed in the source language may be displayed.

3 and 4 have been described on the assumption that Korean is interpreted in Japanese, but when Japanese is interpreted in Korean, FIG. 3 may be performed as a second voice conversion step, and FIG. 4 is a first voice conversion. May be performed in steps.

In the above, as an example, an interpreter between Korean and Japanese has been described as an example, but the present invention is not limited thereto, and it is obvious that the present invention may be applied to other types of languages.

The method according to the invention can be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the present invention has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible.

Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (18)

An interface unit for receiving a user language command from a user and a source language-based voice to be interpreted, and outputting source language text for displaying a sound of the target language-based text in which the source language-based voice is translated as a source language;
A setting unit configured to set information about the source language and the target language in response to the user command; And
The source language-based voice is received through the interface unit to recognize the voice, convert the source language-based text, convert the utterance of the converted source language-based text into the target language text, and convert the source language-based text into the target language. And an interpreter for translating the text into the base text and analyzing the grammatical errors in the source language-based text and the target language-based text.
The method of claim 1, wherein the interpreter
An acoustic and language database unit for storing a recognition network integrating a language model, an acoustic model, and a pronunciation dictionary for the source language and the target language;
A speech recognition unit for analyzing the source language-based voice received from the interface unit based on the recognition network of the sound and language database unit and converting the source language-based speech into source language-based text;
A text translation unit which receives the source language-based text from the speech recognition unit and translates the source language-based text into the target language-based text;
A voice synthesizer configured to receive the target language-based text from the text translator, synthesize a voice corresponding to the target language-based text, generate a synthesized sound, and transmit the synthesized sound to the interface unit; And
A phonetic sound conversion unit for receiving the target language-based text, converting the utterance of the target language-based text into the source language text, and outputting the source language text to the interface unit; Automatic interpretation device comprising a.
The method of claim 2, wherein the voice conversion unit
Receiving the source language-based text from one of the speech recognition unit and the text translation unit, converts the sound of the source language-based text received using the recognition network of the language database unit to the target language text A first phonetic sound converter configured to output to the interface unit; And
A second phonetic sound converter which receives the target language-based text from one of the text translator and the speech synthesizer, converts the voiced sound of the target language-based text into the source language text and outputs it to the interface unit; Automatic interpretation device comprising a.
The method of claim 3, wherein each of the first and second voice conversion unit
And a preprocessor for converting the symbols included in the source language-based text and the target language-based text into corresponding language-based text.
The method of claim 3, wherein the sound and language database unit
The recognition network includes at least one of a pronunciation variation database, a graph to phoneme (g2p) conversion table, a pronunciation band database, and a band word database according to the type of the source language and the target language. Automatic translator device.
6. The automatic interpreter of claim 5, wherein the source language is Korean and the target language is Japanese.
The method of claim 6, wherein the first voice conversion unit
The phonetic variation is performed on the source language-based text to correspond to the phonetic variation according to the characteristics of the source language, sequentially separated into units of words, syllables, and phonemes, and separated using the g2p conversion table. Converts a phoneme into a phonetic symbol of a phoneme, combines the converted phoneme unit phoneme into a syllable unit, converts a phoneme into a syllable of the target language corresponding to the combined syllable, and combines the converted syllables to restore the word Thereby generating the target language text displaying the utterance of the source language based text.
The method of claim 7, wherein the second voice conversion unit
Determine whether there is a band word, which is a vocabulary commonly used in a source language, in the target language-based text, apply a pronunciation of a band word to a vocabulary in which a band word exists, and use a word, syllable and Sequentially separating phonemes, and converting the phonemes separated using the g2p conversion table into phonetic symbols of phoneme units, combining the phoneme units phonetic symbols into syllable units, and then corresponding to the combined syllables. And converting the syllables of the source language into a syllable of the target language based text by restoring the word by combining the pronunciation of the band word and the converted syllables.
The method of claim 2, wherein the interface unit
A voice detector for detecting a voice input by the user and transmitting the voice to the interpreter;
A user input unit implemented as a user command input unit to receive the user command or the source language-based text;
Utterance of the source language text and the source language text in which the source language-based text is translated into the target language-based text, and the source language text in which the voice of the target language-based text is expressed in a source language. A display unit configured to display at least one of the target language texts in which sound is written as a target language; And
And an audio output unit configured to be output as the voice output unit to output the synthesized sound.
In the automatic interpretation method of the automatic interpretation device having an interface unit, a setting unit and an interpretation unit, the automatic interpretation device is
Storing an automatic interpretation setting in response to a user command applied through the interface unit;
Determining whether a source language based voice is applied through the interface unit;
When the source language-based voice is applied, speech recognition is performed using a recognition network integrating a language model, an acoustic model, and a pronunciation dictionary for the source language and target language stored in the sound and language database of the interpreter. Generating the base text;
Converting the sound of the source language-based text into the target language text before outputting the target language-based text; And
Translating the source language-based text into a target language-based text using the recognition network;
Generating the source language-based text and converting the target language-based text to output the text are based on:
Automatic interpretation method for analyzing and correcting a grammar error in the source language-based text and the target language-based text.
delete 11. The method of claim 10 wherein the source language is Korean and the target language is Japanese.
The network of claim 12, wherein the recognition network is
An automatic interpretation comprising a pronunciation variation database, a graph to phoneme (g2p) conversion table, a pronunciation band database and a band language database according to the type of the source language and the target language Way.
The method of claim 13, wherein the converting the target language text into output is performed.
Performing a phonetic variation on the source language-based text to correspond to a phonetic variation according to the characteristics of the source language;
Sequentially dividing the source language-based text on which the pronunciation shift is performed in units of words, syllables, and phonemes;
Converting the phoneme separated using the g2p conversion table into phonetic symbols of phoneme units;
Combining the converted phonetic unit phonetic symbols in syllable units;
Converting into syllables of the target language corresponding to the combined syllables;
Combining the converted syllables to generate the target language text indicating the sound of the source language based text by restoring the word; And
Outputting the target language text through the interface unit; Automatic interpretation method comprising a.
The method of claim 10,
Converting the sound of the translated target language-based text into source language text and outputting the same;
The step of converting to the source language text and outputting
Determining whether the target language-based text includes a band word, which is a vocabulary commonly used in a source language;
Applying a pronunciation of a band word to a vocabulary in which the band word exists;
Sequentially separating words having no band word into words, syllables, and phonemes;
Converting the phoneme separated using the g2p conversion table into phonetic symbols of phoneme units;
Combining the converted phonetic unit phonetic symbols in syllable units;
Converting into syllables of the source language corresponding to the combined syllables;
Generating the source language text indicating the sound of the target language based text by restoring the word by combining the pronunciation of the band word and the converted syllables; And
Outputting the source language text through the interface unit; Automatic interpretation method comprising a.
The method of claim 15, wherein the converting and outputting the converted target language text and outputting the converted target language text are respectively performed.
Analyze and correct grammatical errors in the source language-based text and the target language-based text before performing the phonetic variation conversion and determining whether the band language exists, and corrects the source language-based text and the target language-based text. And a pre-processing step of converting the symbols included in the corresponding language-based text.
The method of claim 10, wherein the automatic interpretation method
If the source language-based voice is not applied before the step of translating the target language-based text, directly receiving the source language-based text through the interface unit; And
After translating to the target language-based text, synthesizing a voice corresponding to the target language-based text to generate a synthesized sound and outputting the synthesized sound through the interface unit; Automatic interpretation method further comprising a.
18. A recording medium having recorded thereon a computer readable program for performing the automatic interpretation method of the automatic interpretation apparatus according to any one of claims 10 and 12.

Images