KR20150105075A - Apparatus and method for automatic interpretation - Google Patents

Abstract

Disclosed are an apparatus and a method for automatic interpretation. The automatic interpretation apparatus comprises an input part receiving pronunciation of a source language; a voice recognition part recognizing the pronunciation of the source language, and generating a voice recognized sentence; an interpretation part converting the voice recognized sentence into a text sentence of a target language; a pronunciation generation part generating a phonetic symbol of each of the target language and the source language for a partner country user to reappear or visually recognize the voice recognized sentence and the text sentence of the target language by using a pronunciation translation database; and an output part outputting the voice recognized sentence, text sentence, phonetic symbol of the target language and source language on a screen or as a voice.

Description

[0001] Apparatus and method for automatic interpretation [

The present invention relates to an automatic interpretation apparatus and method between Korean and Chinese.

Korean-Chinese automatic interpreting is to help communication between Korean and Chinese who do not know each other's language. Basically, it consists of source language utterance, voice recognition sentence generation, automatic translation into target language, and output of target language synthesized sound. The utterance sound of the source language is generated as a sentence of the source language by speech recognition, translated into a sentence of the target language by automatic translation, and synthesized and output as the voice of the target language based on the sentence of the target language. However, since the voiced sound disappears immediately, it is practically impossible to memorize or reproduce the voiced sound and use it for communication. In addition, speech recognition performance deteriorates drastically in a proper noun or noise environment in which a vocal sound is used infrequently. In this case, the user is required to input text directly into the automatic interpretation device, and in particular, it is necessary to learn simple sentences used frequently or in other countries to directly speak. To cope with this situation, additional interfaces should be provided as much as possible in order to actively utilize the automatic interpretation function.

In the present invention, speech sounds of a source language are voice-recognized. In addition to the speech recognition sentence, a speech sound of the sentence is displayed as a character of a target language. In addition to the automatic translation sentence to the target language, And an automatic interpretation device and method for expressing the characters in the language.

According to an aspect of the present invention, an automatic interpretation apparatus is disclosed.

An automatic interpretation apparatus according to an embodiment of the present invention includes an input unit for inputting a speech sound of a source language, a speech recognition unit for generating a speech recognition sentence by voice recognition of a speech sound of the source language, A target language and a source language such that the user of the target station can reproduce or visually recognize the speech recognition sentence and the text sentence in the target language by using a translation unit and a pronunciation band database, And an output unit for outputting a phonetic symbol in the speech recognition sentence, the text sentence, the target language, and the source language on a screen or a voice.

The input unit receives a vocal sound in one of Korean and Chinese through a microphone, or receives a text sentence in one of Korean and Chinese through a character input module.

The pronunciation band database stores Chinese pronunciation word data for expressing the pronunciation sound of the Chinese sentence with the Korean pronunciation symbol and Chinese pronunciation pronunciation data for expressing the pronunciation sound of the Korean sentence with the Chinese pronunciation symbol.

Wherein the Korean pronunciation band word data includes information in which a Chinese pronunciation unit corresponding to a Chinese pronunciation unit is mapped by separating a Chinese word pair into a pronunciation unit and the Chinese word is pronounced as a Chinese roman Is a pronunciation symbol.

The Chinese pronunciation domain word data includes Korean syllables divided into first, second, and last names, and the first, second, and third names are mapped to a phonetic symbol or roman alphabet of a Chinese word.

The automatic interpretation apparatus is implemented as an application of a mobile terminal.

According to another aspect of the present invention, an automatic interpretation method performed by an automatic interpretation apparatus is disclosed.

An automatic interpretation method according to an embodiment of the present invention includes the steps of receiving an utterance sound of a source language, generating a speech recognition sentence by voice recognition of the utterance sound of the source language and outputting the generated speech recognition sentence, Generating and outputting a pronunciation symbol in a target language so that the user of the partner station can recognize or visually recognize the recognition sentence; converting the speech recognition sentence into a text sentence in the target language and outputting the pronunciation sentence; And generating and outputting phonetic symbols in the source language so that the user of the partner station can reproduce or visually recognize the text sentence in the target language using the database.

The step of receiving the utterance sound of the source language includes receiving a utterance sound of one of Korean and Chinese through a microphone or inputting a text sentence of one of Korean and Chinese through a character input module.

The pronunciation band database stores Chinese pronunciation word data for expressing the pronunciation sound of the Chinese sentence with the Korean pronunciation symbol and Chinese pronunciation pronunciation data for expressing the pronunciation sound of the Korean sentence with the Chinese pronunciation symbol.

Wherein the Korean pronunciation band word data includes information in which a Chinese pronunciation unit corresponding to a Chinese pronunciation unit is mapped by separating a Chinese word pair into a pronunciation unit and the Chinese word is pronounced as a Chinese roman Is a pronunciation symbol.

The Chinese pronunciation domain word data includes Korean syllables divided into first, second, and last names, and the first, second, and third names are mapped to a phonetic symbol or roman alphabet of a Chinese word.

In the present invention, speech sounds of a source language are voice-recognized. In addition to the speech recognition sentence, a speech sound of the sentence is displayed as a character of a target language. In addition to the automatic translation sentence to the target language, By writing in the language of the language, the language of the partner country can be directly spoken to help in communication.

Brief Description of the Drawings Fig. 1 schematically illustrates a configuration of an automatic interpretation device between Korean and Chinese. Fig.
2 is a flowchart showing an automatic interpretation method between Korean and Chinese in the automatic interpretation apparatus of FIG.
Fig. 3 illustrates data constructed in the pronunciation band database; Fig.

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals used in the description of the present invention are merely an identifier for distinguishing one component from another.

Also, in this specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate a thorough understanding of the present invention, the same reference numerals are used for the same means regardless of the number of the drawings.

1 is a schematic view illustrating a configuration of an automatic interpretation apparatus between Korean and Chinese.

1, an automatic interpretation apparatus includes an input unit 10, a speech recognition unit 20, a translation unit 30, a voiced sound generation unit 40, an output unit 50, a pronunciation band database 60, And an engine (70).

The input unit 10 receives a sound of a source language. For example, the input unit 10 can receive voices in one of Korean and Chinese through a microphone. Alternatively, the input unit 10 may receive a text sentence in one of Korean and Chinese through a character input module. In this case, the input unit 10 can directly transmit the input sentence to the translator 30.

, 이들이 통합된 인식네트워크를 형성한다. 이어, 음성 인식부(20)는 추출한 음성특징들을 인식네트워크를 통하여 음성인식 텍스트 문장으로 변환하고, 변환한 음성인식 텍스트 문장을 출력부(50)를 통해 사용자 화면에 출력할 수 있다.The speech recognition unit 20 receives a speech sound in the input source language, performs speech recognition, and generates a speech recognition sentence. For example, the speech recognition unit 20 performs a signal process on the speech sound of the input source language to separate the speech segment, and then extracts the speech segment from the speech segment. On the other hand, the speech recognition section 20 constructs an acoustic model, a pronunciation dictionary, and a language model based on the speech database and the language database

, Which form an integrated recognition network. Then, the speech recognition unit 20 converts the extracted speech characteristics into a speech recognition text sentence through the recognition network, and outputs the converted speech recognition text sentence through the output unit 50 to the user screen.

For example, the speech recognition unit 20 can generate a speech recognition sentence such that the result of uttering "Hello" in the source language is output to the user's screen.

The translation unit 30 converts the speech recognition sentence in the source language or the received sentence into a text sentence in the target language by using the translation engine 70. [

"로 변환할 수 있다.For example, when the speech recognition sentence is "Hello ", the translator 30 reads" Hello ","Hello&

"

The utterance tone generator 40 uses the pronunciation band database 60 so that a speech recognition sentence in a source language and a text sentence in which a speech recognition sentence is converted into a target language can be reproduced or visually recognized And generates phonetic symbols in the target language and the source language, respectively.

"인 경우, "니ˇ 하오ˇ"라고 한국어 발음 기호를 생성할 수 있다.For example, when the speech recognition sentence is "Hello ", the utterance tone generator 40 can generate a Chinese pronunciation symbol" (an-nyeong-ha-se-yo) The sentence "

Quot ;, it is possible to generate a Korean pronunciation symbol "Nihao ".

For this purpose, the pronunciation band database 60 stores Chinese pronunciation word data for expressing the pronunciation sound of a Chinese sentence as a Korean pronunciation symbol and Chinese pronunciation pronunciation data for expressing a pronunciation sound of a Korean sentence as a Chinese pronunciation symbol.

For example, FIG. 3 is a diagram illustrating data constructed in the pronunciation band database. Hereinafter, the pronunciation band database 60 will be described with reference to FIG.

Chinese is composed of syllables which have no consonant other than "b, d, ㅇ". These characteristics are reflected in the data built in the sounding band database 60. [

First, Korean pronunciation pronunciation word data for expressing a phonetic note of Chinese sentence with a Korean pronunciation symbol will be described.

"와 같이, "안녕하세요"라는 중국어 한자와 함께 [ ]안에 중국어 고유의 액센트 즉, 4성으로 된 성조가 포함된 중국식 로마자로 표기된 발음 기호가 될 수 있다. 4성을 살펴보면, 제1 성(" ￣ " 또는 "1")은 고음에서 시작하여 계속 같은 높이로 평탄하게 발음하는 것이고, 제2 성(" ´ " 또는 "2")은 중음에서 시작하여 고음으로 상승하며 발음하는 것이고, 제3 성(" ˇ " 또는 "3")은 중저음에서 시작하여 저음으로 내려갔다가 다시 올라가는 음으로 발음하는 것이고, 제4성 (" ｀ " 또는 "4")은 고음에서 시작하여 급격히 최저음으로 내려가면서 발음하는 것이다. 그리고, 중국어는 성조가 없는 경우도 있다.Chinese phonetics is a phonetic symbol in Chinese phonetic transcription. For example, Chinese words "

"As with the Chinese character" Hello ", [], it can be the Chinese accent, that is, the pronunciation symbol written in Chinese romanization, including the four- Quot; or "1") is to start from a treble and continue to be flatter at the same height, and the second ("("" Or "3") starts from the bass and goes down to the bass and back up. The fourth sex ("` "or" 4 " In addition, there are cases where there is no Chinese language.

/ni3 hao3"와 같이 중국어 어휘와 그 발음 기호를 대비시켜 놓은 것인데, 이 것들을 발음 단위로 분리하여, ni3 / 니ˇ, hao3 / 하오ˇ와 같이 중국어 발음 단위에 대한 한국어 발음 기호가 한국어 발음대역어 데이터로 구축된 것이다. 한국어 발음 기호에도 중국어 성조 기호가 포함되어, 발음에 참고될 수 있다. As a result of actual implementation, 2,441 Chinese phonetic sets were extracted from Chinese phonetic dictionary and Chinese phonetic table, and Korean pronunciation words were constructed for each of them. Here, the Chinese pronunciation dictionary is "

/ ni3 hao3 ", which are divided into pronunciation units, and the Korean pronunciation symbols for the Chinese pronunciation units such as ni3 / ni and hao3 / The Korean pronunciation symbol also includes the Chinese character symbol, which can be referenced to pronunciation.

"이 발성되면, 중국어 g2p(grapheme to phoneme) 변환 데이터베이스 및 인식네트워크를 통해 중국어 음성인식 문장으로 "

"이 생성되고, 동시에 중국어 발음사전에 의해 "ni3 hao3"가 생성될 수 있다. 또한, 중한 발음대역어 데이터가 활용되어, 그 발성음의 한국어 발음 기호인 "니ˇ 하오ˇ"가 생성될 수 있다. 여기서, g2p 변환 데이터베이스는 중국어 발음사전에서의 한어병음을 음성인식을 위하여 일부 확장하여 재구성된 것이다.Based on the above-mentioned medium-term pronunciation data,

"When this is spoken, Chinese g2p (grapheme to phoneme) conversion database and Chinese speech recognition sentence through the recognition network"

Quot ;, and "ni3 hao3" can be generated by the Chinese pronunciation dictionary at the same time. Further, the middle pronunciation language word data is utilized, and the Korean pronunciation phonetic symbol "Nihao" . Here, the g2p conversion database is reconstructed by extending a part of Chinese words in the Chinese pronunciation dictionary for speech recognition.

Next, the Chinese pronunciation pronunciation word data for expressing the pronunciation sound of the Korean sentence with the Chinese pronunciation pronunciation symbol will be described.

In order to construct Korean-Chinese pronunciations, the first step is to divide Korean syllables into first, neutral, and last words. In the second step, they are converted into a Korean g2p conversion database. The phonetic symbols of Chinese phonetics and phonetic symbols of Chinese phonetics were used selectively. We will look more closely below.

All possible syllables in Korean are 3192, 19 initials, 21 neutral, 7 representative, and Korean consists of a combination of these, including the case where there is no uterus. As an example, such as "Han / ㅎ,, ro / ㅗ ㅗ". They are constructed by the Korean g2p conversion rules, with Korean g2p conversion tables, such as "Korean / Korean / h a xn" and "Korean / r o / r o". Here, g2p is a phonetic unit phonetic symbol, which is a unit for converting speech to text, and serves as a basis for constructing a phonetic dictionary. Actually, the Chinese g2p unit is used for the Chinese-Korean pronunciation word data.

In constructing Chinese-language pronunciation data for the Korean syllables, the Roman and Chinese words are mixed. In other words, 293 cases of Korean syllables that can be represented by Chinese words are reflected, while 2,899 other words are expressed in Roman numerals. An example of a Chinese word for Chinese g2p is "han / han xn / han", "han / ran / ro / lo". If it is difficult to find the phonetic symbol in Chinese phonetics, it is written in Roman numerals, such as "country / a ㄱ a / g u x g / guk". Thus, the established Chinese-language pronunciation word data has the form of "h a x n / han", "r o / lo", and "g u x g / guk".

The output unit 50 includes a display module or a sound output module. The output unit 50 converts a speech recognition sentence in a source language, a pronunciation symbol in a target language for a speech recognition sentence, and a text sentence converted into a target language into a speech recognition sentence Outputs phonetic symbols in the source language for text sentences on the screen or voice.

For example, such an automatic interpretation apparatus may be applied to an existing automatic translator or may be implemented as an application of a mobile terminal such as a smart phone.

2 is a flowchart illustrating an automatic interpretation method between Korean and Chinese in the automatic interpretation apparatus of FIG.

In step S210, the automatic interpretation apparatus receives the utterance sound of the source language. For example, the automatic interpretation device can receive a vocal sound in one of Korean and Chinese through a microphone. Alternatively, the input unit 10 may receive a text sentence in one of Korean and Chinese through a character input module.

In step S220, the automatic interpretation device receives speech sounds in the input source language, performs speech recognition, and generates a speech recognition sentence. For example, the automatic interpretation device performs signal processing on the speech sound of the input source language to separate speech sections, and then extracts speech features from the speech sections. On the other hand, the automatic interpretation device builds an acoustic model, a pronunciation dictionary, and a language model based on a voice database and a language database, and forms a recognition network in which these are integrated. Then, the automatic interpretation device converts the extracted voice features into a speech recognition text sentence through the recognition network, and outputs the converted speech recognition text sentence through the user screen. If the automatic interpretation apparatus receives a text sentence in either Korean or Chinese through the character input module, step S220 may be omitted.

In step S230, the automatic interpretation device generates a pronunciation symbol in the target language so that the user of the partner station can recognize or visually recognize the speech recognition sentence in the source language, and outputs the pronunciation symbol on the user screen. For example, the automatic interpretation device can generate a Chinese pronunciation symbol "(an-nyeong-ha-se-yo)" when the speech recognition sentence is "Hello".

"로 변환할 수 있다.In step S240, the automatic interpretation device converts the speech recognition sentence in the source language or the received sentence into a text sentence in the target language and outputs it through the user screen. For example, if the speech recognition sentence is "Hello ", the automatic interpretation device will send the word "Hello" to the Chinese word "Hello"

"

"인 경우, "니ˇ 하오ˇ"라고 한국어 발음 기호를 생성할 수 있다.
In step S250, the automatic interpretation device generates a pronunciation symbol in the source language so that the user of the partner station can recognize or visually recognize the text sentence in which the speech recognition sentence is converted into the target language, and outputs the pronunciation symbol on the user screen. For example, an automatic interpretation device may be able to translate a text sentence into Chinese into "

Quot ;, it is possible to generate a Korean pronunciation symbol "Nihao ".

Meanwhile, the automatic interpretation method according to an embodiment of the present invention may be implemented in the form of a program command that can be executed through a variety of means for processing information electronically and recorded in a storage medium. The storage medium may include program instructions, data files, data structures, and the like, alone or in combination.

Program instructions to be recorded on the storage medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of software. Examples of storage media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, magneto-optical media and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The above-mentioned medium may also be a transmission medium such as a light or metal wire, wave guide, etc., including a carrier wave for transmitting a signal designating a program command, a data structure and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as devices for processing information electronically using an interpreter or the like, for example, a high-level language code that can be executed by a computer.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

10: Input unit
20:
30: translation department
40:
50: Output section
60: Pronunciation band database
70: Translation engine

Claims (11)

An input unit for receiving a sound of a source language;
A speech recognition unit for recognizing a speech sound of the source language and generating a speech recognition sentence;
A translation unit for converting the speech recognition sentence into a text sentence in a target language;
A pronunciation language database for generating a pronunciation symbol in the target language and the source language so that the user of the partner station can recognize or visually recognize the speech recognition sentence and the text sentence in the target language, Generating unit; And
And an output unit for outputting the speech recognition sentence, the text sentence, the target language, and pronunciation symbols in the source language on a screen or in a voice.
The method according to claim 1,
Wherein the input unit receives a vocal sound in one of Korean and Chinese through a microphone or receives a text sentence in one of Korean and Chinese through a character input module.
The method according to claim 1,
Wherein the pronunciation band database stores Chinese pronunciation band word data for expressing a pronunciation sound of a Chinese sentence with a Korean pronunciation symbol and Chinese pronunciation pronunciation word data for expressing a pronunciation sound of a Korean sentence with a Chinese pronunciation symbol, .
The method of claim 3,
Wherein the Korean pronunciation band word data includes information in which a Korean pronunciation symbol corresponding to a Chinese pronunciation unit is mapped by separating a Chinese word pair into a pronunciation unit,
Characterized in that the Chinese phonetic symbol is a phonetic symbol for expressing the pronunciation of the Chinese Chinese character in the Chinese roman alphabet including the four-phonetic symbol.
The method of claim 3,
Wherein the Chinese phonetic word data includes information obtained by dividing a Korean syllable into a first sentence, a neutral sentence, and a last sentence, and mapping the first sentence, the neutral sentence, and the last sentence to a phonetic symbol or a roman notation of a Chinese word.
The method according to claim 1,
Wherein the automatic interpretation device is implemented as an application of a mobile terminal.
An automatic interpretation method performed by an automatic interpretation device,
Receiving a speech sound of a source language;
Generating a speech recognition sentence by voice recognition of a speech sound of the source language and outputting the speech recognition sentence;
Generating and outputting a pronunciation symbol in a target language so that the user of the partner station can reproduce or visually recognize the speech recognition sentence using the pronunciation band database;
Converting the speech recognition sentence into a text sentence in the target language and outputting the sentence; And
And generating and outputting phonetic symbols in the source language so that the user of the partner station can reproduce or visually recognize the text sentence in the target language by using the pronunciation band database.
8. The method of claim 7,
The method of claim 1,
Receiving a vocal sound in one of Korean and Chinese through a microphone; or
Inputting a text sentence in one of Korean and Chinese through a character input module.
8. The method of claim 7,
Wherein the pronunciation band database stores Chinese pronunciation band word data for expressing a pronunciation sound of a Chinese sentence with a Korean pronunciation symbol and Chinese pronunciation pronunciation word data for expressing a pronunciation sound of a Korean sentence with a Chinese pronunciation symbol .
10. The method of claim 9,
Wherein the Korean pronunciation band word data includes information in which a Korean pronunciation symbol corresponding to a Chinese pronunciation unit is mapped by separating a Chinese word pair into a pronunciation unit,
Wherein the Chinese phonetic alphabet is a phonetic alphabetic character expressed by a Chinese roman alphabet including a four-phonetic phonetic alphabet.
10. The method of claim 9,
Wherein the Chinese phonetic word data includes information obtained by dividing a Korean syllable into a first sentence, a neutral sentence, and a last sentence, and mapping the first sentence, the neutral sentence, and the last sentence to a phonetic symbol or a roman notation of a Chinese phoneme.

Images