KR20120060666A - Apparatus and method for extracting noun-phrase translation pairs of statistical machine translation - Google Patents

Abstract

PURPOSE: An apparatus and a method for extracting a noun phrase translation pair are provided to automatically extract a translation pair from a parallel language corpus. CONSTITUTION: A source language noun phrase extracting unit(140) extracts a noun phrase from a noun phrase analysis result of a source language sentence. A target language candidate noun phrase extracting unit(130) extracts a target language noun candidate from the analysis result. A translation pair score calculating unit(150) calculates an arrangement probability. A translation pair extracting unit(160) extracts translation pair having high score.

Description

Apparatus and method for extracting noun phrase band pairs from statistical machine translation {APPARATUS AND METHOD FOR EXTRACTING NOUN-PHRASE TRANSLATION PAIRS OF STATISTICAL MACHINE TRANSLATION}

The present invention relates to statistical machine translation, and more particularly, using a part-of-speech information that can form a noun phrase from a corpus of word-aligned source language sentences and target language sentences. By extracting a noun phrase pair that has a high probability of sorting from a noun phrase candidate pair as a noun phrase band pair, if there is a parallel corpus for the source language and the target language, a noun phrase can be automatically extracted from the parallel corpus to extract a band pair without depending on the word alignment. Noun phrase band pair extraction apparatus and method.

Automatic translation technology refers to a software technology that automatically switches from one language to another. This technology has been studied in the United States for military purposes since the middle of the 20th century, and is now being actively researched by numerous research institutes and private companies for the purpose of expanding the scope of information access and innovation of the human interface.

The early stages of automatic translation technology have evolved based on manual bilingual dictionaries written by experts and rules for translating one language into another. However, from the beginning of the 21st century, when the rapid development of computing power has progressed, the development of statistical-based translation technology that automatically learns the translation algorithm from a large amount of data has been actively developed.

Statistical-based translation technology has become an important axis in the field of machine translation along with rule-based machine translation and example-based machine translation. In particular, as statistical machine translation extends from word-based machine translation models to phrase-based machine translation models, example-based machine translation blurs the boundaries between statistical machine translation and rule-based machine translation along with statistical machine translation. It became the main axis.

The important issues in phrase-based statistical machine translation are how to distinguish the boundaries of a phrase and how to find a band phrase. Currently, the most commonly used method is to find a phrase based on a word sorting result according to the definition shown in Equation 1 below.

The band-band pairs found are phrases that match the word alignment, and their boundaries can be distinguished in the same manner as in Table 1 below.

In the figure in Table 1, the hatched cells are word-aligned, and the rectangular areas boxed with bold lines represent the boundaries of the sphere. No? Like did not, if there are multiple pairs of words sorted in one word, all the pairable words are grouped into the same phrase, and if a word in one language belongs to a particular phrase, the words in the other language are sorted. It is a method of including them as band pairs. It also extracts only phrases that contain at least one word alignment.

Table 1 (a) shows the boundary of the phrase that satisfies all of the above conditions, (b) shows the boundary of the phrase that does not meet all the above conditions because it does not include all the words arranged in one word, (c) When a word belongs to a specific phrase, the words of other languages that are sorted also represent the boundary of the phrase that does not meet the criteria for inclusion in the band pair.

This band-band based statistical machine translation shows relatively stable performance in various language pairs without using linguistic information.

However, in the case of a band word having poor performance of word alignment, such as English-Korean, the error is propagated.

1 is a diagram illustrating a word alignment result of an English-Korean sentence pair as an example.

When a region (A, B) that combines several words is a noun phrase to extract, '⑧safety' in the English sentence is linked to '① Naturally' in the Korean sentence. In this case, if a word arranged in one word is a pair, a band phrase is extracted by applying an existing method of enclosing the same phrase, and the 'the cause of the safety accidents' and 'naturally cause of safety accidents' are band pairs. Will be extracted. Also, when extracting the band phrases of 'the safety accidents' or 'the cause', the Korean sentence '④' is connected to '④the' and '⑦the' of the English sentence, so that 'the cause of the safety accident' or 'cause' The bands are extracted as pairs.

As such, when a sphere-based band pair is extracted according to the conventional method, there is a problem in that an appropriate band sphere is not found.

The present invention has been made to solve the above problems, using both parts of the language to extract the noun phrase candidate for each language by using the part-of-speech information that can form a noun phrase in the corpus of words aligned with the source language sentence and the target language sentence By extracting noun phrase candidate pairs that have a high probability of sorting from the noun phrase candidate pairs as a noun phrase band pair, if there is a parallel corpus for the source language and the target language, the noun phrases are automatically extracted from the parallel corpus and the band is not dependent on the word alignment. It is an object of the present invention to provide an apparatus and method capable of pair extraction.

To this end, according to a first aspect of the present invention, an apparatus for band pair extraction in statistical machine translation, comprising: a source language noun phrase extractor for extracting a noun phrase from a result of parsing a source language sentence; A target language noun phrase candidate extractor extracting a noun phrase candidate corresponding to a noun phrase of the source language from a result of analyzing a target language sentence in a morpheme unit; A band pair score calculator configured to calculate a band pair score by calculating an alignment probability between a noun phrase of a source language extracted from the source language noun phrase extractor and a noun phrase candidate of a target language extracted from the target language noun phrase candidate extractor; And a band pair extractor for extracting a band pair having the highest score among scores calculated by the band pair score calculator.

According to a second aspect of the present invention, a terminal device for extracting band pairs in a statistical machine translation of the present invention includes: a memory storing a result of syntax analysis of a source language sentence and a result of morphological analysis of a target language sentence; A source language noun phrase extractor for extracting a noun phrase of a source language from a result of parsing a source language sentence stored in the memory; A target language noun phrase candidate extractor for extracting a noun phrase candidate of a target language corresponding to a noun phrase of a source language from a morphological analysis result of the target language sentence stored in the memory; A band pair score calculator configured to calculate a band pair score by calculating an alignment probability between a noun phrase of a source language extracted from the source language noun phrase extractor and a noun phrase candidate of a target language extracted from the target language noun phrase candidate extractor; And a band pair extractor for extracting a band pair having the highest score among scores calculated by the band pair score calculator.

According to the third aspect of the present invention, the target language noun phrase band pair extractor of the present invention attaches part-of-speech information to the target language corpus analyzed in morpheme units, and uses the part-of-speech information from the target language corpus analyzed in the morpheme units. And extracting a noun phrase candidate.

According to a fourth aspect of the present invention, a method of the present invention includes a method for band pair extraction in statistical machine translation, comprising: (a) extracting a noun phrase from a result of parsing a source language sentence; extracting noun phrase candidates corresponding to a noun phrase of the source language from a result of analyzing a target language sentence in a morpheme unit; (c) calculating a band pair score by calculating an alignment probability between the noun phrase of the source language and the noun phrase candidate of the target language; (d) extracting a band pair having the highest score among the calculated band pair scores.

According to a fifth aspect of the present invention, the method for extracting a noun phrase band pair according to the present invention comprises attaching part-of-speech information to a target language corpus analyzed in morpheme units and from the result of word alignment of the target language corpus analyzed in morpheme units. And extracting a noun phrase candidate using the information.

According to the present invention, if there is a parallel corpus for a source language and a target language, a noun phrase band pair is automatically extracted, and thus, it is possible to automatically extract a band pair of a new word not existing in the parallel corpus.

In addition, if the existing band pair extraction has an error in the word alignment, an error occurs from the extraction of the band pair candidate. However, since the method is not dependent on the word alignment, the band pair extraction can be performed with less influence even when there is an error in the word alignment. It works.

In addition, the present invention basically eliminates the need for a parser for both languages, since band pair extraction is possible using only the parser for the source language. Thus, the present invention has an effect that the performance of the parser can be less affected than using a parser of both languages.

1 is an exemplary diagram illustrating a word alignment result of an English-Korean sentence pair.
2 is an exemplary diagram showing a result of analyzing a sentence structure of an English sentence.
3 is a diagram illustrating an apparatus capable of extracting a noun phrase band pair according to an embodiment of the present invention.
4 is a flowchart illustrating a noun phrase band pair extraction method according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The configuration of the present invention and the operation and effect thereof will be clearly understood through the following detailed description. Prior to the detailed description of the present invention, the same components will be denoted by the same reference numerals even if they are displayed on different drawings, and the detailed description will be omitted when it is determined that the well-known configuration may obscure the gist of the present invention. do.

The present invention is to use the results of the word alignment from the parallel corpus in the statistical machine translation using parallel corpus but to extract the noun phrase candidate from the word alignment results based on this, if there is an error in the word alignment is less affected by the machine It provides a structure that can be analyzed for translation.

The source sentence or source language sentence mentioned below is a sentence of a source language to be translated, and the target sentence or target language sentence is a target language output by translating the source sentence into a desired language. It means a sentence.

2 is an exemplary diagram showing a result of analyzing a sentence structure of an English sentence. For reference, FIG. 2 includes the same example as FIG. 1 to help understand the description.

As a result of the syntax structure analysis of the English sentence of FIG. 2, a word or phrase classified as NP (Noun Phrase) or BNP from a source tree may be viewed as a noun phrase.

That is, 'the cause', 'the safety accidents', and 'the cause of the safety accidents' correspond to the English sentences. In addition, the pronoun 'it' may also be included as a noun phrase, which is omitted in the present invention as a noun phrase consisting of a single word.

As a result of the syntax analysis, the English noun phrase is determined, and the noun phrase extraction of the target language sentence corresponding thereto uses morpheme analysis, parts of speech information, word alignment information, and the like. In particular, if the target language is Korean, Korean is a deadlock, and if the morphological analysis is not performed, the lack of data may become serious. Therefore, in the case of Korean, the results of morphological analysis must be used. In addition, when all possible morphological sequences of Korean language are considered, the noun phrase candidate may be excessive, so that the noun phrase candidate may be extracted using the part of speech information.

The apparatus of the present invention for implementing this is described in detail.

3 is a diagram illustrating an apparatus capable of extracting a noun phrase band pair according to an embodiment of the present invention.

Referring to FIG. 3, an apparatus according to an embodiment of the present invention includes a target morphological analysis corpus DB 110, a source parsing corpus DB 120, a target language noun phrase candidate extractor 130, a source language noun phrase extractor 140, A noun phrase band pair score calculator 150, a band pair extractor 160, and a noun phrase band dictionary DB 170.

Here, the target morphological corpus DB 110, the source parsing corpus DB 120, the noun phrase band dictionary DB 170 is a hard disk or an external device when the apparatus of the present invention is implemented in a computer stand-alone. It is stored in a storage medium such as a memory, and if implemented on a network basis is stored in a server. In addition, the device of the present invention can be stored in the memory of the portable terminal in the case of a high performance, high capacity portable terminal.

The target morpheme analysis corpus DB 110 stores corpus information obtained by morphologically analyzing sentences of a target language, and the source syntax analysis corpus DB 120 stores corpus information parsed a sentence of a source language.

Here, the target morphological corpus DB 110 and the source parsing corpus DB 120 are pre-processed and stored source language sentences and target language sentences in order to extract noun phrases from sentences of each language.

In this case, the target morphological analysis corpus DB 110 is a configuration that is limited to the case where the target language is necessarily necessary due to the characteristics of the language, such as Korean, and the like. You can save the corpus. In this case, the target morpheme analysis corpus DB 110 may further include a target part-of-speech corpus DB (not shown) attached and stored by attaching the part-of-speech information to the target language sentence.

The source language noun phrase extractor 140 receives the parsed source corpus from the source parsing corpus DB 120 and extracts the noun phrase in the source language sentence. This can be extracted by grouping words classified as BNP or NP from the parsed tree as mentioned in FIG. 2.

The target language noun phrase candidate extractor 130 receives the target corpus analyzed in morpheme units from the target morpheme analysis corpus DB 110 and extracts a noun phrase candidate of the target language sentence.

In this case, in the case of the English-Korean corpus, the target language noun phrase candidate extractor 130 applies a union heuristic that considers the word alignment from the English-Korean, Korean-English corpus, and applies the heuristic as a result of applying the heuristic. We apply the old band pair extraction method based on the word alignment described above. As a result, all Korean phrases corresponding to English noun phrases are selected. In addition, a phrase that can be composed of parts of speech extracted by extracting a part-of-speech that can form a noun phrase from the selected phrase is selected as a noun phrase candidate.

Parts of speech that can form noun phrases are parts of speech that can be formed only at the beginning of noun phrases, parts of speech only at the end of noun phrases, parts of noun phrases consisting of three or more morphemes, Possible parts-of-speech and the like.

Morphs that can begin and end noun phrases Noun, investigation, pronoun A morpheme that can only come at the end of a noun phrase Suffix (-s) A morpheme that can only come in the middle of a noun phrase Connection investigation (-and /), tubular screening (-), tubular mother

As such, the target language noun phrase candidate extractor 130 extracts the noun phrase of the source language and the noun phrase candidate of the target language for each sentence of each language, and extracts a proper noun phrase band pair for all possible pairs.

The noun phrase band pair score calculator 150 sorts all the pairs corresponding to the noun phrase candidate of the target language extracted from the target language noun phrase candidate extractor 130 and the noun phrase pair of the source language extracted from the source language noun phrase extractor 140. Analyze the probability. The analyzed sort probability values are called scores.

The method for calculating the score adds up one or more scores from one of the items shown in Table 3 below.

1. The number of links in which source and target nouns are sorted during intersection sorting. 2. Number of links in which source and target nouns are sorted during union sort. 3. The number of nouns in the noun phrase candidate of the target language 4. Number of nouns in the noun phrase of the source language 5. Number of words in noun phrase candidate of target language 6. Number of words in noun phrases in source language 7. Number of word-aligned links across nominal band pair candidates in only one language 8. Number of noun-aligned links across noun phrase band pair candidates in only one language. 9. Total Segment Sort Links / Union Sort Links 10. Number of noun phrases in the source sentence / Noun phrase candidates in the target sentence

In addition, the noun phrase band pair score calculator 150 performs an intersection word sort corpus (stored in the word sort corpus 180) when analyzing the probability of sorting all pairs corresponding to the noun phrase of the source language and the noun phrase candidate of the target language. 182) or extract all possible pairs based on union word sort corpus 184. The intersection sort refers to the intersection of both English and Korean word sort results, and the union sort refers to the union of both English and Korean word sort results.

The band pair extractor 160 extracts a noun phrase candidate of one target language having the highest score for the noun phrase of each source language from the noun phrase band pair score calculator 150. In this case, when there are several noun phrase candidates of the target language having the highest score, all noun phrase candidates having the corresponding score are extracted. The extracted noun phrase band pair may be stored and updated in the noun phrase band dictionary DB 170 to serve as basic data for constructing the noun phrase band dictionary.

As such, the present invention uses the word alignment information, but does not apply the condition to include the words of all the other languages aligned with one word in the same phrase, 'naturally' aligned with 'safety' in the example of FIG. Or 'the' aligned with 'the' is not necessarily included in the same phrase.

Accordingly, the present invention increases the amount of noun phrase pairs that can be extracted in spite of errors in word alignment. However, the number of candidates for Korean noun phrases in one sentence may be excessive, resulting in an excessive operation of the entire system. Accordingly, the present invention is limited to exclude noun phrase candidate pairs of noun phrase-target language of the source language in the following three cases to increase the efficiency of the system.

-Noun phrases in the source language (English noun phrases)-except when there is no union sort in candidate noun phrases in the target language (Korean noun phrases)

-If the number of nouns on one side is more than three times the number of nouns on the other side

-Except when the length of one side (the number of words or morphemes) is at least five times that of the other.

In the above conditions, the criteria for the second and third conditions are English and words are Korean.

As a result of applying the above restrictions, the efficiency of the system is better than before, and the result of the extracted noun phrase pairs does not change much.

Next, a method of extracting a noun phrase band pair using the above apparatus will be described with reference to FIG. 4.

First, as a preprocessing process for the source language sentence, the source corpus 12 stored in the parallel corpus DB 10 is attached with parts of speech information for each word and parsed through a parser based on the parts of speech information (S110). . Then, the parsed result is stored in the source parsing / part-of-speech corpus DB 120 to build a database (S140).

At the same time, the target corpus 14 stored in the parallel corpus DB 10 is morphologically analyzed and the part-of-speech information is attached to the analyzed morphemes (S120). Then, the result is stored in the target morphological analysis / POS attached corpus DB 110 to build a database (S140).

In addition, as another preprocessing process, the parallel corpus DB 10 is word aligned to build a source-target word alignment DB 180 (S130 and S140). Here, the source-target word alignment DB 180 is a union word sort that combines all sorting results in one direction with respect to the source language-target language, and a sorting result for both directions of the source language and the target language. Intersections in mode can be divided into one intersection word order.

The database (110, 120, 180) and corpus DB (10) constructed as described above are stored in a storage medium such as a hard disk or an external memory when the device of the present invention is implemented in a stand-alone computer, and implemented on a network basis. The case is stored on the server to build. In addition, in the case of a portable terminal having high performance and high capacity, the device of the present invention can be stored in a memory of the portable terminal and constructed.

Then, the noun phrase is extracted from the corpus stored in the source parsing / POS attached corpus DB 120 (S150). Its extraction method can be easily obtained by parsing the source corpus.

From the corpus stored in the target morphological analysis / part of speech corpus DB 110, a noun phrase candidate of the target language is extracted (S160). Its extraction method initially selects the noun phrases of the target language that can correspond to the noun phrases of the source language, and extracts the parts of speech based on the extracted parts of speech from the selected phrases based on statistical information. Phrase that can be composed is extracted as a noun phrase candidate.

That is, the parts of the morpheme that can only come in the middle of the noun phrase only at the beginning, end, and end of the noun phrase are distinguished and all the phrases satisfying the conditions are extracted.

Subsequently, a score obtained by analyzing alignment probabilities for all pairs corresponding to the noun phrase candidate of the source language and the target language noun phrase extracted in the above process is calculated (S170).

That is, the number of nouns / words / noun phrases in noun phrases in the source language, the number of nouns / words / noun phrases in noun phrase candidates in the target language, the number of links in which source nouns and target nouns are sorted by union or intersection word sorting, The score is calculated by the number of word / noun alignment links spanning the band pair candidate.

Thereafter, the noun phrase band pair having the highest score is selected by sorting the calculated scores (S180).

Thereafter, the selected noun phrase band pair is stored and updated in the noun phrase band dictionary to build a dictionary (S190).

According to this process, the noun phrase band pairs can be automatically extracted from the parallel corpus for the source language and the target language, so that even if there is an error in word alignment, the band pair extraction can be performed without being affected by the word alignment error. Band pairs of new words (unregistered words) can be automatically extracted from parallel corpus.

Meanwhile, the present invention can be applied to various reproduction apparatuses by implementing the word alignment method described above as a software program and recording it on a computer-readable predetermined recording medium.

Various playback devices may be PCs, laptops, portable terminals, and the like.

For example, the recording medium may be a hard disk, a flash memory, a RAM, a ROM, or the like as an internal type of each playback device, or an optical disc such as a CD-R or a CD-RW, a compact flash card, a smart media, a memory stick, or a multimedia card as an external type. have.

In this case, the program recorded on the computer-readable recording medium, as described above, extracts the noun phrase from the result of parsing the source language sentence and the result of analyzing the target language sentence in morpheme units. Extracting noun phrase candidates corresponding to and, calculating a band pair score by calculating an alignment probability between a noun phrase of a source language and a noun phrase candidate of a target language, and a band pair having the highest score among the calculated band pair scores. It may be performed including the process of extracting.

In addition, the program for extracting noun phrases for Korean may be executed by attaching part-of-speech information to a Korean corpus analyzed in morpheme units and extracting a noun phrase candidate from a result of word alignment of the Korean corpus analyzed in morpheme units.

The foregoing description is merely illustrative of the present invention, and various modifications may be made by those skilled in the art without departing from the spirit of the present invention. Therefore, the embodiments disclosed in the specification of the present invention are not intended to limit the present invention. The scope of the present invention should be construed by the claims below, and all techniques within the scope equivalent thereto will be construed as being included in the scope of the present invention.

In a statistical machine translation system, band pair extraction according to the prior art is dependent on word alignment, and if there is an error in word alignment, an error occurs from band pair candidate extraction, but the present invention extracts part-of-speech information from parallel corpus of source language and target language. By extracting a noun phrase candidate for each language and extracting a pair with a high probability that can be sorted from the noun phrase candidate pairs of the two languages, it is possible to easily extract an old band pair even when there is an error in word alignment. It is possible to automatically extract band pairs of new words (unregistered words) from parallel corpus. As a result, the quality of translation can be improved by using an automatic translator or an automatic dictionary.

110: target stemming corpus DB
120: source parsing corpus DB
130: target language noun phrase candidate extractor
140: source language noun phrase extractor
150: noun phrase band pair score calculator
160: band pair extractor
170: noun phrase band dictionary DB
180: word sort DB
182: Intersection word sort corpus
184: Union word sort corpus

Claims (27)

Apparatus for band pair extraction in statistical machine translation,
A source language noun phrase extractor for extracting a noun phrase of the source language from a result of parsing the source language sentence;
A target language noun phrase candidate extractor for extracting a noun phrase candidate of a target language corresponding to a noun phrase of the source language from a result of analyzing a target language sentence in a morpheme unit;
A band pair score calculator configured to calculate a band pair score by calculating an alignment probability between a noun phrase of a source language extracted from the source language noun phrase extractor and a noun phrase candidate of a target language extracted from the target language noun phrase candidate extractor;
Band pair extractor for extracting the band pair having the highest score among the scores calculated by the band pair score calculator
Noun phrase band pair extraction device in the statistical machine translation, comprising a. The method of claim 1,
The target language noun phrase candidate extractor
An apparatus for extracting noun phrase band pairs in statistical machine translation, comprising extracting a noun phrase candidate by extracting a part-of-speech that can form a noun phrase using the part-of-speech information of the target language sentence. The method of claim 1,
The band pair score calculator
Nouns in the source language and nouns in the target language by number of nouns / words / noun phrases in noun phrases of the source language, nouns / words / noun phrase phrases in noun phrase candidates of the target language, union or intersection word sorting An apparatus for extracting noun phrase band pairs in a statistical machine translation, comprising calculating one or more of the number of sorted links and only one language by adding one or more of the number of words / nouns aligned links within a noun phrase band pair candidate. The method of claim 1,
The band pair extractor
An apparatus for extracting noun phrase band pairs in statistical machine translation, wherein all band pairs are extracted when there are several band pairs having the highest score among the scores. The method of claim 1,
The apparatus for extracting a noun phrase band pair in statistical machine translation, wherein the result of the morphological analysis of the target language sentence includes a result of word alignment through a sorting algorithm. The method of claim 1,
The band pair score calculator
If there is no union of the noun phrases in the source language and the noun phrase candidates in the target language, and the number of nouns in one language is three or more of the nouns in the other language, the length of either language (word B) a noun phrase band pair extraction device in statistical machine translation, wherein the nominal phrase candidate of the target language is removed when the number of morphemes is 5 times or more than that of the other language. The method of claim 1,
And a noun phrase band pair extraction device in statistical machine translation, wherein the source language is English and the target language is Korean. A terminal for band pair extraction in statistical machine translation,
A memory for storing a result of parsing the source language sentence and a result of morphological analysis of the target language sentence;
A source language noun phrase extractor for extracting a noun phrase of a source language from a result of parsing a source language sentence stored in the memory;
A target language noun phrase candidate extractor for extracting a noun phrase candidate of a target language corresponding to a noun phrase of a source language from a morphological analysis result of the target language sentence stored in the memory;
A band pair score calculator configured to calculate a band pair score by calculating an alignment probability between a noun phrase of a source language extracted from the source language noun phrase extractor and a noun phrase candidate of a target language extracted from the target language noun phrase candidate extractor;
Band pair extractor for extracting the band pair having the highest score among the scores calculated by the band pair score calculator
And a second terminal. The method of claim 8,
The target language noun phrase candidate extractor
And extracting a noun phrase candidate by extracting a part-of-speech that can form a noun phrase using the part-of-speech information of the target language sentence. The method of claim 8,
The band pair score calculator
Nouns in the source language and nouns in the target language by number of nouns / words / noun phrases in noun phrases of the source language, nouns / words / noun phrase phrases in noun phrase candidates of the target language, union or intersection word sorting And calculating one or more of the number of sorted links and one or more of the number of words / nouns sorted links in one language only within a noun phrase band pair candidate. The method of claim 8,
The band pair extractor
Terminal for extracting all band pairs when there are several band pairs having the highest score among the scores. The method of claim 8,
The result of the stemming of the target language sentence comprises a result of word alignment through a sorting algorithm The method of claim 8,
The band pair score calculator
If there is no union of the noun phrases in the source language and the noun phrase candidates in the target language, and the number of nouns in one language is three or more of the nouns in the other language, the length of either language (word B) the nominal phrase candidate of the target language is removed when the number of morphemes is 5 times or more than that of the other language. Apparatus for extracting noun phrase band pairs by attaching part-of-speech information to a target language corpus analyzed in morpheme units, and extracting noun phrase candidates from the target language corpus analyzed in morpheme units. 15. The method of claim 14,
And the noun phrase candidate is generated by extracting all parts of speech that can form a noun phrase from the result of word alignment of the target language corpus. The method of claim 15,
Part of speech that can make up a noun phrase includes a part of speech only at the beginning of a noun phrase, a part of speech only at the end of a noun phrase, and a part of speech that is possible in the middle of a noun phrase consisting of three or more morphemes. Noun phrase band pair extraction device characterized in that. 15. The method of claim 14,
When no nouns in the noun phrases of the target language differ from the noun phrase of the target language, when the number of nouns in the target language is three or more of the nouns in the other language, the number of stems of the target language is different. The noun phrase band pair extraction apparatus of claim 5, wherein the noun phrase candidate of the target language is removed. 15. The method of claim 14,
The noun phrase band pair extraction apparatus, characterized in that the target language is Korean. A method for band pair extraction in statistical machine translation,
(a) extracting a noun phrase from a result of parsing a source language sentence;
extracting noun phrase candidates corresponding to a noun phrase of the source language from a result of analyzing a target language sentence in a morpheme unit;
(c) calculating a band pair score by calculating an alignment probability between the noun phrase of the source language and the noun phrase candidate of the target language;
(d) extracting a band pair having the highest score among the calculated band pair scores
Method for extracting noun phrase band pairs in statistical machine translation, comprising: a. The method of claim 19,
Step (b) is
A method for extracting noun phrase band pairs in a statistical machine translation, comprising extracting a noun phrase candidate by extracting a part of speech that can form a noun phrase using the part of speech information of a morphologically analyzed target language sentence. The method of claim 19,
The step (c)
Nouns / words / noun phrases in the noun phrase of the source language, nouns / words / noun phrases in the noun phrase candidate of the target language, number of links in which nouns of the source language and nouns of the target language are sorted by union or intersection A method for extracting noun phrase band pairs in a statistical machine translation, comprising calculating a sum of one or more of the number of words / noun alignment links in a noun phrase band pair candidate. The method of claim 19,
The step (d)
A method for extracting noun phrase band pairs in statistical machine translation, wherein all band pairs are extracted when there are several band pairs having the highest score among the scores. The method of claim 19,
The step (c)
If there is no union of the noun phrases in the source language and the noun phrase candidates in the target language, and the number of nouns in one language is three or more of the nouns in the other language, the length of either language (word And nominal phrase candidates of the target language are removed when the number of morphemes is greater than or equal to five times the other language. Attaching part-of-speech information to a target language corpus analyzed in morpheme units and extracting a target language noun phrase candidate using part-of-speech information from a result of word alignment of the target language corpus analyzed in morpheme units Way. The method of claim 24,
The target language noun phrase candidate extraction is based on parts of speech information that may constitute a target language noun phrase,
The part-of-speech information constituting the target language noun phrase is a part-of-speech that can be done only at the beginning of a noun phrase, a part-of-speech only at the end of a noun phrase, and a part-of-speech that can be located at any position in the middle of a noun phrase consisting of three or more morphemes. Noun phrase band pair extraction method comprising a. The method of claim 24,
When no nouns in the noun phrases of the target language differ from the noun phrase of the target language, when the number of nouns in the target language is three or more of the nouns in the other language, the number of stems of the target language is different. And a noun phrase band pair extraction method for removing a noun phrase candidate of the target language if it is 5 times or more. A computer-readable recording medium having recorded thereon a program for executing the process according to any one of claims 14 to 26.

Images