JP2006251843A - Synonym pair extracting device, and computer program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a synonym pair extracting device capable of accurately extracting a synonym pair even from few data. <P>SOLUTION: The synonym pair extracting device 32 includes a word string extracting part 52 creating synonymous expression pairs comprised of pairs of synonymous expressions included in a synonymous text from each of a plurality of synonymous texts composing a word parallel corpus 30, comparing the created synonymous expression pairs with each other, and extracting a word string pair comprised of two character strings common in each synonymous expression pair, and mutually different word strings between the two character strings as a candidate for the synonym pair, and a synonym pair filtering part 54 and a word trimming part 56 carrying out editing in accordance with predetermined rules with respect to candidates satisfying predetermined conditions from the extracted candidates for the synonym pairs, and outputting them as synonym pairs. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention generally relates to a technique for making machine processing related to a natural language more efficient, and more particularly to a technique for automatically acquiring synonym pairs from a corpus.

  Different notations that indicate the same concept or thing are called synonyms. There are many synonyms in any language, not just Japanese.

  The diversity of synonyms causes inconsistencies in lexical information. Therefore, there is a problem that the efficiency of natural language processing is reduced. This problem is particularly important in fields of application of natural language processing, such as information retrieval, automatic document summarization, document classification, and machine translation preprocessing.

  For example, “ticket”, “kip”, and “ticket” are synonymous. If it is known that these are synonyms, when “ticket” is input as a search term in the information search, data including “kip” and “passage ticket” can be output together. Document summaries are often based on frequency in determining the importance of words. In this case, rather than calculating the frequency of the words “ticket”, “kip”, and “passenger ticket” in the document database and handling them separately, these words can be combined and treated as representing the same concept. As a result, the nature of the word can be accurately captured. In other words, if the synonym cannot be determined correctly, a search omission may occur or the accuracy of document analysis may be reduced.

For example, there is a technique described in Patent Document 1 as a technique related to automatic acquisition of synonyms relating to Japanese. This technology is based on the relationship of sharing many co-occurrence and dependency relationships with known words from a monolingual corpus (corpus consisting of only Japanese sentences) and a thesaurus that stores existing synonyms. Thus, a concept represented by a word not registered in the thesaurus is estimated.
Japanese Patent Application Laid-Open No. 11-212975

  However, since the technique described in Patent Document 1 uses statistical information, a large amount of data is required as a monolingual corpus in order to increase the accuracy of word concept estimation. In addition, even if a large amount of data can be used, there is a problem that the obtained accuracy is not so high.

  Therefore, an object of the present invention is to provide a synonym pair extraction apparatus that can extract synonym pairs with high accuracy even from a small amount of data.

  A synonym pair extraction apparatus according to a first aspect of the present invention is a synonym pair extraction apparatus for extracting a synonym pair from a machine-readable monolingual parallel corpus including a plurality of synonym texts. Each of the texts includes a plurality of expressions representing the same meaning, and synonymous expression pair creating means for creating, from each of the plurality of synonymous texts, a synonym expression pair composed of a pair of synonymous expressions included in the synonymous text. The synonymous expression pairs created by the synonymous expression pair creating means are compared with each other, and a word string pair consisting of two character strings common to the synonymous expression pairs and different word strings sandwiched between the two character strings. Are extracted as synonym pair candidate extraction means from the two synonymous expressions constituting the synonym expression pair, and synonym pair candidates extracted by the synonym pair candidate extraction means Among them, for a given condition is satisfied ones, and a synonym versus output means for outputting as a synonym pairs by performing editing in accordance with a predetermined rule.

  The synonym expression pair creating means creates a synonym expression pair from each of the synonym texts. A synonymous expression pair includes two expressions representing the same meaning. The synonym pair candidate extraction means uses two synonyms for a word string pair consisting of two character strings common to two synonym expressions constituting a synonym expression pair and different word strings sandwiched between the two character strings. Extract from each representation. These become synonymous expression pair candidates. The synonym pair output unit performs editing according to a predetermined rule for synonym pair candidates that satisfy a predetermined condition and outputs the same as a synonym pair. A synonym pair is created from a monolingual parallel corpus, and a synonym pair is created from them. Since the monolingual parallel corpus is already divided into synonymous texts by its nature, synonym pairs can be extracted with high accuracy even with a small amount of data, compared to the case where synonym pairs are extracted from a simple monolingual corpus. Examples of synonymous expressions include synonymous sentences and two sentences or documents representing the same contents. Further, the “character string” mentioned above may be a single character or a plurality of characters. Similarly, the “word string” may be a single word or a plurality of words.

  Preferably, the synonym pair extraction device further includes a part-of-speech information adding unit for adding a corresponding part-of-speech information to a word constituting each expression included in the monolingual parallel corpus, and the synonym pair candidate extracting unit is The word strings constituting the synonymous expression pair created by the synonymous expression pair creating means are compared with each other including the part of speech of each word, the two words of the same part of speech common to the synonymous expression pair, and the two words Means for extracting a word string pair composed of different word strings sandwiched between two from the two synonymous expressions constituting the synonymous expression pair.

  A word string pair is extracted from a synonym pair using part-of-speech information to make a synonym pair candidate. Since the part-of-speech information is used, the extraction accuracy is increased.

  More preferably, the synonym pair output means includes, for each of the synonym pair candidates extracted by the synonym pair candidate extraction means, a word belonging to one of the synonym pair candidates among different word strings. A selection means for selecting only a synonym expression to which the other of the synonym pair candidates belongs and that satisfies a condition that it does not exist in a predetermined region other than a word string different from each other, and selected by the selection means Editing means for executing a editing process specified by the condition for a synonym pair candidate that matches a condition defined by a predetermined editing rule, and outputting a synonym pair.

  When words included in one of the word strings constituting the synonym pair candidate appear in a place that is completely unrelated on the other side, it is highly likely that the synonym pair candidate is not appropriate. By not selecting them, the accuracy of creating synonym pairs can be increased.

  The predetermined editing rule may include a predetermined pruning rule. The editing unit selects at least one of the synonym pair candidates according to the editing process specified by the pruning rule for a synonym pair candidate selected by the selecting unit that matches a condition defined by any of the pruning rules. Means for deleting a word or a word string located at one head or tail may be included.

  Of the synonym pair candidates, a common word is often unnecessary. By deleting these common words according to appropriate rules, the accuracy of the obtained synonym pairs is increased.

  Preferably, the selection unit is configured such that, for each of the synonym pair candidates extracted by the synonym pair candidate extraction unit, a word belonging to one of the synonym pair candidates is different from the synonym pair candidate. Means for selecting a synonymous expression to which the other candidate belongs and that satisfies the condition that it does not exist in any region other than different word strings.

  Of the different word strings, if a word belonging to one of the synonym pair candidates is in an area other than the different word strings of the other synonym expression, this synonym pair candidate may not be appropriate High nature. Therefore, the accuracy of creating synonym pairs can be increased by excluding such candidates.

  More preferably, the selection unit is configured such that, for each of the synonym pair candidates extracted by the synonym pair candidate extraction unit, a word belonging to one of the synonym pair candidates is different from the synonym pair candidate. Means for selecting only a synonymous expression to which the other of the pair candidates belongs and satisfying a condition that it does not exist in any of the adjacent regions of a predetermined length before and after different word strings.

  Of the different word strings, if a word belonging to one of the synonym pair candidates is in an area other than the different word strings of the other synonym expression, this synonym pair candidate may not be appropriate High nature. Therefore, the accuracy of creating synonym pairs can be increased by excluding such candidates. However, in this case, if the area other than the word strings that are different from each other is wide, the above-mentioned condition is often not satisfied. Therefore, accuracy can be increased by limiting the target region to a predetermined length region adjacent to the word string.

  When the computer program according to the second aspect of the present invention is executed by a computer, it causes the computer to operate as any of the above synonym pair extraction devices.

  Therefore, this computer program can achieve the same effects as any of the above synonym pair extraction devices.

<First Embodiment>
[Constitution]
FIG. 1 shows a block diagram of the synonymous expression extraction apparatus 32 according to the first embodiment of the present invention. The synonym expression extraction device 32 extracts an English synonym expression (synonym) from an English monolingual parallel corpus 30 using a morphological analysis dictionary 34 and a word pruning rule group 36 used for word pruning processing to be described later. And has a function of storing in the storage device 38.

  The configuration of the synonym expression extraction device 32 will be described later. First, the configuration of the monolingual parallel corpus 30 will be described. Referring to FIG. 2, the monolingual parallel corpus 30 includes a plurality of synonymous texts 70, 72, 74, 76,. Each synonymous text consists of a plurality of different texts describing the same content.

  In other words, the monolingual parallel corpus includes a plurality of synonymous texts, and each synonymous text has a configuration including two or more synonymous sentences. In addition, although it is called a “synonymous sentence” here, it may be a grammatical unit including a plurality of sentences, such as sentences or documents, as targeted in the second embodiment described later. In short, a monolingual parallel corpus is an expression that constitutes a grammatical unit, such as a sentence or sentence in one language, and a large number of semantically organized pieces are collected and grouped into groups that represent the same meaning. It is. The monolingual parallel corpus needs to be prepared to be machine readable.

  FIG. 3 shows an example of English synonymous text. The synonymous text 90 shown in FIG. 3 consists of three sentences. These three sentences shown in FIG. 3 are different in expression but have the same meaning. The synonymous texts 70, 72, 74, 76,..., 80 shown in FIG. 2 each include a plurality of sentences that represent the same content as the synonymous text 90 shown in FIG.

  The monolingual parallel corpus 30 itself may be created manually or mechanically in some form. For example, a monolingual parallel corpus may be created from a plurality of newspaper articles describing the same content. In the monolingual parallel corpus 30, sentences are classified in advance as synonymous texts. If there are different parts between two synonyms, it is highly likely that they are synonymous expressions, and such parts can be identified relatively easily. Therefore, even if the number of sentences included in the monolingual parallel corpus 30 is small, a relatively large number of synonym pairs can be acquired.

  Referring again to FIG. 1, the synonym expression extraction device 32 performs morphological analysis on each sentence included in the monolingual parallel corpus 30 using the morphological analysis dictionary 34, and provides a morpheme string to which information on parts of speech is given ( A morpheme analysis processing unit 50 for outputting a word string), and a morpheme sequence obtained from two sentences included in the same synonym text output from the morpheme analysis processing unit 50. A word string extraction unit 52 for extracting a plurality of word strings as synonym pair candidates including the preceding and following words, and a predetermined condition among the synonym pair candidates output by the word string extraction unit 52 For each of a plurality of synonym pair candidates output by the synonym pair filtering unit 54 and the synonym pair filtering unit 54 for selecting and outputting only those satisfying Pruning that deletes a common word or word string from synonym pair candidates according to the pruning editing rule defined by the rule for those that match the conditions of the pruning rule stored in the rule group 36 A word trimming unit 56 for performing processing and performing processing to be stored in the storage device 38.

  Hereinafter, each functional block constituting the synonym expression extracting device 32 will be described.

  The morpheme analysis processing unit 50 has a function of performing morphological analysis on each sentence included in the monolingual parallel corpus 30 using the morpheme analysis dictionary 34 and outputting each sentence in the form of a word string to which part-of-speech information is attached. Morphological analysis itself is a well-known technique and can be realized by a readily available program.

  The word string extraction unit 52 compares the word strings of two sentences included in the synonymous text output by the morphological analysis processing unit 50 and extracts different parts as synonym pair candidates including the preceding and following words. It has a function to do. The block diagram is shown in FIG.

  Referring to FIG. 4, this word string extraction unit 52 includes a word string storage unit 100 for storing a word string of each synonymous text given from the morphological analysis processing unit 50 for each synonymous text, and a word string storage unit 100. To select two synonym sentences from the synonym texts stored in, and to output synonym sentence pair creation unit 102 for outputting as synonym sentence pairs, and to store synonym sentence pairs created by synonym sentence pair creation unit 102 For each of the synonym sentence pair storage unit 104 and the synonym sentence pair stored in the synonym sentence pair storage unit 104, the two word strings constituting the synonym sentence pair are compared, and all of the following three types of conditions are compared. It includes a word string comparison unit 106 for extracting a word string pair that satisfies, and a synonym pair candidate storage unit 108 for storing the word string pair extracted by the word string comparison unit 106 as a synonym pair candidate.

  The synonym sentence pair creation performed by the synonym sentence pair creation unit 102 will be described with reference to FIG. Referring to FIG. 5, for example, synonymous text 90 includes word strings (a), (b), and (c) that form three sentences. The synonym sentence pair creation unit 102 creates all possible pairs between these three word strings. In the example shown in FIG. 5, the synonym sentence pair 122 consisting of the word strings (a) and (b), the synonym sentence pair 124 consisting of the word strings (a) and (c), and the word strings (b) and (c). A synonym sentence pair group 120 composed of synonym sentence pairs 126 is created. This process is a simple permutation / combination process.

  For each synonym sentence pair, the word string comparison unit 106 compares synonym sentences included in the synonym sentence pair with each other, and sets different word string pairs that satisfy all of the following three conditions to the preceding and following words. Are extracted as synonym pair candidates.

  The following three conditions are checked by the word string comparison unit 106.

(1) The words existing before the word string pair are equal to each other,
(2) The words existing after the word string pair are equal to each other, and (3) a predetermined condition regarding the part of speech of the word string pair is satisfied.

  The third “predetermined condition for part of speech” varies depending on the target to be collected as a synonym pair, such as “a word string composed of all nouns” or “a word (sequence) composed of one verb”. It means the condition set in.

  In addition, words determined to be equal to each other according to the above conditions (1) and (2) are referred to as common words.

  FIG. 6 shows extraction of synonym pair candidates by the word string comparison unit 106. Referring to FIG. 6, for example, when synonymous sentence pairs 122 are compared with each other, there are two word string pairs that satisfy the above-described conditions. One of them is the word string pair 132 and the other is the word string pair 134. For example, in the case of the word string pair 132, among the two word strings of the synonym sentence pair 122, “is” is equal to each other, and “that” is also equal to each other. The words (sequences) “supposed” and “expected” in the meantime are different from each other. Therefore, the word strings “is-supposed-that” and “is-expected-that” are extracted as synonym pair candidates. The same applies to the synonym pair candidate 134.

  The function of the synonym pair filtering unit 54 shown in FIG. 1 will be described below. The synonym pair filtering unit 54 is such that “a word in one word string of the synonym pair candidates does not exist in any region outside the other word string” among the synonym pair candidates. Select only. An example is shown in FIG.

  Referring to FIG. 7, common words of synonym pair candidates 150 are “REPORT” and “YUAN”. A word string of synonym pair candidates having these two words at the beginning and the end is shown in capital letters in FIG. Differences are indicated by rectangles 152 and 154. These are respectively referred to as “difference portion 152” and “difference portion 154”.

  The two words 160 (“VOICE”) and the word 162 (“AMERICA”) included in the difference 152 are not included in the other difference 154 and exist as the external words 170 and 172. Yes. Therefore, the synonym pair candidate 150 is deleted by the synonym pair filtering unit 54. The synonym pair filtering unit 54 has a function of selecting only those satisfying the above conditions and giving them to the word pruning unit 56.

  Finally, the word pruning unit 56 shown in FIG. 1 uses the rules stored in the word pruning rule group 36 for each of the synonym pair candidates selected by the synonym pair filtering unit 54. Process to trim the word string. Common words included in synonym pair candidates extracted from two word strings often include unnecessary words as synonyms. Therefore, the word trimming unit 56 deletes (prunes) those unnecessary words based on the part of speech of each word.

  Examples of rules included in the word trimming rule group 36 are as follows.

(1) Common words are excluded uniformly.
(2) Among common words, exclude common words with parts of speech that do not match words other than common words, or (3) exclude word string pairs that are part of other word string pairs.

  Among these rules, a combination of one or a plurality of rules that are considered appropriate in consideration of the purpose of processing is adopted. These rules are prepared in a machine-readable form in the form of “condition → processing”. The word trimming unit 56 determines whether or not the given word string matches the condition on the left side of the rule, and applies the process on the right side of the rule to the word string if it matches.

  FIG. 8 shows an example of the above-described word trimming. An example 240 shown in FIG. 8 is an example in which pruning is performed according to the above rule (2). For example, it is assumed that the word strings “the statement aid” and “the announcement aid” are synonym pair candidates. In this case, “statement” and “announcement” are different parts, and the part of speech is a noun. Accordingly, common words included in the synonym pair candidates other than nouns are excluded. As a result, both “the” and “said” are excluded, and “statement” and “announcement” are left as synonym pair candidates.

  The same applies to the other synonym pair candidates in Example 240, and the parts of speech of “Yemeni” and “Yemen” that are the differences in “The Yemeni governance” and “The Yemeni governance” are adjectives and nouns. Therefore, “The” whose part of speech does not match any of these is excluded, and the noun “government” is left.

  Example 242 shows what meets rule (3) above. For example, it is assumed that two synonym pair candidates “Building Materials = Construction Material” and “Building Materials Industry = Construction Material Industry” are extracted from one word string pair. Of these, the former word string is included in the latter word string. Therefore, according to the above rule (3), the former is excluded and only the latter is left.

[Operation]
The synonymous expression extraction device 32 whose configuration has been described above operates as follows. Referring to FIG. 1, first, a monolingual parallel corpus 30 is prepared in advance by hand or some machine processing. The morphological analysis dictionary 34 is also prepared in advance according to the format requested by the morphological analysis processing unit 50. The word trimming rule group 36 is also prepared in advance in a predetermined format according to the purpose by the user.

  The morphological analysis processing unit 50 of the synonym expression extracting device 32 performs morphological analysis on each sentence in each synonymous text included in the monolingual parallel corpus 30 with reference to the morphological analysis dictionary 34 and adds part-of-speech information to each word. . The word string obtained as a result is accumulated in the word string storage unit 100 (see FIG. 4) of the word string extraction unit 52 for each synonymous text.

  The synonym sentence pair creation unit 102 creates a synonym sentence pair from each synonym text and gives the synonym sentence pair storage unit 104.

  The word string comparison unit 106 reads each of the synonym sentence pairs stored in the synonym sentence pair storage unit 104, and extracts a word string pair as a synonym pair candidate from the synonym sentence pair by comparing the word strings. Accumulated in the synonym pair candidate storage unit 108.

  Referring to FIG. 1, synonym pair filtering unit 54 reads the synonym pair candidates stored in synonym pair candidate storage unit 108 of word string extraction unit 52, and as described above, “synonym pair” Only words that are not present in any region outside the other word string are selected and given to the word trimming unit 56.

  The word trimming unit 56 collates the synonym pair candidates given from the synonym pair filtering unit 54 with the conditions of the rules included in the word trimming rule group 36, and sets the rules for the synonym pair candidates that match the conditions. Apply the specified word trimming process. The word trimming unit 56 accumulates the synonym pair candidates after the word trimming process in the storage device 38 as synonym pairs.

  Thus, when processing for all synonymous texts included in the monolingual parallel corpus 30 is completed, a large number of synonym pairs are stored in the storage device 38.

  As described above, according to the synonym expression extracting device 32 according to the first embodiment, a large number of synonym pairs can be extracted from the monolingual parallel corpus 30. Even if the number of synonym pairs included in the monolingual parallel corpus 30 is relatively small, a large number of synonym pairs can be extracted with high accuracy.

[Realization by computer]
The synonym expression extraction device 32 (and functional blocks that are parts thereof) of this embodiment is realized by computer hardware, a program executed by the computer hardware, and data stored in the computer hardware. . FIG. 9 shows the external appearance of the computer system 330, and FIG. 10 shows the internal configuration of the computer system 330.

  Referring to FIG. 9, the computer system 330 includes a computer 340 having an FD (flexible disk) drive 352 and a CD-ROM (compact disk read only memory) drive 350, a keyboard 346, a mouse 348, and a monitor 342. including.

  10, in addition to the FD drive 352 and the CD-ROM drive 350, the computer 340 includes a CPU (central processing unit) 356 and a bus 366 connected to the CPU 356, the FD drive 352, and the CD-ROM drive 350. And a read only memory (ROM) 358 for storing a boot-up program and the like, and a random access memory (RAM) 360 connected to the bus 366 for storing a program command, a system program, work data, and the like. Computer system 330 further includes a printer 344.

  Although not shown here, the computer 340 may further include a network adapter board that provides a connection to a local area network (LAN).

  A computer program for causing the computer system 330 to operate as the synonymous expression extracting device 32 is stored in the CD-ROM 362 or FD 364 inserted in the CD-ROM drive 350 or FD drive 352 and further transferred to the hard disk 354. . Alternatively, the program may be transmitted to the computer 340 through a network (not shown) and stored in the hard disk 354. The program is loaded into the RAM 360 when executed. The program may be loaded directly into the RAM 360 from the CD-ROM 362, from the FD 364, or via a network.

  This program includes a plurality of instructions that cause the computer 340 to operate as the synonymous expression extraction device 32 of this embodiment. Some of the basic functions required to perform this operation are provided by operating system (OS) or third party programs running on the computer 340 or various toolkit modules installed on the computer 340. Therefore, this program does not necessarily include all functions necessary to realize the system and method of this embodiment. This program includes only instructions that execute the operation as the above-described synonymous expression extraction device 32 by calling an appropriate function or “tool” in a controlled manner so as to obtain a desired result. Just go out. The operation of computer system 330 is well known and will not be repeated here.

<Second Embodiment>
In the first embodiment described above, the monolingual parallel corpus 30 is constructed with synonym sentences as units, and the above synonym pairs are extracted for each synonym text. However, the present invention is not limited to such an embodiment. For example, the processing unit can be a document instead of a sentence. In this case, “synonymous documents” means, for example, articles distributed by multiple types of newspapers or news agencies on the same subject, paragraphs of documents translated by the same translator, or longer units than sentences. A document pair (document group) that is considered to represent almost the same content.

  In this case, a plurality of synonymous document groups may be stored in the monolingual parallel corpus 30 and the same processing as that performed in the first embodiment may be performed for each synonymous document group. However, when the above processing is performed in document units, the target range when comparing word strings is wider than in sentence units. As a result, for example, when the processing of the synonym pair filtering unit 54 shown in FIG. 1 is performed, “one of synonym pair candidates among“ synonym pair candidates ”used in the first embodiment. There is a possibility that the word “in the row does not exist in any area outside the other word row” and the condition of “select only those” is not satisfied. In that case, it is difficult to extract many appropriate synonym pairs.

  Therefore, in the second embodiment, instead of the filtering process condition performed by the synonym pair filtering unit 54, only a predetermined number of words before and after the extracted synonym pair candidates are targeted. The condition that a word in one word string of the pair candidate does not exist outside the other word string is applied. That is, among the synonym pair candidates, “a word in a different part in one word string of the synonym pair candidates is adjacent to the other word string, and within the range of N words before and after the word string, and the other word string Is not present at a position outside of "." This condition will be described with reference to FIG.

  Referring to FIG. 11, it is assumed that documents 400 and 402 are selected as a synonymous document pair. It is assumed that the word string 410 in the document 400 and the word string 412 in the document 402 are extracted as synonym pair candidates by comparing the word strings of the two documents.

  In this case, the ranges 420 and 422 including N words before and after the word string 410 and the ranges 430 and 432 including N words before and after the word string 412 are considered. In the present embodiment, a word in one word string (for example, word string 410) is outside the other (word string 412) and exists in N ranges 430 or 432 before and after word string 412. In this case, the synonym pair candidate consisting of the word strings 410 and 412 is not adopted. Only in other cases, this synonym pair candidate is adopted.

  By adopting such rules and performing synonym pair filtering, even when the comparison target is expanded from a sentence to a document unit, a large number of synonym pairs can be extracted with the same accuracy as in the first embodiment. Creating a synonymous document group in document units can be realized by machine processing relatively easily using resources collected in a network. Therefore, compared with the case of the first embodiment, there is an effect that the trouble of preparing the monolingual parallel corpus 30 is reduced.

  In this embodiment, the region where word strings are compared is limited in the form of a range of N words before and after the word strings constituting the synonym pair. However, in the present invention, this limitation may be determined in the form of a predetermined length. The predetermined length in this case may be determined, for example, in the form of 50% of the number of words from the beginning of the synonym pair candidate to the beginning of the sentence (or from the end to the end of the sentence), or the length of the target region may be changed before and after Also good. Further, the unit of length when the predetermined length is used is not necessarily limited to the number of words, but may be limited in the form of the number of content words.

<Modification>
Both the first and second embodiments described above are directed to English. However, the language to which the present invention is applicable is not limited to English. Any language is acceptable. For example, a sticky language such as Japanese or Korean, a refraction word represented by Indo-European language, or an isolated word represented by Chinese may be used.

  Moreover, in the said embodiment, a synonym pair candidate is extracted from the common word between synonym sentence pairs, and the word string of the difference part pinched | interposed by the common word. However, the present invention is not limited to such an embodiment. It is also possible to perform syntactic analysis and extract synonym pair candidates using the dependency relationship obtained as a result.

  In the first embodiment, as shown in FIG. 1, after the processing by the synonym pair filtering unit 54, the word pruning unit 56 performs word pruning. However, the order in which synonym pair filtering and word pruning are performed is not limited to this order. These may be performed in the reverse order. Furthermore, in the above-described embodiment, the morpheme analysis processing by the morpheme analysis processing unit 50 shown in FIG. 1 is performed before the synonym sentence pair creation by the synonym sentence pair creation unit 102 shown in FIG. However, the two processing orders are not limited to this order, and may be reversed.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a block diagram of synonymous expression extraction device 32 concerning a 1st embodiment of the present invention. 2 is a diagram schematically showing a configuration of a monolingual parallel corpus 30. FIG. It is a figure which shows the example of a synonymous text. 3 is a diagram illustrating a configuration of a word string extraction unit 52. FIG. It is a figure for demonstrating operation | movement of the synonym sentence pair preparation part 102 shown in FIG. It is a figure for demonstrating operation | movement of the word string comparison part 106 shown in FIG. It is a figure for demonstrating operation | movement of the synonym pair filtering part 54 shown in FIG. It is a figure for demonstrating operation | movement of the word pruning part 56 shown in FIG. It is an external view of the computer system which implement | achieves the synonymous expression extraction apparatus 32 which concerns on the 1st Embodiment of this invention. FIG. 10 is a block diagram of the computer shown in FIG. 9. It is a figure for demonstrating the synonym pair filtering process in the synonym expression extraction apparatus which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

30 monolingual parallel corpus 32 synonym expression extraction device 34 morpheme analysis dictionary 36 word pruning rule group 38 storage device 50 morpheme analysis processing unit 52 word string extraction unit 54 synonym pair filtering unit 56 word pruning unit

Claims (7)

A synonym pair extraction apparatus for extracting a synonym pair from a machine-readable monolingual parallel corpus including a plurality of synonymous texts, each of the plurality of synonym texts including a plurality of expressions representing the same meaning to each other. ,
Synonymous expression pair creating means for creating a synonymous expression pair consisting of a pair of synonymous expressions included in the synonymous text from each of the plurality of synonymous texts;
The synonymous expression pairs created by the synonymous expression pair creating means are compared with each other, and a word string pair consisting of two character strings common to the synonymous expression pair and different word strings sandwiched between the two character strings Synonym pair candidate extraction means for extracting each as a synonym pair candidate from two synonym expressions constituting the synonym expression pair,
Synonym pair output means for performing editing according to a predetermined rule and outputting as a synonym pair for the synonym pair candidates extracted by the synonym pair candidate extracting means And a synonym pair extraction device. Part-of-speech information giving means for giving corresponding part-of-speech information to words constituting each of the expressions included in the monolingual parallel corpus;
The synonym pair candidate extraction means compares the word strings constituting the synonym expression pair created by the synonym expression pair creation means, including the part of speech of each word, and has the same part of speech common to the synonym expression pair. 2. Means for extracting a word string pair composed of two words and different word strings sandwiched between the two words from two synonymous expressions constituting the synonymous expression pair, respectively. The synonym pair extraction device described. The synonym pair output means includes:
For each of the synonym pair candidates extracted by the synonym pair candidate extraction means, a word belonging to one of the synonym pair candidates among the different word strings belongs to the other of the synonym pair candidates. A selection means for selecting only those satisfying the condition that the synonymous expression does not exist in a predetermined region other than the different word strings;
Among the synonym pair candidates selected by the selection means, for executing a editing process specified by the condition and outputting a synonym pair for a condition that matches a condition defined by a predetermined editing rule The synonym pair extraction device according to claim 1, further comprising an editing unit. The predetermined editing rule includes a predetermined pruning rule,
The editing means, according to the editing process specified by the pruning rule, to the synonym pair candidate selected by the selecting means, which matches the condition specified by any of the pruning rules, The synonym pair extraction device according to claim 3, further comprising means for deleting a word or a word string located at the head or tail of at least one of the pair candidates. The selection means is configured such that, for each of the synonym pair candidates extracted by the synonym pair candidate extraction means, a word belonging to one of the synonym pair candidates among the different word strings is the synonym pair candidate. 5. The method according to claim 3, further comprising means for selecting a synonymous expression to which the other candidate belongs and that satisfies a condition that it does not exist in any of the regions other than the different word strings. Synonym pair extraction device. The selection means is configured such that, for each of the synonym pair candidates extracted by the synonym pair candidate extraction means, a word belonging to one of the synonym pair candidates among the different word strings is the synonym pair candidate. A means for selecting only a synonymous expression to which the other candidate belongs, satisfying a condition that it does not exist in any of the adjacent predetermined length regions before and after the different word strings. The synonym pair extraction device according to claim 3 or claim 4. A computer program that, when executed by a computer, causes the computer to operate as the synonym pair extraction device according to any one of claims 1 to 6.

Images