JP2006190072A - Automatic paraphrasing apparatus, automatic paraphrasing method and paraphrasing process program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic paraphrasing apparatus capable of automatically creating an optimal paraphrase of an inputted original text by means of a simple configuration without being affected by a developer's knowledge and ability. <P>SOLUTION: The automatic paraphrasing apparatus 80 includes an expression element database (DB) D1 for storing expression elements contained in an example corpus C1, together with their frequencies of occurrence; a paraphrase DBD 2 for storing paraphrases of example sentences contained in a bilingual corpus C2, together with paraphrase information showing the aspects of the paraphrases; a paraphrase information DBD 3 for storing information about paraphrasing from the example sentences in the bilingual corpus C2 into the paraphrases, together with their frequencies of application; a similar text search part 84 for searching the paraphrase DBD 2 for paraphrases each of which shares at least one of the expression elements with the original text; an expression verifying part 85 for evaluating the validity scores of each detected paraphrase about the paraphrasing of the original example text and the paraphrasing of the original text; and an original text paraphrasing part 83 for creating a paraphrase by applying the paraphrasing information that corresponds to the paraphrase of the highest validity scores to the original text in the opposite direction. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to natural language processing, and in particular, is used for pre-processing of, for example, a machine translation device, and converts an input original expression into another expression having the same meaning and suitable for later processing (in other words). The present invention relates to an automatic paraphrase device, an automatic paraphrase method, and a paraphrase processing program.

  As a machine translation device, there is known a machine translation device that, when an original sentence is inputted, translates the inputted original sentence into a plurality of sentences having different wordings, and attempts machine translation on any of the sentences after the paraphrase.

  However, in such a machine translation device, the operator needs to select the most appropriate original text for machine translation from a plurality of paraphrased texts. For this reason, the operator needs to have knowledge of both the source language and the target language (for example, the source language in Japanese-to-English translation is Japanese and the target language is English) and have the ability to select a paraphrase accurately. As a result, there is a problem that the users of the machine translation device are limited.

  Depending on the system, for example, "Can you see your passport?", Once translated into English and then translated back into Japanese, "Please show me your passport." In other words, it must be done manually. This means that the machine translation device can be used only by people who have the ability to translate into English.

  In addition, it is possible to obtain various expressions ranging from “Please show me your passport” and “Show me your passport” to “Now you can show me your passport”. The operator must know which of these various expressions is an expression that can be successfully translated by the machine translation device used. In this case, preprocessing of the machine translation apparatus can be appropriately performed only by a person who is familiar with the machine translation apparatus.

  Therefore, as shown in the following document, a technique for automatically editing or rewriting the original text before translation has been proposed.

JP-A-5-225232 JP-A-6-139274 JP 2000-268034 A Satoshi Shirai, Satoru Ikehara, Tsukasa Kawaoka, Yukihiro Nakamura, “Automatic Text Rewriting Translation Method and its Effects in Japanese-English Machine Translation”, Transactions of Information Processing Society of Japan, Vol. 36, no. 1, pp. 12-21 (1995) Yoshimi Akihiko, Sada Ichiko, Yuku Fukumochi, “Original Text Pre-editing for Realizing a Robust English-Japanese Machine Translation System”, Natural Language Processing, Vol. 7, no. 4, pp. 99-117 (2000)

  However, in such a translation apparatus equipped with the automatic source text editing function and the automatic rewriting function, the conditions for knowledge (hereinafter referred to as paraphrase knowledge) as the basis of the paraphrase operation are detailed in advance so that the result of the paraphrase does not become plural. Must be set to For this reason, the burden of development and preparation of paraphrase knowledge increases, and development of a machine translation device requires a large amount of resources, and the performance of the translation device greatly depends on the knowledge ability of the developer. .

  Accordingly, an object of the present invention is to provide an automatic paraphrase device that can automatically generate an optimal paraphrase for an inputted original sentence with a simple configuration and without being influenced by a developer's knowledge ability. is there.

  An automatic paraphrase device according to an aspect of the present invention is for storing an expression element that appears in a first example sentence group of a predetermined language together with an appearance frequency in each first example sentence group of the expression element. One or a plurality of paraphrases for each of the example segment storage means and each example sentence in the second example sentence group in the predetermined language are stored together with the paraphrase information indicating the paraphrase mode when the paraphrase is obtained. Paraphrase storage means for storing, paraphrase information storage means for storing paraphrase information indicating a paraphrase form from the example sentence to the paraphrase in the second example sentence group together with their application frequency, Search for a paraphrase that is stored in the paraphrase storage means for a paraphrase that shares at least one of the descriptive elements stored in the phrase segment storage means with the original sentence. Means and search means For each paraphrase, a paraphrase between the corresponding original example sentence and a paraphrase between the original sentence is calculated by a predetermined calculation method based on the frequency of application of the paraphrase information stored in the paraphrase information storage means. In the evaluation means for evaluating the validity score, and in the paraphrase storage means, the paraphrase information associated with the paraphrase in which the validity score evaluated by the evaluation means satisfies a predetermined condition is reverse to the original sentence. To the original sentence paraphrase means for generating a paraphrase for the original sentence.

  When the original sentence is input, a paraphrase that shares at least one of the original sentence and the expression segment stored in the expression segment storage unit is searched from the paraphrase storage unit by the search unit. Further refine the search from the searched paraphrase. At the time of narrowing down, the validity score evaluated by the evaluation means with respect to the paraphrase between the retrieved paraphrase and the corresponding original example sentence and the paraphrase between the retrieved paraphrase and the original sentence Is used. The validity score is evaluated by a predetermined calculation method based on the frequency of application of the paraphrase information stored in the paraphrase information storage means.

  In this way, the validity of the selected paraphrase is evaluated with respect to the two types of paraphrases, and based on this evaluation, the paraphrase between the original sentence and the paraphrase between the original sentence is considered A paraphrase having paraphrase information that seems to have the highest validity is selected. By applying the paraphrase information associated with the selected paraphrase and stored in the paraphrase storage means to the original sentence in the reverse direction, a paraphrase for the original sentence is generated. The paraphrase obtained in this way is close to the original example sentence corresponding to the paraphrase whose validity score satisfies a predetermined condition. Therefore, in a machine translation apparatus that uses the original example sentence as a resource for language conversion, the result of translating the paraphrase obtained by paraphrasing the original sentence is highly likely to be a correct translation sentence.

  As described above, in this automatic paraphrase device, a paraphrase for the original text that is highly likely to obtain a favorable result when translated is automatically identified. That is, the optimum paraphrase can be obtained automatically without requiring the operator's paraphrase knowledge. In addition, since the optimum paraphrase for the original sentence is specified using the data stored in the expression segment storage means, the paraphrase storage means, and the paraphrase information storage means, the conditions of paraphrase knowledge are set in detail in advance. There is no need, and the apparatus can be realized with a simple configuration.

  The retrieval means receives the original sentence to be paraphrased, and exchanges a paraphrase that shares at least one of the expression elements stored in the expression segment storage means with the original sentence of the paraphrase text stored in the paraphrase storage means. Representing the number of types of expression elements shared with the original sentence or the expression elements shared with the original sentence among the extraction means for extracting from inside and the paraphrase extracted by the extraction means Selection means for selecting an appearance frequency stored in the segment storage means, or a combination of these that satisfies a predetermined condition.

  After extracting a paraphrase that shares the original fragment with the original sentence, the number of types of the expression element shared with the original sentence among them, or the appearance frequency of the expression element shared with the original sentence, or A paraphrase in which these combinations satisfy a predetermined condition is selected. This selection is performed with reference to the appearance frequency of the same expression element stored in the expression element storage unit for the shared expression element of the original sentence and each paraphrase. Considering these, only a paraphrase that satisfies a predetermined condition is selected, so that only a good condition as a paraphrase corresponding to the original sentence can be selected.

  The selection means is based on the number of types of expression segments shared with the original text, the appearance frequency stored in the expression segment storage means regarding the expression segments shared with the original text, or a combination thereof. Means for selecting a predetermined number of paraphrases above the defined score may be included.

  Since a predetermined number of paraphrases with higher scores are selected, the processing time is always stable, and a paraphrase suitable for obtaining a paraphrase for the original sentence can be obtained.

  The evaluation means reduces the relevance of the paraphrase between the paraphrase and the example sentence when the paraphrase part having a small frequency of application of the corresponding paraphrase information includes more paraphrase sentences than the length of the paraphrase sentence or the example sentence. You may make it evaluate.

  Alternatively, the evaluation means evaluates the relevance of the similarity between the original text and the paraphrase lower when the paraphrase text contains more difference parts than the length of the original text or the paraphrase text where the corresponding paraphrase information is applied less frequently. You may make it do.

  By doing so, the weight of the paraphrase information related to each paraphrase can be normalized by the length of the example sentence corresponding to the original sentence, paraphrase, or paraphrase, and it is preferable without being affected by the length of the paraphrase. Can be selected.

  In addition, the paraphrase information stored in the paraphrase information storage unit may include a representation element pair in the paraphrase part of the sample sentence and the paraphrase. This is desirable in that the paraphrase information can be made objective and easy to handle.

  A paraphrase processing program according to another aspect of the present invention, when executed by a computer, causes the computer to operate as any one of the automatic paraphrasing devices described above.

  According to still another aspect of the present invention, an automatic paraphrase method stores expression segments that appear in a first example sentence group of a predetermined language and the frequency of appearance of each expression element in the first example sentence group. And one or a plurality of paraphrases for each example sentence in the second example sentence group in the predetermined language, together with paraphrase information indicating the paraphrase mode when obtaining the paraphrase Using the paraphrase database and the paraphrase information database storing the paraphrase information indicating the mode of paraphrase from the example sentence in the second example sentence group to the paraphrase together with their application frequency, the input original sentence is This is an automatic paraphrase method that paraphrases into one paraphrase, and stores in the paraphrase database a paraphrase that receives the original sentence to be paraphrased and shares at least one of the phrase segments stored in the phrase segment database with the original sentence. A search step for searching from the paraphrased text, and for each paraphrase searched in the search step, a paraphrase between the corresponding original example sentence and a paraphrase between the original text and the paraphrase information An evaluation step for evaluating a validity score calculated by a predetermined calculation method based on the frequency of application of paraphrase information stored in the database, and the validity evaluated in the evaluation step satisfies a predetermined condition in the paraphrase database. Generating a paraphrase for the original sentence by applying paraphrase information associated with the paraphrase in the opposite direction to the original sentence.

[Description of configuration]
As described later, the present embodiment can be realized by a computer and a computer program that operates on the computer. FIG. 1 is a block diagram functionally showing the device realized by the program and the computer as an automatic paraphrase device 80. Referring to FIG. 1, this apparatus 80 includes a source language example corpus C1 in machine translation, and an expression segment collection unit 81 for collecting a large number of expression segments to be described later from the source language example corpus C1. And the expression element database D1 that holds the expression element groups collected by the expression element collection unit 81 as a database together with their appearance frequencies.

  An expression segment is a unit in natural language expression that is preselected to express a difference between two expressions. How this is determined is a design matter and may vary depending on the language, purpose, and field of expression to be processed. In the present embodiment, the target language is Japanese, and the expression segment refers to a partial character string of a predetermined number of characters constituting a sentence. For example, in a character string that composes a sentence, two kanji characters are arranged, and two katakana characters are counted without including small characters “tsu”, “ya”, “yu”, “yo”, and “-”. In the case of including a change of the character, it means a character string including a number of characters determined by a simple standard such as three characters. By scanning the example sentences in order from the top in accordance with this rule, each example sentence can be decomposed into expression segments. In this case, in this embodiment, the last character of the preceding expression element and the first character of the subsequent expression element are shared (overlapped) by the two expression elements. Extract.

  In the case of writing a word such as English, the word can be used as an expression fragment. Even in Japanese, it may be expressed in units of words, etc., but it is necessary to perform word decomposition by performing morphological analysis, etc., which increases the processing load and determines the expression units as described above. Is practical.

  The automatic paraphrasing device 80 further includes a parallel corpus C2 that stores a large number of example sentences for translation between a source language and a target language used in machine translation, and a number of sentences in the source language included in the parallel translation corpus C2 by a known method. It includes a paraphrase generating unit 82 for generating paraphrase text and generating paraphrase information regarding what kind of substitution is performed when performing these paraphrases together with their application frequency.

  In this embodiment, the paraphrase information includes a representation element pair in which a representation replacement mode (that is, a paraphrase mode) is represented by a pair of expression segments. For example, an expression element pair is a different part and corresponds to each other between an expression element group obtained by ordering from a sentence and an expression element group obtained from a paraphrase of the sentence. This is a representation of pairs of expression elements that exist at a certain position.

  The automatic paraphrase device 80 further holds a paraphrase text database D2 that holds the paraphrase text generated by the paraphrase text generation unit 82 as a database together with the paraphrase information composed of pairs of expression units used when generating the paraphrase texts, Including a paraphrase information database D3 that holds paraphrase information composed of pairs of expression units used when the paraphrase text generation unit 82 generates a paraphrase text together with their application frequency.

  The automatic paraphrase device 80 further receives an original sentence from the original sentence paraphrase unit 83 for performing control as described below and a source sentence paraphrase part 83 in order to perform a process of creating one paraphrase for the input original sentence. , By decomposing the original sentence into a plurality of expression elements by referring to the expression element database D1, and among the paraphrases having the same expression element as the obtained expression element, a predetermined number of word sentences similar to the original sentence. Is extracted from the paraphrase text database D2 and the list is given to the original text paraphrase unit 83, and the paraphrase when the paraphrase extracted by the similar sentence search unit 84 is obtained from the original example sentence And the appropriateness of performing the paraphrase between the paraphrase extracted from the similar sentence search unit 84 and the original sentence, using the paraphrase information held in the paraphrase information database D3 and the application frequency thereof, are calculated in a predetermined manner. According to the formula Evaluated as relevance score calculated, and a representation verification unit 85 for providing an evaluation result in the original words 83. The original text paraphrase unit 83 gives the paraphrase text included in the list given from the similar text search unit 84 to the expression verification unit 85, and based on the validity score of each paraphrase text returned from the expression verification unit 85, Select one of them and apply the replacement of the expression unit represented by the pair of expression units used to generate the paraphrase to the original text in the direction opposite to the direction of application when generating the paraphrase. By doing so, it has a function of creating an original sentence.

  Source language examples corpus C1, bilingual corpus C2, expression segment database D1, paraphrase database D2, and paraphrase information database D3 are all storage devices such as a hard disk of a computer that realizes this automatic paraphrase device 80 as will be described later. Stored in

  The expression segment collection unit 81 scans the source language example corpus C1 and collects data for determining the similarity of the source language expressions. Specifically, the expression segment collection unit 81 decomposes each example sentence into expression segments for a large number of example sentences configured in the source language. Then, the appearance frequency of each representation segment when the decomposition processing is performed on all example sentences in the example corpus C1 of the source language is stored in the representation segment database D1 together with the representation segment. However, it does not make much sense to use an expression fragment with an extremely low frequency of appearance or an expression fragment obtained from an incorrect expression. Therefore, in this embodiment, an expression segment with a low appearance frequency is excluded from the targets of similarity determination criteria. Therefore, only the expression elements having an appearance frequency equal to or higher than a predetermined threshold are registered in the expression element database D1.

  For example, it is assumed that an example sentence in the source language of the corpus C1 has “Can you show me your passport?” If there are a lot of other example sentences including "Passport", "Show me" and "Can you please", from this sentence, "Pass", "Spo", "Port", " Representation segment database D1 as a character string (representation segment) in which “show” “show” “set” “te-kake” “kakema” “kema” “masa” “suka” has a frequency equal to or higher than a threshold value Registered and retained.

  Note that the source language example corpus C1 may also be used as the source language side example sentence group of the parallel translation corpus C2.

  The paraphrase sentence generation unit 82 analyzes the example sentence on the source language side of the parallel translation corpus C2 by the same means as that for the original sentence to be translated. Further, the paraphrase text generation unit 82 converts the example sentence into one or a plurality of paraphrase texts by a known paraphrase method such as that used in an existing machine translation device, and outputs the text. Except for the point that the source language and the target language are the same language, the paraphrase generation unit 82 is configured to be equivalent to or a part of the mechanism of an existing machine translation device.

  For example, it is assumed that the example sentence on the source language side of the bilingual corpus C2 has "Can you show me your passport?" The text generated by the text generation unit 82 as the text for this text is “I look at your passport”, “Show me your passport”, “Please give me your passport”, and so on.

  The paraphrase for each example sentence output from the paraphrase sentence generator 82 is registered in the paraphrase database D2. At this time, the paraphrase text generation unit 82 decomposes the generated paraphrase text into expression segments based on the same criteria as the expression segment collection unit 81, and uses each search text as a search key so that the paraphrase text is searched. Register in the phrase database D2.

  In addition, the paraphrase sentence generation unit 82 extracts a difference between each example sentence and the paraphrase generated for the example sentence, and provides information on where and how the example sentence is paraphrased. Are registered in the paraphrase information database D3 as paraphrase information consisting of pairs of corresponding expression segments. At the same time, the paraphrase generation unit 82 registers the expression segment as a pair with the paraphrase used in the paraphrase text database D2.

  FIG. 2 shows the contents of the paraphrase text data registered in the paraphrase text database D2. Referring to FIG. 2, one or a plurality of paraphrases 92 are registered for one example sentence 91. In each paraphrase 92, paraphrase information 93 is stored in association with the difference between the paraphrase and the example sentence 91 in the form of a pair of expression segments. Note that when comparing the paraphrase and the example sentence, there may be a case where one side has a character string and the other side does not exist. In this embodiment, even in such a case, one expression element and a symbol (“Φ” in FIG. 2) indicating that there is no corresponding expression element are associated and stored in the paraphrase database D2.

  In this embodiment, the paraphrase information stored in the paraphrase information database D3 includes an example sentence, a representation element pair in the paraphrase portion of the paraphrase, and an application frequency thereof. For example, consider a case where the example sentence 1 is “Can you show me your passport?” And the paraphrase 1 is “Please show me your passport”. In this example, the example sentence “Can you please” is translated to “Please”. These are broken down into pieces of expression, and “Please give me”, “Kema-sai”, “Masu-φ”, “Suka-φ” are stored as paraphrase information 93.

  By expressing the paraphrase information as a pair of expression elements in this way, the objectivity of the paraphrase information can be ensured and the handling is facilitated.

  In the paraphrase information database D3, the frequency of application of the paraphrase information 93 including the above-described pair of expression elements, that is, the appearance frequency of the paraphrase information that appears when the paraphrase sentence generation unit 82 paraphrases a large amount of example sentences into each paraphrase. A summary of each paraphrase information is registered. For example, in the example shown in FIG. 3, the application frequency of “Please give me” is n1, the application frequency of “Kema-sai” is n2, the application frequency of “Masa-φ” is n3, and “Suka-φ”. The application frequency is n4,.

  When the original sentence is input, the original sentence paraphrase unit 83 outputs the original sentence to the similar sentence search unit 84, and also outputs a paraphrase selected from the similar sentence search unit 84 and a list of paraphrase information to the expression verification unit 85. Further, the original text paraphrase unit 83 finally determines a paraphrase for the input original text based on the validity evaluation output from the expression verification unit 85. This point will be described later.

  The similar sentence search unit 84 breaks down the original sentence sent from the original sentence paraphrase unit 83 into expression units having the same reference number of characters as the expression segment collection unit 81, and uses the expression unit as a search key for the paraphrase database D2. And all the paraphrases that share at least one expression fragment with the input original sentence are extracted.

  The similar sentence search unit 84 further performs an operation of selecting a paraphrase similar to the original sentence from the extracted paraphrases. The selection is performed with reference to the appearance frequency stored in the expression segment database D1 for the expression segments shared by the original sentence and each paraphrase. In this embodiment, specifically, an expression unit having a higher appearance frequency is shared with the original sentence, and a predetermined number is compared with the original sentence from a higher rank of the paraphrase sentence having many types of expression elements to be shared with the original sentence. Select as a paraphrase to be. In addition, the selection method of a paraphrase is not limited to this, The method etc. of selecting the paraphrase with the high sum total of the appearance frequency of the shared expression element may be used.

  Further, the similar sentence search unit 84 outputs the selected paraphrase and the paraphrase information attached to each of the selected paraphrase as a list to the original sentence paraphrase unit 83. At this time, the similar sentence search unit 84 also expresses the difference of the expression segment between the input original sentence and the similar paraphrase sentence in the form of the same expression segment pair as the paraphrase information 93 shown in FIG. The paraphrase information is included in the output list to the original paraphrase unit 83 and output to the original paraphrase unit 83.

  The expression verification unit 85 receives the original sentence, the paraphrase, and the paraphrase information given from the similar sentence search unit 84 to the original sentence paraphrase unit 83 and further sent from the original sentence paraphrase unit 83 to the expression verification unit 85. It evaluates the validity of the paraphrase with the original example sentence for the paraphrase and evaluates the validity of the similarity between the original sentence and the paraphrase.

  The evaluation of the validity of the paraphrase between the paraphrase and the example sentence will be described below. The paraphrase is a paraphrase from the example sentence on the source language side of the bilingual corpus C2, and the sentence before the paraphrase corresponds to the example sentence. In order to evaluate the validity of the paraphrase, refer to the application frequency registered in the paraphrase information database D3 for the same segment segment pair (paraphrase information) as the segment segment pair in the paraphrase portion of the example sentence and the paraphrase. To do.

  It is considered that the expression element pair (paraphrase information) with low application frequency is not often used as a paraphrase expression. Therefore, for example, when the paraphrase information includes a large number of pairs of expression units having a low application frequency compared to the length of the example sentence or the paraphrase, the application frequency and the expression unit are set so that the validity of the paraphrase operation becomes low. The validity score is calculated by a method such as weighting the number of pairs, the example sentence or the length of the paraphrase. In this way, the score calculated from the application frequency of the expression element pair can be normalized with respect to the length of the paraphrase. The specific method of assigning the score is not particularly limited, and the sum or the average value of the frequency of application of the expression element pair in the paraphrase part is larger than the length of the example sentence or the paraphrase. You may make it become a big score. Also, regardless of the length of the example sentence or paraphrase, the larger the sum of the application frequencies or the larger the average value, the larger the score may be.

  Next, the evaluation of the validity of the similarity between the original sentence and the paraphrase text will be described. The expression verification unit 85 is configured so that the difference between the original sentence created by the similar sentence search unit 84 and sent from the original sentence paraphrase part 83 and each paraphrase sentence. Based on the information, the application frequency registered in the paraphrase information database D3 is referred to for the same segment segment pair (paraphrase information) as the segment segment pair in the difference portion. That is, the difference between the original text and the paraphrase is expressed as paraphrase information including a pair of expression elements, and the application frequency thereof is examined.

  It can be determined that pairs of expression segments (paraphrase information) with low application frequency are not often used as paraphrase expressions. Therefore, for example, if the number of expression unit pairs with low application frequency is larger than the length of the original sentence or paraphrase, the application frequency, the number of expression element pairs, Alternatively, the validity score is calculated by a method such as weighting the length of the paraphrase. Again, the score calculated from the pair of expression elements can be normalized with respect to the length of the original sentence or paraphrase. Note that the specific method of assigning the score is not particularly limited, and the larger the sum or average of the frequency of application of the expression element pairs in the difference portion, the greater the length of the original sentence / paraphrase sentence. It is good also as a score. Further, regardless of the length of the original sentence / paraphrase, a higher score may be obtained as the sum of application frequencies and the average value are larger.

  For example, assume that the original text is "Can you see your passport?" As a similar sentence, it is assumed that the paraphrase “I will look at your passport” is selected. Further, it is assumed that this paraphrase is a paraphrase from the example sentence “Can you show me your passport?”. In this case, the phrase “Can you show me your passport?” And the phrase of the phrase “I will see your passport” will be “Can you show me” and “I will see you”. It is assumed that there is a considerable number of application times of paraphrase information between these. In this case, the validity score of the paraphrase for the example sentence for the paraphrase “I will look at the passport” is calculated high.

  In addition, the difference between the original text “Can you see your passport?” And the phrase “I will see your passport” is “Can you do it?” And “I will do it”. It is assumed that there is a considerable number of the frequency of applying paraphrase information between these. Also in this case, a similar validity score for the original sentence for the paraphrase “I will look at the passport” is also calculated high.

  On the other hand, it is assumed that a paraphrase “Can you see the pass card?” Is selected as a similar sentence to the original sentence “Can you see the passport?”. The difference between the two “ports” and “cards” is rarely rephrased, and therefore, there is a high possibility that the application frequency of the paraphrase information between them is not registered. Then, the similar validity score with respect to the original sentence about this paraphrase "Can you see the pass card?" Is calculated low.

  The expression verification unit 85 outputs, to the original sentence paraphrase unit 83, a list in which the paraphrase sent from the original sentence paraphrase unit 83 is combined with the calculated paraphrase and similar two types of validity scores.

  When receiving the list of paraphrases combined with the validity score from the expression verification unit 85, the original paraphrase unit 83 comprehensively determines these two types of scores, and has the highest validity from the paraphrase. One paraphrase is selected as the paraphrase most similar to the original sentence. For example, a product of two validity scores may be used as a new validity score. Then, the paraphrase information registered as a pair with the selected paraphrase is applied in the reverse direction to paraphrase the input original sentence. The reverse direction means applying the paraphrase information so that the sentence after the paraphrase returns to the sentence before the paraphrase (example sentence). The paraphrase here is paraphrased from the example sentence on the source language side of the bilingual corpus C2 by the paraphrase sentence generation unit 82, and the sentence before the paraphrase corresponds to the example sentence. Therefore, the original sentence is paraphrased into a sentence (or example sentence itself) close to the example sentence corresponding to the selected paraphrase sentence.

  For example, consider the case where the original sentence is “Can you see your passport?” And “I will see your passport” is selected as the most appropriate paraphrase. The original example sentence of "I will see your passport" is "Can you show me your passport?" In this case, the same sentence as the original example sentence “Can you show me your passport?” Is generated as a paraphrase for the original sentence “Can you see your passport?”.

[Realization by computer]
−Hardware configuration−
The automatic paraphrase device 80 according to the present embodiment is realized by a computer and software that operates on the computer. Of course, part or all of the above-described functions can be realized by hardware instead of software.

  FIG. 5 shows an external view of the computer system 20 used in the present embodiment, and FIG. 6 shows a block diagram of the computer system 20. The computer system 20 shown here is merely an example, and various other configurations are possible.

  Referring to FIG. 5, the computer system 20 includes a computer 40 and a monitor 42, a keyboard 46, and a mouse 48, all connected to the computer 40. The computer 40 further includes a CD-ROM (Compact Disk Read-Only Memory) drive 50 and an FD (Flexible Disk) drive 52.

  Referring to FIG. 6, computer system 20 further includes a printer 44 connected to computer 40, which is not shown in FIG. The computer 40 further includes a bus 66 connected to the CD-ROM drive 50 and the FD drive 52, a central processing unit (CPU) 56 connected to the bus 66, a boot-up program for the computer 40, and the like. A ROM (Read-Only Memory) 58, a RAM (Random Access Memory) 60 that provides a work area used by the CPU 56 and a storage area for programs executed by the CPU 56, and a hard disk 54 that stores an audio database to be described later. Including.

  The software for realizing the system of the embodiment described below is recorded and distributed on a recording medium such as a CD-ROM 62, and is read into the computer 40 via a reader such as the CD-ROM drive 50. Stored in the hard disk 54. When the CPU 56 executes this program, the program is read from the hard disk 54 and stored in the RAM 60, and an instruction is read from an address designated by a program counter (not shown) and executed. The CPU 56 reads data to be processed from the hard disk 54 and stores the processing result in the hard disk 54 as well.

  Since the operation itself of the computer system 20 is well known, details thereof will not be repeated here.

  Note that the software distribution form is not limited to the form fixed to the storage medium as described above. For example, data may be distributed in the form of receiving data from other computers connected through a network. Further, there may be a distribution form in which a part of software is stored in the hard disk 54 in advance, and the remaining part of the software is taken into the hard disk 54 via the network and integrated at the time of execution.

  In general, modern programs utilize the general purpose functions provided by a computer operating system (OS) and achieve the desired objectives described above by executing them in an organized manner according to the desired objectives. . Accordingly, among the functions of the present embodiment described below, a program (group) that does not include general-purpose functions provided by the OS or a third party, and specifies only a combination of execution orders of these general-purpose functions. Even so, as long as the program (group) has a control structure that achieves a desired object as a whole by using them, it is obvious that they are included in the technical scope of the present invention.

-Program structure-
When the automatic paraphrase device 80 described above is realized by a computer, a computer program that causes the computer to operate as the automatic paraphrase device 80 has the following control structure. Prior to this, the source language example corpus C1, bilingual corpus C2, expression segment database D1, paraphrase database D2, and paraphrase information database D3 are prepared in the hard disk 54 in advance. These are realized by computer programs corresponding to the expression segment collection unit 81 and the paraphrase generation unit 82. Since the functions realized by these computer programs are relatively simple, the details thereof are omitted here.

  Referring to FIG. 4, the paraphrase program that generates a paraphrase from the original text waits for the input of the original text when activated (step 101). When the original text is input (Yes in step 101), the original text is decomposed into expression segments for the input original text, and a paraphrase that shares at least one of the expression segments is the paraphrase database D2. (Step 102).

  Next, from the extracted paraphrase texts, the appearance frequency is examined with reference to the phrase segment database D1 for each of the phrase segments shared by the original sentence and each paraphrase sentence. A predetermined number is selected as a paraphrase similar to the original sentence from among higher paraphrases that are shared with the original sentence and have many types of expression elements to be shared with the original sentence (step 103). Here, the selected paraphrase is also listed together with paraphrase information used when translating the original example sentence into the paraphrase and difference information between the original sentence and the paraphrase.

  Subsequently, in step 104, each of the pair of expression segments in the paraphrase portion of the example sentence and the paraphrase and each of the segment pair (paraphrase information) in the difference portion between the original sentence and each paraphrase is registered in the paraphrase information database D3. By referring to the applied frequency, the validity of the paraphrase between the example sentence and the paraphrase and the validity of the similarity between the original sentence and the paraphrase are evaluated.

  Further, based on the paraphrase and similar two types of validity evaluation results, one paraphrase having the highest validity for the original sentence is selected (step 105).

  Next, the paraphrase information of the original sentence is generated by applying the paraphrase information attached to the single paraphrase selected in the reverse direction to the original sentence (step 106).

[Operation]
The automatic paraphrase device 80 according to the present embodiment operates as follows.

  The operation of the automatic paraphrase device 80 has two phases in total. The first phase is a preparatory stage in which the expression segment database D1, the paraphrase text database D2, and the paraphrase information database D3 are created. The second phase is an operation stage, which is a process of creating a paraphrase from a given original sentence.

-Database creation-
First, as a preparation stage, the expression segment collection unit 81 scans the corpus C1 and decomposes a large number of example sentences configured in the source language into expression segments. The expression segment collection unit 81 examines the frequency of appearance of the same expression segment for each expression segment, and expresses the expression segment having an occurrence frequency equal to or higher than a predetermined threshold and the frequency of occurrence thereof. Register in the piece database D1.

  On the other hand, the paraphrase generation unit 82 generates a paraphrase by a known method for each example sentence in the bilingual corpus C2. These paraphrases are registered in the paraphrase database D2 together with paraphrase information (representative element pairs) used at the time of paraphrase generation. Further, the paraphrase sentence generation unit 82 registers, in the paraphrase information database D3, the application frequency that is the appearance frequency of paraphrase information that appears when paraphrasing the example sentence of the bilingual corpus C2.

  The creation of the expression segment database D1, the paraphrase database D2, and the paraphrase information database D3 may be executed by the same computer as the computer that performs the following paraphrase processing for the original sentence, or may be executed by another computer. Good. However, when the paraphrase is executed as preprocessing of the machine translation device, it is preferable that the bilingual corpus C2 matches that used by the machine translation device.

  Thus, after each database is created, the automatic paraphrase device 80 can create a paraphrase from the original text.

−Preparation of paraphrase from original text−
The original text is given to the original text paraphrase unit 83. The original sentence paraphrase unit 83 gives the original sentence to the similar sentence search unit 84. The similar sentence search unit 84 decomposes the original sentence into expression segments, and extracts a paraphrase that shares at least one original sentence and an expression segment among the paraphrases recorded in the paraphrase database D2. At this time, the similar sentence search unit 84 refers to the expression element database D1 to check the appearance frequency of each expression element, shares the expression element having a higher appearance frequency with the original sentence, and stores the expression element shared with the original sentence. A predetermined number is selected as a paraphrase similar to the original sentence from the top of the paraphrase sentences with many types. The similar sentence search unit 84 supplies the selected paraphrase text and the paraphrase information attached thereto to the original text paraphrase section 83 as a list.

  The original text paraphrase unit 83 gives this list to the expression verification unit 85. For each paraphrase included in the list given from the original paraphrase paraphrase 83, the expression verification unit 85 converts the paraphrase information attached to them and the application frequency of each paraphrase information recorded in the paraphrase information database D3. Based on the original example sentence, the validity of the paraphrase when the paraphrase text is obtained by the paraphrase text generation unit 82 is calculated. At this time, if the number of paraphrasing parts and difference parts with low frequency of application of the corresponding paraphrase information is larger than the length of the paraphrase sentence and the example sentence, the validity of the paraphrase between the paraphrase sentence and the example sentence is evaluated low. Is done. This is because it can be determined that a paraphrase portion or a difference portion where the frequency of application of corresponding paraphrase information is small is not often used as a paraphrase expression.

  The expression verification unit 85 also represents, for each paraphrase included in the list, a difference from the original sentence in the form of a set of expression element pairs. Then, the validity of the paraphrase from the original sentence to the paraphrase expressed by the expression element pair is calculated based on the frequency of application of the paraphrase information recorded in the paraphrase information database D3. At this time, as in the case of calculating the validity of the paraphrase, if the paraphrase part or the difference part where the corresponding paraphrase information is applied less frequently than the length of the original sentence and the paraphrase, the original sentence and the paraphrase are included. The relevance of similarities with sentences is underestimated.

  The expression verification unit 85 forms the form of a list in which each of the paraphrases in the list given from the original paraphrase unit 83 is given the above-described two validity (the validity of the paraphrase and the validity similar to the paraphrase). Is returned to the original paraphrase section 83.

  The original text paraphrase unit 83 selects one paraphrase that seems to be most appropriate based on the two validity values included in the list given from the expression verification unit 85. Then, by applying the paraphrase information attached to the selected paraphrase in the opposite direction to the original sentence, the paraphrase is generated from the original sentence. The paraphrase obtained in this way is close to the example sentence that is the basis of the paraphrase selected by the original paraphrase section 83.

[Effect of the embodiment]
As described above, in the automatic paraphrase device according to this embodiment, the paraphrase required between the validity of the paraphrase when the paraphrase is obtained from the paraphrase obtained by paraphrasing the example sentence and the original sentence. One paraphrase with the highest validity is selected in combination with the validity. The paraphrase information applied when the paraphrase is obtained from the example sentence is applied in the opposite direction to the original sentence. By this operation, one paraphrase is automatically generated for the original sentence. No operator action is required. Therefore, one optimal paraphrase can be obtained automatically without requiring the operator's paraphrase knowledge.

  Moreover, the data required for this specification is the expression segment database D1, the paraphrase database D2, and the paraphrase information database D3, and the optimum paraphrase is specified using these. Therefore, it is not necessary to set in advance knowledge for paraphrasing in detail, and the apparatus can be realized with a simple configuration.

  In the machine translation system to which the automatic paraphrase device according to the present embodiment is applied, when a paraphrase for the original sentence is specified in the original sentence paraphrase unit 83, the original sentence is replaced with the paraphrase and output to a language conversion unit (not shown). A translation process is performed, and a translation process such as shaping the language-converted target language sentence into a predetermined format is performed. At this time, the paraphrase obtained by paraphrasing the original sentence is close to or identical to the example sentences existing in the parallel translation corpus C2. When the language conversion in the language conversion unit is based on an example using the bilingual corpus C2, good results can be obtained by performing language conversion from the paraphrase obtained as described above.

  Therefore, the machine translation system to which the automatic paraphrase device according to the present embodiment is applied automatically performs paraphrase in the form closest to the example sentence that is a source of knowledge that the machine translation system uses for language conversion processing. Can do. Therefore, even if the operator does not have knowledge about what kind of paraphrase is effective, it operates as if the paraphrase is performed in accordance with the machine translation system.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the embodiment described above. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a block diagram which shows the structure of the automatic paraphrasing device of one Embodiment of this invention. It is a figure which shows an example of the data memorize | stored in the paraphrase text database. It is a figure which shows an example of the data memorize | stored in the paraphrase information database. It is a flowchart which shows the content of the paraphrase process by the automatic paraphrase apparatus of FIG. 1 is an external view of a computer system that executes a paraphrase processing program according to an embodiment of the present invention. FIG. FIG. 6 is a block diagram of the computer system of FIG. 5.

Explanation of symbols

40 Computer 80 Automatic Paraphrase Device 81 Representation Fragment Collection Unit 82 Paraphrase Text Generation Unit 83 Original Sentence Paraphrase Unit 84 Similar Text Retrieval Unit 85 Expression Verification Unit 91 Sample sentence 92 Paraphrase 93 Paraphrase Information D1 Representation Fragment Database D2 Paraphrase Text Database D3 Paraphrase Information database C1 Example corpus C2 Bilingual corpus

Claims (8)

A representation element storage means for storing the expression elements appearing in the first example sentence group of the predetermined language together with the appearance frequency of each of the expression elements in the first example sentence group;
Paraphrase storage means for storing one or a plurality of paraphrases for each example sentence in the second example sentence group of the predetermined language, together with paraphrase information indicating a paraphrase mode when the paraphrase is obtained. When,
Paraphrase information storage means for storing paraphrase information indicating a paraphrase form from the sample sentence to the paraphrase in the second example sentence group, together with their application frequency;
In the paraphrase stored in the paraphrase storage means, a paraphrase that receives the original sentence to be paraphrased and shares at least one of the expression segments stored in the phrase segment storage means with the original sentence Search means for searching from,
The frequency of application of the paraphrase information stored in the paraphrase information storage unit with respect to the paraphrase between the corresponding original example sentence and the paraphrase between the original sentence and each paraphrase searched by the search unit An evaluation means for evaluating a validity score calculated by a predetermined calculation method based on
In the paraphrase storage means, by applying paraphrase information associated with a paraphrase in which the validity score evaluated by the evaluation means satisfies a predetermined condition to the original sentence in a reverse direction, the paraphrase for the original sentence An automatic paraphrase device including original sentence paraphrase means for generating a sentence. The search means includes
In the paraphrase stored in the paraphrase storage means, a paraphrase that receives the original sentence to be paraphrased and shares at least one of the expression segments stored in the phrase segment storage means with the original sentence Extraction means for extracting from,
Of the paraphrase extracted by the extraction means, the number of types of expression elements shared with the original sentence, and the expression elements shared with the original sentence are stored in the expression element storage means. The automatic paraphrasing device according to claim 1, further comprising: selecting means for selecting the appearance frequency or a combination thereof satisfying a predetermined condition. The selection means includes the number of types of expression segments shared with the original sentence, the appearance frequencies stored in the expression element storage means regarding the expression elements shared with the original sentence, or these The automatic paraphrase device according to claim 2, further comprising means for selecting a predetermined number of paraphrases having a higher score than the score determined by the combination of the above. The evaluation means determines the validity of the paraphrase between the paraphrase and the example sentence when the paraphrase part having a small frequency of application of the corresponding paraphrase information includes more paraphrase parts than the length of the paraphrase sentence or the example sentence. The automatic paraphrasing device according to any one of claims 1 to 3, which is evaluated low. The evaluation means evaluates the relevance of the similarity between the original sentence and the paraphrase low when the paraphrase text includes more difference parts compared to the length of the original sentence or the paraphrase, where the frequency of application of the corresponding paraphrase information is small. The automatic paraphrasing device according to any one of claims 1 to 4. The automatic paraphrase device according to any one of claims 1 to 5, wherein the paraphrase information stored in the paraphrase information storage unit includes a combination of expression segments in a paraphrase portion of a sample sentence and a paraphrase. A paraphrase processing program that, when executed by a computer, causes the computer to operate as the automatic paraphrase device according to any one of claims 1 to 6. A representation segment database that stores the representation segments that appear in the first example sentence group of the predetermined language, and the frequency of appearance of each representation segment in the first example sentence group;
A paraphrase database that stores one or more paraphrases for each example sentence in the second example sentence group of the predetermined language, together with paraphrase information representing a paraphrase mode when the paraphrase is obtained;
A paraphrase information database that stores paraphrase information indicating a paraphrase mode from a sample sentence to a paraphrase in the second example sentence group, together with their application frequency;
Is an automatic paraphrase method that paraphrases an input original sentence into one paraphrase,
A paraphrase that receives an original sentence that is a paraphrase target and shares at least one of the expression elements stored in the expression fragment database with the original sentence is searched from the paraphrase sentences stored in the paraphrase database. A search step to
For each paraphrase searched in the search step, regarding the paraphrase between the corresponding original example sentence and the paraphrase between the original sentence and the frequency of application of the paraphrase information stored in the paraphrase information database. An evaluation step for evaluating a validity score calculated by a predetermined calculation method based on the above;
In the paraphrase database, by applying paraphrase information associated with the paraphrase in which the validity evaluated in the evaluation step satisfies a predetermined condition to the original sentence in a reverse direction, the paraphrase for the original sentence is An automatic paraphrase method including the step of generating.

Images