JP2001357065A - Method and device for retrieving similar sentence and recording medium having similar sentence retrieval program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and a device for retrieving similar sentences, with which a grammatically or semantically similar sentence can be retrieved, and to provide a recording medium having a similar sentence retrieval program recorded thereon. SOLUTION: First, the information on a part and kind for which grammatically or semantically replacement, deletion and addition can be performed is imparted concerning the similar candidate sentences of an example sentence collection and information on a part and kind which can be replaced or which can be matched with the added part of a similar candidate sentence is imparted similarly to an input sentence as well. Then, in the case of calculating similarity between the input sentence and each of similar candidate sentences, a processing is performed while considering the coincidence of the replaced part of the same kind, the deletion of an unwanted part or addition of a lacking to the different part in each of sentence, and the similar candidate sentence having highest similarity is extracted as a similar sentence together with similarity.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a similar sentence retrieval method and apparatus for retrieving a similar sentence from an input sentence of a natural language, and a recording medium storing a similar sentence retrieval program. If there is a translated sentence corresponding to the searched similar sentence, the translated sentence is extracted. In addition, the present invention can be applied to a part of an example type translation method and an apparatus for generating a translation of an input sentence using the similar sentence and the translation.

[0002]

2. Description of the Related Art As a conventional similar sentence retrieval method, a conventional method 1 "Emmanuel Planas, et al.,
“Formalizing Translation
Memories ", MTSummit VII, Se
ptember, 1999 ". In this method, units that are separated by morphological analysis are used, and processing is performed by expanding the matching target not only in word notation but also in standard form and part of speech, and the percentage of words matched by the notation in the input sentence, The percentage of words that match in standard form, the percentage of words that match in the part of speech in the input sentence, the percentage of words in the candidate sentence that are common to the words in the input sentence, and the percentage of words that are common to the words in the candidate sentence in the input sentence A similar sentence search is performed by comparing the similarity between two sentences in the listed order.

As another similar sentence retrieval method, there is a method described in a conventional method 2 "Translation device for natural language disclosed in Japanese Patent Laid-Open No. 6-290210". In this method, a syntactic surface pattern is generated from an input sentence and a sentence to be searched, and these patterns are compared, and a similar sentence search is performed based on the similarity of the pattern.

As another similar sentence retrieval method, a conventional method 3 "Eiichiro Sumida, Yutaka Tsutsumi," A practical retrieval method of similar examples for translation support ", IEICE Transactions D-II,
Vol. J74-D-II, No. 10, 1991 ". In this method, after performing morphological analysis, a sentence that completely matches the input sentence is searched. If they do not match, the part of speech of the input sentence is generalized to search for a sentence that exactly matches the input sentence.

[0005]

However, in the conventional method 1, since the matching target is a morpheme unit divided by morphological analysis, the expression composed of a plurality of morphemes of a certain sentence is matched with the expression of another sentence to obtain a similarity. The degree calculation cannot be performed.

In the conventional method 2, a verb must always appear in the pattern, and a sentence in which the verb is omitted cannot be handled.

In the conventional method 3, since only the input sentence is generalized and the sentence to be searched is not generalized, the applicable range of the search is narrow.

In the conventional methods 1 and 3, a similar sentence search is performed using a sentence in the same natural language as an input sentence and a bilingual example, and the translated sentence of the similar sentence is edited and used as a part of an example-type translation. In addition, a similar sentence search is not performed in consideration of a case where a missing phrase is added to the similar sentence compared to the input sentence and a corresponding phrase is added to the translated sentence to generate an appropriate translated sentence.

Further, in the conventional methods 1 to 3, the search for grammatically similar sentences is performed, and the search for semantically similar sentences is not considered.

The present invention has been made in view of the above circumstances. Even when notation alone is not high in similarity, a sentence that is grammatically or semantically similar can be searched, and a sentence that does not resemble an input sentence can be searched. The object of the present invention is to provide a similar sentence search method and apparatus capable of shortening the time of similarity calculation by deleting in advance, and a recording medium recording a similar sentence search program.

[0011]

In order to achieve the above object, the present invention provides a similar sentence retrieval method for retrieving a similar sentence of an input sentence from an example sentence collection. Each piece of information is added to places where replacement, deletion, and addition are possible, and each piece of information is added to input sentences in places where replacement is possible and where similar candidate sentences can be added. Above, when calculating the similarity between the input sentence and the similar candidate sentence, perform processing in consideration of the matching of the same type of replacement part with the difference part of each sentence, deletion of unnecessary parts and addition of missing parts, A similar candidate sentence having the highest similarity is extracted as a similar sentence together with the similarity.

Further, the present invention is characterized in that, in the similar sentence search method, a predetermined number of similar candidate sentences having higher similarities are output as similar sentences in addition to the similar candidate sentences having the highest similarity. .

Further, according to the present invention, in the similar sentence search method, data that can be used for general purposes and data that depends on the field of documents are classified as data serving as bases for adding replacement, deletion, and additional information. In the automatic creation of data that depends on the document field, information is added to the example sentences using existing general-purpose or field-dependent data, and from the example sentence collection where parts that can be replaced and deleted are deleted. A similar sentence is collected, and the notation and replacement type of the preceding and succeeding parts in the part where the replacement information is not added in the sentence match, and a new set of the same information and the different notation is added. It is characterized in that it is created as new data to be replaced, and at the same time it is created as new data to be deleted in consideration of the new data to be replaced and the notation before and after.

Further, according to the present invention, in the similar sentence search method, when there are a large number of sentences in the example sentence collection of similar candidate sentences, the number of similar phrases in the input sentence is equal to or greater than a predetermined threshold. As a new similar candidate sentence.

Further, the present invention provides the similar sentence search method, wherein a bilingual translation example, which is a set of each sentence of the example sentence collection and a translated sentence, is used.
It is characterized by extracting a similar sentence of the input sentence and its translation.

Further, the similar sentence retrieval apparatus of the present invention comprises an example section storing a plurality of example sentences, an input means for reading an input sentence,
An example analyzing / information adding means for analyzing a similar candidate sentence obtained from the example sentence of the example unit on a phrase basis, and adding each information to a place where grammatical or semantic replacement, deletion, and addition are possible; Analyze and input information by analyzing the input sentence read by the means on a word-by-phrase basis and assigning each information to a place where grammatical or semantic substitution is possible or a place where a similar candidate sentence can be added. Means for calculating the similarity between the input sentence and the similar candidate sentence with respect to the analyzed similar candidate sentence;
Search means for calculating similarity in consideration of deletion of unnecessary parts and addition of missing parts, and extracting the similar candidate sentence having the highest similarity as a similar sentence, the similar sentence extracted by the search means together with the similarity Output means for outputting.

Further, according to the present invention, in the similar sentence search apparatus, the search means includes a similar candidate sentence having the highest similarity,
It is characterized in that a predetermined number of similar candidate sentences from the one with the highest similarity are extracted as similar sentences.

Further, according to the present invention, in the similar sentence retrieval apparatus, when replacing, deleting, or adding additional information, data that can be used for general purposes and data that depends on the field of documents are divided into basic data. Each data is described, and in the automatic creation of data depending on the field of the document, similar to the sentence of the example sentence collection where parts that can be replaced and deleted are deleted using existing general-purpose or field-dependent data Of the sentence where the replacement information has not been added, the notation and the type of replacement before and after it are the same, and a set of items with the same information but different notations is added as a new replacement target. Data creation means for creating new data to be deleted, and at the same time creating new data to be deleted in consideration of the new data to be replaced and the notation before and after. .

Further, the present invention is characterized in that, in the similar sentence search device, the search means sets in advance a sentence in which the number of phrases equal to the input sentence is equal to or greater than a predetermined threshold as a similar candidate sentence. Is what you do.

Further, according to the present invention, in the similar sentence retrieval apparatus, when a bilingual translation example in which a translated sentence is associated with an example sentence is used, a similar sentence extracted by the retrieval means and an output for outputting the translated sentence are output. Means.

According to the present invention, a similar sentence search program for retrieving a similar sentence of an input sentence from an example sentence collection is recorded on a recording medium. Each information is added to a place where the input sentence is possible, and each information is added to a place where the input sentence can be replaced and a place where a similar candidate sentence can be added. When calculating the similarity with the candidate sentence,
A process that considers matching of the same type of replacement part to the difference part of each sentence, deleting unnecessary parts and adding missing parts, and extracting the similar candidate sentence with the highest similarity as similar sentence together with similarity Is executed by a computer.

Further, according to the present invention, in a recording medium on which the similar sentence search program is recorded, a predetermined number of similar candidate sentences having the highest similarity are output as similar sentences in addition to the similar candidate sentences having the highest similarity. This is for causing a computer to execute processing.

[0023] The present invention also relates to a recording medium on which the similar sentence search program is recorded, which can be generally used as base data for adding replacement, deletion, and additional information, and in the field of documents. Create each data separately for dependent items, and in the automatic creation of data that depends on the document field, add information to example sentences using existing general-purpose or field-dependent data, and specify places that can be replaced and deleted Collect similar sentences from the cut example sentence collection, and in places where replacement information is not added in the sentence, the notation and replacement type of the preceding and following parts match, and the information of the corresponding part is the same and the notation A process of creating a set of different items as new data to be replaced and, at the same time, creating new data to be deleted in consideration of the new data to be replaced and the notation before and after, etc. It is intended to be executed by the data.

Further, according to the present invention, in the recording medium storing the similar sentence search program, when there are a large number of sentences in the example sentence collection of similar candidate sentences, the number of the same phrases as the words of the input sentence is not less than a predetermined threshold value This is for causing the computer to execute a process of setting the similar candidate sentence as a new similar candidate sentence.

Further, according to the present invention, a process for extracting a similar sentence of an input sentence and a bilingual translation thereof on a recording medium on which the similar sentence search program is recorded, using a bilingual example which is a set of each sentence of the example sentence and a translated sentence. It is intended to be executed by a computer.

According to the present invention, in a method of searching for a similar sentence of an input sentence from similar candidate sentences in a bilingual example sentence collection, a grammatically or semantically replaceable portion and a portion where a similar candidate sentence can be added to a possible addition portion Using a similar candidate sentence to which the information of the input sentence to which information is added and a grammatically or semantically replaced, deleted, and addable portion in advance is used, for a portion where the expression differs between the input sentence and the similar candidate sentence, The similarity is calculated by performing replacement (input sentence and similar candidate sentence), and deleting and adding (similar candidate sentence), extracting the similar candidate sentence having the highest similarity as a similar sentence together with the similarity, and simultaneously translating the similar sentence. I tried to extract.

[0027] Further, an example section storing data relating to a translation example as an apparatus configuration, input means for reading an input sentence, and analyzing similar candidate sentences by the example section in terms of words and phrases,
Grammatically or semantically replace, delete, add example and assignable information that can be added and added, and an input analysis unit that analyzes the input sentence by the input unit, and replaces it grammatically or semantically. Analysis that assigns each information to a place where a match is possible and a place where a similar candidate sentence can be added
The information adding means replaces the input sentence and the phrase of the similar candidate sentence, deletes the phrase of the similar candidate sentence, deletes the phrase of the similar candidate sentence, and the like in a portion where the phrase differs between the analyzed similar candidate sentence and the analyzed input sentence. A search unit for calculating a similarity with the input sentence by adding a word to the sentence, extracting a similar candidate sentence having the highest similarity as a similar sentence, and simultaneously extracting a translation, and an output unit for outputting a search result I did it.

In the example section, the example sentence is analyzed in advance, and the analyzed words are replaced, deleted,
A data creation means for automatically or manually setting each information at a place where addition is possible is provided.

[0029]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a processing procedure and an apparatus configuration of a similar sentence retrieval apparatus according to an embodiment of the present invention, wherein 1 is an input section for inputting a first natural language sentence, and 2 is an analysis section.
The information reading unit reads the sentence read by the input unit 1 as shown in FIG.
As shown in (1), the sentence is decomposed into words and phrases by morphological analysis and the like, and a replaceable portion described later and a matchable portion with the addable portion of similar candidate sentences are added. Reference numeral 3 denotes a search unit for comparing the analyzed input sentence with an analyzed example sentence to be described later to search for a similar sentence, and 4 denotes a similar sentence extracted by the search unit 3,
An output unit that outputs a translation of a similar sentence. Reference numeral 5 denotes an example section including a bilingual example collection to be described later.

The search unit 3 analyzes and assigns information by the analysis and information assigning unit 2 as shown in FIG. 13, and searches the analyzed sentence collection 60 for a similar sentence of the input sentence. First, the similar candidate sentence extraction unit 301 narrows down, as similar candidate sentences, example sentences in which the number of words included in the input sentence is equal to or greater than the threshold value. Next, the similar candidate sentence / input sentence processing unit 302 replaces the difference between the input sentence and the similar candidate sentence with a symbol of the same type using the input sentence and the similar candidate sentence to which the information is added by analysis. ,
Unnecessary portions only in the similar candidate sentences are deleted, or words of the missing portions are added to the similar candidate sentences, and processing is performed so that the sentences are grammatically or semantically similar to each other. Next, the input sentence and the similar candidate sentence processed by the similarity calculation unit 303 are calculated for similarity, and the sentence having the highest similarity is extracted together with the similarity.

Reference numerals 10 to 60 denote translation example collections and example analysis / information adding units included in the example section 5 and prepared in advance as a database for similar sentence retrieval.
0 is a bilingual example in which a Japanese sentence (first natural language) with a sentence number and a corresponding English sentence (second natural language) are stored as a plurality of data as illustrated in FIG. FIG. 3 shows a case of Japanese and English bilingual translations of soccer. Such bilingual examples are prepared for each field and for each language to be translated as necessary.

Reference numeral 10 denotes example-independent phrase data (database) commonly used without depending on examples in each field.
FIG. 4 shows an example of example-independent phrase data. In FIG. 4, the dictionary is divided into a dictionary in which words are enumerated as it is, a bilingual pattern in which a plurality of parts of speech and notations are enumerated, and rules for determining an object to be described later according to certain conditions. Replacement targets, deletion targets, and addition targets in the candidate sentence are determined in advance.
The designation of each target is performed manually or automatically in advance. The additional matching target in the input sentence is a target for checking the match with the adding target of the similar candidate sentence, and is common to the deletion target in the similar candidate sentence.

The replacement target in the input sentence and the similar candidate sentence is
Based on the results of the morphological analysis performed by the analysis / information providing unit 2 and the example analysis / information providing unit 50, patterns such as connectives, adverbs, numerals, adnominals, etc. are set as replacement targets. In addition, adjectives and adjective verbs are specified for each conjugation or type according to rules.

The additional matching target in the input sentence and the deletion target in the similar candidate sentence mainly target a word to be modified or an independent word. Based on the results of the morphological analysis performed by the analysis / information addition unit 2 and the example analysis / information addition unit 50, a conjunction, an adverb, an adverb, an adjective adnominal and an adjective verb adjunct, “,”, noun + “ni” Is specified as a deletion target.

The type of the object to be added in the similar candidate sentence and the position to add the phrase are specified. According to the rule of FIG. 4, a pattern including n words before and m words after the deletable portion is extracted from a certain example sentence 1, and a certain example sentence 2 including a pattern in which the deletable portion is missing among them. Search for
If the words corresponding to the m and the last m words match in the part of speech, the type, and the inflected form, respectively, and the element in the same deletable part as the example sentence 1 is not included in the example sentence 2, the example sentence 2
Is added between the n preceding words and the m following words.

For other replacements, deletions, and additions, the required minimum dictionaries, patterns, and rules are designated using a human or an existing general dictionary. The example corresponds to “[time]” or “[time]“ ni ”” in the pattern column of FIG.

Reference numeral 20 denotes example-dependent phrase data (database) prepared for each field. This can be supplemented for field-independent substitutions, deletions, additional matches, and specific expressions not to be added. Example dependent phrase data 2
An example of 0 is shown in FIG. FIG. 5B has the same configuration as FIG.

Reference numeral 30 denotes a data creation unit for automatically creating example-dependent data, for example, an object to be replaced can be obtained. In order to automatically obtain the replacement target, first, the example analyzing / information adding unit 50 uses the bilingual example collection (database) 40 in the example-independent and example-dependent word correspondence data (FIG. 5A) in order. Add information to example sentences. Next, based on such information, replaceable parts and deleteable parts appear in exactly the same range, or deleteable parts that are included in deleteable parts and all independent words such as nouns and adjectives are replaceable parts A possible portion is cut off, and a sentence having a similarity greater than or equal to a threshold T is extracted between the obtained examples by using a similarity calculation between an input sentence and a similar candidate sentence (considering only replacement) described later, and replaced by each example. If the L words that are not the target are the same part-of-speech sequence and the preceding and succeeding K words are the same element, the L word phrase of each example is determined as a new replacement target. FIG. 6 shows an example. As a result, “[Replacement 1]” in FIG. 5B is added. In this case, both "PK" and "free kick" are nouns, so they match grammatically. However, similar sentences are selected from field-dependent example sentences and restricted by matching before and after elements. And more semantically similar. If the obtained replacement target is a noun, “[replacement 1]“ de ”is to be deleted. The threshold value T is initially set to a higher value, and the threshold value is sequentially lowered to a predetermined lower limit to extract a replacement target.

Reference numeral 50 denotes an example-independent phrase data 10, an example-dependent phrase data 20, and a candidate sentence extracted as a similar sentence from the bilingual example collection 40 by morphological analysis or the like as shown in FIG. In the example analysis / information addition unit, replacement / deletion / addition possible portions are checked in the same manner as in the analysis / information addition unit 2 (FIG. 2B) of the input sentence to be processed, and information is added.

Reference numeral 60 denotes an analyzed bilingual example collection (database) for holding the output of the example analyzing / information adding unit 50 and comparing the input sentence with the information with the search unit 3. Are shown in FIGS. 8 and 9.

FIG. 8 shows an example of the Japanese translation of the bilingual example collection shown in FIG. 3 analyzed for each sentence (FIG. 7), and the example-independent phrase data shown in FIG. 4 and FIG. It shows replacement, deletion, and addition locations according to the example-dependent phrase data shown. Information is added from the analysis result in the order of example-independent and example-dependent, in the order of replacement target, deletion target, and addition target. As long as the example-independent and example-dependent phrase data are of the same type, they are treated as the same set regardless of the dependency or dependence (eg, [time]). The parts of speech in FIG. 7 and the replacement, deletion, and addition fields in FIG. 8 are abbreviated as shown in FIG. However, the column of deletion in FIG. 8 shows the range of the deletion phrase. The analyzed translation example book 60 also stores a correspondence table (FIG. 10) between words in a sentence and a sentence number by the process of FIG. 2A.

In the embodiment, the example analysis / information adding unit 50 and the analyzed bilingual example collection 60 will be described in a case where the analyzed data is stored in advance in correspondence with the data 10, 20, and the bilingual example collection 40. Alternatively, a method may be used in which data is extracted from each of the data 10, 20 and the bilingual example collection 40 for each process and information is added.

[0044]

Embodiments of the present invention will be described below with reference to the drawings. In the following embodiments, the input phrase will be described as Japanese and the translated phrase of the searched similar sentence will be described as English, but the present invention is not limited to this.

[Embodiment 1] First, the example section 5 of FIG.
Prepare the data in.

Assuming that FIG. 3 is a bilingual example of the bilingual example collection 40 of FIG. 1, the example analyzing / information adding unit 50 of FIG.
FIG. 7 is created by the morphological analysis processing in the analysis of the example sentence of FIG. FIG. 7 shows “|” for each phrase and “/” for each part of speech.
The part of speech, type number, and inflected form are described. In addition, using FIG. 7, the example-independent phrase data 10 and the example-dependent phrase data 20, a possible replacement, deletion, and addition portion is checked for each example sentence, and information is added, thereby creating FIG. At this time, a correspondence table (FIG. 10) between words and sentence numbers included in the example sentence is also created. 8 and 10
Are stored in the analyzed parallel translation example collection 60 of FIG.

A part of the example-dependent phrase data 20 (FIG. 5B) of FIG. 1 used when creating FIG. 8 is automatically created. First, in order to add a replacement part of the example-dependent phrase correspondence data, the example-independent word data 10 (FIG. 4) and the example-dependent word data 20 (FIG. 5A) are used for the bilingual example collection 40. To give information. Next, as shown in FIG. 6, a part where the replaceable part and the part which can be deleted appear in exactly the same range, and a part which can be deleted where all the independent words included in the part which can be deleted are replaced are deleted. Next, the similarity between the examples obtained by the same method as the similarity calculation between the input sentence and the similar candidate sentence, which will be described later, is calculated (considering only substitution), and a sentence having a similarity equal to or greater than the threshold T is extracted. Next, in a case where the L words that are not replaced in each example are the same part-of-speech sequence and the preceding and succeeding K words are the same element, the L word in each example is determined as a replacement target. As a result, FIG.
“[Replacement 1]” and “Bilingual dictionary: [Replacement 1]” are added to the column of the dictionary to be replaced in (b). When the obtained replacement target [replacement 1] is a noun, “[replacement 1]“ de ”” is added to the pattern to be deleted as in the case of other words. It is also possible to initially set the threshold value T to a higher value and then sequentially lower the threshold value to a predetermined lower limit to extract the replacement target.

Next, in the actual processing, the input sentence “Nakada gave a valuable score in PK” was input from 1 in FIG. 1, and in the analysis and information adding unit 2 in FIG. FIG. 11 is created by the morphological analysis processing in the analysis of the input sentence.
In addition, FIG. 12 is created by examining an additional matching portion between the replaceable portion and the similar candidate sentence and adding information.

Next, the search unit 3 in FIG. 1 searches the input sentence and the example sentence (Japanese example) in FIG. 3 for a similar sentence of the input sentence. The search unit 3 is as shown in FIG.

The similar candidate sentence extracting unit 301 selects a sentence in which the same word as the word of the input sentence exceeds a predetermined threshold value in order to omit the processing of the example sentence that is not very similar.
Actually, the words of the input sentence are checked in FIG. 10, and a sentence in which the ratio of the number of matching words to the number of words of the input sentence is equal to or more than a threshold is selected. Sentence 1 = 8/10 = 0.8, Sentence 2 = 8/10 = 0.8, Sentence 3
= 9/10 = 0.9, sentence 4 = 1/10 = 0.1,
If the threshold is 0.7, sentences 1 to 3 are selected.

The similar candidate sentence / input sentence processing unit 302 generalizes replacement parts, deletes unnecessary phrases, and adds necessary phrases so that the input sentence and the similar candidate sentence are more similar to each other. Is generalized for the replacement part. FIG.
FIG. 4 shows an example of the input sentence and the sentence 3. First, based on the analysis result of the input sentence and the similar candidate sentence, the matching of the notation is checked for each phrase. In FIG. 14, if it is checked whether the same sentence is found in sentence 3 from the head clause of the input sentence, “Nakata”, “PK”,
Since "precious" and "score" match, "1" is stored as the corresponding mark. Next, a match is checked for each of the clauses in which notation does not match and the phrase is replaced. When performing replacement, as shown in sentence 3 of FIG.
Since there may be more than one expression in a clause, the one that contains a lot of notation itself is prioritized, and the total number of elements consisting of words and replaceable parts in the clause is large, and the number of words corresponding to replaceable parts Are stored in parallel in ascending order, and matching is checked in that order. In sentence 3, "in 30 minutes"
There are "in [number] minutes" and "in [time]", but the former takes precedence. As a result, in FIG. 14, the input sentence “[Dynamic_241_for]” does not match. In FIG. 14,
If the phrase does not match even if the phrase is replaced,
A match is checked for all possible words and replacements in the phrase. Here, priorities are sequentially stored in parallel as a replaceable portion where a plurality of words match, a notation of one word, and a replaceable portion of one word, and matching is checked in that order. As a result, in FIG. 14, since “ta” and “.” Match, “1” is stored as the corresponding mark. FIG.
In a similar candidate sentence, if a part that does not match even once is a part that can be deleted, it is deleted. If a part that does not match even once in the input sentence is a part that can be added to the similar candidate sentence, a similar candidate sentence is deleted. Add it to First, the input sentence and sentence 3 are checked for the corresponding parts in
Next, the portion where the sentence 3 can be deleted is checked. First, a check is made to see if there is any part that can be deleted in the whole phrase, and if not, the remaining deleteable parts are combined to select a part that can delete the most words. As a result, "and" and "in 30 minutes" are deleted. Next, the place where addition is possible is examined in sentence 3, but there is no such thing, so the input sentence is “Nakada /
But / PK / in / precious / score / in / up / in /. And sentence 3 are processed into "Nakada / ga / PK // precious / score / was / ta /." Similarly, sentence 1 and sentence 2 are processed as shown in FIG.

Next, the similarity calculator 303 calculates the similarity between the processed input sentence and the similar candidate sentence. here,
The formula for calculating the degree of similarity is as follows, but any other suitable method may be used.

Similarity = 2 × number of matching elements / (number of elements of input sentence + number of elements of similar candidate sentence) As a result, as shown in FIG. 15, sentence 2 is selected as a similar sentence.

Finally, a similar sentence and a translated sentence of the similar sentence are output from the output unit in FIG. In this example, sentence 2 "Nakayama scored a precious score in a free kick."
maadded a variable goal f
roma free kick. Is output. If there are a plurality of elements having the same similarity, as shown in FIG. 15, the elements (words and replaceable parts) after the deletion
The degree of matching, the degree of element matching at, the degree of phrase matching at, and the degree of phrase matching at, are compared in order, and the one with the higher degree of similarity is selected when there is a difference in the degree of similarity. Element matching is performed at the level of a word or replacement part included in a phrase.

If “[Replacement 1]” and “Bilingual dictionary: [Replacement 1]” cannot be automatically added as replacement targets in FIG. 5B, the similarity between sentences 1 to 3 is 2 × 9 / (1
0 + 10) = 0.9. If it is assumed that a plurality of similar sentences are not output as a result, and that a sentence with the same similarity is output at a higher sentence number, the sentence as a whole is more similar to the input sentence than the other sentences. Missing sentence 1 is selected.

In this example, since the data depending on the field of the document is automatically created using the example sentence of the specific field, more detailed points can be obtained by increasing the number of grammatical or semantic replacement parts. It was shown that the similarity calculation can be performed in consideration of the above, and that the time required for the similarity calculation can be reduced by deleting sentences that are not similar to the input sentence in advance.

[Embodiment 2] A description will be given in the same manner as in Embodiment 1.

First, data in the example section 5 of FIG. 1 is prepared in advance.

Assuming that FIG. 16 is a bilingual example of the bilingual example collection 40 of FIG. 1, in the example analyzing / information adding unit 50 of FIG. 1, FIG. 17 is obtained by analyzing the example sentence of FIG. Created. In FIG. 17, each phrase is delimited by “|” and each part of speech is delimited by “/”, and the part of speech, type number, and inflected form are described. In addition, using FIG. 17, the example-independent phrase data 10 and the example-dependent phrase data 20, the possible replacement, deletion, and addition locations are checked for each example sentence, and information is added to create FIG. At this time, at the same time, the correspondence table between the words and the sentence numbers included in each sentence of each translation example (FIG. 19)
Is also created. 18 and 19 are stored in the analyzed bilingual example collection 60 of FIG.

FIG. 5A is used for the phrase data 20 depending on the example because the translation example of this example is shown in FIG.
In addition, the addition target rule (m = 2) in the replacement / deletion points in FIGS. 17 and 18 and the example-independent phrase data in FIG.
18), an addable portion is added to the beginning of sentence 1 in FIG. 18 (the beginning of sentence 2: "and /, / fix_"
190 _ ^{* /} ga ”and the beginning of sentence 1:“ in _190 _ ^{* /} ga ”,“ and /, ”is a part that can be deleted).

Next, in the actual processing, the input sentence "And Nakata scored a valuable score." Is input from the input unit 1 of FIG. 1, and the input sentence of FIG.
FIG. 20 is created by the morphological analysis processing in the analysis of the input sentence of (b). In addition, FIG. 21 is created by examining an additional matching portion between the replaceable portion and the similar candidate sentence and adding information.

Next, the search unit 3 in FIG. 1 searches the input sentence and the example sentence (Japanese example) in FIG. 16 for a similar sentence of the input sentence. This will be described with reference to FIG.

The similar candidate sentence extracting unit 301 selects a sentence containing the same word as the word of the input sentence in a predetermined threshold or more in order to omit the processing of the example sentence that is not very similar in advance.
Actually, the words of the input sentence are checked in FIG. 19, and a sentence in which the ratio of the number of matching words to the number of words of the input sentence is equal to or more than a threshold is selected. Sentence 1 = 7/10 = 0.7, sentence 2 = 8/10 = 0.8, and if the threshold value is 0.7, both sentence 1 and sentence 2 are selected.

The similar candidate sentence / input sentence processing unit 302 generalizes replacement parts, deletes unnecessary words, and adds necessary words so that the input sentence and the similar candidate sentence are more similar to each other. Is generalized for the replacement part. FIG. 22 shows an example of the input sentence and the sentence 1. First, from the analysis result of the input sentence and the similar candidate sentence, when the matching of the notation is checked for each phrase, "precious", "score", and "do" match. Next, when a match is checked for each phrase in which the notation does not match, the phrase is replaced with “[Gu_190]”. When a phrase that does not match even after the replacement of a phrase is checked for a match between all possible words in the phrase and the element of the replacement part, sentence 1
If there is no such, the processing is omitted.
In a similar candidate sentence, a portion that does not match at least once is deleted if it can be deleted.
If there is an addable part in the similar candidate sentence that is the same as the part that does not match the degree, it is added to the similar candidate statement. First, the input sentence,
In the first sentence 1, the corresponding part is checked by. Next, when the deleteable portion of the sentence 1 is examined, there is no deleteable portion in the sentence 1, so the deletion is not performed. Next, there is an input sentence that does not match, and since there is an addable part in sentence 1, it is checked whether it matches. The processing is the same as that of,
The difference is that there is no correspondence on the input sentence side and sentence 1
The side is a match of the addable part. As a result, since “[tangent]” matches, the input sentence is “(tangent,) / [fix_19”
0] / was / precious / score / was / was /. The sentence 1 is processed into “(contact,) / [solid_190] / is / precious / score / is / is / was /.”. Similarly, sentence 2 is processed as shown in FIG.

Next, the similarity calculator 303 calculates the similarity between the processed input sentence and the similar candidate sentence.

As a result, as shown in FIG. 23, sentence 1 is selected as a similar class.

Finally, the output unit 4 of FIG. 1 outputs a similar sentence and a translated sentence of the similar sentence. In this example, sentence 1 “Nakayama scored a valuable score.” And “Nakayama adde”
da value goal. Is output.
If there is a plurality of similarities, the element matching degree of the element (word or replaceable part) after deletion, the phrase matching degree of
Are compared in the order of the degree of phrase matching in the above, and when a difference is found in the similarity, the one with the higher similarity is selected. Element matching refers to matching at the level of a word or replacement part included in a phrase.

If the similar candidate sentence / input sentence processing unit 302 shown in FIG. 13 cannot handle an addition target in the similar candidate sentence, it is not possible to add “[contact,]” to sentence 1.
Is 2.times.8 / (10 + 8) = 0.88, and sentence 2 which is less similar to the input sentence than sentence 1 is selected as the meaning of the entire sentence.

In this example, it has been shown that a more detailed similarity calculation can be performed by considering not only replacement and deletion but also addition.

[Embodiment 3] A description will be given in the same manner as Embodiments 1 and 2.

First, data in the example section 5 of FIG. 1 is prepared in advance.

If FIG. 24 is a bilingual example of the bilingual example set 40 of FIG. 1, the example analysis / information adding unit 50 of FIG. 1 analyzes the example sentence of FIG. Created. In FIG. 25, each phrase is delimited by "|" and each part of speech is delimited by "/", and the part of speech, type number, and inflected form are described. Further, using FIG. 25, the example-independent phrase data 10 and the example-dependent phrase data 20, the replaceable, deleteable, and addable portions are checked for each example, and information is added, thereby creating FIG. 26. At this time, a correspondence table (FIG. 27) between words and sentence numbers included in each sentence of each translation example is also created. 26 and 27 are stored in the analyzed bilingual example collection 60 of FIG.

FIG. 24A is used as the example-dependent phrase data because the bilingual example of this example is shown in FIG.

Next, in the actual processing, the input sentence “Nakayama scored a valuable score in 30 minutes.” Is input from the input unit 1 of FIG. 2
FIG. 28 is created by the morphological analysis processing in the analysis of the input sentence of (b). In addition, FIG. 29 is created by examining an additional matching portion between the replaceable portion and the similar candidate sentence and adding information.

Next, the search unit 3 of FIG. 1 searches the input sentence and the example sentence (Japanese example) of FIG. 24 for a similar sentence of the input sentence. This will be described with reference to FIG.

The similar candidate sentence extracting unit 301 selects a sentence in which the same word as the word of the input sentence exceeds a predetermined threshold value in order to omit the processing of an example sentence that is not very similar in advance.
Actually, the words of the input sentence are checked in FIG. 27, and a sentence in which the ratio of the number of matching words to the number of words of the input sentence is equal to or more than a threshold is selected. Sentence 1 = 9/11 = 0.81, Sentence 2 = 10/11 = 0.90
Assuming that the threshold is 0.7, both sentence 1 and sentence 2 are selected.

The similar candidate sentence / input sentence processing unit 302 generalizes replacement parts, deletes unnecessary words, and adds necessary words so as to make each similar candidate sentence more similar to the input sentence. Generalize the replacement part of the sentence.
As a result of performing processing in the same manner as in Example 1 and Example 2,
The sentence is as shown in FIG.

The similarity calculator 303 calculates the similarity between the processed input sentence and the similar candidate sentence.

As a result, as shown in FIG. 30, sentence 1 is selected as a similar sentence.

Finally, the output unit 4 of FIG. 1 outputs a similar sentence and a translated sentence of the similar sentence. In this example, the sentence 1 “Nakayama scored a valuable score immediately after the start” and “After beg”
inning, Nakayama added a v
available goal. Is output. If there is a plurality of items having the same similarity, as shown in FIG. 30, the element (word or replaceable part) coincidence after deletion, the phrase coincidence with Are compared in the order of the phrase coincidence, and when the similarity differs, the one with the higher similarity is selected. Element matching refers to matching at the level of a word or replacement part included in a phrase.

If the similar candidate sentence / input sentence processing unit 302 in FIG. 13 cannot handle a plurality of morpheme strings as one to be replaced as one to be replaced, the similarity of sentence 1 is 2 ×
9 / (11 + 11) = 0.81, and sentence 2 is selected as the meaning of the entire sentence that is less similar to the input sentence than other sentences.

In this example, it has been shown that the similarity calculation can be performed in consideration of more detailed points by allowing a plurality of words as the unit of the replaceable portion.

The search unit 3 shown in FIG. 1 may extract a predetermined number of similar candidate sentences in descending order of similarity as similar sentences in addition to the similar candidate sentences having the highest similarity.

The similar sentence retrieval method according to the present invention
Specifically, it is executed by a computer such as a personal computer (PC) based on a similar sentence search program recorded in a predetermined computer-readable recording medium in advance. In other words, similar candidate sentences in the example sentence can be grammatically or semantically replaced, deleted, or added in advance on a computer-readable recording medium that records a similar sentence search program that searches the example sentence for a similar sentence of the input sentence. Information is added to the input sentence, and the input sentence and the similar candidate sentence are added after adding each information to a place where the input sentence can be replaced or a similar candidate sentence can be added. When calculating the similarity, the similar candidate sentence with the highest similarity is determined by matching the same type of replacement with the difference of each sentence, deleting unnecessary parts, and adding missing parts. The computer is caused to execute a process of extracting the similar sentence together with the similarity.

According to the present invention, a predetermined number of similar candidate sentences having the highest similarity are output as similar sentences in addition to the similar candidate sentences having the highest similarity in the recording medium storing the similar sentence search program. Causes the computer to execute the processing.

The present invention also relates to a recording medium on which the similar sentence search program is recorded, which can be generally used as data as a base for adding replacement, deletion, and additional information, as well as in the field of documents. Create each data separately for dependent items, and in the automatic creation of data that depends on the document field, add information to example sentences using existing general-purpose or field-dependent data, and specify places that can be replaced and deleted Collect similar sentences from the cut example sentence collection, and in places where replacement information is not added in the sentence, the notation and replacement type of the preceding and following parts match, and the information of the corresponding part is the same and the notation The computer creates a set of different data as new data to be replaced and, at the same time, creates new data to be deleted in consideration of the new data to be replaced and the notation before and after. To be executed.

Further, according to the present invention, in a recording medium on which the similar sentence retrieval program is recorded, when there are a large number of sentences in the example sentence collection of similar candidate sentences, the number of the same phrases as the input sentence exceeds a predetermined threshold value. Is made to be executed by the computer as a new similar candidate sentence.

Further, according to the present invention, a process for extracting a similar sentence of an input sentence and a bilingual translation thereof on a recording medium storing the similar sentence search program, using a bilingual example which is a set of each sentence of the example sentence collection and a translated sentence. Let the computer run.

[0089]

As described above, according to the present invention, the first
When a similar sentence is selected from the input sentence of the first natural language using the bilingual example sentence including only the sentence of the natural language or the corresponding set of sentences of the second natural language, the sentence is analyzed. When comparing the similarity between the input sentence and the analyzed bilingual example sentence, not only the notation itself, replacement or deletion, but also addition is taken into consideration, and replacement, deletion, and addition units are also considered for multiple word strings. Even if the degree of similarity is not high by itself, sentences that are grammatically or semantically similar can be searched. In addition, it is possible to automatically obtain data serving as a base for providing partial replacement, deletion, or additional information from a bilingual translation example in a specific field. In addition, by deleting sentences that are not similar to the input sentence in advance, the time for calculating the similarity can be reduced.

Further, as a part of the process of editing the translation of the bilingual example sentence and translating the input sentence, the present invention can be used to select a bilingual example which is easy to edit and which becomes an appropriate translated sentence.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a processing procedure by a similar sentence search method and a configuration of a similar sentence search device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an analysis process according to the embodiment of the present invention.

FIG. 3 is an explanatory diagram showing an example (1) of a bilingual example book according to the embodiment of the present invention.

FIG. 4 is an explanatory diagram showing an example of example-independent phrase data according to the embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of example-dependent phrase data according to the embodiment of the present invention.

FIG. 6 is an explanatory diagram showing an example of automatic extraction of example-dependent phrase data according to the embodiment of the present invention.

FIG. 7 is an explanatory diagram showing a morphological analysis result of a Japanese example of an example (1) of a bilingual example collection according to the embodiment of the present invention.

FIG. 8 is an explanatory diagram showing an analyzed example collection of Japanese examples in example (1) of a bilingual example collection according to the embodiment of the present invention.

FIG. 9 is an explanatory diagram of part of speech and division according to the embodiment of the present invention.

FIG. 10 is an example of a bilingual example collection according to the embodiment of the present invention (1).
FIG. 8 is an explanatory diagram showing correspondence between words and sentence numbers included in each sentence in the Japanese example of FIG.

FIG. 11 is an explanatory diagram showing a morphological analysis result (1) of an input sentence according to the embodiment of the present invention.

FIG. 12 is an explanatory diagram showing an analysis result (1) of an input sentence according to the embodiment of the present invention.

FIG. 13 is an explanatory diagram illustrating a configuration of a search unit according to an embodiment of the present invention.

FIG. 14 is an explanatory diagram showing an example (1) of processing a similar candidate example sentence and an input sentence according to the embodiment of the present invention.

FIG. 15 is an explanatory diagram showing an example (1) of a sentence used for similarity calculation and a similarity according to the embodiment of the present invention.

FIG. 16 shows an example of a bilingual translation example (2) according to the embodiment of the present invention.
FIG.

FIG. 17 shows an example of a bilingual translation example according to the embodiment of the present invention (2).
FIG. 10 is an explanatory diagram showing a morphological analysis result of the Japanese example.

FIG. 18 is an example (2) of a bilingual translation example book according to the embodiment of the present invention.
It is explanatory drawing which shows the analyzed example collection of Japanese examples.

FIG. 19 is an example of a bilingual translation example according to the embodiment of the present invention (2).
FIG. 8 is an explanatory diagram showing correspondence between words and sentence numbers included in each sentence in the Japanese example of FIG.

FIG. 20 is an explanatory diagram showing a morphological analysis result (2) of an input sentence according to the embodiment of the present invention.

FIG. 21 is an explanatory diagram showing an analysis result (2) of an input sentence according to the embodiment of the present invention.

FIG. 22 is an explanatory diagram showing an example (2) of processing a similar candidate example sentence and an input sentence according to the embodiment of the present invention.

FIG. 23 is an explanatory diagram showing an example (2) of a sentence used for similarity calculation and a similarity according to the embodiment of the present invention.

FIG. 24 is an example of a bilingual example book according to the embodiment of the present invention (3).
FIG.

FIG. 25 is an example (3) of a bilingual translation example book according to the embodiment of the present invention.
FIG. 10 is an explanatory diagram showing a morphological analysis result of the Japanese example.

FIG. 26 is an example (3) of a bilingual translation example book according to the embodiment of the present invention.
It is explanatory drawing which shows the analyzed example collection of Japanese examples.

FIG. 27 is an example (3) of a bilingual translation example book according to the embodiment of the present invention.
FIG. 8 is an explanatory diagram showing correspondence between words and sentence numbers included in each sentence in the Japanese example of FIG.

FIG. 28 is an explanatory diagram showing a morphological analysis result (3) of an input sentence according to the embodiment of the present invention.

FIG. 29 is an explanatory diagram showing an analysis result (3) of an input sentence according to the embodiment of the present invention.

FIG. 30 is an explanatory diagram showing an example (3) of a sentence used for similarity calculation and a similarity according to the embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Input part 2 Analysis and information provision part 3 Retrieval part 4 Output part 5 Example part 10 Example-independent phrase data 20 Example-dependent phrase data 30 Data creation part 40 Parallel translation example collection 50 Example analysis and information provision part 60 Analyzed translation Example collection 301 Similar candidate sentence extraction unit 302 Similar candidate sentence / input sentence processing unit 303 Similarity calculation unit

Claims (15)

[Claims] In a similar sentence search method for searching for a similar sentence of an input sentence from an example sentence collection, each information is stored in a place where a similar candidate sentence of an example sentence can be grammatically or semantically replaced, deleted, or added in advance. After assigning each information to a place where the input sentence can be replaced and a place where a similar candidate sentence can be added, a similarity calculation between the input sentence and the similar candidate sentence is performed. At this time, processing is performed in consideration of matching of the same type of replacement part with respect to the difference part of each sentence, deletion of unnecessary parts and addition of missing parts, and the similar candidate sentence with the highest similarity as a similar sentence along with similarity A similar sentence search method characterized by extraction. 2. The similar sentence search method according to claim 1, wherein a predetermined number of similar candidate sentences having the highest similarity are output as similar sentences in addition to the similar candidate sentences having the highest similarity. 3. Data that can be universally used as base data for adding replacement, deletion, and additional information;
Create each data separately according to the field of the document,
In the automatic creation of data that depends on the document field, information is added to the example sentences using existing general-purpose or field-dependent data, and similar sentences are collected from the example sentence collection in which replaceable and deleteable parts have been deleted. Creates a set of data with the same information but different notation at the place where the replacement information in the sentence is not added and the place before and after it is the same, and the information of the place is the same as the new data to be replaced 3. A similar sentence search method according to claim 1 or 2, wherein the similar sentence search method is created as new data to be deleted in consideration of the new data to be replaced and the notation before and after the new data. 4. When there are a large number of sentences in an example sentence collection of similar candidate sentences, a similar candidate sentence in which the number of words identical to the words of the input sentence is equal to or greater than a predetermined threshold value is determined as a new similar candidate sentence. 4. The similar sentence search method according to claim 1, 2 or 3. 5. A similar sentence of an input sentence and a bilingual translation thereof are extracted by using a bilingual example which is a set of each sentence and a translated sentence of an example sentence collection. Similar sentence search method. 6. An example section in which a plurality of example sentences are stored, input means for reading an input sentence, and a similar candidate sentence obtained from the example sentence of the example section is analyzed in terms of phrases, and grammatically or semantically replaced. Example analysis / information addition means for adding each information to a place where deletion and addition are possible, and an input sentence read by the input means are analyzed in terms of words and phrases, grammatically or semantically replaceable parts or similar. An analysis and information adding means for adding each information to a position where the candidate sentence can be matched with an addable portion; and for calculating the similarity between the input sentence and the similar candidate sentence for the analyzed similar candidate sentence, A search unit that calculates the similarity in consideration of matching of the same type of replacement part with respect to the difference part of the sentence, deletion of an unnecessary part and addition of a missing part, and extracts a similar candidate sentence having the highest similarity as a similar sentence. , Extracted by the search means An output unit for outputting a similar sentence together with the degree of similarity. 7. The similarity according to claim 1, wherein the retrieval means extracts a predetermined number of similar candidate sentences from the one with the highest similarity as similar sentences in addition to the similar candidate sentences with the highest similarity. Sentence search device. 8. When replacing, deleting, and adding additional information, each data is described separately as data that can be used for general purposes and data that depends on the field of the document. In the automatic creation of domain-dependent data, similar sentences are collected from the sentences in the example sentence collection in which replaceable and deleteable parts are deleted using existing general-purpose or field-dependent data, and the replacement information in the sentence is collected. In the parts that are not assigned, the notation and replacement types before and after that are the same, a set of items with the same information but different notations is created as new data to be replaced, and at the same time, a new 8. The similar sentence retrieval apparatus according to claim 6, further comprising data creation means for creating data to be newly deleted in consideration of the data to be replaced and the notation before and after. 9. The search means according to claim 6, 7, or 8, wherein a sentence in which the number of words in the input sentence is equal to or greater than a predetermined threshold value is determined as a similar candidate sentence in advance.
Similar sentence search device. 10. A parallel sentence in which a translated sentence is associated with an example sentence, comprising a similar sentence extracted by the search unit and an output unit for outputting the translated sentence. Item 6, 7, 8 or 9 similar sentence retrieval apparatus. 11. A storage medium storing a similar sentence search program for searching a similar sentence of an input sentence from a set of example sentences, in which a similar candidate sentence of the set of example sentences can be grammatically or semantically replaced, deleted, or added in advance. Each information is added to the location, and each information is added to a location where the input sentence can be replaced and a location where a similar candidate sentence can be added can be matched.
When calculating the similarity between the input sentence and the similar candidate sentence, processing is performed in consideration of the matching of the same type of replacement part with the difference part of each sentence, the deletion of unnecessary parts, and the addition of missing parts. A computer-readable recording medium storing a similar sentence search program for causing a computer to execute a process of extracting a similar candidate sentence having a high similarity as a similar sentence together with the similarity. 12. A storage medium storing the similar sentence search program according to claim 11, wherein a predetermined number of similar candidate sentences having the highest similarity are output as similar sentences in addition to the similar candidate sentences having the highest similarity. And a computer-readable recording medium storing a similar sentence search program for causing a computer to execute a process to be performed. 13. A recording medium on which the similar sentence retrieval program according to claim 11 or 12 is recorded, wherein replacement, deletion,
The data to be used as a basis for adding additional information is divided into data that can be used for general purposes and data that depends on the field of the document. Information is given to example sentences using general-purpose or field-dependent data, and similar sentences are collected from an example sentence collection in which replaceable and deleteable parts are deleted. , The notation and the type of replacement before and after it match, and a set of items with the same information but different notations is created as new data to be replaced, and at the same time, the new data to be replaced A computer-readable recording medium that records a similar sentence search program for causing a computer to execute a process of creating new data to be deleted in consideration of the notation of a document. 14. A recording medium on which the similar sentence retrieval program according to claim 11, 12 or 13 is recorded, wherein, when there are a large number of sentences in an example sentence collection, the number of phrases that are the same as the words in the input sentence for similar candidate sentences. A computer-readable recording medium storing a similar sentence search program for causing a computer to execute a process of setting a similar candidate sentence having a value equal to or greater than a predetermined threshold value as a new similar candidate sentence. 15. A recording medium on which the similar sentence retrieval program according to claim 11, 12, 13, or 14 is recorded,
A computer-readable recording medium which records a similar sentence search program for causing a computer to execute a process of extracting a similar sentence of an input sentence and a translation thereof using a translation example which is a set of each sentence of the example sentence and a translated sentence.