JP6726638B2 - Entailment recognition device, method, and program - Google Patents

Description

The present invention relates to an entailment recognition device, method, and program for recognizing an entailment relationship between two texts t1 and t2.

There is a method of Non-Patent Document 1 as an implication recognition technique for determining whether or not “if t1 is t2” when two texts t1 and t2 are given. In the method of Non-Patent Document 1, when the set E is, for example, a word set, how many elements of the set E commonly appear in t1 and t2 is defined by the following equation.

Here, f(x,t) represents the number of times the element x of the set E appears in t. Using this equation, the degree of similarity representing the degree of the implication relation of “if t1, t2” is calculated by the following equation.

Shohei Hattori, Rifumi Sato, Tomonori Komatani, "Japanese text entailment recognition based on surface similarity," Journal of Japan Society for Artificial Intelligence, Vol.29, No.4, pp.416-426, 2014.

The above conventional method has the following problems.

For the two elements ju and jv in E appearing at t2 and t1, any of the following a), b), c) and d) is established. Here, the sub-sentences are, starting from the verb phrase, the verb phrase, the non-verb phrase related to the verb phrase or related to the verb phrase, and the non-verb phrase related to the non-verb phrase. It is a sentence obtained by connecting and. Alternatively, it is a sentence obtained by connecting the non-verbal phrases in the phrase group excluding such sub-sentences and the non-verbal phrases related thereto.

a) Elements ju and jv are in the same sub-sentence at t2 and in the same sub-sentence at t1.
b) Elements ju and jv are in the same sub-sentence at t2 and not in the same sub-sentence at t1.
c) Elements ju and jv are not in the same sub-sentence at t2, but are in the same sub-sentence at t1.
d) Elements ju and jv are not in the same sub-sentence at t2 and are not in the same sub-sentence at t1.

As an example of a), when t2: “sparrow damages paddy field”, t1: “every year sparrow damages paddy field”, ju: “sparrow”, jv: “paddy field” at t2: It is in the sub-sentence starting from "Troubleshoot." Also, ju: “sparrow” and jv: “paddy” are in the same sub-sentence in the sub-sentence “every year sparrow devastates the paddy field” starting from “disrupt” at t1. Further, the implication relationship of “if t1, t2” is established.

As an example of b), when t2: “Sparrow spoils paddy field”, t1: “Exterminating sparrows spoils paddy field”, ju: “sparrow”, jv: “paddy field” at t2 , In the sub-sentence starting from "Troubleshoot." "Sparrow trolls paddy." and is in the same sub-sentence. In addition, ju: “sparrow” is in the partial sentence “to exterminate sparrow” starting from “disinfect” at t1, and jv: “paddy” starts from “disrupt” in t1. The sub-sentence “to make the rice paddy worse” is present, and ju: “sparrow” and jv: “paddy” are not in the same sub-sentence at t1. In addition, the implication relationship of “if t1, t2” does not hold.

As an example of c), when t2: “Exterminating sparrows damages paddy fields” and t1: “Sparrows damage paddy fields”, ju: “sparrow” selects “disinfect” at t2. There is a sub-sentence that starts from "Being exterminating sparrows", and jv: "Paddy field" is located at a sub-sentence that starts from "Troubleshoot." at t2. "Sparrow" and jv: "Paddy" are not in the same sub-sentence at t2. In addition, ju: “sparrow” and jv: “paddy” are in the same partial sentence as the sub-sentence “sparrow spoils the paddy field” starting from “disrupt” at t1. In addition, the implication relationship of “if t1, t2” does not hold.

As an example of d), when t2: “Exterminating sparrows damages paddy fields” and t1: “Exterminating sparrows damages paddy fields”, ju: “sparrow” is t2. , There is a sub-sentence that begins with “disinfect”, “Periodic extermination of sparrows”, and jv: “Paddy” is a sub-sentence that begins with “disrupt” at t2. , And ju: “sparrow” and jv: “paddy” are not in the same sub-sentence at t2. In addition, ju: "sparrow" is in the partial sentence "disinfect sparrow" starting from "disinfect" at t1, and jv: "paddy" starts in "disrupt" at t1. There is a sub-sentence "to make the rice paddy.", and ju: "sparrow" and jv: "paddy" are not in the same sub-sentence at t1. Further, the implication relationship of “if t1, t2” is established.

When the two elements ju and jv in E appearing at t2 and t1 always satisfy a) or d), there is a high possibility that the implication relation of “if t1 is t2” is established. On the other hand, when a certain ju, jv does not satisfy a) or d) and corresponds to b) or c), it is less likely that the implication relation of “if t1 is t2” is established. In this way, depending on which of the two elements ju, jv in E appearing at t2, t1 is a), b), c), d), a certain degree of implication can be predicted.

However, in the above-mentioned conventional method, since there is no change in the values relating to ju and jv in the similarity calculation formula, information indicating which of ju) and jv corresponds to a), b), c) and d) is given as There was a problem that it could not be utilized at all for recognition of implications.

An object of the present invention is to solve this problem by utilizing information on whether two elements ju and jv in E are in the same sub-sentence in each target text for recognizing an implication relation. An object of the present invention is to provide an entailment recognition device, method, and program for improving the accuracy of entailment recognition.

In order to solve the above problems, the implication recognition device according to the first invention is an implication recognition device that recognizes an implication relationship between two texts t1 and t2, and performs dependency analysis on each of t1 and t2. , Obtained by connecting the adjective phrase, the non-adjective phrase related to the adjective phrase or related to the adjective phrase, and the non-adjective phrase related to the non-verbal phrase starting from the adjective phrase By extracting a partial sentence and extracting a partial sentence obtained by connecting a non-verbal phrase in the phrase group excluding the partial sentence and a non-verbal phrase related to the non-verbal phrase, A subsentence extracting means for extracting a list; a synonym self-supporting portion extracting means for extracting a list of synonymous self-supporting portions in t1 which is synonymous with or synonymous with the self-supporting portion for each self-supporting portion in t2; For each self-supporting part, the alignment selecting means for selecting the alignment between the self-supporting parts by selecting one synonymous self-supporting part from the corresponding synonymous self-supporting part list, and each self-supporting part in t2 in the selected alignment. Alignment similarity calculating means for calculating the similarity of the alignment by averaging the weights of the self-supporting parts of the similarity between the self-supporting part and the synonymous self-supporting part corresponding to the self-supporting part, and the selected alignment In t2, for each pair of independent parts in t2, whether the pair is in the same partial sentence and whether the synonymous independent parts corresponding to each independent part of the pair are in the same partial sentence By obtaining the correction rate of the similarity determined from the similarity correction rate database and taking the average of the similarity correction rates of each pair, which is determined from the weight of each independent part of the pair, the weighting of the pair is performed. An alignment similarity correction unit that calculates the correction similarity of the alignment and calculates the correction similarity of the alignment by multiplying the similarity of the alignment by the correction ratio, and the maximum correction similarity of each alignment. An inter-text similarity calculating means for calculating a value as a similarity between the texts t1 and t2 is included.

The entailment recognition device according to the second invention is a list of sets of texts t1 and t2, for each set, a list of correct sub-sentences of t1 and t2, each independent part in t2, and the independent part. The correct answer alignment which is a list of pairs with the correct one-sense synonymous part in t1 corresponding to and the list of the set of t1 and t2 to which the correct answer label indicating whether or not t1 and t2 have an implication relationship is given. In the set of text t1 and t2 for selecting a set of texts t1 and t2 to be processed and correct alignment of the selected set of t1 and t2. Correct alignment alignment similarity calculation means for calculating the similarity of the correct answer alignment by taking the weighted average of the similarities with the corresponding synonymous independent parts, and during the correct alignment t2, Similarity determined for each pair of self-supporting parts of, whether the pair is in the same correct sub-sentence and whether synonymous self-supporting parts corresponding to each self-supporting part of the pair are in the same correct sub-sentence The correction factor of the degree is unknown, and the similarity correction factor of each pair is averaged with the weight of the pair, which is determined from the weight of each independent part of the pair, to obtain the correction factor of the similarity of the correct alignment. A formula for deriving a formula and a formula for the correction similarity for the correct alignment by multiplying the formula for the correction ratio by the similarity for the correct alignment, and a correct alignment similarity correction means and a set of t1 and t2. Multiple regression analysis means for deriving an optimum value of each similarity correction rate, which is an unknown number, by applying multiple regression analysis to a list of pairs of correct similarity for correct alignment and correct labels of the set. And are included.

Further, the entailment recognition method according to the third aspect of the present invention is a partial sentence extraction means, a synonymous independence part extraction means, an alignment selection means, an alignment similarity calculation means, an alignment similarity correction means, and an inter-text similarity degree. An entailment recognition method in an entailment recognition device for recognizing an entailment relationship between two texts t1 and t2, including calculation means, wherein the partial sentence extraction means performs dependency analysis on each of t1 and t2, A portion obtained by connecting the syllable phrase, the non-syllable phrase related to the syllable phrase, or related to the syllable phrase, and the non-synonym phrase related to the non-syllable phrase with the syllable phrase as a starting point. A list of sub-sentences by extracting sentences and extracting sub-sentences obtained by concatenating non-syllable phrases in the phrase group excluding the sub-sentences and non-syllable phrases related to the non-syllable phrases And extracting the list of synonymous self-supporting parts in t1 which is synonymous with or synonymous with the self-supporting part for each self-supporting part in t2; A step of selecting an alignment between independent parts by selecting one synonymous independent part from a corresponding synonymous independent part list for each independent part in t2; and the alignment similarity calculation means. In the selected alignment, the similarity of the alignment is calculated by taking the weighted average of the similarities between each self-supporting part in t2 and the synonymous self-supporting part corresponding to the self-supporting part. The step of calculating and the alignment similarity correction means, for the selected alignment, for each pair of independence parts in t2, whether the pair is in the same sub-sentence and corresponding to each independence part of the pair. The synonym independence part is obtained from the similarity correction rate database, which determines the similarity correction rate determined by whether or not they are in the same partial sentence, and the weight of each independence part of the similarity correction rate of each pair is obtained. From the weighted average of the pair, the correction rate of the similarity of the alignment is calculated, and the similarity of the alignment is multiplied by the correction rate to calculate the correction similarity of the alignment. And a step in which the inter-text similarity calculating means calculates the maximum value of the corrected similarity of each alignment as the similarity between the texts t1 and t2.

The entailment recognition method according to the fourth aspect of the invention is an entailment recognition device including a set selection unit for texts t1 and t2, a correct alignment similarity calculation unit, a correct alignment similarity correction unit, and a multiple regression analysis unit. An entailment recognition method, which is a list of sets of texts t1 and t2, for each set, a list of correct partial sentences of t1 and t2, each independent part in t2, and t1 corresponding to the independent part. The correct answer alignment, which is a list of pairs of the correct answer in the 1st category independence part, and the list of the set of t1 and t2 to which the correct answer label indicating whether or not t1 and t2 have an implication relationship are input, The step of selecting the set of texts t1 and t2 to be processed by the set selecting means for the texts t1 and t2, and the step t2 in the correct answer alignment of the set of t1 and t2 selected by the correct alignment similarity calculating means. Calculating the similarity of the correct answer alignment by taking the average of the similarities of each independent part and the synonymous independent part corresponding to the independent part, with the weight of the independent part, and the correct alignment In the correct answer alignment, the similarity correction means determines whether or not the pair is in the same correct sub-sentence for each pair of independent parts in t2, and the synonymous independent part corresponding to each independent part of the pair. The similarity correction rate determined by whether or not the same correct sub-sentence is present is used as an unknown number, and the similarity correction rate of each pair is weighted and determined by the weight of each independent part of the pair. By deriving a formula for a correction rate for the similarity of the correct answer, and deriving a formula for a correction similarity for the correct alignment by multiplying the similarity for the correct alignment by the formula for the correction rate. The multiple regression analysis means applies the multiple regression analysis to the list of pairs of correct similarity of correct alignment of each pair of t1 and t2 and the correct answer label of the pair, and thereby the unknown number is obtained. Deriving an optimum value of each similarity correction rate.

The program of the present invention is a program for causing a computer to function as each unit of the implication recognition device of the present invention.

The entailment recognition device according to the first aspect of the invention calculates the degree of the entailment relationship “t1 is t2” for two input texts t1 and t2 as the similarity between t1 and t2. It is an implication recognition device that recognizes the implication relation between t2.

Further, the entailment recognition device according to the second aspect of the invention is a device for acquiring the optimum similarity correction rate to be described in the similarity correction rate database used in the implication recognition from the learning data before performing the implication recognition. ..

The entailment recognition device, method, and program of the present invention have the effect of improving the accuracy of entailment recognition.

It is a block diagram which shows the functional structure of the implication recognition apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the dependency analysis result of a text. It is a figure which shows an example of the correspondence of each self-supporting part in the text t2, and each element of the list of the synonymous self-supporting part in the text t1 extracted with respect to this self-supporting part. It is a figure which shows an example of a thesaurus. It is a figure which shows an example of a word concept base. It is a figure which shows an example of the alignment selected from the synonymous independent part list corresponding to each independent part in the text t2. It is a figure which shows an example of an independent part weight database. It is a figure which shows an example of a similarity correction rate database. It is a flowchart figure which shows the implication recognition processing routine of the implication recognition apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the functional structure of the implication recognition apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of a similarity correction rate database. It is a flowchart figure which shows the implication recognition processing routine of the implication recognition apparatus which concerns on the 2nd Embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment>
FIG. 1 is a configuration example of an implication recognition device showing an example of claim 1 of the present invention. As shown in FIG. 1, an implication recognition device 100 according to the first exemplary embodiment of the present invention includes a CPU, a RAM, a ROM that stores programs and various data for executing each processing routine described later, Can be configured with a computer including. This implication recognition device 100 is functionally provided with an input means 10, a calculation means 20, and an output means 40 as shown in FIG.

The input means 10 receives two texts t1 and t2.

The calculating means 20 includes a partial sentence extracting means 22, a synonymous independent part extracting means 24, an independent part database 25, an alignment selecting means 26, an alignment similarity calculating means 28, an independent part weight database 29, and an alignment similarity. The degree correction unit 30, the similarity correction rate database 31, and the inter-text similarity degree calculation unit 32 are included.

The sub-sentence extraction means 22 receives the two texts t1 and t2 received by the input means 10, and performs dependency analysis on each of t1 and t2. Extracts a partial sentence obtained by connecting a non-verbal phrase related to a phrase or a non-verbal phrase related to the phrase and a non-verbal phrase related to a non-verbal phrase, A partial sentence obtained by connecting the non-verbal phrase and the related non-verbal phrase is extracted, and a list of partial sentences is extracted.

FIG. 2 is a diagram showing the dependency analysis result of the target text “the number of old people living in the rent of wooden structures”. By the dependency analysis, this text is divided into 6 clauses "Wooden", "Building", "For rent", "Live", "Aged", and "Number", and within each clause, , It is divided into a self-supporting part and an accessory part. The independent part of the phrase "wooden" is "wooden", and the attached part is "no". The independence part of the phrase "living" is "living", and the appendix is NULL. Dependency analysis derives the dependency relationship between clauses. “Wooden's” is related to “Building”, “Building” is related to “Rental”, and “Rental” is “Living”. "Living" is related to "old man's", and "old man's" is related to "number".

The sub-sentence extraction unit 22 starts from the verb phrase “live”, the verb phrase “live”, the non-verb phrase “rental” and the non-verb phrase “related to the verb phrase “live”. Non-verbal phrase “building” related to “rent”, non-verbal phrase “wooden construction” related to non-verbal phrase “building”, and non-verbal phrase related to the verbal phrase “live” Extract the partial sentence "Old man living in the rent of a wooden structure" obtained by connecting "Old man's". In this target text, there is one syllable phrase, but if there are other syllable phrases, partial sentences are similarly extracted for each syllable phrase. From the target text, extract the partial sentence "number of old people" obtained by connecting the non-verbal phrase "number" in the phrase group excluding the extracted partial sentences and the related non-verbal phrase "old man's" To do. In this way, the partial sentence lists “of the elderly living in the rent of wooden structures” and “the number of the elderly” are extracted.

The synonymous self-supporting part extraction means 24 extracts, for each self-supporting part in t2, a list of synonymous self-supporting parts in t1 which is synonymous with or synonymous with the self-supporting part. The details will be described below.

Let j1, j2,..., Jm be independent sections during t2. Each ji in this list is an independent part having a different appearance position, and there may be a plurality of independent parts having the same character string in the list. For each ji, in free-standing unit in t1, list of synonymous freestanding unit which is freestanding portion of ji synonymous or synonymous ri1, ri2, ..., extracts rin _i, associated to ji. Each rik in this list is an independent part having a different appearance position, and there may be a plurality of independent parts having the same character string in the list. Moreover, the common synonymous self-supporting part may correspond to the different self-supporting part in t2. When there is no independent part having the same meaning as or synonymous with ji among the independent parts in t1 with respect to ji, ji is associated with NULL indicating that it is empty.

FIG. 3 is a diagram showing a correspondence relationship between each independent part in t2 and each element of the list of synonymous independent parts in t1 extracted for the independent part. The synonymous independent section lists r11, r12, and r13 in t1 correspond to the independent section j1 of t2, the synonymous independent section lists r21 and r22 in t1 correspond to the independent section j2 of t2, and The independent section j3 corresponds to the synonymous independent section lists r31, r32, and r33 in t1, and the independent section j4 at t2 corresponds to NULL.

The self-supporting part in t1 which is the same character string as the self-supporting part ji in t2 becomes a synonymous self-supporting part of ji (However, the meaning of the self-supporting part in t1 is different from the meaning of ji. If this is not the case). The similarity between ji and the self-supporting part in t1 is set to 1.0 and is associated with the pair of ji and the self-supporting part in t1.

It is determined whether or not two independent parts that are not necessarily the same as a character string have the same meaning or a similar meaning using, for example, a thesaurus shown in FIG. If the distance between the nodes on the thesaurus of the two independent parts of interest is less than or equal to a certain value, it is determined to be synonymous or synonymous. Also, based on the distance between the nodes, the similarity between the two target independent sections is calculated and associated with the two target independent sections.

Whether or not two independent parts that are not necessarily the same as a character string are synonymous or synonymous is determined using, for example, the word concept base described in Non-Patent Document 2. The word concept base is a list of pairs of a word and a word concept vector representing the concept of the word. FIG. 5 is a diagram showing an example of a word concept base. The word concept vector of each word is a d-dimensional vector, and the concept vectors of words that are conceptually close are arranged nearby.

[Non-patent document 2] Katsuto Bessho, Toshiro Uchiyama, Tadashi Uchiyama, Ryoji Kataoka, Masahiro Oku, "Generation method of corpus concept base based on co-occurrence between word and semantic attributes," Transactions of Information Processing Society of Japan, Vol.49, No.12, pp.3997-4006, Dec. 2008.

The concept vector of each self-supporting part is generated by synthesizing the word concept vectors of the words forming the self-supporting part. If, for example, the cosine similarity of each of the concept vectors of the two target independent portions is equal to or greater than a certain value, it is determined that they are synonymous or synonymous. In addition, the calculated cosine similarity is associated with two target independent parts.

Databases such as the thesaurus and the word concept base for determining whether the self-supporting parts are synonymous or synonymous are collectively referred to as the self-supporting part database 25. When the self-supporting part in t2 is arbitrarily fixed, the synonymous self-supporting part extraction means 24 refers to the self-supporting part database 25 for each self-supporting part in t1, and the self-supporting part in t2 and the self-supporting part in t1. The degree of similarity with the self-supporting portion is calculated, and when the degree of similarity is equal to or more than a certain value, the self-supporting portion in t1 is recognized as a synonymous self-supporting portion, and the self-supporting portion in t2 and the synonym in t1 are synonymous. The calculated similarity is associated with the pair with the independent section. When there is no corresponding synonymous self-supporting part in t1 with respect to the self-supporting part in t2, the self-supporting part in t2 is associated with NULL indicating that it is empty, and the self-supporting part in t2 and NULL are associated with each other. The similarity of 0.0 is associated with the pair of and. In FIG. 3, a pair of links of each self-supporting part in t2 and the synonymous self-supporting part in t1 corresponding to the self-supporting part are displayed with corresponding degrees of similarity.

The alignment selecting means 26 selects an alignment between independent parts by selecting one synonymous independent part from the corresponding synonymous independent part list for each independent part in t2. The details will be described below.

For each freestanding portion ji in t2, the corresponding synonymous freestanding unit list ri1, ri2, ..., selects one of synonymous freestanding portion from rin _i and rik _i. When the synonymous independence section list is NULL, the synonym independence section list is ri1 and ri1==NULL, and ri1 is selected. Thus, the alignment between the freestanding portion _{(j1, r1k 1), (} j2, r2k 2), ..., (jm, rmk m) is obtained. Each similarity (ji, rik _i ) in the alignment corresponds to the similarity calculated and associated by the synonymous self-supporting portion extraction means 24. There are n ₁ ×n ₂ ×... ×n _m alignments between the self-supporting portions.

When there is an alignment that has not been selected so far, the alignment selecting means 26 selects one of the alignments as a target for subsequent processing, and shifts to the processing of the alignment similarity calculating means 28. When there is no alignment that has not been selected so far, it is determined that the alignment to be processed does not exist, and the process proceeds to the process of the inter-text similarity calculating unit 32.

FIG. 6 is a diagram showing an example of alignment selected from the synonymous self-supporting part list corresponding to each self-supporting part in t2 of FIG. Alignment of FIG. 6, (j1, r11), ( j2, r21), a (j3, r31), (j4 , NULL), the (ji, rik _i) is the similarity in the correspond.

The alignment similarity calculation means 28 calculates the weighted average of the similarities between the self-supporting parts in t2 and the synonymous self-supporting parts corresponding to the self-supporting parts in the selected alignment, The similarity of the alignment is calculated. The details will be described below.

The alignment _{A (j1, r1k 1),} (j2, r2k 2), ..., and (jm, rmk _m). Let si be the similarity corresponding to each (ji, rik _i ). The weight wi of the independent part ji during t2 is acquired from the independent part weight database 29.

FIG. 7 is a diagram showing an example of the independent part weight database 29. Each record is composed of an independent part and a real value of its weight. In FIG. 3 and FIG. 6, the weight of each independent portion acquired from the independent portion weight database 29 is displayed on each independent portion during t2.

The similarity sA of the alignment A is calculated by the following formula.

This alignment similarity means a degree at which the self-supporting part in t2 appears after allowing up to the synonymous self-supporting part at t1, and is a value that is a base of the degree of the implication relationship of “if t1 is t2”. is there.

Regarding the weight of the independence part, for example, when performing entailment recognition between question texts in a Q&A collection, the independence part such as “teach” or “hope” in the question text is premised on “teaching” or “hope”. Therefore, it has no significant meaning. It is not preferable that the alignment similarity is greatly affected by the presence or absence of the self-supporting portion. Therefore, in the self-supporting part weight database 29, the weights of such self-supporting parts are set to 0.1 and the weights of the other self-supporting parts are set to 1.0 as shown in FIG. 7, for example. By doing so, the alignment similarity is hardly affected by the presence or absence of the independent portion such as “teach” or “hope”. As the weight set in the independent part weight database 29, the IDF value of the independent part may be considered.

In the selected alignment, the alignment similarity correction means 30 determines, for each pair of self-supporting parts in t2, whether or not the pair is in the same partial sentence, and a synonymous self-supporting part corresponding to each self-supporting part of the pair. Is obtained from the similarity correction rate database 31 and the similarity correction rate determined by whether or not is in the same sub-sentence is obtained from the similarity correction rate database 31. Is calculated to calculate the correction rate of the similarity of the alignment, and the similarity of the alignment is multiplied by the correction rate to calculate the correction similarity of the alignment. The details will be described below.

The alignment _{A (j1, r1k 1),} (j2, r2k 2), ..., and (jm, rmk _m). The pair (ju, jv) taken from the independent parts j1, j2,..., Jm in t2 is u<v, and the list of pairs is (j1, j2), (j1, j3),..., (j1, jm), (j2, j3), (j2, j4), ... (j2, jm), (j3, j4), ... (j(m-1), jm), and the number of pairs is m(m-. 1)/2.

For each pair (ju, jv), synonymous freestanding portion of the pair _(ruk u, _{rvk v)} corresponding to the freestanding portion of the pair _{Sadamari, (ruk u, rvk v)} a simple (ru, rv) Express.

FIG. 8 is a diagram showing an example of the similarity correction rate database 31. The similarity correction rate for each section of the pair (ju, jv) in t2 and the corresponding pair (ru, rv) in t1 is described. It is classified by whether or not (ju, jv) is in the same sub-sentence in t2 and whether (ru, rv) is in the same sub-sentence in t1. Ru=rv in the sections 1) and 6) means that ru and rv are independent parts having the same appearance position in t1. RU≠rv in sections 2) and 7) means that ru and rv are independent portions having different appearance positions in t1 (there may be the same character string). For the pair (ju,jv) in t2, it is determined which category in the similarity correction rate database 31 corresponds to (ju,jv) and (ru,rv), and the similarity correction rate huv is thereby determined. Determined.

For the pair (ju, jv) in t2, the weight wu of ju and the weight wv of jv, whichever is smaller, is set as the weight wuv of the pair.

The similarity correction rate hA of the alignment A is calculated by the following equation.

When the similarity of the alignment A is sA, the corrected similarity hsA of A is calculated by the following formula.

Regarding the similarity correction rate described in the similarity correction rate database 31, categories 2) and 8) corresponding to cases a) and d) mentioned in the problem to be solved by the invention need to reduce the alignment similarity. Therefore, the similarity correction rate is set to 1.0. In the categories 3) and 7) corresponding to the cases of b) and c) mentioned in the problem to be solved by the invention, the similarity correction rate is set to 0.1 because it is necessary to reduce the alignment similarity.

Regarding the weight of the pair (ju, jv) in t2, if one of ju and jv is an independent part with a small weight, whether or not (ju, jv) and (ru, rv) are in the same sub-sentence Is considered not to affect the alignment similarity correction rate. Therefore, the weight of ju and jv, whichever is smaller, is set as the weight of the pair so that the similarity correction factor is affected by the weight of the independent portion forming the pair.

When the process of the alignment similarity correction unit 30 is completed, the process of the alignment selection unit 26 is performed.

The inter-text similarity calculating means 32 calculates the maximum value of the corrected similarity of each alignment as the similarity between the texts t1 and t2.

Since it is considered that the alignment having the higher correction similarity is the correct answer, the alignment having the maximum correction similarity is adopted as the correct answer, and the maximum value is set as the similarity between the texts t1 and t2.

The output unit 40 outputs the similarity between the texts t1 and t2 calculated by the inter-text similarity calculation unit 32 as a result. The degree of similarity between the texts t1 and t2 output by the output unit 40 represents the degree of the implication relationship of “if t1, t2”.

FIG. 9 is an example of a processing flow of the implication recognition device 100. When the input means 10 receives the two texts t1 and t2, the implication recognition processing routine shown in FIG. 9 is executed.

First, in step S100, the partial sentence extracting unit 22 acquires the two texts t1 and t2 received by the input unit 10.

Then, in step S102, the sub-sentence extracting unit 22 performs dependency analysis on each of the texts t1 and t2 received in step S100, and extracts the verb phrase and the verb phrase from the verb phrase as a starting point. A non-verb in a clause group excluding a sub-sentence obtained by extracting a sub-sentence obtained by concatenating the non-verbal clause related to or related to the non-verbal clause and the non-verbal clause related to the non-verbal clause A partial sentence obtained by connecting the phrase and the non-verbal phrase related thereto is extracted, and a list of partial sentences is extracted.

In step S104, the synonymous self-supporting portion extraction means 24 extracts, for each self-supporting portion in the text t2 received in step S100, a list of synonymous self-supporting portions in t1 that are synonymous with or synonymous with the self-supporting portion. .. Specifically, the synonymous self-supporting part extraction means 24 refers to the self-supporting part database 25 for each self-supporting part in t1 when the self-supporting part in t2 is arbitrarily fixed, and thereby the self-supporting part in t2. The degree of similarity between a section and the self-supporting section in t1 is calculated, and when the degree of similarity is a certain value or more, the self-supporting section in t1 is recognized as a synonymous self-supporting section. Then, the synonymous self-supporting portion extraction means 24 associates the calculated similarity with the pair of the self-supporting portion in t2 and the synonymous self-supporting portion in t1.

In step S106, the alignment selection unit 26 determines whether or not there is an alignment that has not been selected so far among the alignments between the self-supporting portions during the time t1 and the self-supporting portion during the time t2. If there is an alignment that has not been selected so far, the process proceeds to step S108. When there is no alignment that has not been selected so far, it is determined that there is no alignment to be processed, and the process proceeds to step S114.

In step S108, the alignment selection unit 26 selects one alignment from the alignments that have not been selected so far, as a subsequent processing target.

In step S110, the alignment similarity calculation means 28 refers to the self-supporting part weight database 29 in the alignment selected in step S108 and refers to each self-supporting part in t2 and the synonymous self-supporting part corresponding to the self-supporting part. The similarity of the alignment is calculated by taking the average of the similarities weighted by the independent portion.

In step S112, the alignment similarity correction means 30 determines whether or not the pair is in the same sub-sentence for each pair of the independent parts in t2 in the alignment selected in step S108, and each independent part of the pair. The similarity correction rate determined by whether or not the synonymous self-supporting section corresponding to the section is in the same sub-sentence is acquired from the similarity correction rate database 31, and the similarity correction rate of each pair The correction rate of the similarity of the alignment is calculated by taking the average of the weights of the pair determined by the weight of the independent portion. Then, the alignment similarity correction means 30 calculates the correction similarity of the alignment by multiplying the alignment similarity obtained in step S110 by the correction rate.

In step S114, the inter-text similarity calculating means 32 calculates the maximum value of the corrected similarity of each alignment obtained in step S112 as the similarity between the texts t1 and t2, and ends the implication recognition processing routine. ..

The output unit 40 outputs the similarity between the texts t1 and t2 calculated by the inter-text similarity calculation unit 32 as a result.

As described above, according to the implication recognition device according to the embodiment of the present invention, not only is it possible to calculate the alignment similarity that indicates how much the independent parts overlap between the target texts, but When the pair is in the same sub-sentence in one text and not in the same sub-sentence in the other text (that is, in the case of b) and c) mentioned in the problem to be solved by the invention, the similarity correction rate By lowering, the degree of alignment similarity is lowered and the degree of implication is lowered. In this way, the present invention has the effect of improving the accuracy of implication recognition.

<Second Embodiment>
FIG. 10 is a configuration example of an implication recognition device showing an example of claim 2 of the present invention. As shown in FIG. 10, an implication recognition device 200 according to the second exemplary embodiment of the present invention includes a CPU, a RAM, a ROM that stores programs and various data for executing each processing routine described later, Can be configured with a computer including. This implication recognition device 200 is functionally provided with an input means 210, a calculation means 220, and an output means 240, as shown in FIG.

The implication recognition device 200, which is an example of claim 2, is a list of sets of texts t1 and t2, and for each set, a list of correct sub-sentences of t1 and t2, each independent part in t2, and A correct answer alignment which is a list of pairs of correct answers in t1 corresponding to the self-supporting part and a self-supporting part, and a correct answer label indicating whether or not t1 and t2 are in the implication relationship of "if t1, t2" are given. The list of the set of t1 and t2 is set as the input. Number for each t1, t2 set

Is given.

The input unit 210 receives a list of sets of texts t1 and t2. The list of sets of texts t1 and t2 includes, for each set, a list of correct sub-sentences of t1 and t2, each independent part in t2, and one kind of correct answer in t1 corresponding to the independent part. A correct answer alignment, which is a list of pairs with a copy, and a correct answer label indicating whether or not t1 and t2 are in the implication relationship of "if t1, t2" are given.

The computing means 220 includes a set selecting means 224 for the texts t1 and t2, a correct alignment similarity calculating means 228, an independent part database 25, an independent part weight database 29, a correct alignment similarity correcting part 232, and a multiple regression analysis. And means 236.

The set selection means 224 for the texts t1 and t2 selects a set of the texts t1 and t2 to be processed.

That is, if there is a set of t1 and t2 that has not been selected so far, the set selecting means 224 of the texts t1 and t2 selects one set p of t1 and t2 as a target for subsequent processing, and corrects the answer. The process proceeds to the processing of the alignment similarity calculation means 228. If there is no pair of t1 and t2 that has not been selected so far, it is determined that there is no pair of t1 and t2 to be processed, and the multiple regression analysis unit 236 proceeds to the processing.

The correct alignment similarity calculation means 228 calculates the weight of the self-supporting part of the similarity between each self-supporting part in t2 and the synonymous self-supporting part corresponding to the self-supporting part in the correct solution alignment of the selected set of t1 and t2. The similarity of the correct alignment is calculated by taking the added average. The details will be described below.

Let j1, j2,..., Jm be self-supporting portions in t2, and the correct alignment A be (j1, r1), (j2, r2),..., (jm, rm).

With reference to the independent part database 25, the similarity si corresponding to each (ji, ri) is calculated. For example, when the independent part database 25 is a thesaurus as shown in FIG. 4, the similarity si is calculated based on the distance between the nodes on the thesaurus of ji and ri. Alternatively, when the independent part database 25 is based on the word concept as shown in FIG. 5, the concept vectors of ji and ri are generated by synthesizing the word concept vectors of the constituent words, and the cosine similarity between the generated concept vectors is generated. Is the similarity si. When ri is NULL, the similarity si is set to 0.0.

By referring to the independent part weight database 29, the weight wi of each ji is acquired.

The similarity sA of the correct alignment A is calculated by the following formula.

In the correct answer alignment, the correctness alignment similarity correction unit 232 determines, for each pair of independent parts in t2, whether or not the pair is in the same correct partial sentence and a synonym corresponding to each independent part of the pair. The unknown factor is the correction factor of the similarity determined by whether the independent parts are in the same correct sub-sentence, and the weight of the pair determined by the weight of each independent part of the pair is added to the similarity correction factor of each pair. By averaging, the formula of the correction rate of the similarity of the correct answer alignment is derived, and the formula of the correction similarity of the correct answer alignment is derived by multiplying the similarity of the correct answer alignment by the formula of the correction rate. To do. The details will be described below.

When the correct alignment A is (j1, r1), (j2, r2),..., (jm, rm), for each pair (ju, jv) of the self-supporting part in t2, each self-supporting part of the pair. A pair of synonymous self-supporting parts (ru, rv) corresponding to is determined.

FIG. 11 is a diagram showing that the similarity correction rate corresponding to each section g) of the similarity correction rate database 31 according to the first embodiment is set as the unknown number a _g . For the pair (ju,jv) in t2, it is determined which category in the similarity correction rate database 31 corresponds to (ju,jv) and (ru,rv), and the similarity correction rate huv is thereby determined. Determined. huv is any one of a ₁ , a ₂ ,..., A ₁₀ .

For the pair (ju, jv) in t2, the weight wu of ju and the weight wv of jv, whichever is smaller, is set as the weight wuv of the pair.

The formula hA of the similarity correction rate of the correct answer A is derived as follows.

Assuming that the similarity of the correct alignment A is sA, the corrected similarity expression hsA of A is derived as follows. In the following formulas, each unknowns a _g, has placed coefficient when summarizing the section a _g and x _pg. x _pg is a specific real value calculated for each set p of t1 and t2.

In this way, for the set p of t1 and t2, the formula a ₁ x _p1 +a ₂ x _p2 +... +a ₁₀ x _p10 of the correct similarity of correct alignment is derived.

When the processing of the correct answer alignment similarity correction means 232 ends, the processing moves to the processing of the set selection means 224 of the texts t1 and t2.

The multiple regression analysis unit 236 applies the multiple regression analysis to the list of pairs of correct similarity of correct alignment of each pair of t1 and t2 and the correct label of the pair, and each of them is an unknown number. The optimum value of the similarity correction rate is derived. The details will be described below.

When the correct label of the set p of t1 and t2 is “with implication relation”, y _p =1 and when it is “without implication relation”, y _p =0.

each

Then, the following formula is put. a ₁₁ is an unknown constant term.

The multiple regression analysis is performed on the above q equations, and the measured value on the left side

The coefficient _{a 1,} a 2 of residual sums of squares of _{y p} is the _{smallest, ...,} seeking _{a 10,} a _11. Coefficients calculated _{a 1,} a 2, _..., to _{a 10,} corrected similarity _{a 1 x p1 + a 2 x} p2 + ... + a 10 x p10 correct answer alignment set p of the t1, t2 is located "entailment In the case of "," it becomes large, and in the case of "no implication," it becomes small.

In this way, the optimum similarity correction rates a ₁ , a ₂ ,..., A ₁₀ that fit the input learning data are obtained. By storing the calculated optimum similarity correction rates a ₁ , a ₂ ,..., A ₁₀ in the similarity correction rate database 31 and performing the processing of the implication recognition device of claim 1, a new input text t1 is obtained. , T2, an accurate alignment correction similarity can be calculated.

The output means 240 outputs the coefficients a ₁ , a ₂ ,..., A ₁₀ calculated by the multiple regression analysis means 236 as a result. The coefficients a ₁ , a ₂ ,..., A ₁₀ are stored in the similarity correction rate database 31.

FIG. 12 is an example of a processing flow of the implication recognition device 200. The input means 210 is a list of a set of texts t1 and t2, and for each set, a list of correct sub-sentences of t1 and t2, each independent part in t2, and each independent part in t1 corresponding to the independent part. The correct answer alignment, which is a list of pairs with the correct one-sense synonymous independence part, and the correct answer label indicating whether or not there is an implication of whether t1 and t2 are “if t1 is t2”. When the list is accepted, the implication recognition processing routine shown in FIG. 12 is executed.

In step S200, the set selection means 224 for the texts t1 and t2 acquires the list of the sets t1 and t2 received by the input means 210.

In step S202, the set selecting means 224 for the texts t1 and t2 determines whether or not there is a set of t1 and t2 that has not been selected so far from the list of the set of t1 and t2 acquired in step S200. To do. If there is a set of t1 and t2 that has not been selected so far, the process proceeds to step S204. If there is no pair of t1 and t2 that has not been selected so far, it is determined that there is no pair of t1 and t2 to be processed, and the process proceeds to step S210.

In step S204, the set selecting means 224 for the texts t1 and t2 selects one set of t1 and t2 from the set of t1 and t2 which has not been selected so far, as a subsequent processing target.

In step S206, the correct answer alignment similarity calculation unit 228, in the correct alignment of the set of t1 and t2 selected in step S204, the similarity between each self-supporting part in t2 and the synonymous self-supporting part corresponding to the self-supporting part. Of the self-supporting portion is calculated, and the similarity of the correct alignment is calculated.

In step S208, the correct answer alignment similarity correction unit 232 makes the same correct sub-sentence for each pair of the independent sections in t2 in the correct answer alignment of the set of t1 and t2 selected in step S204. The similarity correction rate determined by whether or not there is a synonymous autonomy part corresponding to each autonomy part of the pair is in the same correct sub-sentence, and the similarity correction rate of each pair The weighted average of the pair, which is determined from the weight of each independent part of the pair, is taken to derive an equation for the correction rate of the similarity of the correct answer alignment, and the equation of the correction rate is calculated for the similarity of the correct answer alignment. By multiplying by, the formula of the correction similarity of the correct alignment is derived.

In step S210, the multiple regression analysis unit 236 performs multiple regression for the list of pairs of correct similarity for correct alignment of each pair of t1 and t2 obtained in step S208 and the correct label of the pair. By applying the analysis, the optimal value of each similarity correction rate, which is an unknown number, is derived, and the implication recognition processing routine ends.

The output unit 240 stores the optimum value of each similarity correction rate calculated by the multiple regression analysis unit 236 in the similarity correction rate database 31.

It is possible to construct the processing described above as a program, install the program from a communication line or a recording medium, and execute it by means such as a CPU.

The present invention is not limited to the above embodiments, and various modifications and applications are possible within the scope of the claims.

The present invention can be applied to the entailment recognition technique for recognizing the entailment relationship between two texts t1 and t2.

10, 210 Input means 20, 220 Calculation means 22 Partial sentence extraction means 24 Synonymous independent part extraction means 25 Independent part database 26 Alignment selection means 28 Alignment similarity calculation means 29 Independent part weight database 30 Alignment similarity correction means 31 Similarity Correction ratio database 32 Inter-text similarity calculation means 40, 240 Output means 100, 200 Implication recognition device 224 Text t1, t2 set selection means 228 Correct answer alignment similarity calculation means 232 Correct alignment similarity correction means 236 Multiple regression analysis means

Claims (5)

An entailment recognition device for recognizing an entailment relationship between two texts t1 and t2,
Dependency analysis is performed on each of t1 and t2, and the verb phrase is used as a starting point, and the verb phrase, the non-verb phrase related to the verb phrase, or the non-verb phrase related to the verb phrase, and the non-verb phrase related to It is obtained by extracting a partial sentence obtained by concatenating a non-verbal phrase and a non-verbal phrase in the phrase group excluding the partial sentence and a non-verbal phrase related to the non-verbal phrase. Partial sentence extraction means for extracting a list of partial sentences by extracting partial sentences,
synonym self-sustaining part extraction means for extracting a list of synonym self-supporting parts in t1 which is synonymous with or synonymous with the self-supporting part in t2;
alignment selecting means for selecting an alignment between independent sections by selecting one synonymous independent section from the corresponding synonymous independent section list for each independent section in t2;
In the selected alignment, the similarity of the alignment is calculated by taking the weighted average of the similarities between each self-supporting part in t2 and the synonymous self-supporting part corresponding to the self-supporting part. Alignment similarity calculation means,
In the selected alignment, for each pair of the independent parts in t2, whether the pair is in the same partial sentence and whether the synonymous independent parts corresponding to each independent part of the pair are in the same partial sentence Acquire a similarity correction rate determined by whether or not the similarity correction rate database from the similarity correction rate database, and take the average of the similarity correction rate of each pair, which is determined by the weight of each independent part of the pair. An alignment similarity correction means for calculating the correction similarity of the alignment, and calculating the correction similarity of the alignment by multiplying the alignment similarity by the correction rate.
An implication recognition device, comprising: inter-text similarity calculation means for calculating the maximum value of the corrected similarity of each alignment as the similarity between the texts t1 and t2. A list of sets of texts t1 and t2, for each set, a list of correct answer sub-sentences of t1 and t2, each independence part in t2 and one synonym of correct answer in t1 corresponding to the independence part The input is a correct alignment that is a list of pairs with the independent part and a list of a pair of t1 and t2 to which a correct answer label indicating whether or not t1 and t2 have an implication relationship is input.
Text t1, t2 set selecting means for selecting a set of texts t1, t2 to be processed,
In the correct alignment of the selected set of t1 and t2, by taking the weighted average of the independence parts of the similarity between each independence part in t2 and the synonymous independence part corresponding to the independence part, A correct alignment similarity calculation means for calculating the similarity of the correct alignment;
In the correct answer alignment, for each pair of independent parts in t2, whether the pair is in the same correct partial sentence and whether the synonymous independent part corresponding to each independent part of the pair has the same correct partial sentence The correction rate of the similarity determined depending on whether or not there is an unknown number, and by taking the average of the similarity correction rate of each pair, which is determined by the weight of each independent part of the pair, the correct answer alignment is obtained. A correctness alignment similarity correction means for deriving an expression of the correction rate of the similarity of the correct answer alignment, and multiplying the similarity of the correct answer alignment by the expression of the correction rate,
The optimum value of each similarity correction rate, which is an unknown number, is obtained by applying a multiple regression analysis to the list of pairs of correct similarity of correct alignment of each pair of t1 and t2 and correct labels of the pair. And a multiple regression analysis unit for deriving the implication. It includes a partial sentence extracting unit, a synonymous self-supporting unit extracting unit, an alignment selecting unit, an alignment similarity calculating unit, an alignment similarity correcting unit, and an inter-text similarity calculating unit, and includes two texts t1 and t2. An implication recognition method in an implication recognition device for recognizing an implication relationship between
The sub-sentence extraction unit performs dependency analysis on each of t1 and t2, and uses the syllable phrase and the non-syllable phrase related to the syllable phrase or related to the syllable phrase as a starting point from the syllable phrase. , A partial sentence obtained by concatenating the non-verbal phrase related to the non-verbal phrase, and extracting the non-verbal phrase from the phrase group excluding the partial sentence and the non-verbal phrase related to the non-verbal phrase Extracting a list of partial sentences by extracting partial sentences obtained by connecting and
A step in which the synonymous self-supporting part extraction means extracts, for each self-supporting part in t2, a list of synonymous self-supporting parts in t1 which is synonymous with or synonymous with the self-supporting part;
The alignment selecting means selects an alignment between independent sections by selecting one synonymous independent section from the corresponding synonymous independent section list for each independent section in t2;
The alignment similarity calculation means, in the selected alignment, by taking the weighted average of the self-supporting portion of the similarity between each self-supporting portion in t2 and the synonymous self-supporting portion corresponding to the self-supporting portion, Calculating the similarity of the alignment,
In the selected alignment, the alignment similarity correction means determines whether or not the pair is in the same sub-sentence for each pair of self-supporting parts in t2, and a synonymous self-supporting part corresponding to each self-supporting part of the pair. Is obtained from the similarity correction rate database, the correction rate of the similarity determined by whether or not is in the same sub-sentence, and the similarity correction rate of each pair is determined from the weight of each independent part of the pair. Calculating a correction rate of the similarity of the alignment by taking a weighted average, and calculating a correction similarity of the alignment by multiplying the similarity of the alignment by the correction rate;
The inter-text similarity calculating means includes a step of calculating the maximum value of the corrected similarity of each alignment as the similarity between the texts t1 and t2. An implication recognition method in an implication recognition device including a set selection unit for the texts t1 and t2, a correct alignment similarity calculation unit, a correct alignment similarity correction unit, and a multiple regression analysis unit.
A list of sets of texts t1 and t2, for each set, a list of correct answer sub-sentences of t1 and t2, each independence part in t2 and one synonym of correct answer in t1 corresponding to the independence part The input is a correct alignment that is a list of pairs with the independent part and a list of a pair of t1 and t2 to which a correct answer label indicating whether or not t1 and t2 have an implication relationship is input.
A step of selecting a set of the texts t1 and t2 to be processed by the set selecting means of the texts t1 and t2;
In the correct answer alignment of the selected set of t1 and t2, the correct answer alignment similarity calculating means calculates the weight of the independent part of the similarity between each independent part in t2 and the synonymous independent part corresponding to the independent part. Calculating the degree of similarity of the correct alignment by taking the attached average,
In the correct answer alignment, the correctness alignment similarity correction means determines, for each pair of the independent parts in t2, whether the pair is in the same correct partial sentence and a synonym corresponding to each independent part of the pair. The unknown factor is the correction factor of the similarity determined by whether the independent parts are in the same correct sub-sentence, and the weight of the pair determined by the weight of each independent part of the pair is added to the similarity correction factor of each pair. By averaging, the formula of the correction rate of the similarity of the correct answer alignment is derived, and the formula of the correction similarity of the correct answer alignment is derived by multiplying the similarity of the correct answer alignment by the formula of the correction rate. Steps to
The multiple regression analysis means applies the multiple regression analysis to the list of pairs of correct similarity of correct alignment of each set of t1 and t2 and the correct answer label of the set, and thereby the unknown numbers are obtained. And a step of deriving an optimum value of the similarity correction rate. A program for causing a computer to function as each unit of the implication recognition device according to claim 1.