JP6737151B2 - Synonym expression extraction device, synonym expression extraction method, and synonym expression extraction program - Google Patents

Description

The present invention relates to a synonym expression extraction device, a synonym expression extraction method, and a synonym expression extraction program.

As one of the techniques for extracting synonyms from document data, for the synonym candidates extracted from the document data, the notation similarity between the candidates is calculated based on the context co-occurrence and the notation editing distance, and the notation similarity is calculated. A method of determining whether or not they have the same meaning based on the degree is known (for example, see Patent Document 1). With this type of synonym extraction method, it is possible to extract words used in a specific field or synonyms of compound nouns.

International Publication No. 2014/002776

In the above synonym extraction method, when determining whether a pair of compound nouns is synonymous, it is determined based on the similarity between the notation of one compound noun and the notation of the other compound noun. Therefore, if there are few overlapping parts in the notation of compound nouns in a pair of compound nouns that are actually synonymous, it may be erroneously determined that the pair of compound nouns are not synonymous expressions.

In one aspect, the present invention aims to accurately extract synonymous expressions of compound nouns from document data.

A synonym expression extraction device, which is one aspect, includes a word pair setting unit, a semantic similarity learning unit, a word synonym determination unit, and a compound noun synonym determination unit. The word pair setting unit divides the pair of compound nouns extracted from the document data into a plurality of word pairs, refers to the synonym word dictionary in which synonymous word pairs are registered, and identifies the plurality of word pairs as synonymous word pairs. , It is identified as an undetermined word pair whose synonyms have not been determined. The semantic similarity learning unit performs processing for learning the semantic similarity between words for each of an undetermined word pair and a plurality of synonymous word pairs in document data including synonymous word pairs in a pair of compound nouns. Do multiple times. The word synonym determination unit determines whether the words of the undetermined word pair are synonymous with each other based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity of each of the plurality of synonymous word pairs. Determine whether or not. The compound noun synonym determination unit determines that the compound noun pair is a synonymous expression when all the plurality of word pairs in the compound noun pair are synonymous word pairs. The semantic similarity learning unit in the synonymous expression extraction device adds a case to be used for the learning processing for each learning processing of the semantic similarity for each of a plurality of word pairs to be processed. Further, the word synonym determination unit, the relationship between the number of learning processing and the semantic similarity in the learning result of the semantic similarity of the undetermined word pair, and the number of learning processing in the learning result of the semantic similarity of the synonymous word pair and A correlation coefficient regarding the relationship with the semantic similarity is calculated. The word synonym determination unit determines whether or not the words of the undetermined word pair are synonymous based on the calculated correlation coefficient.

According to the above aspect, it is possible to accurately extract synonymous expressions of compound nouns from document data.

It is a figure which shows the functional structure of the synonymous expression extraction apparatus which concerns on 1st Embodiment. It is a figure which shows the example of a synonym word dictionary. It is a figure explaining the example of the determination method of whether it is synonymous with the example of a pair of compound nouns. It is a flow chart explaining processing which a synonym expression extraction device concerning a 1st embodiment performs. It is a flow chart explaining the contents of synonymous compound noun identification processing. It is a flow chart explaining the contents of semantic similarity learning processing. It is a flowchart (the 1) explaining the content of the synonym determination processing about a word pair. It is a flowchart (the 2) explaining the content of the synonym determination processing about a word pair. It is a flowchart explaining the content of the synonym determination process about the pair of compound nouns. It is a figure which shows the example of a character string and the result of morphological analysis. It is a figure which shows the example of the extraction result of the pair of a compound noun, and the list of word pairs. It is a figure which shows the example of a similarity transition table. It is a figure explaining the determination method of whether an undetermined word pair is a synonymous expression. It is a graph explaining the method of determining the number of learning processes. It is a figure which shows the functional structure of the synonymous expression extraction apparatus which concerns on 2nd Embodiment. It is a flow chart explaining the contents of synonymous compound noun specific processing concerning a 2nd embodiment. 7 is a flowchart illustrating the content of determination threshold setting processing. It is a figure explaining the specific example of the setting method of a determination threshold value. It is a figure which shows the system structure of the synonym dictionary creation system which concerns on 3rd Embodiment. It is a figure which shows the system configuration of the document rewriting system which concerns on 4th Embodiment. It is a figure which shows the functional structure of a document data rewriting device. 9 is a flowchart illustrating a process performed by the document data rewriting device. It is a figure which shows the hardware constitutions of a computer.

[First Embodiment]
FIG. 1 is a diagram showing a functional configuration of a synonym expression extraction device according to the first exemplary embodiment.

As shown in FIG. 1, the synonym expression extraction device 1 according to the present embodiment includes a character string extraction unit 110, a morpheme analysis unit 120, a compound noun extraction unit 130, and a synonymous compound noun identification unit 140. Further, the synonym expression extraction device 1 stores a document set 191, an analysis dictionary 192, an analysis result corpus 193, a synonym word dictionary 194, a similarity transition table 195, and a synonym expression list 196 (a storage unit ( (Not shown).

The character string extraction unit 110 extracts a character string from the document set 191. The document set 191 includes a plurality of document data used for extracting synonymous expressions of compound nouns. Each of the plurality of pieces of document data of the document set 191 includes, for example, a term used in a specific technical field, a specific industry, or the like. The document set 191 is stored in the storage unit of the synonymous expression extraction device 1 via an input reception unit (not shown), for example.

The morpheme analysis unit 120 refers to the analysis dictionary 192 and performs morpheme analysis on the character string extracted by the character string extraction unit 110. The analysis dictionary 192 includes word information used for morphological analysis. The morpheme analysis unit 120 registers the result of the morpheme analysis in the analysis result corpus 193 and passes it to the compound noun extraction unit 130.

The compound noun extraction unit 130 extracts a compound noun in the character string based on the result of the morphological analysis. For example, the compound noun extraction unit 130 extracts a word string that satisfies the compound noun condition based on the order of the parts of speech in the character string. Further, the compound noun extraction unit 130 extracts a pair of compound nouns that may be synonymous expressions from the extracted compound nouns.

Based on the analysis result corpus 193 and the synonym word dictionary 194, the synonymous compound noun specifying unit 140 specifies a pair of compound nouns that are synonymous expressions from the pair of compound nouns extracted by the compound noun extracting unit 130. In the synonym word dictionary 194, a plurality of synonymous word pairs are registered. The synonymous compound noun specifying unit 140 divides each compound noun into words for a pair of compound nouns that may be synonymous expressions, generates a word pair, and refers to the synonym word dictionary 194 to refer to the words of the word pair. Is synonymous with each other. When there is a word pair whose synonym is undetermined among the word pairs in the compound noun pair, the synonymous compound noun specifying unit 140 has the meaning similarity to all the word pairs specified from the result of the morphological analysis. Learn degree. In the following description, a word pair in which the words are synonymous is called a synonymous word pair, and a word pair in which it is undetermined whether the words are synonymous is called an undetermined word pair.

When learning the semantic similarity, the synonymous compound noun identifying unit 140 performs the learning processing (calculation processing) of the semantic similarity based on a plurality of cases multiple times. At this time, the synonymous compound noun specifying unit 140 adds a plurality of cases and performs the next learning process every time one learning process is completed. Here, one case is, for example, the result of morphological analysis for one character string. The synonymous compound noun specifying unit 140 stores the learning result in the similarity estimation table 195 every time the learning processing of the semantic similarity is completed. After that, the synonymous compound noun specifying unit 140 determines whether or not the number of cases (the number of times of processing) in the undetermined word pair and the semantic similarity degree and the relationship between the number of cases and the semantic similarity degree in the synonymous word pair are undetermined. It is determined whether the words of the fixed word pair have the same meaning. When the words of the undetermined word pair are synonymous, the synonymous compound noun specifying unit 140 changes the undetermined word pair to a synonymous word pair. After determining whether or not the undetermined word pair is a synonymous word pair, the synonymous compound noun specifying unit 140 determines that all of the plurality of word pairs in the synonymous expression pair are synonymous word pairs. Or not. When all the plurality of word pairs in the pair of compound nouns are synonymous word pairs, the synonymous compound noun specifying unit 140 determines that the compound nouns in the pair of compound nouns including the plurality of word pairs are synonymous expressions. The synonymous compound noun specifying unit 140 registers the pair of compound nouns determined to be synonymous expressions in the synonymous expression list 196.

The synonymous compound noun specifying unit 140 having the above-described function includes a word pair setting unit 141, a similarity transition table creating unit 142, a semantic similarity learning unit 143, a word synonym determining unit 144, and a compound noun synonymous determining unit 145. And, including.

The word pair setting unit 141 divides each compound noun in a pair of compound nouns that may have synonymous expressions into words to generate a plurality of word pairs, and the plurality of word pairs are defined as undetermined word pairs and synonymous word pairs. Classify into and. The word pair setting unit 141 refers to the synonym word dictionary 194 and classifies a plurality of word pairs into an undetermined word pair and a synonymous word pair. Furthermore, the word pair setting unit 141 refers to the analysis result corpus 193 and the synonym word dictionary 194, and determines from among the words different from the words included in the pair of compound nouns with the possibility of synonymous expression in the document set 191. Extract synonymous word pairs. The word pair setting unit 141 sets the undetermined word pair and the synonymous word pair as a word pair used for determining whether the compound nouns in the pair of compound nouns are synonymous expressions.

The similarity transition table creation unit 142 generates the similarity transition table 195 that stores the learning results of the semantic similarity for the undetermined word pairs and all synonymous word pairs. The similarity transition table calculation unit 142 causes the semantic similarity learning unit 143 to learn (calculate) the semantic similarity of the word pair, and stores the learning result of the semantic similarity of the word pair in the similarity transition table 195. When the total number of word pairs is N and the number of times of performing the semantic similarity learning process is M, the similarity transition table creation unit 142 generates, for example, a similarity transition table 195 of N×M cells. ..

The semantic similarity learning unit 143 generates a semantic vector for the word of the case based on the result (case) of the morphological analysis stored in the analysis result corpus 193, and based on the generated semantic vector, the semantic similarity of the word pair. The process of calculating (learning) the degree is repeated M times. The semantic similarity learning unit 143 adds H cases to be used for calculating the semantic similarity each time the process of calculating the semantic similarity of a word pair is completed. For example, the semantic similarity learning unit 143 sets the number of cases Ht in the m-th (m=1, 2,..., M) process to Ht=m×H.

The word synonym determination unit 144, based on the transition of the semantic similarity regarding the synonymous word pair and the transition of the semantic similarity regarding the undetermined word pair in the learning result of the semantic similarity (similarity transition table 195), It is determined whether or not the words of the undetermined word pair are synonymous expressions. The word synonym determination unit 144 determines, when the correlation coefficient between the transition of the semantic similarity regarding the synonymous word pair and the transition of the semantic similarity regarding the undetermined word pair is equal to or more than a threshold value, the undetermined word pair. The words are determined to be synonymous expressions.

Based on the result of the word synonym determination unit 144, the compound noun synonym determination unit 145 determines whether or not the compound nouns of the pair of compound nouns that are likely to be synonymous are synonymous expressions. The compound noun synonym determination unit 145 determines that the compound nouns of the pair of compound nouns are synonymous expressions when all the word pairs in the pair of compound nouns are synonymous pairs. The compound noun synonym determination unit 145 registers the pair of compound nouns determined to be synonymous expressions in the synonym expression list 196.

FIG. 2 is a diagram showing an example of a synonym word dictionary.
As shown in FIG. 2, the synonym word dictionary 194 stores a plurality of pairs of words that are synonymous expressions (synonymous word pairs). The types of words registered in the synonym word dictionary 194 and the number K of synonymous word pairs can be set as appropriate. For example, the synonym word pair registered in the synonym word dictionary 194 may be a representative synonym word pair having a high appearance frequency in the field of documents included in the document set 191, or the field of documents included in the document set 191. It may be a synonymous word pair that is randomly extracted regardless of.

As described above, the synonym word dictionary 194 is used when determining whether or not compound nouns in a pair of compound nouns that may be synonymous are synonymous.

FIG. 3 is a diagram illustrating an example of a pair of compound nouns and a method of determining whether they have the same meaning.
In FIG. 3, as an example of a pair of compound nouns, a first compound noun (word string) 2 called “fare calculation module” and a second compound noun (word string) 3 called “transportation expense settlement function” are shown. Showing. The first compound noun 2 is a compound noun that combines the first word 201 “fare”, the second word 202 “calculation”, and the third word 203 “module”. On the other hand, the second compound noun 3 is a compound noun that combines the first word 301 “transportation expense”, the second word 302 “settlement”, and the third word 303 “function”. ..

For example, the first compound noun 2 and the second compound noun 3 described above are compound nouns used in the field of application software for calculating a fare (transportation cost). However, the appearance frequency of the first compound noun 2 in the document set 191 for the field is often lower than the appearance frequency of each of the words 201 to 203 in the first compound noun 2. Similarly, the appearance frequency of the second compound noun 3 in the document set 191 for the field is often lower than the appearance frequency of each of the words 301 to 303 in the second compound noun 3. Therefore, it is possible to accurately determine whether the two compound nouns are synonymous with each other based on the appearance frequency and the context similarity between the first compound noun 2 and the second compound noun 3 in the document set 191. Difficult to do.

On the other hand, in the present embodiment, as shown in FIG. 3, the two compound nouns 2 and 3 are each divided into words to generate a plurality of word pairs WP1, WP2, and WP3, and for each word pair, By referring to the synonym word dictionary 194, it is determined whether or not they have the same meaning. In the example shown in FIG. 3, first, the words of the first word 201 (“fare”) in the first compound noun 2 and the first word 301 (“transportation cost”) of the second compound noun 3 It is determined whether the pair WP1 has the same meaning. In the synonym word dictionary 194 shown in FIG. 2, word pairs of “fare” and “transportation expense” are registered. Therefore, the synonymous expression extraction device 1 determines that the word pair WP1 of the first word 201 in the first compound noun 2 and the first word 301 in the second compound noun 3 is synonymous. Similarly, the synonymous expression extraction device 1 uses a word pair of the second word 202 (“calculation”) in the first compound noun 2 and the second word 302 (“settlement”) in the second compound noun 3. It is determined that WP2 is a synonymous word pair.

Here, further, the word pair WP3 of the third word 203 (“module”) in the first compound noun 2 and the third word 303 (“function”) in the second compound noun 3 is shown in FIG. It is assumed that the word is registered in the synonym word dictionary 194. In this case, the synonymous expression extraction device 1 determines that the word pair WP3 of the third word 203 in the first compound noun 2 and the third word 303 in the second compound noun 3 is synonymous. In this way, when it is determined that the plurality of word pairs WP1 to WP3 in the pair of compound nouns are all synonymous word pairs, the synonym expression extraction device 1 determines that the compound nouns in the pair of compound nouns are synonymous. ..

On the other hand, when the word pair WP3 of “module” and “function” is not registered in the synonym word dictionary 194 of FIG. 2, the synonym expression extraction device 1 determines the word pair WP3 in the pair of compound nouns shown in FIG. Judge as an undetermined word pair. In this case, the synonym expression extraction device 1 determines whether or not the word pair WP3 is a synonym word pair in order to determine whether or not the first compound noun 2 and the second compound noun 3 are synonymous expressions. Perform determination processing. The synonymous expression extraction device 1 is based on the transition of the learning result of the semantic similarity for each of the plurality of synonymous word pairs collected from the document set 191, and the transition of the learning result of the semantic similarity for an undetermined word pair. , It is determined whether the undetermined word pair is a synonymous word pair. When it is determined by this process that the word pair WP3 of “module” and “function” is a synonymous word pair, the synonymous expression extraction device 1 determines the first compound noun 2 (“fare calculation module”) and the second compound noun 2 The compound noun 3 (“Transportation expense settlement function”) is determined to be a synonymous expression.

Hereinafter, the processing performed by the synonymous expression extraction device 1 according to the present exemplary embodiment will be described with reference to FIGS. 4 to 8.

FIG. 4 is a flowchart illustrating a process performed by the synonymous expression extraction device according to the first exemplary embodiment.

As shown in FIG. 4, the synonymous expression extraction device 1 of this exemplary embodiment first extracts a character string from the document set 191 (step S1). The processing of step S1 is performed by the character string extraction unit 110. The character string extraction unit 110 extracts a character string from each of a plurality of pieces of document data included in the document set 191 according to a known extraction method.

Next, the synonymous expression extraction device 1 performs morphological analysis of the extracted character string (step S2). The process of step S2 is performed by the morpheme analysis unit 120. The morpheme analysis unit 120 performs a morpheme analysis on each of the extracted plurality of character strings by referring to the analysis dictionary 192 according to a known analysis method. The morpheme analysis unit 120 stores the result of the morpheme analysis in the analysis result corpus 193 and passes it to the compound noun extraction unit 130. The morpheme analysis unit 120 stores the result of the morpheme analysis for each sentence in the document set 191 in the analysis result corpus 193.

Next, the synonymous expression extraction device 1 extracts a compound noun from the character string based on the result of the morphological analysis (step S3). The process of step S3 is performed by the compound noun extraction unit 130. The compound noun extraction unit 130 extracts, from the character string, a word string that satisfies the condition of the compound noun based on the result of the morphological analysis on each character string.

Next, the synonym expression extraction device 1 determines whether or not a plurality of types of compound nouns have been extracted (step S4). The determination of step S4 is performed by the compound noun extraction unit 130, for example. When the compound noun is not extracted, or when the extracted compound noun is one type (step S4; NO), there is no compound noun pair that determines whether or not they are synonymous, so the compound noun extraction unit 130 (Synonym expression extraction device 1) ends the process.

On the other hand, when a plurality of types of compound nouns are extracted (step S4; YES), the synonym expression extraction device 1 next identifies a pair of compound nouns that may be synonymous expressions (step S5). The process of step S5 is performed by the compound noun extraction unit 130, for example. In step S5, the compound noun extraction unit 130 identifies a pair of compound nouns that may be synonymous according to a known identification method. For example, the compound noun extraction unit 130 calculates a context similarity for a compound noun for each of a plurality of combinations of pairs of compound nouns, and based on the context similarity, a compound noun that may have a synonymous expression is extracted. Identify the pair.

Next, the synonymous expression extraction device 1 determines whether or not there is a pair of compound nouns that may be synonymous expressions (step S6). The determination in step S6 is performed by, for example, the compound noun extraction unit 130. When there is no pair of compound nouns that may have synonymous expressions (step S6; NO), there is no pair of compound nouns for determining whether or not they are synonymous, so the compound noun extracting unit 130 (synonymous expression extracting device). In 1), a series of processes for one set of documents 191 ends.

On the other hand, when there is a pair of compound nouns that may be synonymous expressions (step S6; YES), the synonym expression extraction device 1 then performs synonymous compound noun identification processing (step S7). The process of step S7 is performed by the synonymous compound noun specifying unit 140. The synonymous compound noun specifying unit 140 performs the above-described processing to specify a pair of compound nouns that are synonymous expressions. When a pair of compound nouns that are synonymous expressions is specified, the synonym compound noun specifying unit 140 registers the specified pair of compound nouns in the synonym expression list 196. When the synonymous compound noun specifying process (step S7) for the pair of compound nouns that may have the synonymous expression is finished, the synchronous expression extracting device 1 ends the series of processes for one document set 191.

As described above, the process of step S7 (synonymous compound noun specifying process) is performed by the synonymous compound noun specifying unit 140. The synonymous compound noun specifying unit 140 performs, for example, the process shown in FIG. 5 as the synonymous compound noun specifying process.

FIG. 5 is a flowchart illustrating the contents of the synonymous compound noun specifying process.
In the synonymous compound noun specifying process, the synonymous compound noun specifying unit 140 first identifies a synonymous word pair and an undetermined word pair in a pair of compound nouns that may be synonymous expressions (step S71). The process of step S71 is performed by the word pair setting unit 141 of the synonymous compound noun specifying unit 140. As shown in FIG. 3, the word pair setting unit 141 divides a pair of compound nouns that may have synonymous expressions into a plurality of word pairs, and determines whether each word pair is a synonymous word pair or an undetermined word pair. Is identified. The word pair setting unit 141 refers to the synonym word dictionary 194 to identify whether the word pair is a synonymous word pair or an undetermined word pair. Further, in step S71, the word pair setting unit 141 generates, for example, a word pair list indicating whether each of the plurality of word pairs is a synonymous word pair or an undetermined word pair.

Next, the synonymous compound noun specifying unit 140 collects word pairs (synonymous word pairs) of synonymous expressions registered in the synonym word dictionary 194 from the character strings of the document set 191 (step S72). The process of step S72 is performed by the word pair setting unit 141. The word pair setting unit 141 adds, to the above word pair list, synonymous word pairs that do not overlap with the synonymous word pairs in the pair of compound nouns of the synonymous word pairs collected in step S72.

Next, the synonymous compound noun specifying unit 140 performs the semantic similarity learning process (step S73) with the undetermined word pair and the synonymous word pair identified and collected in steps S71 and S72 as the processing target word pair. The semantic similarity learning process of step S73 is performed by the similarity transition table creation unit 142 of the synonymous compound noun specifying unit 140 and the semantic similarity learning unit 143.

In step S73, the similarity transition table creation unit 142 creates the similarity transition table 195 that stores the learning result of the semantic similarity for each word pair to be processed. Further, the similarity transition table calculation unit 142 stores the learning result of the semantic similarity for each word pair to be processed by the semantic similarity learning unit 143 in a predetermined column (cell) of the similarity transition table 195. When the total number of word pairs to be processed is N and learning of the semantic similarity is performed M times, the similarity transition table creation unit 142 may use, for example, a similarity transition table having N×M columns (cells). 195 is generated. In step S73, the semantic similarity learning unit 143 learns the meaning vector for each word of the word pair based on the result (case) of the morphological analysis, and the meaning of the words of the word pair based on the meaning vector. A process of calculating the degree of similarity is performed. At this time, the semantic similarity learning unit 143 repeats the process of calculating the semantic similarity M times for one word pair. Furthermore, the semantic similarity learning unit 143 adds the number of cases used for calculating the semantic similarity each time the process of calculating the semantic similarity for one word pair is completed. For example, the semantic similarity learning unit 143 sets the number of cases Ht in the m-th (m=1, 2,..., M) process for one word pair to Ht=m×H (for example, H=1000). And

After finishing the process of step S73, the synonymous compound noun specifying unit 140 then performs the synonym determination process (step S74) for the undetermined word pair. The process of step S74 is performed by the word synonym determination unit 144 of the synonymous compound noun specifying unit 140. The word synonym determination unit 144 refers to the similarity transition table 195, calculates the correlation coefficient between the transition of the semantic similarity of the synonymous word pair and the transition of the semantic similarity of the undetermined word pair, and the correlation coefficient Is greater than or equal to the threshold, it is determined that the words in the undetermined word pair are synonymous. The word synonym determination unit 144 changes the undetermined word pair determined to be synonymous with each other into a synonymous word pair.

Next, the synonym compound noun specifying unit 140 performs the synonym determination process for the pair of compound nouns (step S75). The process of step S75 is performed by the compound noun synonym determination unit 145 of the synonym compound noun identification unit 140. The compound noun synonym determination unit 145 determines whether or not each of the plurality of word pairs in the pair of compound nouns that may be synonymous is a synonymous word pair. When all the word pairs in the pair of compound nouns are synonymous word pairs, the compound noun synonym determination unit 145 determines that the pair of compound nouns is the pair of compound nouns of the synonymous expression, and the pair of compound nouns is the synonymous expression. Register in list 196.

When the process of step S75 ends, the synonymous compound noun specifying unit 140 ends the synonymous compound noun specifying process (step S7).

The semantic similarity learning process (step S73) of the synonymous compound noun identifying process performed by the synonymous compound noun identifying unit 140 is performed by the similarity transition table creating unit 142 and the semantic similarity learning unit 143 as described above. .. The similarity transition table creation unit 142 and the semantic similarity learning unit 143 perform, for example, the process shown in FIG. 6 as the semantic similarity learning process.

FIG. 6 is a flowchart illustrating the contents of the semantic similarity learning process.
In the semantic similarity learning process, as shown in FIG. 6, first, a similarity transition table 195 that stores M learning results for each of N word pairs is prepared (step S7301). The process of step S7301 is performed by the similarity transition table creation unit 142. The similarity transition table creation unit 142 creates, for example, a table having N×M data storage columns (cells).

After the process of step S7301, the similarity transition table creation unit 142 and the semantic similarity learning unit 143 repeat the first loop process (steps S7302 to S7309) M times. One processing in the first loop processing is the second loop processing (steps S7303 to S7308). At the loop end of the first loop process (steps S7302, S7309), a process of counting the number of times m of the second loop process and a determination of whether or not the number of processes m is m≧M are performed. The similarity transition table creation unit 142 counts and determines the number of times of processing m. When the second loop process is repeated M times, the similarity transition table creation unit 142 ends the first loop process and ends the semantic similarity learning process.

One process (steps S7304 to S7307) in the second loop process is performed by learning the semantic vector of each word in one set of word pairs to calculate the semantic similarity, and calculating the calculated semantic similarity as the similarity transition table. The processing is stored in 195. At the loop end of the second loop processing (steps S7303 and S7308), the processing of selecting one word pair out of the plurality of word pairs and the processing of steps S7304 to S7307 for all word pairs have been performed. Whether or not it is determined. The similarity transition table creation unit 142 performs selection and determination at the loop end of the second loop processing. When the processes of steps S7304 to S7307 have been performed for all word pairs, the similarity transition table creation unit 142 ends the second loop process once.

In one processing in the second loop processing, first, the results of H morphological analysis are acquired from the analysis result corpus 193 as learning examples (step S7304). The processing of step S7304 is performed by the semantic similarity learning unit 143. The semantic similarity learning unit 143 acquires, from the results of the morphological analysis registered in the analysis result corpus 193, H results of the morphological analysis that have not yet been acquired by the current semantic similarity learning process. When the second loop process is the second to Mth times, the semantic similarity learning unit 143 adds the acquired H learning cases to the already acquired learning cases.

The semantic similarity learning unit 143 that has acquired the learning case continues to learn the meaning vector of the word pair based on the acquired learning case (step S7305), and determines the semantic similarity of the word pair based on the learned meaning vector. It is calculated (step S7306). In step S7305, the semantic similarity learning unit 143 learns a semantic vector for each word in the currently processed one word pair according to a known learning method. Further, in step S7305, the semantic similarity learning unit 143 calculates the semantic similarity of the word pair based on the learned semantic vector of each word. The semantic similarity learning unit 143 sends the semantic similarity calculated for the word pair currently being processed to the similarity transition table creation unit 142. Upon receiving the calculation result of the semantic similarity, the similarity transition table creation unit 142 stores the received value of the semantic similarity in the similarity transition table 195 (step S7307). In step S7307, the similarity transition table creation unit 142 identifies the currently processed word pair and the number of times m of processing out of the N×M data storage columns (cells) of the similarity transition table 195. The calculated result of the semantic similarity is stored in the cell.

When the processes of steps S7304 to S7307 are performed on all the word pairs, the similarity transition table creation unit 142 ends the second loop process. After that, the similarity transition table 142 updates the number of processing times m to m=m+1 at the loop end of the first loop processing (step S7302 or S7309), and repeats the second loop processing until m≧M. Then, when the number of times of processing m becomes m≧M, the similarity transition table creation unit 142 ends the first loop processing and ends the semantic similarity learning processing.

When the semantic similarity learning processing (step S73) by the similarity transition table creation unit 142 and the semantic similarity learning unit 143 is completed, the synonymous compound noun specifying unit 140 then performs the synonym determination processing on the word pair (step S74). I do. The process of step S74 is performed by the word synonym determination unit 144 of the synonymous compound noun specifying unit 140. The word synonym determination unit 144 performs the processing shown in FIGS. 7A and 7B as the synonym determination processing in step S74.

FIG. 7A is a flowchart (part 1) explaining the content of the synonym determination process for a word pair. FIG. 7B is a flowchart (part 2) explaining the content of the synonym determination process for a word pair.

In the synonym determination process for a word pair, the word synonym determination unit 144 first acquires the list of word pairs collected from the document set 191, and the similarity transition table 195, as shown in FIG. 7A (step S7401). ..

Next, the word synonym determination unit 144 performs the first loop processing (S7402 to S7411). One processing (steps S7403 to S7410) in the first loop processing includes the second loop processing (steps S7403 to S7406) and the processing of steps S7407 to S7410 performed after the second loop processing. At the loop end of the first loop processing (steps S7402, S7411), the word synonym determination unit 144 performs the processing of selecting undetermined word pairs and the processing of steps S7403 to S7410 for all undetermined word pairs. Whether or not it is determined. When the processes of steps S7403 to S7410 are performed on all the undetermined word pairs, the word synonym determination unit 144 ends the first loop process and ends the synonym determination process for the word pair.

One processing (steps S7404 and S7405) in the second loop processing is performed by calculating the correlation coefficient of the transition of the semantic similarity between the undetermined word pair currently selected as the processing target and one synonymous word pair. It is a process of calculating. The word synonym determination unit 144 first acquires, from the similarity transition table 195, the semantic similarity of the undetermined word pair that is the current processing target and the semantic similarity of the synonymous word pair (step S7404). Subsequently, the word synonym determination unit 144 calculates the correlation coefficient between the acquired semantic similarities (step S7405). In step S7405, the word synonym determination unit 144 calculates the correlation coefficient by the following formula (1), for example.

In Expression (1), x _m is the result of the m-th learning process of the semantic similarity with respect to the undetermined word pair (semantic similarity), and xa is the addition of the M-th learning process results with respect to the undetermined word pair. Average. Further, in the equation (1), y _m is the result of a learning process of the m-th mean similarity to synonymous word pairs (meaning similarity), ya is additive results M times of learning process for the synonymous word pairs Average.

At the loop end of the second loop process (steps S7403, S7406), a process of selecting a synonymous word pair to be processed from all synonymous word pairs and a process of calculating a correlation coefficient for all synonymous word pairs are performed. It is determined whether or not it has been performed. After performing the process of calculating the correlation coefficient for all synonymous word pairs, the word synonym determination unit 144 ends the second loop process.

When the second loop processing is completed, the word synonym determination unit 144 then calculates the average value of the correlation coefficients of the semantic similarity for the undetermined word pair that is the current processing target (step S7407). In step S7407, the word synonym determination unit 144 calculates the average value of the correlation coefficient of the semantic similarity with each synonymous word pair calculated in the second loop processing.

Next, as shown in FIG. 7B, the word synonym determination unit 144 determines whether the calculated average value of the correlation coefficients is equal to or more than the threshold value (step S7408). When the average value of the correlation coefficient is equal to or more than the threshold value (step S7408; YES), the word synonym determination unit 144 changes the undetermined word pair that is the current processing target of the word pair list to the synonymous word pair (step S7409). .. On the other hand, when the average value of the correlation coefficient is smaller than the threshold value (step S7408; NO), the word synonym determination unit 144 deletes the undetermined word pair that is the current processing target from the word pair list (step S7410).

When the second loop process for the undetermined word pair that is the current processing target and the processes of steps S7407 to S7410 are finished, the word synonym determination unit 144 determines whether or not there is an undetermined word pair that has not been subjected to these processes. Is determined (step S7402 or S7411). If there is an undetermined word pair that has not been subjected to the second loop processing and steps S7407 to S7410, the word synonym determination unit 144 performs the second loop processing for the unprocessed undetermined word pair, and steps S7407 to. The process of S7410 is performed. Then, when the second loop process and the processes of steps S7407 to S7410 are performed on all the undetermined word pairs, the word synonym determination unit 144 ends the first loop process, and the synonym determination on the word pairs is performed. The process ends.

When the word synonym determination process (step S74) by the word synonym determination unit 144 ends, the synonym compound noun determination unit 140 then performs the synonym determination process (step S75) for the pair of compound nouns. The process of step S75 is performed by the compound noun synonym determination unit 145 of the synonym compound noun determination unit 140. The compound noun synonym determination unit 145 performs the process shown in FIG. 8 as the synonym determination process of step S75.

FIG. 8 is a flowchart illustrating the contents of the synonym determination process for a pair of compound nouns.

In the synonym determination process for the pair of compound nouns, the compound noun synonym determination unit 145 performs a loop process (S7501 to S7505). One process (steps S7502 to S7504) in the loop process is performed when a plurality of word pairs in a pair of compound nouns that may be synonymous are all synonymous word pairs. This is a process of registering in the synonym expression list 196. The compound noun synonym determination unit 145 performs the processes of steps S7502 to S7504 for each of all compound noun pairs that may be synonymous. At the loop end of the loop processing (steps S7501 and S7505), the compound noun synonym determination unit 145 selects the compound noun pairs, and steps S7502 to all compound noun pairs that may be synonymous. It is determined whether or not the process of S7504 has been performed.

In one process in the loop process, the compound noun synonym determination unit 145 first refers to the result of the synonym determination process for each of the plurality of word pairs in the pair of compound nouns currently selected as the processing target (step S7502).

Next, the compound noun synonym determination unit 145 determines whether all the plurality of word pairs in the pair of compound nouns are synonymous word pairs (step S7503). If all are synonymous word pairs (step S7503; YES), the compound noun synonym determination unit 145 registers the pair of compound nouns currently being processed in the synonym expression list 196 (step S7504) and terminates the loop process. It is determined whether or not (step S7505). On the other hand, when a plurality of word pairs include a word pair that is not synonymous (step S7503; NO), the compound noun synonym determination unit 145 skips step S7504 and determines whether to end the loop processing. Yes (step S7505). When the processes of steps S7502 to S7504 have been performed for all pairs of compound nouns, the compound noun synonym determination unit 145 ends the loop process and ends the synonym determination process for the compound noun.

As described above, in the synonymous expression determination process for the compound noun according to the present embodiment, each compound noun in the pair of compound nouns is divided into words to generate a plurality of word pairs, and all of the plurality of word pairs have the same meaning. A compound noun pair that is a word pair of is determined to be a synonymous expression. In addition, for compound noun pairs that may have synonymous expressions, based on the learning results of the semantic similarity performed for each word pair whose synonym or not is undetermined and each synonymous word pair. Then, it is determined whether or not the word pair whose synonym is undetermined is synonymous. At this time, the learning of the semantic similarity of each word pair is performed in a mode in which a learning case is added every one of a plurality of times, and the relationship between the number of cases and the semantic similarity is calculated. When the semantic similarity of a word pair is learned by such a method, the value of the semantic similarity increases as the number of cases increases in the synonymous word pair. Moreover, the same tendency is observed in the relationship between the number of cases and the semantic similarity in a plurality of synonymous word pairs. Therefore, if the relationship between the number of cases and meaning similarity in a word pair whose synonym is undetermined has the same tendency as the relationship between the number of cases and meaning similarity in a synonymous word pair, it is undetermined. It is possible to determine that the words in the word pair have the same meaning. Therefore, according to the present embodiment, even if the number of pieces of document data in the document set 191 is small or there is a document including a compound noun used only in a specific field, the synonymous expression of the compound noun can be accurately extracted. It will be possible.

FIG. 9 is a diagram showing an example of a character string and a result of morphological analysis. FIG. 10 is a diagram showing an example of a result of extracting a pair of compound nouns and a list of word pairs.

The table 401 in FIG. 9A shows an example of character strings extracted from the document set 191. In the synonym expression extraction device 1 according to the present exemplary embodiment, the morpheme analysis unit 120 performs morpheme analysis (step S2) on each of the character strings extracted from the document set 191. In the table 402 of FIG. 9B, the result of the morphological analysis for the character string “execute the fare calculation module to obtain the transportation cost” shown in the table 401 of FIG. The result of the morphological analysis for the character string "I carry out the transportation expense reimbursement function in order to obtain" is specifically shown. The symbol “/” in the table 402 indicates a morpheme delimiter. In the analysis result corpus 193, for example, the result of morphological analysis for each sentence (each character string) is accumulated in a format like the table 402.

After performing the morphological analysis, the synonymous expression extraction device 1 extracts a compound noun from the character string in the compound noun extraction unit 130 (step S3), and further identifies a pair of compound nouns that may be synonymous expressions ( Step S5). When the condition of a word string in which a plurality of nouns are consecutive is set as one of the conditions of the compound noun, from the character string "Run the fare calculation module to obtain the transportation cost", "Fare calculation module" is displayed. A compound noun "" is extracted. In addition, a compound noun “transportation expense settlement function” is extracted from the character string “execute by the transportation expense settlement function to obtain transportation expenses”. Although not shown, compound nouns are extracted from other character strings according to the same conditions.

After extracting the compound noun, the compound noun extraction unit 130 identifies a pair of compound nouns that may be synonymous based on the context similarity or the like of the compound noun. The above-mentioned two character strings illustrated are both sentences “To obtain transportation expenses (compound noun)... ”, and have high context similarity. Therefore, the compound noun output unit 130 sets the “fare calculation module” and the “transportation expense settlement function” into a pair of compound nouns that may be synonymous, as in the table 403 of FIG. Identify.

Then, in the synonym expression extraction device 1, the word pair setting unit 141 identifies a plurality of word pairs in the pair of compound nouns as synonymous word pairs or determination target word pairs, and collects synonymous word pairs from the document set 191 ( Steps S71 and S72).

As described above, the “fare calculation module” and the “transportation expense settlement function” are compound nouns in which three words (nouns) are combined. Therefore, from the pair of the "fare calculation module" and the "transportation expense settlement function", as shown in the word pair list 404 of FIG. 10B, the word pair of "fare" and "transportation expense", Word pairs of "calculation" and "settlement" and word pairs of "module" and "function" are generated. The word pair setting unit 141 refers to the synonym word dictionary 194 and identifies whether these three word pairs are synonymous word pairs or determination target word pairs. When referring to the synonym word dictionary 194 of FIG. 2, the word pair of “fare” and “transportation expense” and the word pair of “calculation” and “settlement” are synonymous word pairs. On the other hand, the word pair of "module" and "function" is an undetermined word pair. Therefore, the synonym word dictionary 194 registers three word pairs and the attributes (undetermined or synonymous) of each word pair in the word pair list 404. Further, when registering a word pair in the word pair list 404, as shown in FIG. 10B, an array ID for identifying each word pair is given.

Furthermore, the determination target identifying unit 141 collects synonymous word pairs registered in the synonymous word dictionary 194 from the results of morphological analysis for all character strings, and registers them in the word pair list 404. In the synonym word dictionary 194 of FIG. 2, a pair of “execution” and “implementation” is registered as synonymous word pairs. Further, the result of the morphological analysis shown in the table 402 of FIG. 9B includes a character string including the word “execution” and a character string including the word “implementation”. For this reason, the determination target identifying unit 141 registers the word pair of “execution” and “implementation” as a synonymous word pair, as in the word pair list 404 of FIG. The determination target identifying unit 141 also collects all synonymous word pairs registered in the synonym word dictionary 194 from the results of morphological analysis for all character strings, and registers them in the word pair list 404.

After registering the undetermined word pair in the pair of compound nouns and the synonym word pair registered in the synonym word dictionary 194 in the word pair list 404, the synonym expression extraction device 1 performs the semantic similarity degree learning process (step S73). I do. In the semantic similarity learning process, the similarity transition table creation unit 142 and the semantic similarity learning unit 143 perform the processes of steps S7301 to S7309 of FIG. 6 to generate the similarity transition table 195 as shown in FIG. create.

FIG. 11 is a diagram showing an example of the similarity transition table.
As shown in FIG. 11, the similarity transition table 195 has a data storage column (cell) that stores the results of M learning processes for each of N word pairs. The result of the learning process performed M times for each word pair is stored in the data storage field specified by the array ID assigned to the word pair and the learning count m. Further, in the similarity transition table 195 of FIG. 11, the number H of cases acquired in one learning process of the semantic similarity is H=1000. That is, the result of the m-th learning process in the similarity transition table 195 of FIG. 11 is the result of the learning process performed using m×1000 (m=1, 2,..., M) learning cases. Become.

After creating the similarity transition table 195, the synonym expression extraction device 1 causes the word synonym determination unit 144 to perform synonym determination processing for a word pair (step S74). The process of step S74 is a process of determining whether or not the undetermined word pair is a synonymous word pair. The word synonym determination unit 144 performs the processes of steps S7401 to S7411 of FIGS. 7A and 7B and determines whether the undetermined word pair is a synonymous word pair.

FIG. 12 is a diagram illustrating a method of determining whether or not an undetermined word pair is a synonymous expression.
In FIG. 12A, three straight lines showing the relationship between the number of cases of synonymous word pairs and the meaning similarity in the similarity transition table 195, the number of cases and the meaning similarity of undetermined word pairs. And two straight lines showing the relationship of. A synonym word pair is a pair of words defined as synonymous in the synonym word dictionary 194. Further, each word in the synonymous word pair has a higher appearance frequency in the document set 191 than a compound noun in which a plurality of words are combined. Therefore, the semantic similarity of the synonymous word pair calculated based on the learning result of the semantic vector of each word in the synonymous word pair is determined by the number of cases (learning data amount) as shown in (a) of FIG. Tends to be proportionally higher. That is, when two words in a word pair are synonymous, it can be said that the semantic similarity of the word pair tends to increase in proportion to the number of cases (learning data amount).

Therefore, when two words in an undetermined word pair have the same meaning, the relationship between the number of cases and the semantic similarity for the word pair has the same tendency as the relationship between the number of cases and the meaning similarity for the synonymous word pair. It is thought that there is. That is, when two words in an undetermined word pair are synonymous, the semantic similarity of the word pair is proportional to the number of cases (learning data amount) as shown by the thick solid line in FIG. It is thought that it will become high. On the other hand, when the two words in the undetermined word pair are not synonymous, the relationship between the number of cases and the semantic similarity for the word pair is synonymous as shown by the thick dotted line in (a) of FIG. It can be said that there is a tendency that the relationship between the number of cases and semantic similarity in word pairs differs.

Therefore, in the present embodiment, the processing of steps S7403 to S7407 in FIG. 7A is performed to calculate the correlation coefficient between the transition of the semantic similarity of the undetermined word pair and the transition of the semantic similarity of the synonymous word pair. .. For example, in the similarity transition table 195 in FIG. 11, the word pair with the array ID “1” is an undetermined word pair, and the word pairs with other array IDs are synonymous word pairs (see (b) in FIG. 10). reference). For this reason, the word synonym determination unit 144 calculates the correlation coefficient of the semantic similarity for each pair of array IDs in which one array ID is “1”, as in the table 405 of FIG. 12B. To do. After that, the word synonym determination unit 144 calculates the average value of the calculated correlation coefficients of the plurality of semantic similarities, and if the average value is equal to or more than the threshold value, the word whose array ID of one word is “1” The pair is determined to be a synonymous word pair. In the example shown in FIG. 12B, the average value of the correlation coefficient is 0.68. Therefore, when the determination threshold used in step S7408 of FIG. 7B is 0.68 or less, it is determined that the word pair with the array ID “1” (the pair of “module” and “function”) is synonymous. It In this case, the word synonym determination unit 144 changes the attribute of the word pair “module” and “function” in the word pair list 404 of FIG. 10B from undetermined to synonymous (step S7409). .. On the other hand, when the determination threshold value used in step S7408 of FIG. 7B is larger than 0.68, it is determined that the word pair having the array ID “1” is not synonymous. In this case, the word synonym determination unit 144 deletes the word pair of “module” and “function” from the word pair list 404 (step S7410).

After the processing of step S74, in the synonym expression extraction device 1, the compound noun synonym determination unit 145 determines whether a pair of compound nouns that may be synonymous expressions are synonymous expressions. As described above, a pair of compound nouns that may be synonymous with each other includes an undetermined word pair among a plurality of word pairs in the pair of compound nouns. With respect to the undetermined word pair, it is already determined whether or not it is a synonymous word pair by the synonym determination process for the word in step S74. When the undetermined word pair is a synonymous word pair, the attribute of the undetermined word pair in the word pair list 404 is changed from undetermined to synonymous. On the other hand, when the undetermined word pair is not a synonymous word pair, the undetermined word pair is deleted from the word pair list 404. Further, synonymous word pairs in the pair of compound nouns are registered in the word pair list 404. Therefore, when a plurality of word pairs in a pair of compound nouns are all registered as synonymous word pairs in the word pair list 404, the compound noun synonym determination unit 145 determines that the pair of compound nouns is a synonymous expression, Register in the synonymous expression list 196.

As described above, in the synonym expression extraction device 1 according to the present exemplary embodiment, a synonym word pair and a word pair whose synonym is undetermined in the pair of compound nouns extracted from the document set 191 are used for learning. The semantic similarity is learned multiple times while adding cases (data). Then, if the average value of the correlation coefficient for the relationship between the learning frequency and the semantic similarity in the synonymous word pair and the relationship between the learning frequency and the semantic similarity in the undetermined word pair is equal to or greater than the threshold, Definite word pairs are determined to be synonymous word pairs. Then, in the present embodiment, it is determined whether or not the pair of compound nouns is a synonymous expression depending on whether or not all of the plurality of word pairs in the pair of compound nouns are synonymous word pairs. Therefore, according to the present embodiment, it is possible to improve the accuracy of determination as to whether or not the synonymous expression is for a pair of compound nouns having a low appearance frequency in the document set 191.

The number M of times of learning processing of the semantic similarity of each word pair in this embodiment can be set as appropriate, but is preferably 20 times or more (M≧20).

FIG. 13 is a graph illustrating a method of determining the number of learning processes.
In the graph of FIG. 13, the horizontal axis represents the number of times the semantic similarity learning process is executed. In the graph of FIG. 13, the left vertical axis is the significance test value for the correlation coefficient of the semantic similarity, and the right vertical axis is the processing time required for learning processing.

The relationship between the number of executions of the learning process for each word pair and the significance test value is that the significance test value for the correlation coefficient of the semantic similarity increases as the number of executions increases, as shown by the thick solid line in FIG. Decrease. The significance test value is a result of testing whether or not the correlation coefficient is statistically significant, and is calculated by a known test method. This significance test value is related to the reliability of the correlation coefficient of the semantic similarity, and the smaller the significance test value, the higher the reliability of the correlation coefficient of the semantic similarity. That is, as shown in FIG. 13, when the number of executions of the learning process for each word pair is increased, the reliability of the correlation coefficient of the semantic similarity between two word pairs is increased. However, if the number of times the learning process is executed for each word pair is increased, the processing time required for the learning process increases as shown by the thick dotted line in FIG. Therefore, the number of times M the learning process is executed for each word pair is determined by the reliability of the correlation coefficient of the desired semantic similarity (the determination accuracy of whether or not the word pair is a synonymous expression) and the processing of the synonymous expression extraction device 1. It may be appropriately set based on the capability, the desired response, and the like. For example, when the reliability of the significance test value (reliability of 99% or more) being 0.5 or less is desired, the number of times M the learning process is executed for each word pair may be set to about 30 times. Further, for example, when the reliability of the significance test value (reliability of 99% or more) being 0.6 or less is desired, the number of times M the learning process is executed for each word pair can be reduced to about 20 times. Becomes

The flowcharts of FIGS. 4 to 8 are merely examples of the synonymous expression extraction processing according to the present embodiment. The synonymous expression extraction processing according to the present embodiment is not limited to the flowcharts of FIGS. 4 to 8, and the processing content can be appropriately changed without departing from the scope of the present embodiment. For example, in the semantic similarity learning processing (step S73), instead of generating the similarity transition table 195 as shown in FIG. 11, N pieces of the learning processing results for each of the N word pairs are sequentially stored N times. May generate an array of. Further, for example, in the synonym determination process for a word pair (step S74), the semantic similarity of the undetermined word pair which is the current processing target may be acquired before the second loop process in FIG. 7A is started. In this case, in step S7404 in FIG. 7A, only the semantic similarity of the synonymous word pair that is the current processing target needs to be acquired. Therefore, in the second loop processing, it is possible to suppress an increase in processing time caused by acquiring the semantic similarity of the undetermined word pair each time.

The synonymous expression extraction device 1 according to the present embodiment is not limited to the configuration shown in FIG. 1, and may be, for example, a device including an acquisition unit that acquires the document set 191 from an external device. Further, the synonym expression extraction device 1 may be a device that includes a display unit that displays the processing result of the compound noun synonym identification process (step S7), the synonym expression list 196, or the like, or an output unit that outputs the result to an external device. Good.

[Second Embodiment]
FIG. 14 is a diagram showing a functional configuration of the synonymous expression extraction device according to the second exemplary embodiment.

As shown in FIG. 14, the synonym expression extraction device 1 according to the present embodiment includes a character string extraction unit 110, a morpheme analysis unit 120, a compound noun extraction unit 130, and a synonymous compound noun identification unit 140. Further, the synonym expression extraction device 1 stores a document set 191, an analysis dictionary 192, an analysis result corpus 193, a synonym word dictionary 194, a similarity transition table 195, and a synonym expression list 196 (a storage unit ( (Not shown).

The character string extraction unit 110, the morpheme analysis unit 120, the compound noun extraction unit 130, and the synonymous compound noun identification unit 140 in the synonymous expression extraction device 1 of this exemplary embodiment each have the functions described in the first exemplary embodiment. The synonym compound noun specifying unit 140 in the synonym expression extraction device 1 of the present exemplary embodiment has the determination threshold used for determining whether or not the undetermined word pair is a synonymous pair in addition to the function described in the first exemplary embodiment. With the function to set.

The synonym compound noun specifying unit 140 in the synonym expression extracting device 1 of the present exemplary embodiment includes a word pair setting unit 141, a similarity transition table creating unit 142, a semantic similarity learning unit 143, a word synonym determining unit 144, and a composite. And a noun synonym determination unit 145. The word pair setting unit 141, the similarity transition table creation unit 142, the semantic similarity learning unit 143, the word synonym determination unit 144, and the compound noun synonym determination unit 145 in the synonymous compound noun specifying unit 140 according to the present embodiment, respectively. It has the function described in the first embodiment.

Moreover, the synonymous compound noun specifying unit 140 according to the present embodiment further includes a determination threshold setting unit 146. The determination threshold setting unit 146 sets a determination threshold used for determining whether or not the undetermined word pair is a synonymous pair. The determination threshold setting unit 146 sets the determination threshold based on the learning result of the semantic similarity in each of the plurality of synonymous word pairs collected from the document set by the word pair setting unit 141. That is, the determination threshold setting unit 146 sets the determination threshold based on the learning result of the synonymous word pair in the similarity transition table 195 created in the semantic similarity learning process (step S73) described in the first embodiment. ..

The synonymous expression extraction device 1 according to the present exemplary embodiment extracts the synonymous expression of the compound noun from the document set 191 by the processing of steps S1 to S7 shown in FIG. 4, for example. The processes of steps S1 to S6 of the processes of steps S1 to S7 performed by the synonym expression extraction device 1 according to the present exemplary embodiment may have the same processing content as described in the first exemplary embodiment. On the other hand, the synonymous expression extraction device 1 according to the present embodiment is used for determining whether or not the undetermined word pair is a synonymous pair in the semantic similarity learning process included in the synonymous compound noun identification process of step S7. Set the judgment threshold. Therefore, the synonym expression extraction device 1 according to the present exemplary embodiment performs, for example, the process shown in FIG. 15 as the synonymous compound noun specifying process in step S7.

FIG. 15 is a flowchart illustrating the contents of the synonymous compound noun specifying process according to the second embodiment.

Also in the present embodiment, the synonymous compound noun identifying process is performed by the synonymous compound noun identifying unit 140. As shown in FIG. 15, the synonymous compound noun specifying unit 140 first identifies the synonymous word pair and the undetermined word pair in the compound noun pair (step S71), and registers the synonymous word dictionary from the document set. And processing for collecting synonymous word pairs (step S72). The word pair setting unit 141 performs the processing of steps S71 and S72. The word pair setting unit 141 performs the processing described in the first embodiment, and generates the word pair list 404 as shown in FIG. 10B, for example.

Next, the synonymous compound noun specifying unit 140 performs the semantic similarity learning process (step S73) with the undetermined word pair and the synonymous word pair obtained by the processes of steps S71 and S72 as the processing target word pair. The semantic similarity learning process of step S73 is performed by the similarity transition table creation unit 142 of the synonymous compound noun specifying unit 140 and the semantic similarity learning unit 143. For example, the similarity transition table creation unit 142 and the semantic similarity learning unit 143 perform the processing described in the first embodiment (see FIG. 6), and the learning result of the semantic similarity in each of a plurality of word pairs. The similarity transition table 195 (see FIG. 11) showing the transition of is created. That is, also in the present embodiment, the semantic similarity learning unit 143 executes the process of learning the semantic similarity with respect to one set of word pairs a plurality of times, and each time the processing is performed, a case used for learning (morphological analysis Result) is added. For example, the semantic similarity learning unit 143 sets the number of cases Ht in the m-th (m=1, 2,..., M) process for one word pair to Ht=m×H (for example, H=1000). And

When the semantic similarity learning process of step S73 is finished, the synonymous compound noun specifying unit 140 then performs a determination threshold setting process (step S76) used for synonymous determination of word pairs. The determination threshold setting unit 146 performs the determination threshold setting process in step S76. The determination threshold setting unit 146 sets the determination threshold based on the learning result of the semantic similarity regarding the plurality of synonymous word pairs in the similarity transition table 195 generated in step S73. Specifically, the determination threshold setting unit 146 calculates the correlation coefficient for the learning result of the semantic similarity between the synonymous word pairs in the similarity transition table 195, and sets the average value of the correlation coefficient as the determination threshold. To do.

After finishing the process of step S76, the synonymous compound noun specifying unit 140 then performs the synonym determination process (step S74) for the undetermined word pair. The process of step S74 is performed by the word synonym determination unit 144 of the synonymous compound noun specifying unit 140. The word synonym determination unit 144 refers to the similarity transition table 195, calculates the correlation coefficient between the transition of the semantic similarity of the synonymous word pair and the transition of the semantic similarity of the undetermined word pair, and the correlation coefficient Is greater than or equal to the threshold, it is determined that the words in the undetermined word pair are synonymous. The word synonym determination unit 144 changes the undetermined word pair determined to be synonymous with each other into a synonymous word pair. In the process of step S74 according to the present embodiment, the word synonym determination unit 144 determines whether the undetermined word pair is a synonym word pair based on the determination threshold set in step S76. That is, the word synonym determination unit 144 uses the determination threshold set in step S76 for the determination in step S7408 of FIG. 7B.

Next, the synonym compound noun specifying unit 140 performs the synonym determination process for the pair of compound nouns (step S75). The process of step S75 is performed by the compound noun synonym determination unit 145 of the synonym compound noun identification unit 140. The compound noun synonym determination unit 145 determines whether or not each of the plurality of word pairs in the pair of compound nouns that may be synonymous is a synonymous word pair. When all the word pairs in the pair of compound nouns are synonymous word pairs, the compound noun synonym determination unit 145 determines that the pair of compound nouns is the pair of compound nouns of the synonymous expression, and the pair of compound nouns is the synonymous expression. Register in list 196.

When the process of step S75 ends, the synonymous compound noun specifying unit 140 ends the synonymous compound noun specifying process (step S7).

As described above, in the synonymous compound noun specifying process (step S7) according to the present embodiment, the determination threshold value used for determining whether or not the word pair is synonymous based on the learning result of the semantic similarity in the plurality of word pairs. To set. The determination threshold setting process (step S76) is performed by the determination threshold setting unit 146 as described above. The determination threshold setting unit 146 performs, for example, the processing illustrated in FIG. 16 as the determination threshold setting processing.

FIG. 16 is a flowchart illustrating the content of the determination threshold setting process.
The determination threshold setting unit 146 extracts the learning result of the semantic similarity of the synonymous word pairs from the similarity transition table 195 (step S7601). In step S7601, the determination threshold setting unit 146 acquires, for example, the array ID of a word pair having the same attribute from the word pair list 404 of FIG. 10B, and obtains the learning result of the semantic similarity of the array ID. It is extracted from the similarity transition table 195.

Next, the determination threshold value setting unit 146 performs a loop process (steps S7602 to S7604) of calculating the correlation coefficient between the semantic similarities of the synonymous word pairs. One process in the loop process is a process of calculating a correlation coefficient between semantic similarities in each of two synonymous word pairs (step S7603). At the loop end (steps S7602 and S7604) in the loop processing, the determination threshold setting unit 146 selects the two synonymous word pairs for which the correlation coefficient is calculated and the correlation coefficient for all the synonymous word pair combinations. It is determined whether or not it has been calculated. When the correlation coefficient is calculated for all of the combinations when two synonymous word pairs are extracted from the plurality of synonymous word pairs that are the processing targets, the determination threshold setting unit 146 ends the loop processing.

When the loop processing (steps S7602 to S7604) of calculating the correlation coefficient of the semantic similarity between two synonymous word pairs is completed, the determination threshold setting unit 146 then determines the correlation coefficient of the calculated similarities. An average value is calculated (step S7605). In step S7605, the determination threshold setting unit 146 calculates the average value of the correlation coefficients from the plurality of correlation coefficients calculated in the loop processing (steps S7602 to S7604).

After that, the determination threshold setting unit 146 sets the average value of the calculated correlation coefficients as the determination threshold of the synonymous expression for the word pair (step S7606), and ends the determination threshold setting process.

After completing the determination threshold setting process (step S76), the synonym compound noun specifying unit 140 performs the synonym determination process (step S74) for the word pair as described above, and the undetermined word pair is a synonymous word pair. Determine whether there is. The processing in step S74 is performed by the word synonym determination unit 144. The word synonym determination unit 144, for example, performs the processes of steps S7401 to S7411 illustrated in FIGS. 7A and 7B. At this time, the word synonym determination unit 144 uses the determination threshold set in step S76 as the determination threshold in the determination in step S7408.

FIG. 17 is a diagram illustrating a specific example of the method of setting the determination threshold.
FIG. 17A shows the word pair list 404 exemplified in the first embodiment. As described above, the word pair list 404 is created by the word pair creation unit 141 by performing the processing of steps S71 and S72 in the synonymous compound noun identification processing (step S7). In the word pair list 404, the word pairs extracted from the compound noun pairs and the synonymous word pairs collected from the document set 191 are registered. Further, in the word pair list 404, an attribute of “undetermined” or “synonymous” indicating whether or not each word pair is synonymous, and an array ID for identifying the word pair are assigned.

In the determination threshold setting process according to the present embodiment, the determination threshold setting unit 146 determines that the undetermined word pair is a synonymous word pair based on the correlation coefficient between the learning results (transitions) of the semantic similarity of the synonymous word pair. Determine whether there is. Therefore, the determination threshold value setting unit 146 calculates the correlation coefficient of the semantic similarity for all combinations (array ID pairs) when selecting two array IDs from the array IDs of the synonymous word pairs. .. For example, in the word pair list 404 of FIG. 17A, the attribute of the word pair with the array ID 1 is “undetermined”, and the attribute of the word pair with the array IDs 2 to 4 and N is N. Is "synonymous". Therefore, as shown in the table 406 of FIG. 17B, the determination threshold setting unit 146 generates the array ID pairs {2,3}, {2,4} and the like assigned to the synonymous word pairs. , A correlation coefficient of semantic similarity is calculated for each pair of array IDs (steps S7602 to S7604). The determination threshold setting unit 146 calculates the correlation coefficient of the semantic similarity by the above formula (1). When the correlation coefficient of the semantic similarity is calculated for all the pairs of array IDs, the determination threshold setting unit 146 calculates the average value of the correlation coefficients from the calculated plurality of correlation coefficients (step S7605).

As described above, in the present embodiment, the determination threshold used for determining whether or not the undetermined word pair extracted from the pair of compound nouns is synonymous is the semantic similarity of the synonymous word pairs collected from the document set 191. It is set automatically based on the correlation coefficient. As shown in (a) of FIG. 12, the relationship between the number of cases and the meaning similarity in the learning result of the meaning similarity for each of a plurality of synonymous word pairs becomes higher as the number of cases increases. Although the same tendency is seen, there is a difference in the amount of change (the slope of the straight line). Therefore, by setting the determination threshold value based on the correlation coefficient of the semantic similarity of the synonymous word pairs in the document set 191 including the document for extracting the compound noun of the synonymous expression, the content of the document of the document set 191 can be adjusted. It is possible to determine whether the word pair is synonymous with the threshold value. Therefore, according to the present embodiment, it is possible to improve the accuracy of determining whether or not undetermined word pairs have the same meaning.

The flowcharts of FIGS. 15 and 16 are merely an example of the synonymous compound noun specifying process according to the present embodiment. The synonymous compound noun specifying process according to the present embodiment is not limited to the processes shown in FIGS. 15 and 16, and the processing content can be appropriately changed without departing from the scope of the present embodiment.

[Third Embodiment]
FIG. 18 is a diagram showing the system configuration of the synonym dictionary creating system according to the third embodiment.

As shown in FIG. 18, the synonym dictionary creating system 5 according to the present embodiment includes a synonym expression extracting device 1, a first storage device 6, and a second storage device 7. The synonymous expression extraction device 1 is connected to each of the first storage device 6 and the second storage device 7 by a transmission cable or the like, and data can be transmitted and received between the devices. The synonym expression extraction device 1, the first storage device 6, and the second storage device 7 are connected to a network 8 such as the Internet or a Local Area Network (LAN), and the terminal device 9 is connected via the network 8. (9A to 9C) is communicatively connected.

The synonym expression extraction device 1 according to the present exemplary embodiment extracts synonymous expressions for compound nouns included in a document set, as described in the first or second exemplary embodiment. The synonym expression extraction device 1 according to this exemplary embodiment includes a communication unit that performs communication (data transmission/reception) with each of the first storage device 6 and the second storage device 7.

The first storage device 6 is a device for storing a document set for extracting synonymous expressions of compound nouns. For example, in the first storage device 6, as shown in FIG. 18, a plurality of document data of a predetermined field including a first field document set 601 and a second field document set 602 are stored. Store a set of documents. In the plurality of document sets 601, 602, for example, document data collected by the user of the synonym dictionary creating system 5 using the terminal device 9 and transferred from the terminal device 9 to the first storage device 6 is accumulated. It

The second storage device 7 is a device for storing a synonym dictionary including a pair of compound nouns that are synonymous expressions. For example, as shown in FIG. 18, the second storage device 7 stores a plurality of synonym dictionaries including a synonym dictionary 701 of the first field and a synonym dictionary 702 of the second field. The first-field synonym dictionary 701 is a dictionary in which synonymous expressions about compound nouns extracted from the first-field document set 601 are registered. The synonym dictionary 702 of the second field is a dictionary in which synonymous expressions about compound nouns extracted from the document set 701 of the second field are registered. In the plurality of synonym dictionaries 701 and 702, for example, pairs of compound nouns that are synonymous expressions extracted from the document sets 601 and 602 by the synonym expression extraction device 1 are registered. The user of the synonym dictionary creating system 5 accesses the synonym dictionaries 701 and 702 of the second storage device 7 by using the terminal device 9 and displays another expression (synonymous expression) about the compound noun in the document. It is possible to look it up. The synonym dictionary stored in the second storage device 7 may include not only synonyms for compound nouns but also synonyms for various words.

In the synonym dictionary creating system 5 according to the present embodiment, for example, the user of the system performs a process of collecting various document data using the terminal device 9 and accumulating them in the document sets 601 and 602 as needed. In addition, the synonym expression extraction device 1 in the synonym dictionary creation system 5 receives, for example, a command transmitted by the system user to the synonym expression extraction device 1 using the terminal device 9 or a preset date and time. Upon arrival, the process as shown in FIG. 4 is performed. At this time, the synonymous expression extraction device 1 performs a process of acquiring a document set of one field from the first storage device 6 and extracting a pair of synonymous compound nouns. After extracting the synonymous compound noun pairs, the synonym expression extracting device 1 registers the extracted compound noun pairs in a predetermined synonym dictionary of the second storage device 7. When registering the extracted pair of compound nouns in the synonym dictionary, the synonym expression extracting device 1 refers to the pair of compound nouns registered in the synonym dictionary and adds only unregistered pairs in the synonym dictionary. sign up.

As described above, the synonym dictionary creating system 5 according to the present embodiment extracts synonymous expressions of compound nouns used in a specific field based on a document set in which only documents in a specific field are collected, and Register (store) in the prepared synonym dictionary. Further, in the synonym dictionary creating system 5 according to the present embodiment, as described in the first and second embodiments, the compound noun pair is divided into a plurality of word pairs, and the plurality of word pairs are divided into a plurality of word pairs. All compound noun pairs that are synonymous word pairs are determined to be synonymous expressions. Therefore, the synonym dictionary creating system 5 according to the present embodiment can efficiently and accurately extract synonymous expressions of compound nouns used only in a specific field to create a synonym dictionary. ..

The system configuration of FIG. 18 is merely an example of the system configuration of the synonym dictionary creating system 5 according to this embodiment. The synonym dictionary creation system 5 according to the present embodiment is not limited to the system configuration of FIG. 18, and can be changed as appropriate. For example, the first storage device 6 may store only a set of documents in one specific field, and the second storage device 7 may store only the synonym dictionary in the specific field. In addition, even if one synonym dictionary is prepared in the second storage device 7 and synonymous expressions of compound nouns extracted from each of a plurality of document sets are collectively registered in the one synonym dictionary. Good. Further, the synonym dictionary creating system 5 is configured to include one storage device that stores a document set and a synonym dictionary instead of the first storage device 6 and the second storage device 7. Good.

[Fourth Embodiment]
FIG. 19 is a diagram showing the system configuration of the document rewriting system according to the fourth embodiment.

As shown in FIG. 19, the document rewriting system 10 according to the present embodiment includes a synonym expression extracting device 1, a first storage device 6, a second storage device 7, and a document data rewriting device 11. .. The synonymous expression extraction device 1 is connected to each of the first storage device 6 and the second storage device 7 by a transmission cable or the like, and data can be transmitted and received between the devices. Further, the document data rewriting device 11 is connected to the second storage device 7 by a transmission cable or the like, and data can be transmitted and received between the devices. Further, the synonym expression extracting device 1, the first storage device 6, the second storage device 7, and the document data rewriting device 11 are connected to a network 8 such as the Internet or a Local Area Network (LAN), and the network 8 The terminal device 9 (9A to 9C) is communicatively connected via the.

The synonym expression extraction device 1 according to the present exemplary embodiment extracts synonymous expressions for compound nouns included in a document set, as described in the first or second exemplary embodiment. The synonym expression extraction device 1 according to this exemplary embodiment includes a communication unit that performs communication (data transmission/reception) with each of the first storage device 6 and the second storage device 7.

The first storage device 6 is a device for storing a document set for extracting synonymous expressions of compound nouns. For example, in the first storage device 6, as shown in FIG. 18, a plurality of document data of a predetermined field including a first field document set 601 and a second field document set 602 are stored. Store a set of documents. In the plurality of document sets 601, 602, for example, document data collected by the user of the document rewriting system 10 using the terminal device 9 and transferred from the terminal device 9 to the first storage device 6 is accumulated.

The second storage device 7 is a device for storing a synonym dictionary including a pair of compound nouns that are synonymous expressions. For example, as shown in FIG. 18, the second storage device 7 stores a plurality of synonym dictionaries including a synonym dictionary 701 of the first field and a synonym dictionary 702 of the second field. The first-field synonym dictionary 701 is a dictionary in which synonymous expressions about compound nouns extracted from the first-field document set 601 are registered. The synonym dictionary 702 of the second field is a dictionary in which synonymous expressions about compound nouns extracted from the document set 701 of the second field are registered. In the plurality of synonym dictionaries 701 and 702, for example, pairs of compound nouns that are synonymous expressions extracted from the document sets 601 and 602 by the synonym expression extraction device 1 are registered. The user of the document rewriting system 10 uses the terminal device 9 to access the synonym dictionaries 701 and 702 of the second storage device 7 to check other expressions (synonymous expressions) about the compound noun in the document. Is possible. The synonym dictionary stored in the second storage device 7 may include not only synonyms for compound nouns but also synonyms for various words.

The document data rewriting device 11 is a device that refers to the synonym dictionaries 701 and 702 stored in the second storage device 7 and rewrites a compound noun included in the document data into a synonymous expression. The document data rewriting device 11 extracts, for example, a compound noun from the document data transmitted to the document data rewriting device 11 by the user of the document rewriting system 11 using the terminal device 9. Further, the document data rewriting device 11 searches the synonym dictionary of the second storage device 7 and acquires another expression (synonymous expression) of the compound noun extracted from the document data. When there is a synonymous expression for the compound noun extracted from the document data, the document data rewriting device 11 determines whether to rewrite the compound noun with another synonymous expression. For example, the document data rewriting device 11 determines whether to rewrite the compound noun based on the priority of the compound noun extracted from the document data and the synonymous expression acquired from the synonym dictionary. The priority of the synonymous expression is set based on, for example, the frequency of appearance in the document set. When the synonym expression acquired from the synonym dictionary has a higher priority, the document data rewriting device 11 rewrites the compound noun in the document data with the synonym expression acquired from the synonym dictionary. When the above processes are completed, the document rewriting data device 11 returns the document data to the terminal device 9.

FIG. 20 is a diagram showing a functional configuration of the document data rewriting device.
As shown in FIG. 20, the document data rewriting device 11 includes a document data acquisition unit 1110, a character string extraction unit 1120, a morpheme analysis unit 1130, a compound noun extraction unit 1140, a synonym expression search unit 1150, and a synonym expression. A rewriting unit 1160 and a document data replying unit 1170 are provided.

The document data acquisition unit 1110 receives the input of the document data transmitted from the terminal device 9 to the document data rewriting device 11. The document data reply unit 1170 returns the document data, which has been rewritten by the document data rewriting device 11, to the terminal device 9.

The character string extraction unit 1120 and the morpheme analysis unit 1130 have the same functions as the character string extraction unit 110 and the morpheme analysis unit 120 of the synonymous expression extraction device 1 according to the first embodiment, respectively. The character string extraction unit 1120 extracts a character string from the document data that is the target of the rewriting process. The morpheme analysis unit 1130 refers to the analysis dictionary 1190 and performs morpheme analysis on the extracted character string.

The compound noun extraction unit 1140 extracts a compound noun included in the character string of the document data based on the result of the morphological analysis.

The synonym expression search unit 1150 searches the synonym dictionary of the second storage device 7 and acquires the synonym expression for the compound noun extracted from the character string.

The synonym expression rewriting unit 1160 rewrites the compound noun included in the character string of the document data into the synonym expression acquired from the synonym dictionary. The synonym expression rewriting unit 1160 rewrites the compound noun to the synonym expression when the synonym expression obtained from the synonym dictionary has a higher priority than the compound noun included in the character string of the document data.

When a command for rewriting a compound noun in document data is input from the terminal device 9 or the like, the document data rewriting device 11 in the document rewriting system 10 according to the present exemplary embodiment acquires the document data designated by the command. 21 to perform the processing shown in FIG.

FIG. 21 is a flowchart illustrating the processing performed by the document data rewriting device.
The document data rewriting device 11 that has acquired the document data that is the target of the rewriting process first extracts a character string from the document data (step S11). The processing of step S11 is performed by the character string extraction unit 1120.

Next, the document data rewriting device 11 performs morphological analysis on the extracted character string (step S12). The process of step S12 is performed by the morpheme analysis unit 1130. The morpheme analysis unit 1130 refers to the analysis dictionary 1190 and divides the character string into morphemes according to a known analysis method.

Next, the document data rewriting device 11 extracts a compound noun from the character string based on the result of the morphological analysis (step S13). The process of step S13 is performed by the compound noun extraction unit 1140. The compound noun extraction unit 1140 extracts a compound noun from the character string according to the compound noun extraction condition. For example, the compound noun extraction condition includes a condition that a word string in which a plurality of noun words (morphemes) are consecutive is extracted as a compound noun.

Next, the document data rewriting device 11 performs a loop process (steps S14 to S19) of retrieving the synonym dictionary and rewriting the compound noun extracted from the document data into a synonymous expression. The loop process is performed by the synonym expression search unit 1150 and the synonym expression rewriting unit 1160. One process (steps S15 to S18) in the loop process is a process of retrieving and rewriting a synonymous expression for one type of compound noun extracted from the document data. At the loop end (steps S14 and S19) in the loop processing, the synonym expression search unit 1150 selects one type of compound nouns extracted from the document data and the synonyms for all compound nouns extracted from the document data. It is judged whether or not the expression is searched. When the synonym expression is searched for in all of the extracted one or more types of compound nouns and rewriting is performed as necessary, the synonym expression search unit 1150 ends the loop processing.

In one processing in the loop processing (steps S14 to S19), first, the synonym expression search unit 1150 searches the synonym dictionary of the second storage device 7 using the compound noun selected as the processing target as a keyword ( Step S15).

Next, the synonym expression search unit 1150 determines whether another synonym expression for the compound noun to be processed is registered in the synonym dictionary (step S16). When another synonym expression is not registered in the synonym dictionary (step S16; NO), the synonym expression search unit 1150 determines whether or not to end the loop process at the loop end (step S19) of the loop process. ..

When another synonym expression for the compound noun to be processed is registered in the synonym dictionary (step S16; YES), the synonym expression search unit 1150 next selects a synonym other than the compound noun extracted from the document data. It is determined whether the expression has a higher priority (step S17). When the compound noun extracted from the document data has a higher priority expression (step S17; NO), the synonym expression search unit 1150 determines whether or not to end the loop processing at the loop end (step S19) of the loop processing. To judge.

On the other hand, when another synonym expression has a higher priority (step S17; YES), the synonym expression retrieving unit 1150 then causes the synonym expression rewriting unit 1160 to add the compound noun in the document data to another synonym The expression is rewritten (step S18). After that, the synonym expression search unit 1150 determines whether or not to end the loop processing at the loop end of the loop processing (step S19).

When it is determined that the loop processing is to be ended at the loop end of the loop processing (step S19), the synonym expression search unit 1150 ends the loop processing. After completing the loop processing, the synonym expression search unit 1150 causes the synonym expression rewriting unit 1160, for example, to return the document data in which the compound noun is rewritten to another synonymous expression to the terminal device 9 (step S20).

When the document data is returned to the terminal device 9, the document data rewriting device 11 finishes the rewriting process for one piece of document data.

As described above, in the document rewriting system 10 according to the present exemplary embodiment, the synonym dictionary created by the method described in the first exemplary embodiment or the second exemplary embodiment is referred to based on the document set for each field, and the document data Rewrite the compound noun of to another synonymous expression. Further, in the document rewriting system 10, as described above, when another synonymous expression of the compound noun has a higher priority than the compound noun extracted from the document data, the compound noun of the document data is changed to another. Rewrite as a synonym for. Therefore, according to the present embodiment, it is possible to unify the compound nouns of the synonymous expressions existing in the document data into one notation.

The flowchart of FIG. 21 is only an example of the process performed by the document data rewriting device 11 according to the present exemplary embodiment. The processing performed by the document data rewriting device 11 according to the present embodiment is not limited to the processing shown in FIG. 21, and the processing content can be appropriately changed without departing from the scope of the present embodiment.

Further, the system configuration of FIG. 19 is merely an example of the system configuration of the document rewriting system 10 according to the present exemplary embodiment. The document rewriting system 10 according to the present embodiment is not limited to the configuration shown in FIG. 19, and for example, the synonym expression extracting device 1 and the document data rewriting device 11 may be integrated. Further, the document set and the synonym dictionary may be stored in one storage device.

In addition, the synonymous expression extraction device 1 according to each of the above embodiments can be realized by a computer and a program executed by the computer. Hereinafter, the synonym expression extraction device 1 realized by a computer and a program will be described with reference to FIG.

FIG. 22 is a diagram showing a hardware configuration of a computer.
As shown in FIG. 22, the computer 15 includes a processor 1501, a main storage device 1502, an auxiliary storage device 1503, an input device 1504, an output device 1505, an input/output interface 1506, a communication control device 1507, and a medium. And a driving device 1508. These elements 1501 to 1508 in the computer 15 are connected to each other by a bus 1510, and data can be transferred between the elements.

The processor 1501 is a Central Processing Unit (CPU), a Micro Processing Unit (MPU), or the like. The processor 1501 controls the overall operation of the computer 15 by executing various programs including an operating system. Further, the processor 1501 executes, for example, a synonym expression extraction program including a process of extracting a pair of compound nouns of synonymous expressions shown in FIGS. 4 to 8.

The main storage device 1502 includes a Read Only Memory (ROM) and a Random Access Memory (RAM) which are not shown. In the ROM of the main storage device 1502, for example, a predetermined basic control program read by the processor 1501 when the computer 15 is started is recorded in advance. On the other hand, the RAM of the main storage device 1502 is used by the processor 1501 as a work storage area as needed when executing various programs. The RAM of the main storage device 1502 can be used to store, for example, a character string extracted from the document set 191, an analysis result corpus 193, a list of undetermined word pairs and synonymous word pairs, a similarity transition table 195, and the like. The RAM of the main storage device 1502 can be used to store, for example, the analysis dictionary 192, the synonym word dictionary 194, the synonym expression list 196, and the like.

The auxiliary storage device 1503 is a storage device having a larger capacity than the RAM of the main storage device 1502, and includes, for example, a hard disk drive (HDD) and a nonvolatile memory (Solid State Drive (SSD)) such as a flash memory. ) Etc. The auxiliary storage device 1503 can be used to store various programs executed by the processor 1501 and various data. The auxiliary storage device 1503 can be used, for example, for storing a synonym expression extraction program including a process of extracting a pair of synonymous compound nouns shown in FIGS. 4 to 8. The auxiliary storage device 1503 can be used to store, for example, a character string extracted from the document set 191, an analysis result corpus 193, a list of undetermined word pairs and synonymous word pairs, a similarity transition table 195, and the like. Further, the auxiliary storage device 1503 can be used to store, for example, the analysis dictionary 192, the synonym word dictionary 194, the synonym expression list 196, and the like.

The input device 1504 is, for example, a keyboard device, a touch panel device, or the like. When an operator (user) of the computer 15 performs a predetermined operation on the input device 1504, the input device 1504 transmits the input information associated with the operation content to the processor 1501. The input device 1504 can be used, for example, to input a command to start a process of extracting a synonymous expression for a compound noun, a command related to another process that can be executed by the computer 15, and various setting values.

The output device 1505 is, for example, a display device such as a liquid crystal display device or a printing device such as a printer. The output device 1505 can be used for outputting the result of the synonym expression extraction process for the compound noun and the content of the synonym expression list.

The input/output interface 1506 connects the computer 15 to another electronic device. The input/output interface 1506 includes, for example, a Universal Serial Bus (USB) standard connector and the like. The input/output interface 1506 can be used to connect the computer 15 to the storage devices 6 and 7, for example.

The communication control device 1507 is a device that connects the computer 15 to a network such as the Internet and controls various types of communication between the computer 15 and other communication devices via the network. The communication control device 1507 can be used for communication between the computer 15 and the terminal device 9, for example.

The medium driving device 1508 reads programs and data recorded in the portable storage medium 16 and writes data and the like stored in the auxiliary storage device 1503 to the portable storage medium 16. For the medium driving device 1508, for example, a memory card reader/writer compatible with one or a plurality of types of standards can be used. When a memory card reader/writer is used as the medium driving device 1508, the portable storage medium 16 is a memory card (flash memory) of a standard supported by the memory card reader/writer, for example, Secure Digital (SD) standard. ) Etc. can be used. As the portable recording medium 16, for example, a flash memory having a USB standard connector can be used. Further, when the computer 15 is equipped with an optical disk drive that can be used as the medium driving device 1508, various optical disks that can be recognized by the optical disk drive can be used as the portable recording medium 16. Optical discs that can be used as the portable recording medium 16 include, for example, Compact Disc (CD), Digital Versatile Disc (DVD), and Blu-ray Disc (Blu-ray is a registered trademark). The portable recording medium 16 can be used, for example, for storing a synonym expression extraction program including a process of extracting a pair of compound nouns of synonymous expressions shown in FIGS. 4 to 8. Further, the portable recording medium 16 can be used to store, for example, a character string extracted from the document set 191, an analysis result corpus 193, a list of undetermined word pairs and synonymous word pairs, a similarity transition table 195, and the like. Further, the portable recording medium 16 can be used to store, for example, the analysis dictionary 192, the synonym word dictionary 194, the synonym expression list 196, and the like.

For example, when the operator uses the input device 1504 or the like to input a command to start the extraction processing of the synonym expression to the computer 15, the processor 1501 stores the synonym expression stored in a non-temporary recording medium such as the auxiliary storage device 1503. Read and execute the extraction program. In this process, the processor 1501 functions (operates) as the character string extraction unit 110, the morpheme analysis unit 120, the compound noun extraction unit 130, and the synonymous compound noun identification unit 140 in the synonymous expression extraction device 1. Further, while the processor 1501 is executing the synonym expression extraction program, the RAM of the main storage device 1502, the auxiliary storage device 1503, and the like function as a storage unit (not shown) of the synonym expression extraction device 1. That is, the RAM of the main storage device 1502, the auxiliary storage device 1503, and the like store the document set 191, the analysis dictionary 192, the analysis result corpus 193, the synonym word dictionary 194, the similarity transition table 195, the synonym expression list 196, and the like. Functions as a storage unit.

The computer 15 that operates as the synonymous expression extraction device 1 does not need to include all the elements 1501 to 1508 shown in FIG. 22, and some elements may be omitted depending on the application and conditions. For example, the computer 15 may be one in which the communication control device 1507 and the medium driving device 1508 are omitted.

A storage device such as the auxiliary storage device 1503 in the computer 15 can also be used as, for example, the first storage device 6 and the second storage device 7 in the synonym dictionary creating system shown in the third embodiment. Is.

Further, the computer 15 can be operated not only as the synonym expression extraction device 1 but also as the document data rewriting device 11 of the document rewriting system 10 shown in the fourth exemplary embodiment. Further, in the document rewriting system 10, it is possible to operate one computer 15 as the synonymous expression extracting device 1 and also as the document data rewriting device 11.

The following supplementary notes will be further disclosed regarding each of the embodiments described above.
(Appendix 1)
Divide the compound noun pair extracted from the document data into a plurality of word pairs, and refer to the synonymous word dictionary in which synonymous word pairs are registered, and refer to the synonymous word pairs as synonymous word pairs. A word pair setting unit for identifying an undetermined word pair whose presence or absence has not been confirmed,
For each of the plurality of synonymous word pairs in the document data, including the synonymous word pair in the compound noun pair and the undetermined word pair, a process of learning the semantic similarity between words is performed a plurality of times. A semantic similarity learning unit,
Based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity in each of the plurality of synonymous word pairs, whether the words of the undetermined word pair are synonymous or not. A word synonym determination unit for determination,
When the plurality of word pairs in the pair of compound nouns are all synonymous word pairs, a compound noun synonym determination unit that determines that the pair of compound nouns are synonymous expressions, and
The semantic similarity learning unit adds a case to be used for the learning process every time the semantic similarity learning process is performed on each of a plurality of word pairs to be processed,
The word synonym determination unit, the relationship between the number of learning processing and the semantic similarity in the learning result of the semantic similarity of the undetermined word pair, and the number of learning processing in the learning result of the semantic similarity of the synonymous word pair and Calculate a correlation coefficient for the relationship with the semantic similarity, based on the correlation coefficient, to determine whether the words of the undetermined word pair are synonymous,
A synonymous expression extraction device characterized by the above.
(Appendix 2)
The word synonym determination unit,
In each of the plurality of synonymous word pairs, the relationship between the number of learning processes for the undetermined word pair and the semantic similarity, the number of learning processes for the synonymous word pair and the semantic similarity Calculate the correlation coefficient for and
When the average value of the plurality of calculated correlation coefficients is equal to or greater than a threshold value, it is determined that the words of the undetermined word pair are synonymous with each other,
The synonymous expression extracting device according to appendix 1, characterized in that.
(Appendix 3)
The synonymous expression extraction device, based on the relationship between the number of learning processing and the semantic similarity in the learning result of the semantic similarity in each of the plurality of synonymous word pairs, the words of the undetermined word pair are synonymous Further comprising a determination threshold setting unit for setting a determination threshold of whether or not,
The word synonym determination unit determines that the words of the undetermined word pair are synonymous when the average value of the calculated correlation coefficient is equal to or more than the determination threshold set by the determination threshold setting unit. ,
The synonymous expression extraction device as described in appendix 2, wherein.
(Appendix 4)
The synonym expression extraction device further includes a storage unit that stores a synonym expression list in which the pair of compound nouns that are synonymous expressions are registered.
The synonymous expression extracting device according to appendix 1, characterized in that.
(Appendix 5)
The synonym expression extraction device further includes a synonym expression rewriting unit that rewrites a compound noun extracted from document data into a synonym expression, based on the synonym expression of the compound noun registered in the synonym expression list,
The synonymous expression extracting device according to appendix 1, characterized in that.
(Appendix 6)
Computer
Divide the compound noun pair extracted from the document data into a plurality of word pairs, and refer to the synonymous word dictionary in which synonymous word pairs are registered, and refer to the synonymous word pairs as synonymous word pairs. Identified as an undetermined word pair with undetermined whether
From the document data, collect the synonym word pairs registered in the synonym word dictionary,
The process of learning the semantic similarity between words is executed a plurality of times for each of the plurality of synonymous word pairs in the document data including the synonymous word pair in the compound noun pair and the undetermined word pair. Then
Based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity in each of the plurality of synonymous word pairs, whether the words of the undetermined word pair are synonymous or not. Judge,
When the plurality of word pairs in the pair of compound nouns are all synonymous word pairs, it is determined that the pair of compound nouns are synonymous expressions, a process is executed,
In the process of learning the semantic similarity, the computer adds a case to be used for the learning process every time the learning process of the semantic similarity is performed for each of a plurality of word pairs to be processed,
In the process of determining whether the words of the undetermined word pair are synonymous with each other, the computer has a relationship between the number of learning processes in the learning result of the semantic similarity of the undetermined word pair and the semantic similarity. , Calculating a correlation coefficient for the relationship between the number of learning processes and the semantic similarity in the learning result of the semantic similarity of the synonymous word pair, based on the correlation coefficient, the words of the undetermined word pair Determines whether is synonymous,
A synonym expression extraction method characterized by the above.
(Appendix 7)
In the process of determining whether the words of the undetermined word pair are synonymous, the computer,
In each of the plurality of synonymous word pairs, the relationship between the number of learning processes for the undetermined word pair and the semantic similarity, the number of learning processes for the synonymous word pair and the semantic similarity Calculate the correlation coefficient for and
When the average value of the plurality of calculated correlation coefficients is equal to or greater than a threshold value, it is determined that the words of the undetermined word pair are synonymous with each other,
The synonymous expression extraction method according to appendix 6, characterized in that
(Appendix 8)
Before performing the process of determining whether the words of the undetermined word pair are synonymous with each other, the computer, the number and the meaning of the learning process in the learning result of the semantic similarity in each of the plurality of synonymous word pairs Based on the relationship with the degree of similarity, further comprising a process of setting a determination threshold whether or not the words of the undetermined word pair are synonymous,
In the process of determining whether the words of the undetermined word pair are synonymous with each other, the computer, when the average value of the correlation coefficient is greater than or equal to the determination threshold, the words of the undetermined word pair Is determined to be synonymous,
The synonymous expression extraction method according to appendix 7, characterized in that
(Appendix 9)
Divide the compound noun pair extracted from the document data into a plurality of word pairs, and refer to the synonymous word dictionary in which synonymous word pairs are registered, and refer to the synonymous word pairs as synonymous word pairs. Identified as an undetermined word pair with undetermined whether
From the document data, collect the synonym word pairs registered in the synonym word dictionary,
The process of learning the semantic similarity between words is executed a plurality of times for each of the plurality of synonymous word pairs in the document data including the synonymous word pair in the compound noun pair and the undetermined word pair. Then
Based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity in each of the plurality of synonymous word pairs, whether the words of the undetermined word pair are synonymous or not. Judge,
A synonym expression extraction program that causes a computer to execute a process of determining that the pair of compound nouns are synonymous expressions when all of the plurality of word pairs in the pair of compound nouns are the synonymous word pairs,
The process of learning the semantic similarity includes a process of adding a case used for the learning process every time the learning process of the semantic similarity is performed on each of a plurality of word pairs to be processed,
The process of determining whether or not the words of the undetermined word pair are synonymous with each other, the relationship between the number of learning processes and the semantic similarity in the learning result of the semantic similarity of the undetermined word pair, and the synonymous word The correlation coefficient is calculated for the relationship between the number of learning processes and the semantic similarity in the learning result of the semantic similarity of the pair, and the words of the undetermined word pair are synonymous based on the correlation coefficient. Including the process of determining whether or not
A synonym expression extraction program characterized by the following.
(Appendix 10)
The process of determining whether the words of the undetermined word pair are synonymous,
In each of the plurality of synonymous word pairs, the relationship between the number of learning processes for the undetermined word pair and the semantic similarity, the number of learning processes for the synonymous word pair and the semantic similarity Calculate the correlation coefficient for and
When the average value of the calculated plurality of correlation coefficients is equal to or more than a threshold value, it is determined that the words of the undetermined word pair are synonymous with each other, including processing.
The synonymous expression extraction program described in appendix 9.
(Appendix 11)
The number of learning processes and the semantic similarity in the learning result of the semantic similarity in each of the plurality of synonymous word pairs, which is executed before the process of determining whether or not the words of the undetermined word pair are synonymous, Based on the relationship of, further includes a process of setting a determination threshold of whether the words of the undetermined word pair are synonymous,
The process of determining whether or not the words of the undetermined word pair are synonymous with each other, when the average value of the correlation coefficient is greater than or equal to the determination threshold, the words of the undetermined word pair are synonymous. Judge,
The synonymous expression extraction program according to appendix 10, characterized in that.

1 Synonym expression extraction device 2, 3 Compound nouns 401-403, 405, 406 Table 404 Word pair list 5 Synonym dictionary creation system 6, 7 Storage device 8 Network 9 (9A-9C) Terminal device 10 Document rewriting system 11 Document data Rewriting device 15 Computer 16 Portable recording medium 110, 1120 Character string extraction unit 120, 1130 Morphological analysis unit 130, 1140 Compound noun extraction unit 140 Synonymous compound noun identification unit 141 Word pair setting unit 142 Similarity transition table creation unit 143 Semantic similarity Degree learning unit 144 Word synonym determination unit 145 Compound noun synonym determination unit 146 Determination threshold setting unit 191, 601, 602 Document set 192, 1190 Analysis dictionary 103 Analysis result corpus 194 Synonym word dictionary 195 Similarity transition table 196 Synonym expression list 701 , 702 Synonym dictionary 1110 Document data acquisition unit 1150 Synonym expression search unit 1160 Synonym expression rewriting unit 1170 Document data replying unit 1501 Processor 1502 Main storage device 1503 Auxiliary storage device 1504 Input device 1505 Output device 1506 Input/output interface 1507 Communication control device 1508 Auxiliary storage

Claims (6)

Divide the compound noun pair extracted from the document data into a plurality of word pairs, and refer to the synonymous word dictionary in which synonymous word pairs are registered, and refer to the synonymous word pairs as synonymous word pairs. A word pair setting unit for identifying an undetermined word pair whose presence or absence has not been confirmed,
For each of the plurality of synonymous word pairs in the document data, including the synonymous word pair in the compound noun pair and the undetermined word pair, a process of learning the semantic similarity between words is performed a plurality of times. A semantic similarity learning unit,
Based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity in each of the plurality of synonymous word pairs, whether the words of the undetermined word pair are synonymous or not. A word synonym determination unit for determination,
When the plurality of word pairs in the pair of compound nouns are all synonymous word pairs, a compound noun synonym determination unit that determines that the pair of compound nouns are synonymous expressions, and
The semantic similarity learning unit adds a case to be used for the learning process every time the semantic similarity learning process is performed on each of a plurality of word pairs to be processed,
The word synonym determination unit, the relationship between the number of learning processing and the semantic similarity in the learning result of the semantic similarity of the undetermined word pair, and the number of learning processing in the learning result of the semantic similarity of the synonymous word pair and Calculate a correlation coefficient for the relationship with the semantic similarity, based on the correlation coefficient, to determine whether the words of the undetermined word pair are synonymous,
A synonymous expression extraction device characterized by the above. The word synonym determination unit,
In each of the plurality of synonymous word pairs, the relationship between the number of learning processes for the undetermined word pair and the semantic similarity, the number of learning processes for the synonymous word pair and the semantic similarity Calculate the correlation coefficient for and
When the average value of the plurality of calculated correlation coefficients is equal to or more than a threshold value, it is determined that the words of the undetermined word pair are synonymous with each other,
The synonymous expression extracting device according to claim 1, wherein The synonymous expression extraction device, based on the relationship between the number of learning process and the semantic similarity in the learning result of the semantic similarity in each of the plurality of synonymous word pairs, the words of the undetermined word pair are synonymous Further comprising a determination threshold setting unit for setting a determination threshold of whether or not,
The word synonym determination unit determines that the words of the undetermined word pair are synonymous when the average value of the calculated correlation coefficient is equal to or more than the determination threshold set by the determination threshold setting unit. ,
The synonymous expression extracting device according to claim 2, wherein. The synonym expression extraction device further includes a storage unit that stores a synonym expression list in which the pair of compound nouns that are synonymous expressions are registered.
The synonymous expression extracting device according to claim 1, wherein Computer
Divide the compound noun pair extracted from the document data into a plurality of word pairs, and refer to the synonymous word dictionary in which synonymous word pairs are registered, and refer to the synonymous word pairs as synonymous word pairs. Identified as an undetermined word pair with undetermined whether
From the document data, collect the synonym word pairs registered in the synonym word dictionary,
The process of learning the semantic similarity between words is executed a plurality of times for each of the plurality of synonymous word pairs in the document data including the synonymous word pair in the compound noun pair and the undetermined word pair. Then
Based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity in each of the plurality of synonymous word pairs, whether the words of the undetermined word pair are synonymous or not. Judge,
When the plurality of word pairs in the pair of compound nouns are all synonymous word pairs, it is determined that the pair of compound nouns are synonymous expressions, a process is executed,
In the process of learning the semantic similarity, the computer adds a case to be used for the learning process every time the learning process of the semantic similarity is performed for each of a plurality of word pairs to be processed,
In the process of determining whether the words of the undetermined word pair are synonymous with each other, the computer has a relationship between the number of learning processes in the learning result of the semantic similarity of the undetermined word pair and the semantic similarity. , Calculating a correlation coefficient for the relationship between the number of learning processes and the semantic similarity in the learning result of the semantic similarity of the synonymous word pair, based on the correlation coefficient, the words of the undetermined word pair Determines whether is synonymous,
A synonym expression extraction method characterized by the above. Divide the compound noun pairs extracted from the document data into a plurality of word pairs, and refer to the synonymous word dictionary in which synonymous word pairs are registered, and refer to the synonymous word pairs as synonymous word pairs. Identified as an undetermined word pair with undetermined whether
From the document data, collect the synonym word pairs registered in the synonym word dictionary,
The process of learning the semantic similarity between words is executed a plurality of times for each of the plurality of synonymous word pairs in the document data including the synonymous word pair in the compound noun pair and the undetermined word pair. Then
Based on the learning result of the semantic similarity of the undetermined word pair and the learning result of the semantic similarity in each of the plurality of synonymous word pairs, whether the words of the undetermined word pair are synonymous or not. Judge,
A synonym expression extraction program that causes a computer to execute a process of determining that the pair of compound nouns are synonymous expressions when all of the plurality of word pairs in the pair of compound nouns are the synonymous word pairs,
The process of learning the semantic similarity includes a process of adding a case used for the learning process every time the learning process of the semantic similarity is performed on each of a plurality of word pairs to be processed,
The process of determining whether or not the words of the undetermined word pair are synonymous with each other, the relationship between the number of learning processes and the semantic similarity in the learning result of the semantic similarity of the undetermined word pair, and the synonymous word The correlation coefficient is calculated for the relationship between the number of learning processes and the semantic similarity in the learning result of the semantic similarity of the pair, and the words of the undetermined word pair are synonymous based on the correlation coefficient. Including the process of determining whether or not
A synonym expression extraction program characterized by the following.

Images