JP5448744B2 - Sentence correction program, method, and sentence analysis server for correcting sentences containing unknown words - Google Patents

Description

  The present invention relates to a sentence correction program, method, and sentence analysis server for correcting sentences including unknown words.

  With the spread of the Internet, various texts are described on blogs, bulletin boards, or Web sites that publish reviews. A “blog” is a site that is generally run by an individual and can be updated in a diary to express their opinions on current news and specialized topics. A “bulletin board” is a site for sequentially exchanging discussions with other people on various themes. Furthermore, “review comments” refers to a site where comments about things such as people's rumors can be described. These sites are often described in colloquial “quick expressions” because of their friendliness.

  In recent years, Web documents published by such sites have also been targeted for information extraction, search, and statistical processing. In order to analyze the text content of these Web documents, at least morphological analysis is required. The morphological analysis program divides a sentence to be analyzed into morphemes. English sentences are divided into words such as "This is a pen." On the other hand, Japanese sentences are not separately written, and therefore processing by morphological analysis is required prior to syntax analysis or the like.

  A “morpheme” refers to the smallest meaningful unit among the constituent elements of a sentence. The morphological analysis program has a dictionary in which “part of speech” and “reading” are registered for each “word”. The divided morphemes are given “part of speech” and “reading” information using a dictionary, and the sentence is analyzed using grammatical rules that define the word arrangement. Thus, the analysis accuracy of the morphological analysis program depends on the dictionary. Therefore, a sentence including words that are not registered in the dictionary cannot obtain sufficient analysis accuracy.

On the other hand, a Web document published by a blog, a bulletin board, or a word-of-mouth comment generally includes many “descriptive expressions” as follows.
(1) Notation combined with the changing tendency of pronunciation in conversations such as “Usoso” and “Sugoi” (2) Notation that intentionally katakana is used for words that are originally written in hiragana, such as “cool” (3) Peculiar notation such as “Kai-yi” (read as “cute”), “Kita U” (read as “I am”)

  Since these expressions are generally not registered in the dictionary of the morphological analysis program, they are processed as “unknown words”. A word that is not registered in the dictionary is divided into units corresponding to morphemes, and only information “unknown word” is given instead of “part of speech” and “reading” information.

  Of course, if all the words processed as “unknown words” can be manually registered in the dictionary, the accuracy of morphological analysis can be improved. However, the registration of “unknown words” requires specialized human skills, such as registration of parts of speech and usage forms, and maintenance of compatibility with existing dictionaries.

  On the other hand, there is a technique for correcting a sentence including an unknown word into a sentence suitable for morphological analysis (for example, see Non-Patent Document 1). According to this technique, a morphological analysis is performed on an analysis target sentence that is a sentence including an unknown word, on the assumption that the analysis fails and is output as an “unknown word”. Based on the result of the morphological analysis, a search key is generated by replacing the unknown word part with an arbitrary character string. A correction candidate character string is automatically searched from the correction reference text by using the generated search key, and one or more correction rules are generated from the correction candidate character string. The generated correction rules are either (1) an expression that is frequently used in the same context, (2) whether it has changed too much from the expression of the sentence to be analyzed, or (3) the corrected sentence is in Japanese Scoring is performed using an index such as whether natural or not, and an optimal correction rule is selected. The analysis target sentence is corrected to a sentence suitable for morphological analysis according to the selected correction rule.

Kazufumi Ikeda, Tadashi Yanagihara, Kazunori Matsumoto, Yasuhiro Takishima, “Proposal and Evaluation of Unsupervised Learning Method for Correcting Expressions”, 8th Information Science and Technology Forum, 2009

  According to the technique described in Non-Patent Document 1, the correction rule is generated based on the correction candidate character string searched from the correction reference sentence, and thus the analysis target sentence is corrected depending on the correction reference sentence. . As a result, even if the analysis target sentence is corrected to a sentence that is correctly analyzed for morpheme, it may be corrected to a sentence having a different meaning or may be overcorrected.

  Further, according to the technique described in Non-Patent Document 1, a sentence with high analysis accuracy of morphological analysis such as a sentence of a newspaper article is used as the correction reference sentence. In general, the sentences of newspaper articles often have “standard expressions”. Since the “standard expression” has a high probability of being registered in the dictionary of the morphological analysis program, the sentences of newspaper articles have high morphological analysis accuracy.

  For example, the expression “e-yu is cool” in the sentence to be analyzed is corrected as “the hero is cool” when the sentence of the newspaper article is the correction standard sentence. If the analysis target sentence is a sentence related to IT or a computer, it is desirable that this expression is corrected as “au is cool”.

  Further, for example, the expression “no such relationship” in the sentence to be analyzed is corrected as “no such relationship”. However, if the sentence to be analyzed is a sentence reflecting a trend several years ago, it is desirable that this expression is not corrected as it is.

  Therefore, according to the present invention, even sentence information described in colloquial “complex expression” can be corrected to a sentence suitable for sentence analysis that is not determined as an unknown word by morphological analysis. An object is to provide a sentence correction program, method, and sentence analysis server.

According to the present invention, for a sentence information to be analyzed including an unknown word, a sentence correction program that causes a computer to function to correct the unknown word,
A reference sentence storage means for storing a plurality of corrected reference sentence information transmitted to an unspecified number of third parties , classified into a plurality of categories, and stored;
An unknown word extraction means for dividing the sentence information to be analyzed into morphemes and extracting unknown words;
Search key generating means for generating a search key comprising a wild card corresponding to an unknown word and at least one of the adjacent morphemes in front and rear of the unknown word;
Correction rule search means for searching for one or more correction rule candidates searched with a wild card using a search key for each category of correction reference sentence information;
Correction rule candidate selection means for selecting one correction rule candidate based on a sentence composition index from correction rule candidates for each category of correction reference sentence information ;
Category selection means for selecting a category having the highest score based on the sentence composition index of the correction rule for the correction rule candidate selected for each category of the correction reference sentence information;
A correction rule selection means for selecting a correction rule based on the selected category;
Causing the computer to function as a correction rule applying means for correcting the sentence information to be analyzed by applying the selected correction rule ;
The sentence composition index includes (1) the higher the appearance frequency of the character string corresponding to the correction rule candidate, (2) the shorter the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, and / or (3) The index is a higher score as the difference between the morphological analysis cost values before and after the correction is smaller.
The computer is made to function as described above .

According to another embodiment of the sentence correction program of the present invention, the reference sentence storage means is configured such that the plurality of categories for classifying the plurality of corrected reference sentence information are based on the feature words and / or the date / time information. It is also preferable that there is.

  According to another embodiment of the sentence correction program of the present invention, the sentence information to be analyzed is described by an unspecified number of users in a blog (Weblog), bulletin board, and / or word-of-mouth comment published via a network. It is also preferable that it is text information.

According to the present invention, a sentence correction method for correcting unknown words using a computer for analysis target sentence information including unknown words,
A reference sentence storage unit that stores a plurality of corrected reference sentence information transmitted to an unspecified number of third parties , classified into a plurality of categories, and stored.
An unknown word extraction means for dividing the sentence information to be analyzed into morphemes and extracting unknown words;
A second step of generating a search key consisting of a wild card corresponding to the unknown word and at least one of the front and rear adjacent morphemes for the unknown word;
A third step of searching for one or more correction rule candidates searched with a wild card using a search key for each category of correction reference sentence information;
A fourth step of selecting one correction rule candidate based on a sentence composition index from among correction rule candidates for each category of correction reference sentence information;
A fifth step of selecting a category having the highest score based on the sentence composition index of the correction rule for the correction rule candidate selected for each category of the correction reference sentence information;
A sixth step of selecting a correction rule based on the selected category;
The analyzed sentence information, have a <br/> a seventh step of modifying by applying the selected modified rules,
The sentence composition index includes (1) the higher the appearance frequency of the character string corresponding to the correction rule candidate, (2) the shorter the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, and / or (3) The index is a higher score as the difference between the morphological analysis cost values before and after the correction is smaller .

According to the present invention, a sentence analysis server that acquires analysis target sentence information including an unknown word from another public server via a network, and corrects the unknown word,
A reference sentence storage means for storing a plurality of corrected reference sentence information transmitted to an unspecified number of third parties , classified into a plurality of categories, and stored;
An unknown word extraction means for dividing the sentence information to be analyzed into morphemes and extracting unknown words;
Search key generating means for generating a search key comprising a wild card corresponding to an unknown word and at least one of the adjacent morphemes in front and rear of the unknown word;
Correction rule search means for searching for one or more correction rule candidates searched with a wild card using a search key for each category of correction reference sentence information;
Correction rule candidate selection means for selecting one correction rule candidate based on a sentence composition index from correction rule candidates for each category of correction reference sentence information ;
Category selection means for selecting a category having the highest score based on the sentence composition index of the correction rule for the correction rule candidate selected for each category of the correction reference sentence information;
A correction rule selection means for selecting a correction rule based on the selected category;
Causing the computer to function as a correction rule applying means for correcting the sentence information to be analyzed by applying the selected correction rule ;
The sentence composition index includes (1) the higher the appearance frequency of the character string corresponding to the correction rule candidate, (2) the shorter the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, and / or (3) The index is a higher score as the difference between the morphological analysis cost values before and after the correction is smaller .

  According to the sentence correction program, method, and sentence analysis server of the present invention, even sentence information described in colloquial “kudaku expression” is not judged as an unknown word by morphological analysis. It can be corrected to a sentence suitable for.

It is a functional block diagram of the text correction program in this invention. It is explanatory drawing of selection of the category and correction rule in this invention. It is a system configuration | structure figure of the text analysis server in this invention. It is a sequence diagram of a system in the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a functional configuration diagram of a sentence correction program according to the present invention.

  According to FIG. 1, the sentence correction program 1 includes a reference sentence storage unit 11, an unknown word extraction unit 12, a search key generation unit 13, a correction rule search unit 14, a correction rule candidate selection unit 15, and a category selection. Unit 16, correction rule selection unit 17, and correction rule application unit 18. These functional units other than the reference text storage unit 11 can be realized by executing a program that causes a computer installed in the apparatus to function. The processing flow of each functional unit can be executed as a text correction method using a computer.

  The reference sentence storage unit 11 stores a set (category) of corrected reference sentence information. The corrected reference text includes texts in various fields such as technical documents, blog texts, magazine articles, and newspaper articles. The reference sentence storage unit 11 may classify the corrected reference sentence information according to the sentence content category or feature word, or may be classified according to the sentence content date and time (or creation date and time). According to the present invention, all categories (a set of corrected reference sentence information) stored in the reference sentence storage unit 11 are compared with the correction target sentence information in a brute force manner. The reference text storage unit 11 is searched by the correction rule search unit 14.

The unknown word extraction unit 12 inputs analysis target sentence information. The analysis target sentence information may be sentence information described by an unspecified number of users such as a blog, a bulletin board, or a word-of-mouth comment published on a website. The unknown word extraction unit 12 divides the analysis target sentence information into morphemes by morphological analysis. Here, an analysis target sentence including a complicated expression will be described as an example.
Text to be analyzed: E-yu is cool morphological analysis results: E-yu (unknown word) / ha / cool expressions are often not registered in the morphological analysis dictionary. Therefore, the expression “e-yu” that is not registered in the morphological analysis dictionary is processed as an unknown word.

  When the unknown word extraction unit 12 detects an unknown word from the analysis target sentence, the unknown word extraction unit 12 outputs a character string obtained by combining the unknown word extracted by the morphological analysis and the morpheme adjacent to the unknown word to the search key generation unit 13. To do. Further, the unknown word extraction unit 12 outputs the analysis target sentence information in which the unknown word is detected to the correction rule candidate selection unit 15 and the correction rule application unit 18.

The search key generation unit 13 generates a “search key” composed of an unknown word and at least one of the adjacent morphemes ahead and behind the unknown word. Here, it is assumed that the unknown word and each of the front and rear morphemes adjacent to the unknown word are used for generating the search key.
Input string: e-yu (unknown word) / ha / cool search key: * is cool (where "*" indicates one or more arbitrary character strings)

  The search key generation unit 13 sets an unknown word as an arbitrary character string (for example, a wild card), and generates a “search key” that combines the arbitrary character string and a character string adjacent to the unknown word. Of course, the unknown word may be two or more consecutive words. Moreover, the adjacent morpheme should just have at least one of the front and back with respect to an unknown word. Similarly, two or more adjacent morphemes may be continuous. The search key generation unit 13 outputs the generated search key to the correction rule search unit 14.

  The correction rule search unit 14 extracts a correction candidate character string including a “search key” for each category stored in the reference text storage unit 11. Then, the correction rule search unit 14 determines that the portion corresponding to the arbitrary character string in the extracted correction candidate character string is a portion that approximates an unknown word, and extracts it as a correction rule candidate. The correction rule is a rule for converting a character string from an unknown word (for example, “Kanyi”) into a correction candidate character string (for example, “cute”).

The correction rule search unit 14 obtains the following correction candidate character strings, for example. The correction rule search unit 14 extracts one or more correction rule candidates for each category from the portion corresponding to the arbitrary character string in the extracted correction candidate character string.
Search key: “* is cool”
Category: 1
Candidates for correction: Hero / Ha / Cool Good Candidates for correction: E-yu ⇒ Hero Category: 2
Correction candidate character string: au / ha / cool Good correction rule candidate: e-yu ⇒au
Category: 2
Correction candidate string: Aiyu / Ha / Cool Correction rule candidate: E-yu ⇒ Aiyu

  Two or more correction rule candidates may be obtained by a search using a search key. The correction rule search unit 14 outputs all the correction rules obtained by the search using the search key to the correction rule candidate selection unit 15.

  If there are two or more correction rule candidates extracted based on the same category, the correction rule candidate selection unit 15 selects one correction rule candidate suitable for the context based on the sentence composition index, one for each category.

  The sentence composition index includes (1) the appearance frequency in the correction rule candidate, (2) the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, and / or (3) before and after the correction. An index calculated from the difference between morphological analysis cost values. The correction rule candidate selection unit 15 selects one correction rule candidate based on the sentence composition index.

For example, it is assumed that the correction rule candidate selection unit 15 inputs the following correction rule candidates.
Correction rule candidate: Category 2 e-yu ⇒ au
Category 2 E-yu ⇒ Aiyu

(1) frequency of occurrence frequency modification rule candidate in the modified rule candidate refers to how often Rubun string to correspond to the found modified rule candidate has emerged. The following table represents an example of a scoring based on the appearance frequency of the string.

  A character string having a high appearance frequency is considered to be an expression that is frequently used in a context similar to the context in which the unknown word appears, and is likely to be a correction candidate character string. On the other hand, an expression that is not frequently used in a similar context is highly likely not to be a correction candidate character string. Therefore, a correction rule candidate with a high appearance frequency has a high score. The score may be independent of the number of searches by dividing the appearance frequency by the number of searches and normalizing the score.

(2) Edit distance between an unknown word and a modified morpheme based on a modification rule candidate An edit distance is an index indicating how different two character strings are, and one character string is replaced with the other character string. Is given as the minimum number of insertions, deletions and substitutions required to convert to. In many cases, a modified morpheme based on a modification rule is obtained by inserting, deleting, or replacing a small number of characters in an unknown word. For example, in the editing from “Forum” to “Farm”, the method of replacing “o” with “a” and deleting “La” is the minimum number of editing times, so the editing distance is 2 It is. The following table shows an example of scoring based on edit distance.

  A correction rule candidate with a small editing distance has a high score. In many Web documents, words that should be originally written in hiragana, such as “Yabai” or “Cool”, are often written in katakana. Therefore, for example, a weighted editing distance that reduces the editing distance for replacing katakana with hiragana may be used.

(3) Difference in morpheme analysis cost value The morpheme analysis cost value is an index representing how probable the word break is among a plurality of word breaks. The morpheme analysis cost value is calculated from, for example, the appearance probability (occurrence cost) of a single word or the probability that a plurality of words appear continuously (concatenation cost). The morphological analysis cost value is used as an index for evaluating the fitness in the context of the correction rule candidate.

  The morpheme analysis cost value of the entire sentence is calculated by accumulating the sum of the concatenation cost of each morpheme from the beginning of the sentence to the end of the sentence and the word occurrence cost (accumulated cost). When an expression having a high degree of fitness in context is generated by applying the correction rule, the occurrence cost and the concatenation cost around the expression are reduced, so the morphological analysis cost value of the entire sentence is reduced. On the other hand, when an expression with low fitness in the context is generated, the occurrence cost and the connection cost around the expression increase, and the morphological analysis cost value of the entire sentence increases.

  Here, a score based on the morphological analysis cost value is calculated from the difference between the morphological analysis cost value of the entire sentence after application of the correction rule candidate and the morphological analysis cost value of the entire sentence before the correction. If the expression generated by the correction rule adapts to the context, the calculated score will be high.

The sentence composition index (score) includes (1) the appearance frequency (freq) in the correction rule candidate, (2) the edit distance (dist) between the unknown word and the correction morpheme based on the correction rule candidate, and / or ( 3) From the difference (cost) of the morpheme analysis cost value, for example, the following formula is used.
score = α ・ freq + β ・ dist + γ ・ cost
Here, α, β, and γ are weighting functions, and optimum values can be calculated by applying and learning correction rules. The correction rule can set a threshold value to be applied. When the threshold is set low, the number of correction rules to be applied increases, but the erroneous application of the correction rules included therein also increases. On the other hand, when the threshold value is set high, the number of correction rules to be applied decreases, but the erroneous application of the correction rules included therein can also be reduced.

  The correction rule candidate selection unit 15 selects one correction rule candidate suitable for the context based on the sentence composition index for each category. Here, the correction rule candidate selection unit 15 may select a correction rule candidate having a maximum score that is equal to or greater than the threshold. The correction rule candidate selection unit 15 outputs the selected correction rule candidate to the category selection unit 16.

  The category selection unit 16 compares scores based on the sentence composition index for the correction rule candidates selected for each category of the correction reference sentence information. The category selection unit 16 selects a category (a set of correction reference sentence information) from which the correction rule candidate with the highest score is extracted. The category selection unit 16 outputs the selected category to the correction rule selection unit 17.

The following table shows an example of scoring based on the sentence composition index.

  In general, a word that is strongly related to a specific category tends to appear biased in that category, and thus the score based on the sentence composition index is high. For example, when category 1 is a newspaper article sentence and category 2 is an IT-related sentence, “au” is present in the IT-related sentence rather than the probability that a common word “hero” appears in the newspaper article sentence. The probability of appearing is higher. Therefore, the score based on the sentence composition index of the correction rule candidate “eyu ⇒ au” is higher than the score of the correction rule candidate “eyu ⇒ hero”, and category 2 is selected.

  The correction rule selection unit 17 selects a correction rule based on the selected category. The correction rule selection unit 17 outputs the correction rule to the correction rule application unit 18.

  The correction rule application unit 18 applies the correction rule to the analysis target sentence. The corrected text information output from the correction rule application unit 18 is suitable for various text analysis.

  A feature of the present invention resides in that an appropriate correction reference sentence category is selected based on a sentence composition index for an analysis target sentence. Therefore, an embodiment of the correction rule search unit 14, the correction rule candidate selection unit 15, the category selection unit 16, and the correction rule selection unit 17 will be described with reference to FIG.

  FIG. 2 is an explanatory diagram of selection of categories and correction rules in the present invention.

(S21) The reference sentence storage unit 11 stores a set of corrected reference sentence information classified into a plurality of categories according to contents, feature words, and / or date and time information. The correction rule search unit 14 searches the reference text storage unit 11 for each category.
(S22) The correction rule search unit 14 extracts a correction candidate character string including a search key for each category. Then, the correction rule search unit 14 determines that the portion corresponding to the arbitrary character string in the extracted correction candidate character string is a portion that approximates an unknown word, and extracts it as a correction rule candidate. Here, the extracted correction rule candidates are associated with the category of the extraction source. For example, a correction rule candidate extracted based on a correction reference sentence classified as “category 1” is associated with “correction rule candidate 1”. There may be two or more correction rule candidates associated with a category for each category.
(S23) Each correction rule candidate is output to the correction rule candidate selection unit 15.
(S24) The correction rule candidate selection unit 15 selects one correction rule candidate suitable for the context based on the sentence composition index, one for each category. The correction rule candidate selection unit 15 outputs the selected correction rule candidate to the category selection unit 16.
(S25) The category selection unit 16 outputs the selected category to the correction rule selection unit 17. The correction rule selection unit 17 selects a correction rule based on the selected category.

  FIG. 3 is a system configuration diagram of the sentence analysis server in the present invention.

  According to FIG. 3, the sentence analysis server 2 includes a communication interface unit 20, an analysis target sentence input unit 21, and a sentence correction function unit 22. The sentence analysis server 2 connects to the Internet via the communication interface unit 20.

  Moreover, according to FIG. 3, the text analysis server 2 can communicate with the Web server 3 via the Internet. The Web server 3 is connected from the poster terminal 4 for contributors.

  The Web server 3 publishes Web documents such as blog texts and word-of-mouth comments, which are analysis target sentences, received from the terminal 4 for contributors. The sentence analysis server 2 acquires the Web document as an analysis target sentence from the Web server 3 via the Internet.

  The analysis target sentence input unit 21 receives the analysis target sentence via the communication interface unit 20. The analysis target sentence is output to the sentence correction function unit 22.

  The sentence correction function unit 22 is exactly the same as the function configuration unit described above with reference to FIG. The sentence correction function unit 22 inputs the analysis target sentence from the analysis target sentence input unit 21 and outputs the corrected sentence.

  FIG. 4 is a sequence diagram of the system according to the present invention.

(S401) The contributor terminal 4 posts the blog text, which is the analysis target sentence, to the Web server 3.
(S402) The sentence analysis server 2 acquires the analysis target sentence information ("E-yu is cool") from the Web server 3. The analysis target sentence is input to the sentence correction function unit 22.
(S403) The sentence correction function unit 22 divides the analysis target sentence information into morphemes. If an unknown word is included in the morpheme, the unknown word is extracted.
(S404) The sentence correction function unit 22 generates a search key including the unknown word extracted in S403 and at least one of the adjacent morphemes ahead and behind the unknown word.
(S405) The sentence correction function unit 22 searches for one or more correction rule candidates that approximate an unknown word using a search key for each category of the correction reference sentence information.
(S406) The sentence correction function unit 22 selects one correction rule candidate from the correction rule candidates for each category of the correction reference sentence information based on the sentence composition index.
(S407) The sentence correction function unit 22 selects a category having the highest score based on the sentence composition index of the correction rule for the correction rule candidate selected for each category of the correction reference sentence information.
(S408) The sentence correction function unit 22 selects a correction rule based on the category selected in S407. The sentence correction function unit 22 applies the selected correction rule to the analysis target sentence, and outputs the corrected sentence (“au is cool”).

  As described above in detail, according to the sentence correction program, method, and sentence analysis server of the present invention, even sentence information described in colloquial “complex expression” can be identified as unknown words by morphological analysis. It can be corrected to a sentence suitable for sentence analysis without being judged.

  By using a set (category) of correction reference sentence information that is similar in content to the analysis target sentence, the analysis target sentence can be corrected to a sentence having relevant semantic content. Further, by using a set of correction standard sentence information close to the description date or creation date and time of the analysis target sentence, the sentence can be corrected to reflect the trend at the time of sentence creation. As a result, the sentence can be corrected without being corrected to a sentence having a different meaning or excessively corrected.

  In addition, compared to the case where the correction reference sentence is a fixed sentence (for example, only newspaper articles) with high accuracy of morphological analysis, the proportion of unknown words in the corrected sentence can be reduced, and the accuracy of sentence correction can be reduced. Can be raised.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Text correction program 11 Standard text memory | storage part 12 Unknown word extraction part 13 Search key generation part 14 Correction rule search part 15 Correction rule candidate selection part 16 Category selection part 17 Correction rule selection part 18 Correction rule application part 2 Text analysis server 20 Communication Interface section 21 Analysis target sentence input section 22 Sentence correction function section 3 Web server 4 Posting terminal

Claims (5)

A sentence correction program that makes a computer function to correct an unknown word for analysis target sentence information including an unknown word,
A reference sentence storage means for storing a plurality of corrected reference sentence information transmitted to an unspecified number of third parties , classified into a plurality of categories, and stored;
An unknown word extracting means for dividing the analysis target sentence information into morphemes and extracting unknown words;
Search key generating means for generating a search key consisting of a wild card corresponding to the unknown word and at least one of the adjacent morphemes forward and backward with respect to the unknown word;
Correction rule search means for searching for one or more correction rule candidates searched by the wild card using the search key for each category of the correction reference sentence information;
Correction rule candidate selection means for selecting one correction rule candidate based on a sentence composition index from among the correction rule candidates for each category of the correction reference sentence information ;
For a correction rule candidate selected for each category of the correction reference text information, category selection means for selecting the category having the highest score based on the text composition index of the correction rule;
Correction rule selection means for selecting a correction rule based on the selected category;
Causing the computer to function as a correction rule applying means for correcting the analysis target sentence information by applying the selected correction rule ;
The sentence composition index is (1) the more frequently the character string corresponding to the correction rule candidate appears, and (2) the shorter the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, And / or (3) an index with a higher score as the difference between the morphological analysis cost values before and after the correction is smaller
Sentence correction program characterized by making a computer function like 2. The sentence correction according to claim 1, wherein the plurality of categories for classifying the plurality of corrected reference sentence information are based on the feature words and / or the date / time information thereof. program.   The analysis target sentence information is sentence information described by an unspecified number of users in a blog (Weblog), bulletin board, and / or word-of-mouth comment published via a network. The sentence correction program described in. A sentence correction method for correcting unknown words using a computer with respect to analysis target sentence information including unknown words,
A reference sentence storage unit that stores a plurality of corrected reference sentence information transmitted to an unspecified number of third parties , classified into a plurality of categories, and stored.
An unknown word extracting means for dividing the analysis target sentence information into morphemes and extracting unknown words;
A second step of generating a search key comprising a wildcard corresponding to the unknown word and at least one of the adjacent morphemes forward and backward with respect to the unknown word;
A third step of searching for one or more correction rule candidates searched by the wild card using the search key for each category of the correction reference sentence information;
A fourth step of selecting one correction rule candidate based on a sentence composition index from among the correction rule candidates for each category of the correction reference sentence information;
For a correction rule candidate selected for each category of the correction reference text information, a fifth step of selecting the category having the highest score based on the text composition index of the correction rule;
A sixth step of selecting a correction rule based on the selected category;
The analyzed sentence information, have a <br/> a seventh step of modifying by applying the selected said modified rule,
The sentence composition index is (1) the more frequently the character string corresponding to the correction rule candidate appears, and (2) the shorter the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, And / or (3) the sentence correction method, wherein the index is a higher score as the difference between the morphological analysis cost values before and after the correction is smaller . A sentence analysis server that acquires analysis target sentence information including an unknown word from another public server via a network, and corrects the unknown word,
A reference sentence storage means for storing a plurality of corrected reference sentence information transmitted to an unspecified number of third parties , classified into a plurality of categories, and stored;
An unknown word extracting means for dividing the analysis target sentence information into morphemes and extracting unknown words;
Search key generating means for generating a search key consisting of a wild card corresponding to the unknown word and at least one of the adjacent morphemes forward and backward with respect to the unknown word;
Correction rule search means for searching for one or more correction rule candidates searched by the wild card using the search key for each category of the correction reference sentence information;
Correction rule candidate selection means for selecting one correction rule candidate based on a sentence composition index from among the correction rule candidates for each category of the correction reference sentence information ;
For a correction rule candidate selected for each category of the correction reference text information, category selection means for selecting the category having the highest score based on the text composition index of the correction rule;
Correction rule selection means for selecting a correction rule based on the selected category;
Causing the computer to function as a correction rule applying means for correcting the analysis target sentence information by applying the selected correction rule ;
The sentence composition index is (1) the more frequently the character string corresponding to the correction rule candidate appears, and (2) the shorter the edit distance between the unknown word and the correction morpheme based on the correction rule candidate, And / or (3) a sentence analysis server having an index that has a higher score as the difference between the morphological analysis cost values before and after the correction is smaller .

Images