JP5363178B2 - Correction candidate acquisition device, correction candidate acquisition system, correction candidate acquisition method, correction candidate acquisition program - Google Patents

Description

  The present invention relates to a correction candidate acquisition device, a correction candidate acquisition system, a correction candidate acquisition method, and a correction candidate acquisition program for acquiring correction candidates when correcting an expression without changing the semantic content of the original text.

  2. Description of the Related Art Conventionally, morpheme analyzers that analyze text-based sentence information such as blogs published on the Internet as analysis targets and divide them into morphemes, which are the smallest units constituting sentences, have been used. For example, such a morphological analyzer stores in advance dictionary information in which a predetermined word (known word) and a part of speech of the word are associated with each other. Then, the sentence information to be analyzed is read to extract words constituting the sentence, and each of the extracted words is compared with words included in dictionary information stored in advance. By performing the analysis in this way, for example, it can be used for statistical processing such as what is being taken up on the Internet.

  However, when analyzing text information containing unknown words with unique text expressions such as blogs and news published on the Internet, electronic bulletin board services, etc., it is impossible to detect the corresponding word from the dictionary information. There is a case where morpheme cannot be divided.

  Therefore, the inventors of the present application have proposed a technique for correcting text information including unknown words by a unique text expression so that morphological analysis can be appropriately performed (Non-Patent Document 1 and Non-Patent Document 2). In these Non-Patent Document 1 and Non-Patent Document 2, in order to perform correct determination and evaluation (scoring) of the correction rule, the presence / absence information of the unknown word when the sentences before and after the application of the correction rule are analyzed is used. Yes.

Kazufumi Ikeda, Tadashi Yanagihara, Kazunori Matsumoto, Yasuhiro Takishima, “Proposal and Evaluation of Automatic Rule Generation Method to Normalize Blog-like Notation”, DEIM Forum, 2009 Kazufumi Ikeda, Tadashi Yanagihara, Kazunori Matsumoto, Yasuhiro Takishima, “Proposal of Automatic Rule Generation System for Correcting Shaking of Notations in Blogs”, Information Processing Society of Japan National Convention 5H-6, 2009

  In the methods of Non-Patent Document 1 and Non-Patent Document 2, a large number of correction rules are automatically generated based on a small number of general-purpose correction rules (primitive rules) given in advance by hand. However, these conventional methods have a problem in that the accuracy decreases when the number of given primitive rules is small. In addition, the conventional method has a problem that only a combination of correction rules given manually can be generated, and there are few cases that can be applied. Furthermore, in order to increase the number of cases, it is necessary to write a large number of correction rules manually, but there is a problem that much labor is required.

These problems of the prior art will be described with specific examples.
Assuming that the original text is “good weather today” and “ゎ” is an unknown word. In the conventional technique, if both of the correction rules “ゎ” → “wa” and “ゎ” → “ha” exist, the statistical learning function will change the “good weather today” instead of “good weather today”. Can be determined to be correct as a sentence, and the sentence can be corrected. However, if “ゎ → ha” does not exist as a correction rule, there is no other option, and an incorrect correction is made, such as “weather weather today”.
Also, when the original text is "I'm in trouble", and "Cha" is an unknown word, with the conventional technology, if there is a correction rule "Cha"->"Tetsu", it can be corrected as "I'm in trouble" . However, giving all the rules by hand required a great deal of effort.

  An object of the present invention is to provide a correction candidate acquisition device, a correction candidate acquisition system, a correction candidate acquisition method, and a correction candidate acquisition program that can acquire correction candidates without manually giving correction examples and correction rules. .

  The present invention proposes the following matters in order to solve the above problems. In addition, in order to make an understanding easy, although the code | symbol corresponding to embodiment of this invention is attached | subjected and demonstrated, it is not limited to this.

(1) The present invention is a correction candidate acquisition device that acquires correction candidates for correcting sentence information including unknown words, and includes a known word storage means (30) for storing known words and a plurality of words. The unknown word extracting means (40) for extracting an unknown word that is not stored in the known word storage means from the first sentence information that is generated, the corpus storage means (50) for storing a document set, and the unknown A search means (60) for searching and acquiring a correction candidate character string approximating an unknown character string including an unknown word extracted by the word extraction means from the corpus storage means , and the search means includes the unknown word extraction means Proposes a correction candidate acquisition device that searches the corpus storage means for a character string that matches a character string adjacent to at least one of the front and rear of the unknown word extracted by using the unknown word portion as a wild card .

According to this invention, the known word storage means stores a known word. The unknown word extraction means extracts unknown words that are not stored in the known word storage means from the first sentence information composed of a plurality of words. The corpus storage means stores a document set. The search means searches the corpus storage means for a correction candidate character string that approximates an unknown character string that includes the unknown word extracted by the unknown word extraction means and acquires it. The search means searches the corpus storage means for a character string that matches a character string adjacent to at least one of the front and rear of the unknown word extracted by the unknown word extraction means, using the unknown word portion as a wild card. Accordingly, correction candidates can be acquired without manually giving correction examples and correction rules. In addition, correction candidates can be easily searched.

(2) The present invention provides a correction rule for correcting the unknown word into a known word based on the correction candidate character string searched and acquired by the search means (60) in the correction candidate acquisition device according to (1) A correction candidate acquisition device characterized in that it includes a correction rule generation means (70) for generating.

  According to this invention, the correction rule generating means generates a correction rule for correcting an unknown word to a known word based on the correction candidate character string searched and acquired by the search means. Therefore, a correction rule can be automatically generated and output.

(3) In the correction candidate acquisition device according to (2) , the present invention provides a frequency evaluation value for the correction rule based on an appearance frequency at which the correction candidate character string is searched from the corpus storage means (50). It proposes a correction candidate acquisition device characterized by comprising frequency evaluation means (91) for providing

  According to this invention, the frequency evaluation means assigns a frequency evaluation value to the correction rule based on the appearance frequency at which the correction candidate character string is searched from the corpus storage means. Therefore, the generated correction rule can be scored.

(4) The present invention provides the correction candidate acquisition device according to (2) or (3) , wherein the character string distance between the unknown word and the corrected word after correcting the unknown word by the correction rule A character string distance calculating means (92a) for calculating the distance and a distance evaluating means (92) for assigning a distance evaluation value to the correction rule based on the character string distance calculated by the character string distance calculating means. The correction candidate acquisition apparatus characterized by this is proposed.

  According to this invention, the character string distance calculating means calculates the character string distance between the unknown word and the corrected word after the unknown word is corrected by the correction rule. The distance evaluation means assigns a distance evaluation value to the correction rule based on the character string distance calculated by the character string distance calculation means. Therefore, the generated correction rule can be scored.

(5) According to the present invention, in the correction candidate acquisition device according to any one of (2) to (4) , the first sentence information is corrected according to the correction rule, Correction means (80) for generating sentence information, morpheme analysis means (20) for performing morphological analysis on the second sentence information, and the morpheme for the correction rule used for generating the second sentence information A correction candidate acquisition device is provided, characterized by comprising morpheme analysis evaluation means (93) for assigning a morpheme analysis evaluation value according to the analysis result.

  According to this invention, the correction means performs the correction according to the correction rule on the first sentence information and generates the second sentence information. The morpheme analysis means performs morpheme analysis on the second sentence information. The morpheme analysis evaluation unit assigns a morpheme analysis evaluation value corresponding to the morpheme analysis result to the correction rule used for generating the second sentence information. Therefore, the generated correction rule can be scored.

(6) The present invention provides the correction candidate acquisition apparatus according to (5) , wherein the morpheme analysis means includes the unknown word extraction means and the known word storage means. A device is proposed.

  According to this invention, the morpheme analyzing means includes the unknown word extracting means and the known word storage means. Therefore, the calculation of the morphological analysis result of the original sentence and the unknown word extraction from the original sentence can be performed simultaneously, and the calculation time can be shortened.

(7) The present invention is a correction candidate acquisition system for acquiring correction candidates for correcting sentence information including unknown words, and includes a known word storage means (30) for storing known words and a plurality of words. The unknown word extracting means (40) for extracting an unknown word that is not stored in the known word storage means from the first sentence information that is generated, the corpus storage means (50) for storing a document set, and the unknown A search means (60) for searching and acquiring a correction candidate character string approximating an unknown character string including an unknown word extracted by the word extraction means from the corpus storage means, and the search means includes the unknown word extraction means a string that matches the character string which is adjacent to at least one of but the front and rear of the extracted unknown word, the unknown word portion as a wildcard, has proposed a modified candidate acquisition system for retrieving from the corpus storage means .

  According to this invention, the known word storage means stores a known word. The unknown word extraction means extracts unknown words that are not stored in the known word storage means from the first sentence information composed of a plurality of words. The corpus storage means stores a document set. The search means searches the corpus storage means for a correction candidate character string that approximates an unknown character string that includes the unknown word extracted by the unknown word extraction means and acquires it. Accordingly, correction candidates can be acquired without manually giving correction examples and correction rules.

(8) The present invention is a correction candidate acquisition method for acquiring a correction candidate for correcting sentence information including an unknown word, the known word is stored in the known word storage means (30), and is composed of a plurality of words An unknown word that is not stored in the known word storage means is extracted from the first sentence information to be stored, and a correction candidate character string that approximates an unknown character string that includes the extracted unknown word is stored as a document set The character string matching the character string adjacent to at least one of the front and rear of the extracted unknown word in the search for the correction candidate character string is obtained by searching from the corpus storage means (50). As a wild card, a correction candidate acquisition method for searching from the corpus storage means has been proposed.

  According to the present invention, the correction candidate acquisition method stores a known word in the known word storage unit, and the unknown word that is not stored in the known word storage unit from the first sentence information composed of a plurality of words. A word is extracted, and a correction candidate character string that approximates an unknown character string including the extracted unknown word is retrieved from a corpus storage unit that stores a document set and acquired. Accordingly, correction candidates can be acquired without manually giving correction examples and correction rules.

(9) The present invention is a correction candidate acquisition program for acquiring a correction candidate for correcting sentence information including an unknown word, the computer including a known word storage means (30) for storing a known word, An unknown word extraction means (40) for extracting an unknown word that is not stored in the known word storage means from the first sentence information composed of words, and a corpus storage means (50) for storing a document set Search means (60) for searching and acquiring a correction candidate character string approximating an unknown character string including an unknown word extracted by the unknown word extraction means from the corpus storage means , wherein the search means includes the unknown a string that matches the character string which is adjacent to at least one of the front and rear of the unknown word word extracting means has extracted, the unknown word portion as a wild card, as a search means for searching from said corpus memory means It has proposed a modification candidate acquisition program of the order to function.

  According to this invention, the known word storage means stores a known word. The unknown word extraction means extracts unknown words that are not stored in the known word storage means from the first sentence information composed of a plurality of words. The corpus storage means stores a document set. The search means searches the corpus storage means for a correction candidate character string that approximates an unknown character string that includes the unknown word extracted by the unknown word extraction means and acquires it. Accordingly, correction candidates can be acquired without manually giving correction examples and correction rules.

  According to the present invention, correction candidates can be acquired without manually giving correction examples and correction rules. In addition, correction rules can be automatically generated and stored. Furthermore, the generated correction rule can be evaluated accurately.

It is a figure which shows the structure of embodiment of the correction candidate acquisition apparatus by this invention. It is a flowchart which shows the flow of operation | movement of a correction candidate acquisition apparatus.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Note that the constituent elements in the present embodiment can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the present embodiment does not limit the contents of the invention described in the claims.

(Embodiment)
FIG. 1 is a diagram showing a configuration of an embodiment of a correction candidate acquisition apparatus according to the present invention.
The correction candidate acquisition apparatus of the present embodiment includes an analysis target sentence input unit 10, a morpheme analysis unit 20, a known word storage unit 30, an unknown word extraction unit 40, a corpus storage unit 50, a search unit 60, and a correction. A rule generation unit 70, a correction unit 80, a correction rule evaluation unit 90, and a correction rule score output unit 100 are provided.

  The analysis target sentence input unit 10 inputs sentences with many broken expressions such as blogs and Web texts as analysis target sentences (first sentence information: hereinafter referred to as original sentences). The analysis target sentence input unit 10 sends the input original sentence to the morpheme analysis unit 20 and the unknown word extraction unit 40.

The morpheme analysis unit 20 performs morpheme analysis on the sentence. The morphological analyzer 20 of the present embodiment uses a general Mecab as a morphological analyzer, but other types of morphological analyzers may be used. The morpheme analysis unit 20 performs morphological analysis on the original sentence obtained from the analysis target sentence input unit 10 and the corrected sentence (second sentence information) corrected by the correction unit 80.
The morpheme analysis unit 20 sends the morpheme analysis result for the original sentence obtained from the analysis target sentence input unit 10 to the unknown word extraction unit 40. Further, the morpheme analysis unit 20 sends the morpheme analysis result for the original sentence obtained from the analysis target sentence input unit 10 and the morpheme analysis result for the corrected sentence to the correction rule evaluation unit 90.

  The known word storage unit 30 is a dictionary database that stores a large number of known words (after being known). In the present embodiment, a word that is not stored in the known word storage unit 30 is handled as an unknown word (unknown word). For example, the above-mentioned “teru” of “being troubled” is after being known, but “chau” of “being troubled” is an unknown word.

  The unknown word extraction unit 40 refers to the known word storage unit 30 to check whether or not an unknown word is included in the original sentence obtained from the analysis target sentence input unit 10, and extracts the unknown word. The original text including the unknown word extracted by the unknown word extraction unit 40 is sent to the search unit 60.

  The corpus storage unit 50 stores a large number of document sets. The document stored in the corpus storage unit 50 of the present embodiment is preferably a document with few broken expressions, and for example, newspaper article data can be used.

The search unit 60 searches the corpus storage unit 50 for and acquires a correction candidate character string that approximates an unknown character string that includes the unknown word extracted by the unknown word extraction unit 40.
In order to generate a search sentence, for example, there are methods such as extracting several nearby phrases including unknown words and using the unknown word part as a wild card (arbitrary character string).
Original: “I do n’t know if it ’s OK”, Unknown: “K”
Sentence breaks: Good / Kid / How / Kan / Understanding / Ne / Non Extracting one phrase on both sides of an unknown word and using the unknown word as a wildcard (character string of arbitrary length) generates a search sentence it can.
Search text: “How to understand *”
Note that the search text generation method is not limited to this, and two phrases adjacent to the unknown word may be extracted, or the search text may be generated by other methods.
The search unit 60 generates such a search sentence, and acquires all the sentences that partially match the search sentence from the corpus storage unit 50 as correction candidate character strings.

The correction rule generation unit 70 generates a correction rule for correcting an unknown word to a known word based on the correction candidate character string acquired by the search by the search unit 60. That is, the correction rule is created by using the search result that matches the wild card portion in the search result of the search unit 60 as the corrected character string of the correction rule.
Here, the correction rule indicates a character string replacement rule for correcting a sentence including an unknown word into a literary expression that does not include an unknown word, such as “Kai-i → cute”. It should be noted that the above-described symbol “→” represents a correction rule for correcting from the word on the left side of the symbol to the word on the right side. The correction rule generation unit 70 newly generates this correction rule and sends the correction rule to the correction rule evaluation unit 90.

  The correction unit 80 corrects the original text including the unknown word extracted by the unknown word extraction unit 40 in accordance with the correction rule output from the correction rule generation unit 70. The correction unit 80 applies the correction rule to the original sentence, and generates a corrected sentence after the correction rule is applied. The corrected sentence generated by the correction unit 80 is sent to the morphological analysis unit 20.

The correction rule evaluation unit 90 includes a frequency evaluation unit 91, a distance evaluation unit 92, and a morpheme analysis evaluation unit 93, and evaluates the correction rule generated by the correction rule generation unit 70.
The correction rule score output unit 100 outputs the score of the correction rule evaluated by the correction rule evaluation unit 90.

As described above, in the present embodiment, the correction rule generation unit 70 generates a correction rule for correcting an unknown word to a known word based on the correction candidate character string acquired by searching by the search unit 60.
Hereinafter, the search for the correction candidate character string by the search unit 60 and the correction rule generated by the correction rule generation unit 70 and its evaluation (scoring) will be described in detail.

(1) The search unit 60 generates a sentence Q in which the unknown word X is replaced with a wild card for the sentence A (unknown character string) including the unknown word X, and a sentence B that partially matches the sentence Q Are all acquired from the corpus storage unit 50 as correction candidate character strings. Specific examples are shown below.
Original: “I don't know how,” Unknown: “K”
All sentences that partially match the sentence in which the unknown word “ka” is replaced with an arbitrary character string are acquired from the corpus.
Search text: “how * understanding”, where “*” indicates a wild card.
Search result 1: How do you know how?
Search result 2: I don't know how it is ...
Search result 3: I know how to do it ... I don't know how to do it ...
Search result 4: How do you know ... How do you know ...
Search result 5: I understand how it is ... I don't know if it is true ...
Search result 6: I don't know how to use ... I don't know how to use ...
Search result 7: I don't know how it is ... I don't know if it is ...

(2) The correction rule generation unit 70 generates a correction rule from the search result acquired in (1) above. In the above example, the part enclosed by “” is the character string after the unknown word “K” is corrected by the correction rule.
Amendment rule 1: “K” → “Kah”
Amendment rule 2: “K” → “K”
Amendment rule 3: “K” → “What should I do?”

(3) The correction rule evaluation unit 90 scores the correction rule generated by the correction rule generation unit 70.
First, the frequency evaluation unit 91 summarizes, for each correction rule, the frequency at which search results corresponding to the correction rule appear by the search by the search unit 60 in a table.
Table 1 shows a table compiled by the frequency evaluation unit 91.

  In the present embodiment, the appearance frequencies compiled by the frequency evaluation unit 91 are used as an index for scoring the correction rule.

本実施形態では、距離評価部９２の文字列距離算出部９２ａがこのようにして求めた文字列距離を修正ルールのスコアリングの一指標とする。 (4) For each correction rule generated in (2) above, the character string distance calculation unit 92a of the distance evaluation unit 92 acquires the distance between the character strings before and after the correction. Acquisition of the distance of the character string can use an existing method such as calculation of the Levenshtein distance.
Here, in order to obtain the character string distance, a weighted cost is given to insertion, deletion, replacement, etc. of the character string. For insertion / deletion of character strings, for example, the following costs are given.
Amendment rule 1: “K” → “Kah”, insertion / deletion cost: 0
Amendment rule 2: “K” → “K”, insertion / deletion cost: 1
Amendment rule 3: “K” → “What should I do?”, Insertion / deletion cost: 5
It is assumed that the correction rule having a smaller value obtains a higher score.
For character string replacement, for example, the following costs are given.
Correction rule: “：” → “ji”, replacement cost: 2
Correction rule: “ぢ” → “ni”, replacement cost: 4
Correction rule: “ぢ” → “e”, replacement cost: 8
In the replacement cost, although the character strings are more similar like “列 → ji”, the replacement cost is reduced.
The calculation of the character string distance is performed using the insertion / deletion cost and the replacement cost as described above.
Table 2 below shows an example of calculating the character string distance.

In the present embodiment, the character string distance obtained in this way by the character string distance calculation unit 92a of the distance evaluation unit 92 is used as an index for scoring the correction rule.

(5) The morpheme analysis unit 20 performs morpheme analysis on the post-correction sentence corrected by the correction unit 80 by applying the correction rules, and sends the disassembly element analysis result to the morpheme analysis evaluation unit 93. The morpheme analysis evaluation unit 93 obtains a morpheme analysis score (morpheme analysis evaluation value) based on the morpheme analysis result.
Here, as the morpheme analysis score (morpheme analysis evaluation value), for example, a technique used in the prior art (Non-Patent Documents 1 and 2) is used. In Non-Patent Document 1, the correction rule is applied to a large corpus and the correction rule is scored by a statistical value by comparing whether or not an unknown word is included in a sentence before and after the application of the rule. Is going.
As the morpheme analysis score, for example, morpheme analysis cost can also be used. Morphological analysis cost is an index that expresses how probable the phrase break is among multiple phrase breaks. The Morphological analysis costs are used in many existing morphological analyzers.

In the present embodiment, scoring is performed based on the correct answer rate, assuming that there are no unknown words as disclosed in the related art (Non-Patent Documents 1 and 2) as described above. Table 3 below shows an example of the morphological analysis score of the corrected sentence when scoring of the corrected sentence is performed by the conventional method.

  In the prior art (Non-Patent Documents 1 and 2), the process of specializing the correction rule is performed to improve the score. For example, “how to → how” has a higher morphological analysis score than “how to →”. Here, specialization refers to generating more specific correction rules based on general-purpose correction rules. When the morphological analysis score is obtained by the conventional technique, it is necessary to specialize the “Kai → Kaha” rule obtained in (2) to “How → Ka” as in the prior art. In this case, the appearance frequency and the character string distance score are the same as those of “Ka → Kaha”. In Table 3 above, this specialized correction rule 4 is also shown.

(6) Table 4 summarizes specific examples of the scores of the above (2) to (5).

By combining the scores obtained as shown in Table 4, the score of the correction rule is obtained, for example, as in the following equation.
Correction rule score = appearance frequency × α + character string distance × β + morphological analysis score × γ
Here, α, β, and γ are weighting parameter variables, and optimal values may be calculated by experiments or the like.
The correction rule score obtained in this way can be used to improve the accuracy of the correction rule or speed up the process by, for example, deleting a rule with a low score or determining the order in which the correction rule is applied. You can.

FIG. 2 is a flowchart showing an operation flow of the correction candidate acquisition apparatus.
Hereinafter, the flow of the operation of the correction candidate acquisition apparatus of the present embodiment will be described with reference to FIG.
Step (hereinafter referred to as S) 10 indicates an initial state in which the process is started. At this stage, the analysis target sentence input unit 10 inputs a sentence such as a blog to be analyzed. Also, a document set with few broken expressions is prepared in the corpus storage unit 50.
In S20, the unknown word extraction unit 40 extracts an unknown word from a sentence such as a blog to be analyzed and acquires a sentence including the unknown word.

In S30, the search unit 60 generates a search sentence from the unknown word sentence acquired in S20.
In S40, the search unit 60 searches the search sentence generated in S30 from the corpus storage unit 50, and acquires the search result. Further, the correction rule generation unit 70 generates a correction rule from the search result.

In S50, the frequency evaluation unit 91 acquires the appearance frequency for each correction rule.
In S60, the character string distance calculation unit 92a calculates the distance between the character strings before and after the correction for each correction rule.

In S70, the corrected sentence corrected by the correction unit 80 according to the correction rule is analyzed by the morpheme analysis unit 20, and a morpheme analysis score is obtained based on the morpheme analysis result evaluation value.
In S80, a comprehensive score of the correction rule is calculated based on the score of the correction rule acquired in S50 to S70.

  As described above, according to the present embodiment, the search unit 60 searches the corpus storage unit 50 for a correction candidate character string that approximates an unknown character string including an unknown word. The correction candidate can be acquired without giving Therefore, the correction rule can be automatically generated. Further, since the correction rule evaluation unit 90 performs scoring for each correction rule, the usefulness of the generated correction rule can be accurately evaluated, and a correction rule that can be corrected accurately can be automatically generated.

  The correction candidate acquisition apparatus, correction candidate of the present invention is recorded by recording the processing of the correction candidate acquisition apparatus on a computer-readable recording medium, causing the correction candidate acquisition apparatus to read and execute the program recorded on the recording medium. An acquisition system and a correction candidate acquisition method can be realized. The computer system here includes an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW (World Wide Web) system is used. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the embodiments, and includes designs and the like that do not depart from the gist of the present invention.

(Deformation)
(1) In the present embodiment, an example of a single device as a correction candidate acquisition device has been described. However, the present invention is not limited to this. For example, a correction candidate acquisition system that combines a plurality of devices may be configured. .

(2) In the present embodiment, the morphological analysis unit 20, the known word storage unit 30, and the unknown word extraction unit 40 have been described as independent forms. For example, the morphological analysis unit 20 may include the known word storage unit 30 and the unknown word extraction unit 40.

  In addition, although embodiment and a deformation | transformation form can also be used in combination as appropriate, detailed description is abbreviate | omitted.

DESCRIPTION OF SYMBOLS 10 Analysis object text input part 20 Morphological analysis part 30 Known word memory | storage part 40 Unknown word extraction part 50 Corpus memory | storage part 60 Search part 70 Correction rule production | generation part 80 Correction part 90 Correction rule evaluation part 91 Frequency evaluation part 92 Distance evaluation part 92a Character Column distance calculation unit 93 Morphological analysis evaluation unit 100 Correction rule score output unit

Claims (9)

A correction candidate acquisition device for acquiring a correction candidate for correcting sentence information including an unknown word,
Known word storage means for storing known words;
An unknown word extracting means for extracting an unknown word that is not stored in the known word storage means from the first sentence information composed of a plurality of words;
Corpus storage means for storing a document set;
Search means for searching for and obtaining a correction candidate character string that approximates an unknown character string including an unknown word extracted by the unknown word extraction means;
Equipped with a,
The search means searches the corpus storage means for a character string that matches a character string adjacent to at least one of the front and rear of the unknown word extracted by the unknown word extraction means, using the unknown word portion as a wild card. Correction candidate acquisition device. In the correction candidate acquisition apparatus according to claim 1 ,
A correction rule generating means for generating a correction rule for correcting the unknown word to a known word based on the correction candidate character string obtained by the search by the search means;
The correction candidate acquisition apparatus characterized by this. In the correction candidate acquisition apparatus according to claim 2 ,
Comprising frequency evaluation means for assigning a frequency evaluation value to the correction rule based on the appearance frequency at which the correction candidate character string is searched from the corpus storage means;
The correction candidate acquisition apparatus characterized by this. In the correction candidate acquisition device according to claim 2 or claim 3 ,
A character string distance calculating means for calculating a character string distance between the unknown word and the corrected word after correcting the unknown word by the correction rule;
Distance evaluation means for giving a distance evaluation value to the correction rule based on the character string distance calculated by the character string distance calculation means;
A correction candidate acquisition device comprising: In the correction candidate acquisition device according to any one of claims 2 to 4 ,
Correction means for correcting the first sentence information according to the correction rule and generating second sentence information;
Morphological analysis means for performing morphological analysis on the second sentence information;
A morpheme analysis evaluation unit that assigns a morpheme analysis evaluation value corresponding to the morpheme analysis result to the correction rule used to generate the second sentence information;
A correction candidate acquisition device comprising: In the correction candidate acquisition apparatus according to claim 5 ,
The morpheme analysis means includes the unknown word extraction means and the known word storage means;
The correction candidate acquisition apparatus characterized by this. A correction candidate acquisition system for acquiring correction candidates for correcting sentence information including unknown words,
Known word storage means for storing known words;
An unknown word extracting means for extracting an unknown word that is not stored in the known word storage means from the first sentence information composed of a plurality of words;
Corpus storage means for storing a document set;
Search means for searching for and obtaining a correction candidate character string that approximates an unknown character string including an unknown word extracted by the unknown word extraction means;
With
The search means searches the corpus storage means for a character string that matches a character string adjacent to at least one of the front and rear of the unknown word extracted by the unknown word extraction means, using the unknown word portion as a wild card. Correction candidate acquisition system. A correction candidate acquisition method for acquiring a correction candidate for correcting sentence information including an unknown word,
Storing a known word in a known word storage means;
Extracting unknown words that are not stored in the known word storage means from the first sentence information composed of a plurality of words,
A candidate correction character string that approximates an unknown character string including the extracted unknown word is retrieved from a corpus storage means that stores a document set, and is acquired .
In the search for the correction candidate character string, a correction candidate that searches the corpus storage means for a character string that matches a character string that is adjacent to at least one of the front and rear of the extracted unknown word, using the unknown word part as a wild card Acquisition method. A correction candidate acquisition program for acquiring correction candidates for correcting sentence information including unknown words,
Computer
Known word storage means for storing known words;
An unknown word extracting means for extracting an unknown word that is not stored in the known word storage means from the first sentence information composed of a plurality of words;
Corpus storage means for storing a document set;
Search means for searching for and obtaining a correction candidate character string approximating an unknown character string including an unknown word extracted by the unknown word extraction means from the corpus storage means , the search means comprising the unknown word extraction means Correction candidate acquisition program for causing a character string that matches a character string adjacent to at least one of the extracted unknown words to the front and rear to function as a search unit that searches the corpus storage unit using the unknown word part as a wild card .

Images